Unmanned store sales device and unmanned store system

Abstract

To provide an unmanned store sales device and unmanned store sales system that suggest dietary improvements suited to customer's health conditions during shopping in the store to support customer's health promotion.SOLUTION: An unmanned store operation system 1 detects the entry of a customer into an unmanned store with an optical radar 50, authenticates the customer having entered with an authentication display 40, detects a weight change when the authenticated customer takes out a commodity from a commodity shelf 20 and registers the commodity as an unpurchased commodity, generates a radar chart based on five major nutrients of all unpurchased commodities registered at that time, furthermore, generates a radar chart of the five major nutrients based on customer's health check data provided by a health management system 80, and outputs these two types of radar charts to a customer's user terminal 90 connected via a network 200 for display output.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to, for example, an unmanned store sales device and an unmanned store sales system for supporting health promotion.

Background Art

[0002] In companies, systems for regularly conducting health checkups for employees and collecting and analyzing the health check data and health-related data obtained there are generally widespread. The analysis information thus obtained may be utilized as guidance and advice to employees through industrial physicians and experts as part of trend promotion.

[0003] In efforts towards health promotion, not only for company employees but also for general individuals, businesses are making proposals to improve eating habits. As one example, there is a proposal to input nutritional analysis information into IC tags attached to foods in a convenience store and read the information with a sensor and display it on a display (Patent Document 1).

[0004] As another example, there is a proposal to diagnose the health condition from eating habits over a certain period and recommend products (Patent Document 2).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] In the case of Patent Document 1 described above, the nutritional balance of the food itself placed in the store serves as the standard for health promotion. In the case of Patent Document 2 described above, the diagnosis result of the diet serves as the standard for health promotion, and proposals for health promotion are made from a perspective far removed from the customer's own health condition.

[0007] The present invention has been made in view of such a background. In order to solve the above-described problems and support the health promotion of customers, it is an object of the present invention to provide an unmanned store sales device and an unmanned store system capable of proposing an improvement in a diet suitable for the health condition of a customer during shopping in a store.

Means for Solving the Problems

[0008] In order to solve the above-described problems and achieve the above object, an embodiment of the present invention is an unmanned store sales device that sells products containing nutrients in an unmanned store, the unmanned store sales device including: a first registration unit that registers, for each customer registered in advance, by using a plurality of nutrients, nutrient data indicating a health condition in association with the customer; a second registration unit that registers, for each product sold in the unmanned store, nutrient data belonging to the same classification as the nutrient data in association with the product; a first determination unit that determines the entry and exit status of a customer in the unmanned store; a recognition unit that recognizes whether a customer whose entry status has been determined by the first determination unit is a customer registered in advance; a second determination unit that determines a take-out operation or a return operation of a product sold in the unmanned store for a customer recognized by the recognition unit; a generation unit that, when a take-out operation or a return operation of a product sold in the unmanned store is determined by the second determination unit, generates graph data based on the nutrient data registered in the first registration unit for the recognized customer and generates graph data based on the nutrient data for each taken-out product with reference to the second registration unit; and a display processing unit that outputs the two types of graph data generated by the generation unit as displayable data.

[0009] In addition, another embodiment of the present invention is an unmanned store operation system for implementing unmanned store operation, including an unmanned store sales device for selling products containing nutrients in the unmanned store, an authentication display for receiving customer photography and operation input, a product shelf for displaying products and detecting weight changes related to taking out and returning products, an optical radar for performing detection for customer traffic analysis, and an authentication platform for authenticating customers. The unmanned store sales device includes a first registration unit for registering, for each pre-registered customer, nutrient data indicating a health state by using a plurality of nutrients; a second registration unit for registering, for each product sold in the unmanned store, nutrient data belonging to the same classification as the nutrient data; a first determination unit for determining the entry and exit states of customers in the unmanned store based on the detection of the optical radar; a recognition unit for recognizing whether a customer whose entry state has been determined by the first determination unit according to the customer photography and operation input by the authentication display using the authentication platform is a pre-registered customer; a second determination unit for determining a product taking-out operation or a return operation of a product sold in the unmanned store for a customer recognized by the recognition unit based on the detection of weight changes by the product shelf; a generation unit for generating graph data based on the nutrient data registered in the first registration unit for the recognized customer and generating graph data based on the nutrient data for each taken-out product with reference to the second registration unit when the product taking-out operation or the return operation of a product sold in the unmanned store is determined by the second determination unit; and a display processing unit for outputting the two types of graph data generated by the generation unit as displayable data.

Effect of the Invention

[0010] According to the present invention, it is possible to propose an improvement in a diet suitable for a customer's health state during shopping in a store in order to support the improvement of the customer's health.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0012] Hereinafter, Embodiment 1 of the present invention will be described with reference to the drawings. Note that the following description and drawings are merely examples for explaining the present invention, and for the sake of clarity of explanation, omissions and simplifications are made as appropriate. Also, the present invention can be implemented in various other forms. Also, unless otherwise specified, each component may be singular or plural.

[0013] <Embodiment 1> First, Embodiment 1 will be described with reference to FIGS. 1 to 8. First, the overall system will be described with reference to FIG. 1. FIG. 1 is a configuration diagram showing an example of the configuration of an unmanned store system according to Embodiment 1.

[0014] The unmanned store operation system 1 shown in FIG. 1 includes an unmanned store sales device 10A, a commodity shelf 20 provided with a weight sensor 21, a monitor 30, an authentication display 40 having a camera 41 and a touch panel 42, an optical radar 50, a settlement platform 60, an authentication platform 70, etc., and each is connected to a network 200. This network 200 is a communication network such as the Internet, a VPN (Virtual Private Network), or a wireless communication network on the premise of security.

[0015] In this unmanned store operation system 1, each component is connected to the network 200, but the present invention is not limited to this, and it may be configured as one system. Also, although the settlement platform 60 and the authentication platform 70 are configured to be included in the unmanned store operation system 1, since they are not components that need to be installed in the unmanned store, they may be connected via the network 200 as external devices without being included in the unmanned store operation system 1.

[0016] The unmanned store sales device 10A is a computer that efficiently configures and operates an unmanned store without a gate and supports health promotion for users, that is, customers, who are shopping in the unmanned store. As shown in FIG. 1, this unmanned store sales device 10A is further connected to a user terminal 90 such as a smartphone, which is an external terminal, and a health management system 80 via the network 200, and communicates data, signals, etc. to realize unmanned store operation.

[0017] The authentication display 40 is a device installed in the entry area with a predefined area within the space constituting the unmanned store. The camera 41 of the authentication display 40 is an imaging device that not only detects the entry and exit of customers who visit the entry area of the unmanned store, but also captures biometric parts, for example, the face, and acquires and generates face image data or its feature quantity data of the customer.

[0018] In addition, the touch panel 42 of the authentication display 40 receives an input of identification information (hereinafter referred to as "ID") for identifying the customer from the customer whose face is to be photographed by the above-mentioned camera 41.

[0019] It is assumed that each customer of the unmanned store has previously executed a user registration process with the authentication infrastructure 70 via the unmanned store sales device 10A (or the user terminal 90).

[0020] In this case, the unmanned store sales device 10A executes face image capture of the customer with the camera 41 and inputs the ID held in advance to the touch panel 42. The unmanned store sales device 10A sets the face image data (or its feature quantity data) and the ID thus obtained and performs the user registration process on the authentication infrastructure 70. For such a user registration process that sets biometric information and the ID, existing methods may be appropriately adopted.

[0021] Therefore, the customer for whom the above-mentioned user registration has been completed proceeds with the user authentication procedure on the authentication infrastructure 70 via the unmanned store sales device 10A by performing face capture with the camera 41 and ID input with the touch panel 42 in front of the authentication display 40.

[0022] Note that the information defining the above-mentioned entry area includes the coordinate values of each vertex constituting the shape of the area. Also, the origin coordinates of the coordinate system including the coordinate values indicate a point at any location (predefined) in the space constituting the unmanned store.

[0023] Subsequently, the optical radar 50 is, for example, a 3D LiDAR, which is a sensor capable of irradiating infrared laser light in a three-dimensional manner and measuring the shape of the irradiation target area and the objects present therein and the distance thereto.

[0024] The optical radar 50 in the present embodiment is responsible for grasping the space constituting the unmanned store, the entry area and the purchase area in the space, the location of the customer, and the specific processing of the reach target position (i.e., the product located at that position) on the product shelf. The observation values obtained by the optical radar 50 are distributed to the unmanned store sales device 10A via the network 200.

[0025] Also, the weight sensors 21 are installed at each product storage position on the product shelf 20, and their measurement results are distributed to the unmanned store sales device 10A via the network 200. The unmanned store sales device 10A holds and manages information regarding the weight of each product stored or displayed in each section of the product shelf 20 in a product management database (not shown).

[0026] The monitor 30 is a display device installed around the above-described product shelf 20, i.e., in the purchase area, and also includes a speaker. The monitor 30 outputs the information distributed from the unmanned store sales device 10A on the screen or outputs it as audio.

[0027] The information output on the screen includes, in addition to advertisements for customers, the status of the customer's purchase behavior (any one of being in the store, making a purchase, and in the settlement process).

[0028] Also, the information output as audio includes a message for guiding actions that optimizes the actions of the customer according to the status of the purchase behavior classified for the customer, the outstanding or success / failure of user authentication for the customer, and the product removal operation of the customer.

[0029] The user terminal 90 is a terminal used by customers of the unmanned store when using the service. Specifically, it is a device such as a smartphone, a tablet terminal, or a personal computer. This user terminal 90 shows an example of one terminal, and it is assumed that each customer (ID owner) who visits the unmanned store but is not shown in the figure owns one.

[0030] In addition, in the first embodiment, the authentication infrastructure 70 is a device that authenticates the user by biometric authentication, for example, and adopts authentication infrastructure technology that achieves both protection of privacy and high security. The authentication infrastructure 70 receives a user authentication request from the unmanned store sales device 10A through an API (Application Programming Interface) and responds with the authentication result.

[0031] The payment infrastructure 60 is, for example, a platform for credit card payment, and also receives a payment request from the unmanned store sales device 10A (regarding the goods taken out by the customer from the product shelf 20 and carried out from the purchase area) through an API (Application Programming Interface) and responds with the payment result.

[0032] The health management system 80 reads and registers health examination data from a server managed by an operator such as a hospital, a clinic, or a company that the customer with an ID has entrusted with the management of health examination data, and shares the health examination data with the unmanned store sales device 10A via the network 200.

[0033] Regarding the sharing of health examination data, it is assumed that the unmanned store sales device 10A cannot identify the ID or the customer and view the health examination data itself, and is limited to being used for processing for health promotion between each customer's user terminal 90.

[0034] Note that the unmanned store sales device 10A may be configured to perform a simple health check in the unmanned store to obtain health check data. In any case, since the health check data is subject to the handling of personal information, it is assumed that, on the premise of obtaining the permission of the individual, it can be handled with restrictions.

[0035] Next, the unmanned store sales device according to Embodiment 1 will be described with reference to FIGS. 2 to 4B. FIG. 2 is a configuration diagram showing an example of the hardware configuration of the unmanned store sales device according to Embodiment 1, FIG. 3 is a block diagram showing an example of the unmanned store sales device according to Embodiment 1, FIG. 4A is a table diagram showing an example of data management according to Embodiment 1, and FIG. 4B is an explanatory diagram for explaining an example of the five major nutrients obtained from the health check data according to Embodiment 1.

[0036] First, the hardware configuration of Embodiment 1 will be described with reference to FIG. 2. As shown in FIG. 2, the unmanned store sales device 10A includes a CPU 11 that controls the entire device, a memory 13 that stores a program 12 such as the processing according to Embodiment 1 to be executed by the CPU 11 and stores various data during execution, an operation device 14 including a keyboard and a display device, an external storage device 15 that registers and manages various data in the form of a table or the like, a communication IF 16 that is connected to the network 200 and controls communication with external terminals (each device such as the user terminal 90 shown in FIG. 1), and a bus 17 that is connected to each unit in the device and controls communication of data, signals, etc. inside the device. The external storage device 15 registers and manages tables shown in FIGS. 4A and 4B.

[0037] Subsequently, the functional blocks of Embodiment 1 will be described with reference to FIGS. 3, 4A, and 4B. As shown in FIG. 3, the functional block configuration of the unmanned store sales device 10A includes a communication control unit 101, an entrance / exit determination unit 102 that is a first determination unit, a user recognition unit 103, an action determination unit 104 that is a second determination unit, a data storage unit 105, an un-purchased determination unit 109, a nutrient analysis unit 110, a graph generation unit 111, a settlement processing unit 112, and a display processing unit 113.

[0038] The communication control unit 101 is the function of the communication IF 16 that controls communication with each device in FIG. 1 connected to the network 200. The entry / exit determination unit 102, user recognition unit 103, behavior determination unit 104, non-purchase determination unit 109, graph generation unit 111, payment processing unit 112, and display processing unit 113 are functions realized by the program 12 executed by the CPU 11.

[0039] The entry / exit determination unit 102 determines whether a customer has entered or exited the store based on the acquired data of the optical radar 50 when a person enters or exits the store. The user recognition unit 103 performs face recognition of the customer based on the captured data of the camera 41 and acquires an ID from the operation of the touch panel 42 by the customer to recognize the user.

[0040] The behavior determination unit 104 determines the customer's movement route based on the acquired data of the optical radar 50, and determines whether a product is taken out from the product shelf or returned to the product shelf based on the position and movement of the customer in the store. The non-purchase determination unit 109 determines whether the product has not been purchased based on the determination result of the behavior determination unit 104. Here, during the process of taking out a product, it is determined as the state of the product not being purchased.

[0041] The data storage unit 105 is provided in the external storage device 15 and includes a health score table (hereinafter referred to as "TBL") 106, a user information TBL 107 of the second registration unit, and a health examination TBL 108 of the first registration unit.

[0042] As shown in FIG. 4A for example, the health score TBL 106 stores values obtained by evaluating, for example, the five major nutrients of carbohydrates, lipids, proteins, vitamins, and minerals as health items in, for example, 10 levels in association with the store inventory and the IDs of the products arranged on the product shelves. In the case of the product ID "P0125", as shown in FIG. 4A for example, the carbohydrate is "3", the lipid is "1", the protein is "2", the vitamin is "7", and the mineral is "4". This is an example of the product ID "P0125", and the absolute values of the health items are determined for each product.

[0043] As shown in, for example, FIG. 4A, the user information TBL107 stores authentication result data, in-store location data, purchase behavior status, observation time data, unpurchased item data, and overall evaluation data associated with the corresponding ID for pre-registered customers. The authentication result data is registered in the "OK" state when the user recognition unit 103 can authenticate the user at the time of entering the store, and in the "NG" state otherwise.

[0044] The location data is coordinate data indicating the position of the customer in the store based on the acquisition data of the optical radar 50 by the behavior determination unit 104. The observation time data indicates the time measured, for example, by the timer control of the CPU 11.

[0045] The unpurchased item data is data that is managed such that the number of registered data increases according to the number of unpurchased items when taken out from the product shelf for purchase, and the data for the corresponding product is deleted when returned to the product shelf. This unpurchased item data registers the sum of the points attached to each health item of the copy of the health score (the arrangement shown by the arrow AR0 in FIG. 4A) associated with the product ID. The overall evaluation data is data that calculates and registers the total value of points for each health item for all unpurchased items, and also adds the total points and registers them as bonus points.

[0046] For example, for the customer with ID "001", as shown in FIG. 4A, the authentication result is "OK" and the purchase behavior status is "in the store", indicating that the customer is still shopping. At 12:10 on September 12, 2023, it can be seen that there are 2 unpurchased items, namely 1 "product ID: P0125" and 2 "product ID: P0303", and the settlement has not been made yet. The overall evaluation data is the bonus points "57", which is the sum of the points "17" and "40" of all unpurchased items.

[0047] And in the health check TBL108, as shown in FIG. 4B, the health check results of each customer are registered as data and are imported from the health management system 80 via the network 200.

[0048] Furthermore, in the health check TBL108, the five major nutrients required for each customer are evaluated based on the data indicating the health check results, and 10 - level points are calculated for each nutrient. As a result, each nutrient is registered corresponding to the health items.

[0049] Since the evaluation of the five major nutrients is taken as an example of 10 levels, for each nutrient, with the point "5" being the preferred evaluation value, when the value is 4 or less, the smaller the value, the more it indicates under - intake, and when the value is 6 or more, the larger the value, the more it indicates over - intake. Of course, regarding the evaluation of the five major nutrients, 10 levels are just an example, and it can be less than 10 levels or more than 10 levels. The appropriate number of points should be the median value.

[0050] Carbohydrates are evaluated based on the total intake amount and its type. Specifically, the measurement of blood glucose level and HbA1c is used to evaluate carbohydrate metabolism. These numerical values indicate the metabolic state of carbohydrates. For example, on a 10 - level evaluation scale, with point "1" indicating under - intake and point "10" indicating over - intake, appropriate carbohydrate intake is evaluated with the intermediate point "5".

[0051] In the first embodiment, although not shown in the figure, it also includes controlling the display on the display screen of the user terminal 90 that under - intake means the risk of hypoglycemia and energy deficiency, and over - intake means the risk of obesity and diabetes.

[0052] Lipids are evaluated based on the total lipid intake, especially the balance between saturated fatty acids and unsaturated fatty acids. Specifically, a lipid profile test is conducted, and numerical values such as total cholesterol, LDL cholesterol, HDL cholesterol, and triglyceride are used as references for evaluation. For example, when excessive intake of saturated fatty acids is the case, the value is close to point "1" on the evaluation scale, and when the intake is small, the point is higher.

[0053] It is desirable that appropriate lipid intake is evaluated with the intermediate value, that is, point "5". Since an appropriate lipid balance is important for cardiovascular health, it is also necessary to control the display of this information on the display screen of the user terminal 90.

[0054] Protein is evaluated based on appropriate intake and protein quality. Specifically, to evaluate the protein status, the concentration of albumin obtained from a blood test is measured.

[0055] Appropriate protein intake is evaluated at the intermediate value, i.e., point "5", and insufficient intake approaches point "1" according to the intake amount, while excessive intake approaches point "10". Insufficient intake needs to indicate risks such as muscle loss, and it is also necessary to perform display control of this information on the display screen of the user terminal 90.

[0056] In the case of vitamins and minerals, the results of blood tests are referred to evaluate respective deficiencies and excessive intakes. Appropriate intake of vitamins and minerals is evaluated at the intermediate value, i.e., point "5", and insufficient intake approaches point "1" according to the intake amount, while excessive intake approaches point "10".

[0057] In the example shown in FIG. 4B, for the customer with ID "100", the points for health items are: carbohydrates are "8", lipids are "5", protein is "4", vitamins are "5", and minerals are "4". It can be seen that carbohydrates are in a state of excessive intake, and the others show appropriate values.

[0058] Note that the function of obtaining the points for each of the five major nutrients based on the above-mentioned health check results is performed by the nutrient analysis unit 110.

[0059] Furthermore, the non-purchase determination unit 109 determines that it is in a non-purchased state indicating the stage where the product is taken out from the product shelf according to the determination of the action determination unit 104, and additionally registers it in the user information TBL107. On the other hand, the non-purchase determination unit 109 cancels the non-purchased state and deletes the registration in the user information TBL107 when the non-purchased product is returned to the product shelf.

[0060] As shown in FIG. 5 for example, the graph generation unit 111 generates and transmits a radar chart (graph data) while being connected via the customer's user terminal 9 and the network 200. The graph generation unit 111 calculates the points of each health item corresponding to the five major nutrients, namely, "carbohydrates", "lipids", "proteins", "vitamins", and "minerals", and generates a pentagonal radar chart (graph data) that can be displayed on the screen of the user terminal 90.

[0061] Note that the basic health items are not limited to five, and one or more elements such as total protein (TP), albumin (ALB), transthyretin (TTR), total lymphocyte count (TLC), transferrin (Tf), cholinesterase (CH-E), and total cholesterol (TC) may be added. In that case, the radar chart will be a polygon with five or more sides.

[0062] Also, since the more health items there are, the more points for the customer to check, the customer may be able to make selections through operations on the user terminal 90 or the touch panel 42 in the store.

[0063] Then, the settlement processing unit 112 obtains the data of the unpurchased items determined to be unpurchased by the unpurchased determination unit 109 and collaborates with the settlement infrastructure 60 to execute the settlement of the unpurchased items. The trigger for executing the settlement occurs at the timing when the entrance / exit determination unit 102 detects the customer's leaving the store, that is, moving outside the store. In the first embodiment, there is no gate for settlement provided as a facility within the store. Therefore, when a customer who has been authenticated exits the store, there is no need to perform a settlement procedure at the gate, and it is possible to exit the store while carrying the unpurchased items.

[0064] The display processing unit 113 outputs to the communication control unit 101 in order to transmit the radar chart (graph data) received from the graph generation unit 111 to the user terminal 90, and also outputs display data for notifying the user terminal 90 of the settlement result of the settlement processing unit 112 to the communication control unit 101.

[0065] Next, the operation of the unmanned store sales device 10A will be described with reference to FIGS. 5 to 8. FIG. 5 is an explanatory diagram for explaining an example of a screen transition during product purchase according to the first embodiment, and FIG. 6 is a flowchart for explaining an example of the operation of the unmanned store sales device according to the first embodiment. FIG. 7 is a flowchart for explaining an example of the operation related to the display of the unmanned store sales device according to the first embodiment, and FIG. 8 is a flowchart for explaining an example of the operation related to the settlement of the unmanned store sales device according to the first embodiment.

[0066] Regarding the entry of a customer, the entry / exit determination unit 102 determines whether the customer has entered the store (step S601). If it is determined that the customer has entered the store (YES route in step S601), the user recognition unit 103 identifies the customer through user authentication from the authentication infrastructure 70 (step S602) and obtains a result indicating that the user, that is, the customer, has been recognized (step S603). Note that the determination as to whether the customer is a registered customer is made by collating the user information TBL107 in which registered customers have been registered in advance.

[0067] When the customer is recognized (YES route in step S603), the movement analysis of the customer within the store is started, and "in the store" is set in the purchase behavior status of the user information TBL107 while the customer is moving within the store (step S604).

[0068] Then, for the recognized customer, the weight sensor 21 detects the taking in and out of the products placed on the product shelf 20, and the user information TBL107 is updated by specifying the products to be taken in and out by the non-purchase determination unit 109 and associating them with the customer taking the action.

[0069] When the customer takes out a product from the product shelf 20 and, for example, the taking-out state continues for a certain period of time (for example, 10 seconds) or more, the corresponding product becomes a purchase candidate and the taking out of the product is confirmed (YES route in step S606), and the corresponding product is additionally set (stored) in the user information TBL108 as an un-purchased product (step S607).

[0070] Thus, it is difficult to instantly determine whether a product has been taken out. Therefore, since there are actions such as a customer just picking up a product, it is preferable to make a judgment after observing for a certain period of time in light of human behavior principles, which can reduce the processing load.

[0071] Also, when the weight sensor 21 detects the taking in and out of products placed on the product shelf 20, and when an unpurchased product is returned to the product shelf 20, if the customer returns the product from the product shelf 20 and the state continues for a certain period of time (for example, 10 seconds) or more, similar to the above-mentioned taking out, the return of the corresponding product is confirmed (NO route in step S606), and the data of the returned unpurchased product is deleted from the user information TBL108 (step S612). Then, the process returns to step S605.

[0072] Thus, observing for a certain period of time for the return of a product also leads to a reduction in the processing load.

[0073] The above-mentioned step S606 is a trigger for determining the taking in and out of products, and the purchase candidates for the products change at this timing. At this timing, the point calculation of the five major nutrients based on the customer's health status and the point calculation of the five major nutrients based on the unpurchased products are executed as nutrient analysis processing (step S608).

[0074] First, the health items and points of the unpurchased products stored in association with the corresponding customer are read from the user information TBL107. Subsequently, the health diagnosis data of the corresponding customer is read from the health diagnosis TBL108, and the points for each health item are calculated by the nutrient analysis unit 110 as described above.

[0075] Regarding the data being processed, the data to be temporarily stored is stored in the memory 13, and the data to be added or updated as table data is appropriately stored in the user information TBL107 and the health diagnosis TBL108.

[0076] In this way, when all the points of the unpurchased items and the health check results are aligned, the process proceeds to step S609. In this step S609, the overall evaluation and graph display shown in FIG. 7 are executed.

[0077] First, with reference to the user information TBL107, a process is executed to total and store the scores for each health item for all unpurchased items that are purchase candidates (step S701). As a result, the total points for each of "carbohydrates", "lipids", "proteins", "vitamins", and "minerals" are obtained. Further, the total points calculated for each health item are totaled, and a process is executed to store that total value as a bonus point (step S702).

[0078] Then, based on the total points for each health item in the unpurchased items obtained in step S702, a 10 - level radar chart (graph data) is generated. Further, with reference to the health check TBL108, a 10 - level radar chart (graph data) is generated based on the points for each health item in the health check results of the corresponding customer (step S703). The two types of radar charts described above are generated by the graph generation unit 111.

[0079] The two types of radar charts (graph data) and the bonus points generated in this way are output to the communication control unit 101 by the display processing unit 113 in the form of display data in order to be displayed on the user terminal 90 (step S704).

[0080] Regarding the connection to the user terminal 90, it is possible to achieve it by logging in with the customer's ID etc. through a browser or an app. Since this is general technology, a specific explanation is omitted here. Note that since it involves handling data related to an individual's health, it remains on the premise of protecting personal information.

[0081] Supplementary explanations will be given for the radar chart and bonus point display in step S704 with reference to FIG. 5. In FIG. 5, 90A, 90B, and 90C respectively represent the display screens of the user terminal 90, and for the sake of explanation, it is assumed that the screens transition in the order of display screens 90A, 90B, and 90C.

[0082] In FIG. 5, "nutrients you need" is a graph (radar chart indicated by a dashed line) based on the points of health items according to the health check results already explained, and "nutrients of the products you picked up" is a graph (radar chart indicated by a dotted line) based on the points of health items according to the unpurchased products already explained.

[0083] As shown in the display screen 90A of FIG. 5, in the initial display stage, the radar chart may be set to "0" without displaying the bonus points, and only the two types of radar charts may be displayed on the display screen 90A of the user terminal 90. In this case, when the display process of step S609 is executed next due to the weight change in step S605, the bonus points may be displayed together with the radar chart.

[0084] Here, the display screens 90B and 90C indicated by the display transitions shown by the arrows AR1 and AR2 are just examples. The display screen 90C where each point of the health item satisfies "5" has a higher point value.

[0085] In the stage of exchanging or adding unpurchased products before settlement, a process of displaying bonus points may be performed when there is a change such that the points of each health item approach "5" on average compared to just before.

[0086] After that, the bonus points may also be updated when it is determined that the points approach "5" on average more than the previous time. Of course, the bonus points may be displayed when it is determined that the points are "5" or more on average from the first display. This display method of bonus points is just a display mode for enhancing the customer's health awareness, and various designs are possible.

[0087] Regarding the bonus points, if the customer completes the payment without changing the unpurchased items, the bonus points are finalized and stored as they are, providing benefits such as being able to be used when the customer purchases the next item.

[0088] In this way, for customers, they can check the nutrients lacking from the radar chart based on the health check results, so it becomes possible to encourage them to supplement the lacking nutrients while exchanging the unpurchased items. In particular, since the two types of radar charts are overlapped and displayed so that the status of the health items can be compared, the index of which nutrients should be supplemented is visually intuitive and easy to understand. Furthermore, when an unpurchased item is taken out from the product shelf 20, the radar chart changes in real time, so the action of reading and understanding the nutritional components of each item can be omitted.

[0089] After the graph display in step S609 is completed, if there are still customers in the store, the process returns to step S605 (the NO route of step S610), and the same process is repeatedly executed. On the other hand, if the customer has left the store, the payment process is executed (step S611).

[0090] Specifically, as shown in FIG. 8, when the user information TBL108 is referenced and there are unpurchased items for the corresponding customer (the YES route of step S801), the payment process is executed in cooperation with the payment platform 60, and this process ends (step S802). When the customer has left the store, the purchase behavior status is updated to "payment completed", like the customer with ID "002" shown in FIG. 4A.

[0091] As described above, according to the first embodiment, by proposing foods that promote the health of customers from the information related to the health of customers who visit the store and the products displayed in the store, that is, the nutritional information of foods, it becomes possible to propose an improvement in the diet suitable for the health status of customers during shopping in the store. As a result, it is possible to support the improvement of customers' health.

[0092] In particular, for customers who visit the store, just by picking up the products they want to buy, they can visually and real-time check in the form of a radar chart whether the products contribute to improving their own health. As a result, the rhythm of the action of purchasing products in the store is not disrupted, and no extra time is taken from the customers.

[0093] Also, in the process where the radar chart optimal for health improvement changes in conjunction with the customer taking products from the shelf and putting them back, during that time, the customer will enjoy shopping in a game-like feeling, which also leads to an improvement in the purchasing desire. On the other hand, for the store side, proposing health improvement to the customer enables the recommendation of products with benefits to the customer, which also leads to the promotion of purchases.

[0094] Furthermore, during shopping, since the customer only needs to view the radar chart from the screen of the user terminal, the customer can enjoy simplicity throughout the overall operation after entering the store.

[0095] Also, since an unmanned store is used, there is no need to face the store employees, and it is possible to complete the purchase of products without interacting with a third party.

[0096] Furthermore, since there is an incentive for customers in the form of points, it can be expected to enhance the reach effect, which also leads to an increase in the average customer spending.

[0097] <Embodiment 2> Next, Embodiment 2 of the present invention will be described with reference to FIG. 9. FIG. 9 is a flowchart for explaining an example of the operation related to the health judgment mode of the unmanned store sales device according to Embodiment 2. Since Embodiment 2 has some additional processing in the same configuration as Embodiment 1 described above, the additional part will be described with reference to the drawings.

[0098] In the aforementioned Embodiment 1, the health examination results were adopted as the customer's health condition. However, the present invention is not limited to this, and nutrient analysis may be performed in consideration of the current health condition of the customers who enter the store to generate a radar chart. This process shall be carried out after the customer enters the store and completes user authentication.

[0099] In the case of the second embodiment, the health judgment mode is set, and the customer can arbitrarily operate and execute the mode from the touch panel 42 of the user terminal 90 or the authentication display 40.

[0100] When the health judgment mode starts by the customer's operation, guidance such as voice and display is output from the authentication display 40, and the customer is guided to move in front of the camera 41. The face image of the customer is captured by the camera 41, and the face image data is acquired (step S901). The facial expression is analyzed from the face image data according to the analysis logic prepared in advance, and the current health condition of the customer is determined according to the analysis result (step S902). This analysis process is executed by the nutrient analysis unit 110.

[0101] Regarding the method of the above-mentioned analysis logic, generally, there are the following methods. For facial expression analysis, there is a method of using a machine learning algorithm that detects the feature points of the face and identifies specific patterns related to facial expressions. Facial expression analysis is realized by associating specific patterns related to facial expressions with health conditions and emotions. Also, regarding the evaluation of health, the health condition can be evaluated based on the result of facial expression analysis. For example, information such as stress, fatigue, and anxiety can be extracted from the facial expression analysis result of the face and used as one of the health examinations.

[0102] Also, regarding data collection and machine learning, it is necessary to train a learning model using a large-scale dataset to improve performance. The training data needs to include images representing various health conditions and emotions.

[0103] After the current health state is determined in step S902, health items related to the determined health state are determined (step S903).

[0104] If the result of the facial expression analysis is stress, fatigue, or anxiety, for example, the vitamin B group becomes one of the appropriate nutrients. The vitamin B group includes B1 (thiamine), B2 (riboflavin), B3 (niacin), B6 (pyridoxine), B9 (folic acid), B12 (cobalamin), and the like.

[0105] When the health item is determined in step S903, a process of determining a coefficient to be applied to the health item and storing it in the data storage unit 105 is executed (step S904). This coefficient shall be used in the graph display of step S609.

[0106] In the determination of step S902, if the above-described coefficient is set to correspond to the negative elements of the facial expression, it may be set to a value of 1 or less to reduce the points to 1 times or less. On the other hand, if the positive elements of the facial expression are also taken into account, for example, it may be set to a value of 2 or less at a maximum of 2 times so that the points can be doubled.

[0107] As described above, according to the second embodiment, since the radar chart is generated by taking into account the current health state in the customer's health diagnosis result, it is possible to support health promotion in a more realistic manner than in the first embodiment described above.

[0108] <Embodiment 3> Next, Embodiment 3 of the present invention will be described with reference to FIGS. 10 and 11. FIG. 10 is an explanatory diagram for explaining an example of one-screen transition at the time of purchasing a product according to Embodiment 3, and FIG. 11 is a flowchart for explaining an example of an operation related to the display of the unmanned store sales device according to Embodiment 3. Since Embodiment 3 has an additional part in the same configuration as Embodiment 1 described above, the additional part will be described with reference to the drawings.

[0109] In the foregoing Embodiment 1, if there is a point exceeding the point "5" (predetermined value) considered appropriate in the radar chart, it was expressed in 10 levels as it was. However, the present invention is not limited to this. As in this Embodiment 3, the display magnification may be adjusted so as to fall within the display range of the point "5" to generate a radar chart (graph data).

[0110] In FIG. 10, 90R and 90S respectively show the display screens of the user terminal 90, and AR3 indicates the direction changing from the foregoing Embodiment 1 to this Embodiment 3. For example, in the foregoing Embodiment 1, as in the display screen 90R, even if the nutrients of the product taken in hand exceed the range of the point "5", the radar chart is displayed as it is. In this Embodiment 3, as in the display screen 90R, when the nutrients of the product taken in hand exceed the range of the point "5", the display magnification is changed and the display of the radar chart is changed so as to fall within the range of the point "5".

[0111] The difference between Embodiments 1 and 3 lies in the overall evaluation and graph display in step S609, so the difference will be described with reference to FIG. 11. The content described in the foregoing Embodiment 1 is omitted here. When the processing up to the bonus point calculation is completed in step S702, in the graph display of the radar chart, it is determined whether there is any health item exceeding the point "5" serving as the reference as the specified size.

[0112] If there is even one, the health item having the largest point among them becomes the target for determining the scale ratio (YES route in step S1104). In the example of FIG. 9, among the "nutrients of the product taken in hand", vitamin has the largest point, so the scale ratio at which the point of this vitamin falls within "5" is calculated and applied. All the sizes of the health items are uniformly reduced at the ratio calculated in this way (0.75 is adopted as a reasonable value in FIG. 10) (step S1105).

[0113] In this way, as shown in FIG. 9, a display screen 90S with the size of the radar chart changed is created and displayed and output to the user terminal 90 (steps S703 and S704).

[0114] As described above, according to the third embodiment, even when one of the five major nutrients (health items) is over-intaken, it is possible to reduce the display size and present a display considering the balance of the five major nutrients. It is also possible to support health promotion in this third embodiment.

[0115] The arrangement forms of the various functional units, various processing units, and various databases of the unmanned store sales device described above are merely examples. The arrangement forms of the various functional units, various processing units, and various databases can be changed to an optimal arrangement form from the viewpoints of the performance, processing efficiency, communication efficiency, etc. of the hardware and software provided in these devices.

[0116] In addition, the configuration (such as schema) of the database for storing the various data described above can be flexibly changed from the viewpoints of efficient use of resources, improvement of processing efficiency, improvement of access efficiency, improvement of search efficiency, etc.

[0117] Also, in an unmanned store, since there is no need to line up at the cash register, the purchased goods can be purchased without being seen by store clerks or other customers visiting the store, so the privacy of customers is protected. On this premise, it is possible to manage data in time series at each stage, such as product selection during shopping, movement analysis within the unmanned store, and purchase history, linked to individual customers, and it is also possible to support health promotion in the context of the customer's diet throughout. In this case, it is possible to judge whether food has been purchased with reference to the radar chart presented during shopping, and as one method, it is possible to appeal to customers by incentives such as reduction points so that the motivation for further health promotion increases at the next shopping.

[0118] In addition, in each of the above-described embodiments, the user terminal 90 is caused to display a radar chart to support the improvement of the customer's health. However, the present invention is not limited to this, and the monitor 30 may be caused to display a radar chart to support the improvement of the customer's health. In this case, since it is the monitor 30 installed in the store, the movement analysis of other customers in the store may be performed by the action determination unit 104 to draw attention to the screen so that the radar chart cannot be seen by a third party. Further, when other customers are nearby, a device may be provided so as not to display. Of course, a structure for preventing peeping may be provided on the screen of the monitor 30.

[0119] Also, as an application of each of the above-described embodiments, when a health item that results in excessive intake is found from the radar chart, a message recommending a product (food) that supplements other nutrients and suppresses excessive intake may be displayed on the screen of the user terminal. Therefore, when exchanging for an un-purchased product to be recommended, an incentive may be provided by multiplying the bonus points by a coefficient prepared in advance.

[0120] In addition, in each of the above-described embodiments, the health diagnosis result is directly adopted to generate a radar chart of the five major nutrients. However, the present invention is not limited to this, and by focusing on the insufficient intake and excessive intake of each nutrient, the number of points lacking or the surplus number of points with respect to the appropriate point "5" may be presented in a graph. Of course, a presentation focusing on either insufficient intake or excessive intake may be used.

[0121] In addition, regarding the point that the health diagnosis result is directly adopted to generate a radar chart of the five major nutrients in each of the above-described embodiments, in addition, based on the points of the five major nutrients based on the health diagnosis result, it is determined which nutrients should have their intake increased, and for example, the radar chart may be generated in such a way that it becomes an effort target, such as increasing it by 20%. In this case, the display mode such as changing the display color and emphasizing the characters may be used for the nutrients to be increased.

[0122] In addition, in each of the above-described embodiments, a radar chart has been described as an example of a graph. However, the present invention is not limited to this, and other graphs such as bar graphs may be employed. For example, in the case of a stacked bar graph, each nutrient is expressed as a percentage of the whole.

[0123] Also, in each of the above-described embodiments, support for health promotion based on health check results has been exemplified. However, the present invention is not limited to this, and it can also be applied to fields that support customers' health promotion by focusing on dietary habits aimed at purposes such as weight loss and veganism as part of the health that customers aim for. In these cases as well, a radar chart is used to present the health status corresponding to health items to the customers.

[0124] In addition, each of the above-described configurations, functional units, processing units, processing means, etc. may be realized in hardware by designing part or all of them, for example, by using an integrated circuit. Also, each of the above-described configurations, functions, etc. may be realized in software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, and files for realizing each function can be stored in a recording device such as a memory, a hard disk, an SSD (Solid State Drive), a recording medium such as an IC card, an SD card, or a DVD.

Explanation of Reference Numerals

[0125] 1 Unattended Store Operation System 10A Unattended Store Sales Device 10B Unattended Store Sales Device 11 CPU 12 Program 13 Memory 14 Operation Device 15 External Storage Device 16 Communication IF 17 Bus 20 Product Shelf 21 Weight Sensor 30 Monitor 40 Authentication Display 41 Camera 42 Touch panel 50 Optical radar 60 Settlement infrastructure 70 Authentication infrastructure 90 User terminal 101 Communication control unit 102 Entry / exit determination unit 103 User recognition unit 104 Behavior determination unit 105 Data storage unit 106 Health score TBL 107 User information TBL 108 Health diagnosis TBL 109 Unpurchased determination unit 110 Nutrient analysis unit 111 Graph generation unit 112 Settlement processing unit 113 Display processing unit

Claims

1. An unmanned store vending device that sells products containing nutrients in an unmanned store, a first registration unit that links and registers nutrient data indicating a health condition using a plurality of nutrients for each pre-registered customer; A second registration unit that links and registers nutrient data having the same classification as the nutrient data to each product sold at the unmanned store; A first determination unit that determines a customer entry / exit state in the unmanned store; a recognition unit that recognizes whether the customer whose entry status has been determined by the first determination unit is a customer registered in advance; a second determination unit that determines whether or not a customer recognized by the recognition unit takes out or returns a commodity sold at the unmanned store; a generating unit that generates graph data based on nutrient data registered in the first registering unit for the recognized customer when the second determining unit determines that the customer has taken out or returned a commodity sold in the unmanned store, and that generates graph data based on the nutrient data for each of the taken out commodities by referring to the second registering unit; a display processing unit that outputs the two types of graph data generated by the generation unit as displayable data; An unmanned store sales device comprising:

2. 2. The unmanned store vending device according to claim 1, wherein the generating unit generates graph data of a radar chart.

3. 2. The unmanned store sales device according to claim 1, wherein a product shelf on which each of the products is displayed is installed within the unmanned store, and the product shelf is provided with a weight sensor that detects a change in weight when each of the products is taken out or returned, and the second determination unit determines the take-out or return action of a product sold in the unmanned store based on the weight change detected by the sensor.

4. 4. The unmanned store vending device according to claim 3, wherein the second determination unit determines whether to take out or return a product sold in the unmanned store after a certain period of time has elapsed since the weight change was detected.

5. 4. The unmanned store sales device according to claim 3, further comprising a payment processing unit that processes payment for products that have been confirmed to have been taken out by the second judgment unit when the first judgment unit judges that a customer who has been recognized by the recognition unit has left the store.

6. 2. The unmanned store sales device according to claim 1, further comprising a communication control unit that communicates with an external terminal via a network, and the communication control unit transmits data output by the display processing unit to the external terminal via the network.

7. 2. The unmanned store vending device according to claim 1, further comprising a nutrient analysis unit which inputs health checkup data for each of the preregistered customers, and performs a nutrient analysis based on the health checkup data to obtain the nutrient data.

8. 8. The unmanned store sales device according to claim 7, wherein the nutrient analysis unit inputs facial expression data of a customer whose entry status has been determined by the first determination unit, and performs nutrient analysis taking into account the input facial expression data.

9. In the unmanned store sales device described in claim 1, the multiple nutrients are represented by nutrient data consisting of numerical values, and the generation unit is characterized in that when the nutrient data for any of the multiple nutrients exceeds a predetermined value, the display size of the entire graph based on the nutrient data is reduced.

10. An unmanned store operation system for operating an unmanned store, An unmanned store vending device that sells products containing nutrients in an unmanned store; an authentication display that accepts photographs and operation inputs from customers; a product shelf that displays products and detects weight changes occurring when products are taken out and returned; An optical radar that detects customer movement patterns and analyzes them; An authentication infrastructure that authenticates customers; Equipped with The unmanned store sales device includes: A first registration unit that links and registers nutrient data indicating a health condition using a plurality of nutrients for each pre-registered customer; A second registration unit that links and registers nutrient data having the same classification as the nutrient data to each product sold at the unmanned store; a first determination unit that determines a state of customers entering and leaving the unmanned store based on detection by the optical radar; a recognition unit that recognizes whether a customer whose entry status has been determined by the first determination unit according to the photograph and operation input of the customer by the authentication display using the authentication infrastructure is the pre-registered customer; a second determination unit that determines whether the customer recognized by the recognition unit takes out or returns a commodity sold in the unmanned store by detecting a weight change due to the commodity shelf; a generating unit that generates graph data based on nutrient data registered in the first registering unit for the recognized customer when the second determining unit determines that the customer has taken out or returned a commodity sold in the unmanned store, and that generates graph data based on the nutrient data for each of the taken out commodities by referring to the second registering unit; a display processing unit that outputs the two types of graph data generated by the generation unit as displayable data; An unmanned store operation system comprising:

11. In the unmanned store operating system described in claim 10, the unmanned store sales device further has a communication control unit that communicates with an external terminal via a network, and the communication control unit transmits data output by the display processing unit to the external terminal via the network.

12. 11. The unmanned store operating system of claim 10, further comprising a monitor, wherein the unmanned store sales device transmits data output by the display processing unit to the monitor, and the monitor displays a graph based on the data.

Images