Face recognition self-service weighing consumption system

Abstract

The present invention relates to the technical field of catering systems, and discloses a facial recognition self-service weighing consumption system. The key points of the technical solution are that the system comprises a dish settlement device and a selection area. The dish settlement device comprises a weighing platform, a display, a first camera, a second camera and an inductive sensor. The first camera is aimed at the weighing platform, and the second camera and the inductive sensor are aimed at the selection area. The present invention designs a facial self-service weighing consumption system. Compared with the existing consumption system, the present invention does not need to use a QR code tray or an RFID tag, and the pain point of binding a tray to register personnel information before picking up a meal can be eliminated. The step of customizing the tray can be omitted, so that the personnel can automatically settle the bill when they leave, and the entire consumption process achieves a seamless consumption experience.

Description

Technical Field

[0001] The present invention relates to the technical field of catering systems, and more specifically to a face recognition self-service weighing consumption system. Background Art

[0002] At present, in order to improve the dining experience and health needs of diners while reducing the number of service staff, fast food restaurants on the market or in companies will use facial recognition and food recognition to complete self-checkout and publicize the nutritional content of dishes.

[0003] In the prior art, the self-service checkout method is composed of a weighing scale, a facial recognition terminal and a tray. This type of system first requires a unique identity tag on the tray, such as a QR code or RFID tag. Before picking up the food, the tray must be bound to the corresponding person who picks up the food. Then, when picking up the food, facial recognition settlement is performed based on the weight of the food. This method requires a tray and binding, which increases the cost of the tray.

[0004] In the prior art, the price calculation and nutritional information disclosure of dishes are obtained through simple weighing and then combined with quantitative data in the database. However, the same dish has different soup content in different dishes, which leads to large differences in the actual dry weight and nutritional content of each dish, which is bound to lead to non-objective and inaccurate food nutritional data and food price data, especially for some indivisible foods such as fish heads and pork chops. The soup causes a large error in the actual weight of the dish, which greatly affects people's choice in the process of taking dishes, and ultimately leads to no one choosing some dishes with more soup. Summary of the invention

[0005] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a face recognition self-service weighing consumption system to overcome the above-mentioned defects in the prior art.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solution: a face recognition self-service weighing consumption system, comprising a dish settlement device and a selection area, characterized in that: the dish settlement device comprises a weighing platform, a display, a first camera, a second camera and an inductive sensor, the first camera is aimed at the weighing platform, the second camera and the inductive sensor are aimed at the selection area;

[0007] It also includes a human body sensing module, a face recognition module, a startup module, an image acquisition module, an image reconstruction module, an image recognition module, a data calculation module, a display module and a database;

[0008] The database stores facial information, account information, a first weight corresponding to each dish, a unit price of each dish, and a unit nutritional value of each dish;

[0009] The human body sensing module recognizes the environmental status of the selected area in real time, and generates face recognition information if dynamic data is generated;

[0010] The face recognition module is configured with a face recognition algorithm. The face recognition module obtains the face recognition information in real time, calculates dynamic data through the face recognition algorithm, and matches it with the face information in the database. If the match is successful, the account information corresponding to the face data in the database is read and user binding information is generated. If the match fails, the calculation of the face data is stopped;

[0011] The startup module acquires the user binding information in real time and is connected to the weighing platform for communication. When the user binding information is acquired and the weight data of the weighing platform increases, startup information is generated;

[0012] The image acquisition module receives the startup information, acquires the image data on the weighing platform through the first camera, and generates image information to be processed;

[0013] The image reconstruction module is configured with a first image marking algorithm and a first preset interval value, obtains the image information to be processed, identifies the dish image in the image information to be processed, obtains a number of distance data values ​​from the dish in the container to the first camera through the first camera, and when the overlap rate of the most overlapping distance data values ​​to the number of distance data values ​​is within the first preset interval value, the data area of ​​the most overlapping distance data values ​​is marked by the first image marking algorithm, and the first dish image information containing the first marked area is generated, and when the overlap rate is not within the first preset interval, the second dish image information without the first marked area is generated;

[0014] The image recognition module is configured with a dish image data model. If the first dish image information is obtained, the image is recognized by the dish image data model, the food type in the first dish image information is confirmed, the first weight, unit price and unit nutritional value of the corresponding food type are obtained, and the ratio of the area of ​​the first marked area to the dish area in the container is calculated to generate the first dish information containing the ratio value. If the second dish image information is obtained, the image is recognized by the dish image data model, the food type in the second dish image information is confirmed, and the unit price and unit nutritional value of the corresponding food type are obtained to generate the second dish information;

[0015] The data calculation module, if the first dish information is obtained, reads the dish weight value on the weighing table, assigns the first weight to the proportional value, and calculates the dish dry weight value excluding the weight of the soup, and combines the dish dry weight value with the unit price and the unit nutritional value to generate the corresponding dish price and dish nutritional value; if the second dish information is obtained, reads the dish weight value on the weighing table, and combines the dish weight value with the unit price and the unit nutritional value to generate the corresponding dish price and dish nutritional value;

[0016] The display module obtains the price of the dish and the nutritional value of the dish, and displays them on the display;

[0017] The settlement module, if the weight data on the weighing platform is cleared and the second camera and the induction sensor detect that the dish is placed on the plate and the person leaves, the deduction is completed according to the price of the dish and the settlement is ended; if the weight data on the weighing platform is cleared and the second camera does not detect that the dish is placed on the plate, the deduction is canceled and the settlement is ended.

[0018] As a further improvement of the present invention, the image recognition module is configured with a second image marking algorithm and a second preset interval value. If the first dish image information is obtained, the food area in the image is marked by the second image marking algorithm to generate a second marked area, and the image clarity of the overlapping area of ​​the first marked area and the second marked area is calculated. If the image clarity of the overlapping area is within the second preset interval value, the first marked area is cancelled for the overlapping area. If the image clarity of the overlapping area is not within the second preset interval value, the first marked area is retained for the overlapping area.

[0019] As a further improvement of the present invention, the first image marking algorithm is configured with a first marking interval value, and the edges of the most overlapping distance data values ​​are marked according to the first marking interval value to form the first marking area with a polygonal shape at the edge.

[0020] As a further improvement of the present invention, the image recognition module is further configured with a first area algorithm, which obtains any point in the first marking area as an origin, connects the origin with each corner at the edge of the first marking area, so that the first marking area is divided into a plurality of triangles, and measures the length of each of the connecting lines;

[0021]

[0022]

[0023] Wherein, S is the area of ​​a single triangle, t is the first marking interval value, a is the length of one of the connecting lines, and b is the length of another adjacent connecting line. The area of ​​the first marking area is obtained by summing up the areas S of several triangles.

[0024] As a further improvement of the present invention, the second image marking algorithm is configured with a second marking interval value, and the edge of the food area is marked according to the marking interval value to form the second marking area with a polygonal edge.

[0025] As a further improvement of the present invention, both the first preset interval value and the second preset interval value can be changed.

[0026] As a further improvement of the present invention, the first mark interval value is equal to the second mark interval value.

[0027] As a further improvement of the present invention, it also includes an abnormal alarm module, which monitors the state of the consumption system in real time. When an abnormal state occurs, the abnormal alarm module performs an audible and visual alarm in real time.

[0028] As a further improvement of the present invention, the database also includes corresponding weight data of each tableware;

[0029] The image recognition module further includes a tableware recognition unit, which is configured with a tableware image data model. If the first dish image information or the second dish image information is obtained, the image in the first dish image information or the second dish image information is recognized by the tableware image data model to obtain the tableware weight data of the corresponding tableware;

[0030] The data calculation module subtracts the tableware weight data when reading the dish weight value on the weighing platform.

[0031] As a further improvement of the present invention, the dish settlement device also includes a third camera and a shopping table, the shopping table is used to place dishes, and the third camera is aimed at the shopping table.

[0032] Beneficial effects of the present invention: The present invention designs a consumption system with face self-service weighing. Compared with the existing consumption system, the present invention does not need to use a QR code tray or RFID tag. The pain point of binding the tray to register the personnel information before taking the meal can be omitted. The step of customizing the tray can be omitted, and the settlement can be automatically settled when the personnel leave, and the whole consumption process can achieve a non-sensory consumption experience. At the same time, the weighing system can achieve on-demand consumption and nutritional analysis. Weighing buffets can solve the nutritional problems that individuals care about and prevent food waste. On-demand consumption, traditional buffets cannot perform nutritional analysis. Through accurate weighing and measurement, the nutritional components of the dishes taken can be counted, allowing consumers to match nutritionally balanced dishes according to their physical conditions. And by processing the image of the dishes, the dry weight value of the dishes in the image can be quickly determined. In particular, for dishes such as fish and steaks that are full of soup but cannot be divided, weighing their dry weight not only determines the price of the food more accurately, avoids the situation where no one chooses dishes with too much soup, but also makes the judgment of the nutritional components of the dishes more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the process of the present invention;

[0034] Figure 2 is an explanatory diagram of the dish settlement device of the present invention;

[0035] Figure 3 is an explanatory diagram of the consumption system of the present invention;

[0036] Figure 4 It is an algorithm diagram of the first area algorithm of dishes of the present invention.

[0037] Attached figures: 101, dish settlement device; 102, weighing platform; 103, display; 104, first camera; 105, second camera; 106, induction sensor; 107, third camera; 108, shopping platform; 109, vibrator; 201, human body sensing module; 202, start module; 203, image reconstruction module; 204, data calculation module; 205, settlement module; 206, database; 207, face recognition module; 208, image acquisition module; 209, image recognition module; 210, dish image data model; 211, tableware recognition unit; 212, tableware image data model; 213, display module; 214, abnormal alarm module. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The same parts are represented by the same reference numerals. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "bottom surface" and "top surface", "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0039] The face recognition self-service weighing and consumption system of this embodiment includes a dish settlement device 101 and a selection area. The dish settlement device 101 includes a weighing platform 102, a display 103, a first camera 104, a second camera 105 and an inductive sensor 106. The first camera 104 is aimed at the weighing platform 102, and the second camera 105 and the inductive sensor 106 are aimed at the selection area.

[0040] Specifically, the selected area includes a face area, wherein the second camera 105 is directly aimed at the face to obtain a face image for facilitating face recognition.

[0041] Specifically, at least one dish settlement device 101 is provided. In order to meet the needs of a restaurant for a rich variety of dishes, a plurality of dish settlement devices 101 can sell a large number of dishes at the same time, and these dish settlement devices 101 share a server.

[0042] The consumption system further includes a human body sensing module 201, a face recognition module 207, a start module 202, an image acquisition module 208, an image reconstruction module 203, an image recognition module 209, a data calculation module 204, a display module 213 and a database 206;

[0043] Specifically, the human body sensing module 201, face recognition module 207, startup module 202, image acquisition module 208, image reconstruction module 203, image recognition module 209, data calculation module 204, display module 213 and database 206 in the consumer system are all in the server and are collaboratively controlled by the CPU and GPU main control chips built into the server.

[0044] Specifically, the server is connected to the weighing platform 102, the display 103, the first camera 104, the second camera 105 and the induction sensor 106 through various communication methods such as Ethernet, WiFi, 4G, etc.

[0045] Database 206 stores facial information, account information, first weights corresponding to each dish, unit price of each dish, and unit nutritional value of each dish, wherein the facial information and the account information are matched. In some company cafeterias, the facial information and the account information are directly entered into the server. For some commercial cafeterias, the server can be connected to the Alipay system, and facial payment can be completed through the Alipay system. The unit price of each dish and the unit nutritional value of each dish are preset values, which can be adjusted according to actual needs;

[0046] Specifically, the first weight is the weight ratio of the food to the soup in each dish. By assigning the first weight to a proportional value, the proportional value of the identified area can be converted into a proportional value of the weight, so that the specific ratio of the dry weight value of the dish in a bowl of food to the weight of the dish can be confirmed, so that the price and nutritional content of the bowl of food can be calculated more accurately. The weight of the food in different dishes is different at the same volume, and the soup part of each dish will also have different weights at the same volume due to different cooking methods. Therefore, the first weights for different dishes are different, and the first weights of different dishes are adjustable.

[0047] The human body sensing module 201 recognizes the environmental status of the selected area in real time and generates face recognition information if dynamic data is generated;

[0048] Specifically, when no object passes through the environmental state of the selected area, static data is generated. When an object passes through the selected area, dynamic data is generated, that is, the picture taken by the second camera 105 changes or the sensing sensor 106 detects that an object enters the selected area.

[0049] The face recognition module 207 is configured with a face recognition algorithm. The face recognition module 207 obtains face recognition information in real time, calculates dynamic data through the face recognition algorithm, and matches it with the face information in the database 206. If the match is successful, the account information corresponding to the face data in the database 206 is read and user binding information is generated. If the match fails, the calculation of the face data is stopped;

[0050] Specifically, the face recognition algorithm includes but is not limited to convolutional neural network (CNN), eigenfaces Eigenfaces, etc. How the face recognition algorithm completes face recognition is not within the scope of protection required by the present invention and will not be elaborated in detail.

[0051] The startup module 202 acquires user binding information in real time and communicates with the weighing platform 102. When the user binding information is acquired and the weight data of the weighing platform 102 increases, the startup information is generated;

[0052] The image acquisition module 208 receives the start information, acquires the image data on the weighing platform 102 through the first camera 104, and generates image information to be processed;

[0053] The image reconstruction module 203 is configured with a first image marking algorithm and a first preset interval value, obtains the image information to be processed, identifies the image of the dish in the image information to be processed, obtains a number of distance data values ​​from the dish in the container to the first camera 104 through the first camera 104, and when the overlap rate of the most overlapping distance data values ​​among the several distance data values ​​is within the first preset interval value, the data area of ​​the most overlapping distance data values ​​is marked through the first image marking algorithm, and the first dish image information containing the first marked area is generated, and when the overlap rate is not within the first preset interval, the second dish image information without the first marked area is generated;

[0054] Specifically, the number of distance data values ​​obtained may be based on the distance data values ​​collected at a number of points per unit area. The unit area is one square centimeter in the default value, and the number of collection times is ten.

[0055] Furthermore, both the unit area and the number of acquisitions can be modified.

[0056] Specifically, the initial interval of the first preset interval value is 20% to 95%. The setting of the first preset interval value can effectively distinguish soups and dishes with low soup content from dishes that need to calculate the food price and nutritional components separately (especially fish, pork chops and other dishes that cannot be divided but contain a lot of soup). Dishes with an overlap rate higher than 95% are identified as soups, and dishes with an overlap rate lower than 20% are identified as dishes with low soup content.

[0057] The image recognition module 209 is configured with a dish image data model 210. If the first dish image information is obtained, the image is recognized through the dish image data model 210, the food type in the first dish image information is confirmed, and the first weight, unit price and unit nutritional value of the corresponding food type are obtained, wherein the unit in the unit price and the unit nutritional value can be selected as required, including but not limited to grams, kilograms, catties, ounces, etc., wherein the density of the soup is different from the density of the food. If the dry weight of the ingredients and the dry weight of the soup are directly calculated according to the area ratio, it is easy to cause the measurement value to be inaccurate, and the significance of the existence of the first weight is to eliminate the impression of different densities, and then the ratio of the area of ​​the first marked area to the area of ​​the dish in the container is calculated to generate the first dish information containing the ratio value. If the second dish image information is obtained, the image is recognized through the dish image data model 210, the food type of the second dish image information is confirmed, and the unit price and unit nutritional value of the corresponding food type are obtained to generate the second dish information;

[0058] Specifically, the method for calculating the area of ​​the dish is: first determine the shape of the tableware, usually one of round, rectangular, and elliptical, then determine the corresponding calculation formula according to the shape of the tableware (the calculation formula for round, rectangular, and elliptical shapes), and then measure the various values ​​of the area occupied by the dish in the tableware and substitute them into the calculation formula to obtain the algorithm for the area of ​​the dish.

[0059] The data calculation module 204, if the first dish information is obtained, reads the dish weight value on the weighing platform 102, assigns the first weight to the proportional value, and calculates the dish dry weight value excluding the weight of the soup, and combines the dish dry weight value with the unit price and the unit nutritional value to generate the corresponding dish price and dish nutritional value; if the second dish information is obtained, reads the dish weight value on the weighing platform 102, and combines the dish weight value with the unit price and the unit nutritional value to generate the corresponding dish price and dish nutritional value;

[0060] Specifically, it should be noted that during the process of reading the weight value of the dish, the reading will not begin until the value stops changing for at least one second.

[0061] Specifically, the manner of assigning the first weight to the proportional value includes but is not limited to multiplication, division, and the like.

[0062] Furthermore, the dry weight value of the dish = the weight value of the dish - the weight value of the dish * the first weight * the ratio value.

[0063] Display module 213, obtains dish prices and dish nutritional values, and displays them through display 103;

[0064] Settlement module 205, if the weight data on the weighing platform 102 is cleared and the second camera 105 and the induction sensor 106 detect that the dishes are placed on the plate and the person leaves, the deduction is completed according to the dish price and the settlement is ended. If the weight data on the weighing platform 102 is cleared and the second camera 105 does not detect that the dishes are placed on the plate, the deduction is canceled and the settlement is ended. In this case, the dishes are put back into the dish selection area.

[0065] The present invention designs a consumption system with face self-service weighing. Compared with the existing consumption system, the present invention does not need to use a QR code tray or RFID tag. The pain point of binding the tray to register the personnel information before taking the meal can be omitted. The step of customizing the tray can be omitted, and the settlement can be automatically settled when the personnel leave, and the whole consumption process can achieve a non-sensory consumption experience. At the same time, the weighing system can achieve on-demand consumption and nutritional analysis. Weighing buffets can solve the nutritional problems that individuals care about and prevent food waste. On-demand consumption, traditional buffets cannot perform nutritional analysis. Through accurate weighing and measurement, the nutritional components of the dishes taken can be counted, allowing consumers to match nutritionally balanced dishes according to their physical conditions. And by processing the image of the dishes, the dry weight value of the dishes in the image can be quickly determined. In particular, for dishes such as fish and steaks that are full of soup but cannot be divided, weighing their dry weight not only determines the price of the food more accurately, avoids the situation where no one chooses dishes with too much soup, but also makes the judgment of the nutritional components of the dishes more accurate.

[0066] In one embodiment, the first image marking algorithm is configured with a first marking interval value, and the edges of the distance data values ​​with the most overlap are marked according to the first marking interval value to form a first marking area with a polygonal shape at the edge. The formation of the polygonal first marking area can facilitate the subsequent calculation of the area of ​​the dish, and the preference between marking efficiency and accuracy can also be adjusted according to the size of the first marking interval value.

[0067] Specifically, the first mark interval value represents a line segment of length. During the marking process, the edge of the image is marked according to the line segment assigned by the first mark interval value, so that a first marked area with a polygonal shape at the edge can be obtained. It should be noted that the length of the line segment is consistent during the marking process and the angle is adjustable.

[0068] Furthermore, the first mark interval value can be changed. The larger the first mark interval value, the higher the accuracy, and the higher the requirements of the image marking on the CPU and GPU. The smaller the first mark interval value, the lower the accuracy, and the lower the requirements of the image marking on the CPU and GPU.

[0069] In one embodiment, the image recognition module 209 is further configured with a first area algorithm, which obtains any point in the first marking area as the origin, connects the origin with each corner at the edge of the first marking area, so as to divide the first marking area into a plurality of triangles, and measures the length of each connecting line;

[0070]

[0071]

[0072] Wherein, S is the area of ​​a single triangle, t is the first marking interval value, a is the length of one of the connecting lines, and b is the length of another adjacent connecting line. The area of ​​the first marking area is obtained by summing up the areas S of several triangles.

[0073] Specifically, the lines connecting the origin and each corner at the edge of the first marking area can be connected in a clockwise or counterclockwise direction. After such a setting, it is less likely that the lines will cross, thereby improving the accuracy of area calculation.

[0074] Specifically, the image recognition module 209 is also configured with an origin determination strategy, and the steps of the origin determination strategy are: S1: determine the two corners with the farthest distance between the polygons, S2: connect the two selected corners and mark the midpoint of the connection line, S3: connect the marked midpoint with each corner, S4: measure the length of each connection line, and define the corner with the shortest connection length as the origin. (As shown in the attached figure, Figure 4 As shown, L represents the line connecting the two angles of the polygon that are farthest apart, with the origin being P)

[0075] Furthermore, the origin determination strategy is set up so that the two farthest corners can be determined by measurement. The midpoint after determination is the center of the circumscribed circle of the polygon. After the center of the circle is confirmed, it can be defined as the center of the image. After the image center is confirmed, the corner closest to the image center is defined as the origin. The advantage of this is that it reduces the problem of inaccurate area calculation caused by the line connecting the origin and the corners at the edge of the first marked area exceeding the marked first marked area.

[0076] Furthermore, the image recognition module 209 is also configured with an origin offset strategy, which is: when the connection line at least partially exceeds the first marked area, the origin is replaced with the adjacent corner in the opposite direction of the connection line as the origin, and the connection is reconnected with the new origin. That is, if the connection direction is clockwise, when the connection line exceeds the first marked area, the origin is automatically replaced with the adjacent corner in the counterclockwise direction of the origin as the origin.

[0077] Furthermore, after the origin is offset by the origin offset strategy, if there is still a line beyond the first marked area, the length of the line within the first marked area can be measured, and then the image area within the first marked area can be calculated. Alternatively, after the origin offset strategy fails, the image of the area enclosed by the line beyond the first marked area can also be defaulted to the first marked area. Both of the above methods can solve the problem of how to calculate the area within the first marked area after the origin offset strategy fails. At the same time, the calculation accuracy of the area of ​​the first area is less affected.

[0078] In one embodiment, the image recognition module 209 is configured with a second image marking algorithm and a second preset interval value. If the first dish image information is obtained, the food area in the image is marked by the second image marking algorithm to generate a second marked area, and the image clarity of the overlapping area of ​​the first marked area and the second marked area is calculated. If the image clarity of the overlapping area is within the second preset interval value, the first marked area is cancelled for the overlapping area. If the image clarity of the overlapping area is not within the second preset interval value, the first marked area is retained for the overlapping area. The second image marking algorithm can be used to determine whether the food hidden under the surface of the soup should be calculated as the first marked area, thereby improving the accuracy of calculating the dry weight value of the dish, and also making the calculation of the actual nutritional value of the dish more accurate.

[0079] Specifically, the second image marking algorithm is configured with a second marking interval value, and the edge of the food area is marked according to the marking interval value to form a second marking area with a polygonal shape at the edge.

[0080] Specifically, for the re-identified first marked area, the area may be calculated solely according to the first area algorithm.

[0081] Specifically, the image clarity calculation method includes but is not limited to Brenner gradient function, Laplacian gradient function, SMD (grayscale difference) function, etc. The above calculation method can detect the clarity of the image, so as to judge the approximate depth of the food hidden under the soup. For the image whose clarity is within the second preset interval, it can be determined that the food is close to the surface of the soup, and it should be determined that the food does not belong to the first marked area. For the image whose clarity is not within the second preset interval, it can be determined that the food is far from the surface of the soup, and it should be determined that the food belongs to the first marked area.

[0082] Furthermore, the value of the second preset interval is stored in the database 206, and the value of the second preset value interval of each dish is different. For example, the value of the second preset interval of braised dishes is smaller than the value of the second preset area of ​​steamed dishes, that is, when the clarity of braised dishes is at a lower state, they can also be identified as belonging to the first marking area.

[0083] Furthermore, the first preset interval value and the second preset interval value can be changed as needed.

[0084] In one embodiment, the first mark interval value and the second mark interval value are equal by default, which facilitates the calculation of the area after separation.

[0085] In one embodiment, an abnormality alarm module 214 is also included to monitor the consumption system status in real time. When an abnormal state occurs, the abnormality alarm module 214 issues an audible and visual alarm in real time.

[0086] Specifically, if the weight data on the weighing table 102 is reset and the second camera 105 does not detect that the dishes are placed on the plate, the third camera 107 can be used to confirm whether the dishes are put back on the purchasing table 108. If the dishes are placed on the purchasing table 108, the settlement is completed. If the dishes are not placed on the purchasing table 108, the abnormal alarm module 214 can sound an alarm.

[0087] Specifically, if the weight data on the weighing table 102 is cleared and the third camera 107 does not detect that the dishes are put into the purchasing table 108, the abnormal alarm module 214 can issue an alarm.

[0088] Specifically, the abnormal alarm module 214 may sound an alarm in situations including but not limited to insufficient balance, picking up food and leaving without recognizing the face, and other abnormal situations.

[0089] Specifically, when the abnormal alarm module 214 issues an alarm, the cause of the alarm may be broadcasted through a sound device.

[0090] In one embodiment, the database 206 also includes corresponding weight data of each dishware;

[0091] The image recognition module 209 further includes a tableware recognition unit 211, which is configured with a tableware image data model 212. If the first dish image information or the second dish image information is obtained, the image in the first dish image information or the second dish image information is recognized by the tableware image data model 212 to obtain the tableware weight data of the corresponding tableware;

[0092] The data calculation module 204 subtracts the tableware weight data when reading the dish weight value from the weighing platform 102 .

[0093] Specifically, the database 206 may also store data of each tableware as required, including but not limited to tableware weight data and tableware plate size data.

[0094] Furthermore, the tableware identification unit 211 also includes a method for calculating the area of ​​a dish. The method for calculating the area of ​​a dish is as follows: first, the tableware plate size data is obtained, then the dish size value is measured, and finally the ratio of the dish size value to the tableware plate size data is calculated to obtain the area of ​​the dish. In this embodiment, the tableware plate size data includes but is not limited to the area, length, center of a circle, major axis, minor axis, etc., and the tableware plate size data of each type of tableware is determined according to the shape of the tableware. The measurement of the dish size value can also be measured and calculated according to the shape of the tableware, which is more convenient. For example, if the tableware is round, the radius of the dish size value is calculated. For the ratio calculation of the dish area and the tableware area, it is the square of the measured value. Compared with the above-mentioned dish area calculation, the dish area calculation in this embodiment greatly reduces the data required to be measured, thereby improving the measurement speed.

[0095] In one embodiment, the dish settlement device 101 also includes a third camera 107 and a shopping table 108. The shopping table 108 is used to place dishes. The third camera 107 is aimed at the shopping table 108, and several identical dishes on the shopping table 108 are divided into a training set, a verification set and a test set for training the dish image data model 210 and the tableware image data model 212.

[0096] Specifically, the ratio of the training set, the validation set, and the test set includes but is not limited to 6:2:2.

[0097] In one embodiment, a vibrator 109 is provided at the bottom of the shopping table 108. When the vibrator 109 is turned on, the shopping table 108 is driven to vibrate. The vibration image of the soup in the dish is obtained through the third camera 107. The vibration image of the soup is identified through the trained dish image data model 210 to determine the viscosity and color of the dish in the soup, and to determine some of the nutritional components of the soup. In this embodiment, the viscosity of the soup can reflect the content of starch, cooking oil, soy sauce, sugar and other substances in the soup. At the same time, the vibration image of the soup is different after the above substances are in the dish. The unit nutritional value of the dish can be modified through the trained dish image data model 210.

[0098] In one embodiment, the first camera 104, the second camera 105, and the third camera 107 can all adopt binocular cameras or multi-eye cameras to obtain spatial information of faces and dishes, thereby realizing face recognition and measurement of distance data values. Of course, the server of the present invention is also configured with relevant distance measurement algorithms.

[0099] In one embodiment, the sensing sensor 106 includes but is not limited to one or more of an infrared light sensor and a laser sensing sensor 106 , and can quickly determine whether a person has entered the selected area before the second camera 105 performs face recognition.

[0100] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A facial recognition self-service weighing consumption system, comprising a dish settlement device (101) and a selection area, characterized in that: The dish settlement device (101) comprises a weighing platform (102), a display (103), a first camera (104), a second camera (105) and an inductive sensor (106), wherein the first camera (104) is aimed at the weighing platform (102), and the second camera (105) and the inductive sensor (106) are aimed at the selected area; It also includes a human body sensing module (201), a face recognition module (207), a start module (202), an image acquisition module (208), an image reconstruction module (203), an image recognition module (209), a data calculation module (204), a display module (213), a settlement module (205) and a database (206); The database (206) stores facial information, account information, first weights corresponding to each dish, unit prices of each dish, and unit nutritional values ​​of each dish; The human body sensing module (201) recognizes the environmental status of the selected area in real time, and generates face recognition information if dynamic data is generated; The face recognition module (207) is configured with a face recognition algorithm. The face recognition module (207) obtains the face recognition information in real time, calculates dynamic data through the face recognition algorithm, and matches it with the face information in the database (206). If the match is successful, the account information corresponding to the face data in the database (206) is read and user binding information is generated. If the match fails, the calculation of the face data is stopped; The startup module (202) acquires the user binding information in real time and is connected to the weighing platform (102) for communication, and generates startup information when the user binding information is acquired and the weight data of the weighing platform (102) increases; The image acquisition module (208) receives the startup information, acquires image data on the weighing platform (102) through the first camera (104), and generates image information to be processed; The image reconstruction module (203) is configured with a first image marking algorithm and a first preset interval value, obtains the image information to be processed, identifies the dish image in the image information to be processed, obtains a number of distance data values ​​from the dish in the container to the first camera (104) through the first camera (104), and when the overlap ratio of the most overlapping distance data values ​​to the number of distance data values ​​is within the first preset interval value, the data area of ​​the most overlapping distance data values ​​is marked by the first image marking algorithm, and the first dish image information containing the first marked area is generated; when the overlap ratio is not within the first preset interval, the second dish image information without the first marked area is generated; The image recognition module (209) is configured with a dish image data model (210). If the first dish image information is obtained, the image is recognized by the dish image data model (210), the food type in the first dish image information is confirmed, the first weight, unit price and unit nutritional value of the corresponding food type are obtained, and the ratio of the area of ​​the first marked area to the dish area in the container is calculated to generate the first dish information containing the ratio value. If the second dish image information is obtained, the image is recognized by the dish image data model (210), the food type in the second dish image information is confirmed, the unit price and unit nutritional value of the corresponding food type are obtained to generate the second dish information; The data calculation module (204), if the first dish information is obtained, reads the dish weight value on the weighing table (102), assigns the first weight to the proportional value, and calculates the dish dry weight value excluding the weight of the soup, the dish dry weight value is combined with the unit price and the unit nutritional value to generate the corresponding dish price and dish nutritional value, and if the second dish information is obtained, reads the dish weight value on the weighing table (102), the dish weight value is combined with the unit price and the unit nutritional value to generate the corresponding dish price and dish nutritional value; The display module (213) obtains the price of the dish and the nutritional value of the dish, and displays them through the display (103); The settlement module (205) completes deductions according to the price of the dish and ends the settlement if the weight data on the weighing platform (102) is cleared and the second camera (105) and the induction sensor (106) detect that the dish is placed on the plate and the person leaves; if the weight data on the weighing platform (102) is cleared and the second camera (105) does not detect that the dish is placed on the plate, the deductions are canceled and the settlement ends.

2. The face recognition self-service weighing consumption system according to claim 1 is characterized by: The image recognition module (209) is configured with a second image marking algorithm and a second preset interval value. If the first dish image information is obtained, the food area in the image is marked by the second image marking algorithm to generate a second marked area. The image clarity of the overlapping area of ​​the first marked area and the second marked area is calculated. If the image clarity of the overlapping area is within the second preset interval value, the first marked area is cancelled for the overlapping area. If the image clarity of the overlapping area is not within the second preset interval value, the first marked area is retained for the overlapping area.

3. The face recognition self-service weighing consumption system according to claim 2 is characterized by: The first image marking algorithm is configured with a first marking interval value, and the edges of the most overlapping distance data values ​​are marked according to the first marking interval value to form the first marking area with a polygonal shape at the edge.

4. The face recognition self-service weighing consumption system according to claim 3 is characterized by: The image recognition module (209) is further configured with a first area algorithm, wherein the first area algorithm obtains any point in the first marking area as an origin, connects the origin with each corner at the edge of the first marking area, so as to divide the first marking area into a plurality of triangles, and measures the length of each of the connecting lines; Wherein, S is the area of ​​a single triangle, t is the first marking interval value, a is the length of one of the connecting lines, and b is the length of another adjacent connecting line. The area of ​​the first marking area is obtained by summing up the areas S of several triangles.

5. The face recognition self-service weighing consumption system according to claim 3 or 4, characterized in that: The second image marking algorithm is configured with a second marking interval value, and the edge of the food area is marked according to the marking interval value to form the second marking area with a polygonal edge.

6. The face recognition self-service weighing consumption system according to claim 2 is characterized by: Both the first preset interval value and the second preset interval value can be changed.

7. The face recognition self-service weighing consumption system according to claim 2 is characterized by: The first mark interval value and the second mark interval value are equal.

8. The face recognition self-service weighing consumption system according to claim 1 is characterized by: It also includes an abnormal alarm module, which monitors the status of the consumption system in real time. When an abnormal state occurs, the abnormal alarm module will issue an audible and visual alarm in real time.

9. The face recognition self-service weighing consumption system according to claim 1 is characterized by: The database (206) also includes corresponding weight data of each tableware; The image recognition module (209) further comprises a tableware recognition unit (211), wherein the tableware recognition unit (211) is configured with a tableware image data model (212), and when the first dish image information or the second dish image information is obtained, the image in the first dish image information or the second dish image information is recognized by the tableware image data model (212), and tableware weight data of the corresponding tableware is obtained; The data calculation module (204) subtracts the tableware weight data when reading the dish weight value from the weighing platform (102).

10. The face recognition self-service weighing consumption system according to claim 1 is characterized in that: The dish settlement device (101) further comprises a third camera (107) and a shopping table (108), wherein the shopping table (108) is used for placing dishes, and the third camera (107) is aimed at the shopping table (108).

Images