Intelligent dining nutrition management system

Abstract

The invention relates to an intelligent dining nutrition management system, belongs to the technical field of intelligent restaurants, and solves the problem that in the prior art, individual needs of dietary nutrition management of users cannot be met. The system comprises a nutrition data module which stores recommended intake of nutrition information required by a user; a weighing platform; the shooting device is used for shooting a human body behavior image and a human face image; the behavior recognition module is used for recognizing the food taking behavior based on the human body coordinate frame and the key points marked on the human body behavior image and the weight change of the dishes; the identity recognition module is used for acquiring identity information of a meal taker; the nutrition management module is used for calling the recommended intake of the required nutrition information according to the identity information of the meal taker; and the display device is used for displaying the nutritional ingredient information and the weight change of the dishes and the recommended intake of the nutrition information required by the meal taker. According to the invention, a meal taker can be accurately identified, so that an accurate personalized nutrition management service is provided for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent restaurants, and in particular to an intelligent dining nutrition management system. Background Art

[0002] In hospitals or nursing homes, it is necessary to manage the nutritional and dietary health of inpatients. With the rapid development of the Internet, the model of intelligence + Internet + products has been applied in all walks of life, and smart restaurants (smart restaurants) have emerged. Smart restaurants have realized the automation and intelligence of restaurant management through an integrated management platform, improving operational efficiency and service quality. However, the existing smart restaurant management system mainly focuses on improving the management efficiency of restaurants. Although some systems provide nutritional recommendations, the existing systems cannot provide accurate personalized nutrition management services and cannot meet the personalized needs of users.

[0003] At the same time, the existing smart restaurant system uses the facial recognition camera to capture facial images to determine the identity of the person picking up food, and uses the entry and exit of the face as the basis for determining the start and end time of picking up food. There are problems such as inaccurate recognition of the picking up behavior due to multiple people picking up food, and misjudgment of the end of picking up food due to loss of facial signals. Therefore, it is impossible to accurately obtain information such as "who picked up food, what dishes were taken, and how much was taken", and thus it is impossible to accurately manage the user's dietary nutrition and health. Summary of the invention

[0004] In view of the above analysis, an embodiment of the present invention aims to provide an intelligent dining nutrition management system to solve the problem that the existing technology cannot meet the personalized needs of users for dietary nutrition management.

[0005] On the one hand, an embodiment of the present invention provides an intelligent dining nutrition management system, the system comprising:

[0006] A nutrition data module stores the recommended intake of the user's required nutrition information;

[0007] A weighing table, used to hold dishes and detect weight changes of the dishes;

[0008] A shooting device, used for shooting human behavior images and human face images;

[0009] A behavior recognition module processes the human behavior image to identify a human coordinate frame and key points, and obtains position information of the key points; then recognizes the meal-taking behavior according to the human coordinate frame, the position information of the key points and the weight change of the dish; the key points include the key points of the human limb joints and the tableware;

[0010] An identity recognition module, when only one of the meal-picking behaviors is recognized, obtains the identity information of the person who picks up the meal according to the coordinate frame of the human body where the meal-picking behavior occurs and the facial image;

[0011] A nutrition management module, which retrieves the recommended intake of the nutrition information required by the meal picker according to the identity information of the meal picker;

[0012] The display device displays the nutritional information and weight changes of the dishes, as well as the recommended intake of nutritional information required by the diners.

[0013] Based on the further improvement of the above system, the system further includes:

[0014] Data storage module, used to store the user's personal meal collection records;

[0015] The personal meal pick-up record includes the user's ID number, the name of the dish picked up each time, the weight of the meal, the time of meal pick-up completion, and the amount of nutrients taken.

[0016] Based on the further improvement of the above system, the nutrition management module determines the completion time of the meal picking by tracking the coordinate frame of the human body where the meal picking behavior occurs, and uses the weight change of the dish when the meal picking is completed as the meal picking weight, calculates the nutritional component consumption of the dish based on the meal picking weight, and then sends the nutritional component consumption of the dish to the personal meal picking record of the person who picks up the meal in the data storage module for storage.

[0017] Based on further improvements of the above system, when the meal pick-up is completed, the nutrition management module also calculates the cumulative amount of nutrients taken of all dishes currently taken by the diner, and compares the cumulative amount of nutrients with the recommended intake. If the cumulative amount exceeds the recommended intake, a reminder message is sent to the diner.

[0018] Based on the further improvement of the above system, the nutrition management module determines the meal picking completion time of this meal picking by tracking the coordinate frame of the human body where the meal picking behavior occurs, including the following steps:

[0019] The ID number of the person picking up the meal is obtained according to the identity information obtained, and the ID number is continuously and dynamically bound to the coordinate frame of the human body where the meal picking behavior occurs, and then the coordinate frame of the human body is tracked according to the ID number to determine the completion time of meal picking.

[0020] Based on further improvement of the above system, the nutrition management module also calculates the current intake amount of each nutrient component of the dish in real time according to the weight change of the dish detected by the weighing platform, and displays it through a display device.

[0021] Based on the further improvement of the above system, the system further includes:

[0022] The dish recognition module is used to recognize the dish name according to the dish image, obtain the nutritional information of the dish according to the dish name, and send the dish name and the nutritional information to the display device for display.

[0023] Based on the further improvement of the above system, the number of the weighing platforms is set to multiple, a dish is set on each weighing platform, each weighing platform is equipped with a display device, and each weighing platform is respectively provided with a group of the shooting devices to shoot the food picking area of ​​the weighing platform to obtain human behavior images and facial images.

[0024] Based on the further improvement of the above system, the method of identifying the meal-taking behavior according to the position information of the human body coordinate frame and the key points and the weight change of the dish includes the following steps:

[0025] Setting a meal-taking reference line on the processed human behavior image, wherein the meal-taking reference line is located between the edge line of the weighing platform and the edge line of the dining table; and a range defined between the meal-taking reference line and the bottom edge line of the human behavior image is used as a reference area;

[0026] Determining whether a meal-taking behavior occurs within the human body coordinate frame according to the reference area;

[0027] If the preset key point in the human body coordinate frame is located in the reference area, and the weight of the dish changes and lasts for a preset time, it is determined that the meal picking behavior occurs.

[0028] Based on the further improvement of the above system, the identity recognition module obtains the identity information of the person who takes the meal according to the coordinate frame of the person who takes the meal and the face image, including the following steps:

[0029] Input the face image into a face recognition model for processing, identify the face coordinate frame and obtain the identity information of the person;

[0030] According to the coordinates, the coordinate frame of the human body that takes the meal is matched with the coordinate frame of the human face in the face image to obtain the identity information of the person who takes the meal.

[0031] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0032] 1. In the present invention, when the meal-picking behavior is recognized based on the human behavior image, the key points of the human limb joints and the tableware are taken as the key points to be recognized, so that the meal-picking behavior can be accurately recognized, and then the accurate association between the meal-picking behavior and the identity information is realized according to the human face image and the human coordinate frame marked on the human behavior image, and the identity information of the person who picks up the meal is accurately recognized, so that the recommended intake of the nutritional information required by the person who picks up the meal can be displayed to the person in real time, and accurate personalized nutrition management services can be provided to the user to meet the personalized nutritional needs of the user.

[0033] 2. In the present invention, by setting up a data storage module, the personal meal-taking record of the person who takes the meal can be accurately collected, which is helpful for professionals to analyze the user's dietary preferences and provide personalized services.

[0034] 3. In the present invention, after identifying the meal-picking behavior, the meal-picking completion time can be accurately determined by tracking the coordinate frame of the human body where the meal-picking behavior occurs, which can avoid the problem of misjudging the end of meal-picking due to the loss of facial signals, thereby facilitating the accurate acquisition of the meal-picking weight and the nutritional content of the dishes.

[0035] 4. In the present invention, when the cumulative intake of nutrients exceeds the recommended intake, an alarm message is sent to the user, thereby realizing real-time nutritional intervention reminders.

[0036] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0038] Figure 1 This is a module diagram of an intelligent dining nutrition management system according to an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of human limb joints and a human coordinate frame according to an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of the positions of a meal pickup reference line and a screening line according to an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of an optimal recognition range defined in an embodiment of the present invention; DETAILED DESCRIPTION

[0042] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0043] The embodiment of the present invention discloses an intelligent dining nutrition management system. Figure 1 The system comprises:

[0044] A nutrition data module stores the recommended intake of the user's required nutrition information;

[0045] A weighing table, used to hold dishes and detect weight changes of the dishes;

[0046] A shooting device, used for shooting human behavior images and human face images;

[0047] A behavior recognition module processes the human behavior image to identify a human coordinate frame and key points, and obtains position information of the key points; then recognizes the meal-taking behavior according to the human coordinate frame, the position information of the key points and the weight change of the dish; the key points include the key points of the human limb joints and the tableware;

[0048] An identity recognition module, when only one of the meal-picking behaviors is recognized, the identity recognition module obtains the identity information of the person who picks up the meal according to the coordinate frame of the human body where the meal-picking behavior occurs and the facial image;

[0049] A nutrition management module, which retrieves the recommended intake of the nutrition information required by the meal picker according to the identity information of the meal picker;

[0050] The display device displays the nutritional information and weight changes of the dishes, as well as the recommended intake of nutritional information required by the diners.

[0051] During implementation, a container or space for placing dishes is provided on the weighing platform, for example, the weighing platform may adopt a buffet stove with a weighing function. The recommended intake of the nutritional information required by the user is set by doctors or professionals in the hospital or health care center according to the user's personal situation.

[0052] Compared with the prior art, in the embodiment of the present invention, when identifying the meal-picking behavior based on the human behavior image, the key points of the human limb joints and the tableware are all taken as the key points that need to be identified, so that the meal-picking behavior can be accurately identified, and then the meal-picking behavior and the identity information are accurately associated according to the human coordinate frame marked on the face image and the human behavior image, and the identity information of the person who picks up the meal is accurately identified, so that the recommended intake of the nutritional information required by the person who picks up the meal can be displayed to the person in real time, and accurate personalized nutrition management services can be provided to the user to meet the user's personalized nutritional meal needs.

[0053] In one embodiment, the system further comprises:

[0054] Data storage module, used to store the user's personal meal collection records;

[0055] The personal meal pick-up record includes the user's ID number, the name of the dish picked up each time, the weight of the meal, the time of meal pick-up completion, and the amount of nutrients taken.

[0056] In the embodiment of the present invention, by setting up a data storage module, the personal meal-taking record of the person who takes the meal can be accurately collected, which is helpful for professionals to analyze the user's dietary preferences and provide personalized services.

[0057] Specifically, the nutrition management module determines the completion time of the meal picking by tracking the coordinate frame of the human body that performs the meal picking behavior, and uses the weight change of the dish when the meal picking is completed as the meal picking weight, calculates the nutritional component consumption of the dish based on the meal picking weight, and then sends the nutritional component consumption of the dish to the personal meal picking record of the person who picks up the meal in the data storage module for storage.

[0058] Compared with the prior art, in the embodiment of the present invention, after identifying the meal picking behavior, the meal picking completion time can be accurately determined by tracking the coordinate frame of the human body where the meal picking behavior occurs, thereby avoiding the problem of misjudging the end of meal picking due to loss of facial signals, thereby facilitating accurate acquisition of the meal picking weight and the nutritional content of the dishes for this meal picking.

[0059] Specifically, when the meal is picked up, the nutrition management module also calculates the cumulative amount of nutrients taken of all dishes currently taken by the person picking up the meal, and compares the cumulative amount of nutrients taken with the recommended intake. If the cumulative amount exceeds the recommended intake, a reminder message is sent to the person picking up the meal.

[0060] In the embodiment of the present invention, when the cumulative intake of nutrients exceeds the recommended intake, an alarm message is sent to the user, thereby realizing real-time nutritional intervention reminders.

[0061] Specifically, the alarm information may be a text signal, a light signal or a sound signal, and is sent out through a display device, or a device such as a warning light or an alarm.

[0062] Specifically, the nutrition management module determines the meal picking completion time of this meal picking by tracking the coordinate frame of the human body where the meal picking behavior occurs, including the following steps:

[0063] The ID number of the person picking up the meal is obtained according to the identity information obtained, and the ID number is continuously and dynamically bound to the coordinate frame of the human body where the meal picking behavior occurs, and then the coordinate frame of the human body is tracked according to the ID number to determine the completion time of meal picking.

[0064] In this embodiment, each user is respectively set with an ID number, and the ID number of the person picking up the meal is obtained after obtaining the identity information of the person picking up the meal. The ID number is continuously and dynamically bound to the coordinate frame of the human body where the meal picking behavior occurs, so that the meal picking behavior and the identity information of the person picking up the meal are timely bound, so that the position of the person picking up the meal and the entire meal picking process can be accurately tracked. When the meal picking behavior is no longer recognized in the human coordinate frame, the meal picking is completed, and this time is the meal picking completion time of this meal picking, and then the accurate meal picking weight of the person picking up the meal is obtained according to the meal picking completion time.

[0065] Specifically, the nutrition management module also calculates the current amount of each nutrient component of the dish in real time according to the weight change of the dish detected by the weighing platform, and displays it through a display device.

[0066] In the embodiment of the present invention, the current amount of each nutrient component of the dish is displayed in real time to the person picking up the meal, so that the person picking up the meal can control the intake of each nutrient component and realize real-time reminder of dietary nutrition.

[0067] Specifically, the system further includes:

[0068] The dish recognition module is used to recognize the dish name according to the dish image, obtain the nutritional information of the dish according to the dish name, and send the dish name and the nutritional information to the display device for display.

[0069] During implementation, during the restaurant's dish preparation period, the system starts the serving mode. After the restaurant staff places the dishes on the weighing table, the camera automatically takes an image of the dish. Then the dish recognition module uses the dish image to recognize the name of the dish, obtains the nutritional information of the dish, and displays it through the display device. There is no need for the staff to manually input the dish information, thereby reducing the labor intensity of the staff and improving work efficiency.

[0070] Furthermore, the dish recognition module processes the dish image through a dish recognition model to recognize the dish name. Specifically, the dish recognition model may adopt a neural network model.

[0071] In a specific embodiment, the number of the weighing platforms is set to multiple, each weighing platform is provided with a dish, each weighing platform is respectively configured with a display device, and each weighing platform is respectively provided with a group of the shooting devices to shoot the food picking area of ​​the weighing platform to obtain human behavior images and facial images.

[0072] Specifically, each group of the shooting devices includes a first camera and a second camera. The first camera is used to shoot human behavior images and / or food images, and the second camera is used to shoot human face images.

[0073] During implementation, the first camera is arranged above the weighing platform of the same group, and the first camera faces the weighing platform, so that the weighing platform is located in the shooting picture of the first camera. The second camera is arranged above the weighing platform of the same group, and the second camera faces the head of the person located in front of the dining table, and the shooting center of the second camera and the center line of the weighing platform of the same group are located on the same vertical plane.

[0074] In an embodiment of the present invention, a plurality of weighing tables, photographing devices and display devices are arranged on the dining table, so that nutritional dietary management of a variety of dishes can be achieved.

[0075] The first camera is used to capture images of human behavior and dishes. Therefore, the person picking up the food, the weighing platform, and the dishes on the weighing platform should be in the picture taken by the first camera, so as to ensure that the person picking up the food can be reflected in the human behavior image taken by the first camera. The second camera is used to capture facial images. The second camera accurately monitors the customer's head area, which helps to capture the customer's facial features. Its shooting center is on the same vertical plane as the center line of the weighing platform, ensuring the consistency and accuracy of the shooting picture and reducing the error caused by angle deviation.

[0076] Specifically, when the behavior recognition module identifies the meal-picking behavior based on the human behavior image, the human body and tableware are regarded as a whole. The key points of the human limb joints and the tableware are all key points that need to be identified. Then, the meal-picking behavior is comprehensively identified based on the position of the preset key points and the weight changes of the dishes. This enables the meal-picking behavior to be identified more accurately and reduces the misjudgment of the meal-picking behavior of multiple people.

[0077] Among them, when identifying the meal-taking behavior, the behavior recognition module processes the human behavior image through the key point recognition model, identifies the human coordinate frame and key points, and obtains the position information of the key points.

[0078] Specifically, the key point model is a YOLOV-POSE model, such as a YOLOV7-POSE model, a YOLOV9-POSE model, etc. The YOLOV-POSE model is an extension based on the YOLO target detection framework, and is used for human posture estimation. It adds a key point detection function based on target detection, and can identify key points in an image, such as joint points, etc. The output of the YOLOV-POSE model is a set of coordinate points and target boxes representing the key points of objects in the image.

[0079] like Figure 2 As shown in , the model can detect the limb joints of the human body, the key points of the tableware and a human coordinate frame. During implementation, 17 limb joints of the human body and 3 key points on the tableware are used as targets for recognition.

[0080] In a specific embodiment, specifically, the YOLOV7-POSE model is used as the key point recognition model. The limb joint detection algorithm of the YOLOV7-POSE model is implemented based on the top-down method, that is, first detect the target frame of all people, and then estimate the posture of each frame and output the result. The YOLOV7-POSE model accepts the input image and generates a feature map, which is then used to predict the position of each person's limb joints. It can estimate the posture of multiple people in real time and is suitable for the scene of picking up food.

[0081] First, the human behavior images are acquired by shooting the food pickup area with a behavior recognition camera, and the acquired behavior images are input into the YOLOV7-POSE model.

[0082] It should be noted that the key point model in the embodiment of the present invention is a trained model. Specifically, training the key point model includes the following steps:

[0083] Obtain several human behavior images in different food pickup scenarios;

[0084] Marking key points and human coordinate frames on each of the human behavior images; wherein the marked key points include human limb joints and tableware key points;

[0085] Divide the labeled human behavior images into training sets and test sets according to a preset ratio;

[0086] Training the key point model using the training set;

[0087] The trained key point model is verified through the test set to obtain the trained key point recognition model.

[0088] After the behavior recognition module processes the human behavior image through the key point recognition model, the behavior recognition module recognizes the meal-taking behavior according to the human coordinate frame and the position information of the key points and the weight change of the dish.

[0089] In a specific embodiment, the behavior recognition module recognizes the meal-taking behavior according to the position information of the human body coordinate frame and the key points and the weight change of the dish, including the following steps:

[0090] Setting a meal-taking reference line on the processed human behavior image, wherein the meal-taking reference line is located between the edge line of the weighing platform and the edge line of the dining table; and a range defined between the meal-taking reference line and the bottom edge line of the human behavior image is used as a reference area;

[0091] Determining whether a meal-taking behavior occurs within the human body coordinate frame according to the reference area;

[0092] If the preset key point in the human body coordinate frame is located in the reference area, and the weight of the dish changes and lasts for a preset time, it is determined that the meal picking behavior occurs.

[0093] In the embodiment of the present invention, Figure 3 As shown in the figure, a meal-taking reference line is drawn between the edge line of the weighing table and the edge line of the dining table, and some of the identifiable key points (key points of human limb joints and tableware) are selected as preset key points. When the preset key points exceed the meal-taking reference line and enter the reference area, and the weight of the dish changes and lasts for a preset time, for example, within 1 second, it is determined that a meal-taking behavior has occurred.

[0094] Among them, in this embodiment, the meal-taking behavior is determined to have occurred only after the weight of the dish has changed and lasted for a preset time, so as to avoid misidentification of the meal-taking behavior caused by measurement errors.

[0095] More specifically, the dishes include dishes that require tableware to be served and dishes that do not require tableware to be served;

[0096] Wherein, when the dish is a dish that needs to be picked up with tableware, the hand joint points among the human limb joint points and the tableware key points are selected as the preset key points;

[0097] When the dish is a dish that does not require tableware to be taken, the hand joints among the human limb joints are selected as the preset key points.

[0098] For example, dishes that require tableware to be served include: meat dishes, vegetarian dishes, soups, etc.; dishes that do not require tableware to be served include fruits, drinks, etc.

[0099] At the same time, for dishes that require tableware to pick up, if the person picking up the food does not take a spoon, it will not be judged as a meal picking behavior, which can avoid the problem of errors in the weight of the meal due to the weight of the tableware.

[0100] Specifically, the edge line of the dining table is parallel to the edge line of the weighing table, and the interval between the edge line of the weighing table and the edge line of the dining table is greater than or equal to a preset distance.

[0101] In the embodiment of the present invention, considering that if the weighing platform is placed relatively close to the edge line of the dining table, the hand joints of people passing by this position are likely to exceed the meal picking reference lines (the first meal picking line and the second meal picking line), thereby causing misjudgment of the meal picking behavior, the weighing platform should not be placed too close to the edge line of the dining table to avoid misjudgment of the meal picking behavior.

[0102] Preferably, before judging whether a meal-taking behavior occurs in the human body coordinate frame according to the reference area, the behavior recognition module recognizes the meal-taking behavior according to the human body coordinate frame, the key point position information and the weight change of the dish, and further comprises the following steps:

[0103] Setting a screening line on the human behavior image, wherein the screening line is located outside the dining table;

[0104] Judging whether a human coordinate frame in the human behavior image may have a meal-taking behavior according to the screening line;

[0105] If the bottom of the human coordinate frame exceeds the screening line, it is determined that a meal-taking behavior may occur in the human coordinate frame, and then whether a meal-taking behavior occurs in the human coordinate frame is determined according to the reference area;

[0106] If the bottom of the human body coordinate frame does not exceed the screening line, it is determined that the meal-taking behavior is not possible in the human body coordinate frame, and then it is no longer determined whether the meal-taking behavior occurs in the human body coordinate frame based on the reference area.

[0107] In the embodiment of the present invention, by setting a screening line to remove the human coordinate frame where it is impossible for the human body to take the food, the number of key point position judgments can be reduced and the recognition efficiency can be improved.

[0108] In one embodiment, the identity recognition module obtains the identity information of the person who takes the meal according to the coordinate frame of the person who takes the meal and the face image, including the following steps:

[0109] Input the face image into a face recognition model for processing, identify the face coordinate frame and obtain the identity information of the person;

[0110] According to the coordinates, the coordinate frame of the human body that takes the meal is matched with the coordinate frame of the human face in the face image to obtain the identity information of the person who takes the meal.

[0111] In the embodiment of the present invention, face recognition is performed through a face recognition model to obtain the identity information of all persons appearing in the meal pickup area, and then position matching is performed based on the coordinates, so that the meal pickup behavior can be matched with the identity information.

[0112] During implementation, the face recognition model may adopt a neural network model, such as a DeepFace model or a FaceNet model.

[0113] Specifically, the step of matching the coordinate frame of the human body that takes the meal with the coordinate frame of the human face in the face image according to the coordinates includes the following steps:

[0114] Acquire the position coordinate information of the target weighing platform in the shooting picture of the first camera; wherein the target weighing platform is a weighing platform in the same group as the first camera;

[0115] Defining the best recognition range of the target weighing platform according to the position coordinate information of the target weighing platform, and extracting the image in the best recognition range as the human behavior image;

[0116] Align the vertical center line of the human behavior image with the vertical center line of the human face image.

[0117] In the embodiment of the present invention, there is no need to strictly regulate the relative position between the first camera and the second camera. The optimal recognition range is obtained by detecting the position coordinate information of the target weighing platform in the picture taken by the first camera, such as Figure 4 As shown in the figure, the image in the optimal recognition range is extracted as the human behavior image, and then the center line of the human behavior image is aligned with the center line of the face image along the Y axis, that is, the coordinate alignment of the two images is completed, which can ensure the accurate matching of the human coordinate frame and the face coordinate frame, and accurately obtain the identity information of the person who picks up the meal.

[0118] Among them, when the optimal recognition range of the target weighing platform is defined according to the position coordinate information of the target weighing platform, the target weighing platform should be located on the center line of the optimal recognition range. Specifically, in the two-dimensional coordinate system of the shooting picture of the first camera, the bottom edge of the shooting picture is the X-axis, the height of the shooting picture is H, and the coordinates of the four corners of the target weighing platform are defined as (x1, y1), (x2, y1), (x1, y2), (x2, y2), wherein; then the coordinates of the four corners of the corresponding optimal recognition range are (x1-Δx, 0), (x2+Δx, 0), (x1-Δx, H), (x2+Δx, H), wherein Δx>0, Δx is the deviation value, specifically, Δx is a preset value, or Δx is the distance between the target weighing platform and the adjacent weighing platform.

[0119] It should be noted that, in actual applications, the shooting range of the second camera is smaller, and the shooting range of the first camera is larger. Taking the above situation into consideration, in an embodiment of the present invention, the optimal recognition range is defined in the shooting picture of the first camera to extract the human behavior image.

[0120] In addition, if the alignment of the images captured by the first camera and the second camera cannot be completed automatically, an alarm signal is issued to prompt the staff to adjust the installation position or installation angle of the two cameras.

[0121] It should be noted that in the embodiments of the present invention, the weighing platform, the first camera, the second camera and the display device are hardware devices of the system, and the behavior recognition module, the identity recognition module, the nutrition management module, the dish recognition module are software modules of the system and can be installed in a computer system or an Internet platform.

[0122] The intelligent dining nutrition management system of the embodiment of the present invention is an intelligent solution that integrates biometric recognition software such as face recognition and behavior recognition and intelligent equipment. It is an important part of the smart restaurant. It meets the user's personalized dietary nutrition needs through intelligent equipment and artificial intelligence algorithms and improves the efficiency of restaurant management.

[0123] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0124] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An intelligent dining nutrition management system, characterized in that: The system comprises: A nutrition data module stores the recommended intake of the user's required nutrition information; A weighing table, used to hold dishes and detect weight changes of the dishes; A shooting device, used for shooting human behavior images and human face images; A behavior recognition module processes the human behavior image to identify a human coordinate frame and key points, and obtains position information of the key points; identifies the meal-taking behavior according to the human coordinate frame and the position information of the key points and the weight change of the dish; the key points include the key points of human limb joints and tableware; An identity recognition module, when only one of the meal-picking behaviors is recognized, obtains the identity information of the person who picks up the meal according to the coordinate frame of the human body where the meal-picking behavior occurs and the facial image; A nutrition management module, which retrieves the recommended intake of the nutrition information required by the meal picker according to the identity information of the meal picker; The display device displays the nutritional information and weight changes of the dishes, as well as the recommended intake of nutritional information required by the diners.

2. The system according to claim 1, characterized in that The system further comprises: Data storage module, used to store the user's personal meal collection records; The personal meal pick-up record includes the user's ID number, the name of the dish picked up each time, the weight of the meal, the time of meal pick-up completion, and the amount of nutrients taken.

3. The system according to claim 2, characterized in that The nutrition management module determines the completion time of the meal picking by tracking the coordinate frame of the human body that performs the meal picking behavior, and uses the weight change of the dish when the meal picking is completed as the meal picking weight, calculates the nutritional component consumption of the dish based on the meal picking weight, and then sends the nutritional component consumption of the dish to the personal meal picking record of the person who picks up the meal in the data storage module for storage.

4. The system according to claim 3, characterized in that When the meal is picked up, the nutrition management module also calculates the cumulative amount of nutrients taken by the person who has picked up the meal, and compares the cumulative amount of nutrients taken with the recommended intake. If the cumulative amount exceeds the recommended intake, a reminder message is sent to the person who has picked up the meal.

5. The system according to claim 3, characterized in that The nutrition management module determines the meal picking completion time of this meal picking by tracking the coordinate frame of the human body where the meal picking behavior occurs, including the following steps: The ID number of the person picking up the meal is obtained according to the identity information obtained, and the ID number is continuously and dynamically bound to the coordinate frame of the human body where the meal picking behavior occurs, and then the coordinate frame of the human body is tracked according to the ID number to determine the completion time of meal picking.

6. The system according to any one of claims 1 to 5, characterized in that: The nutrition management module also calculates the current intake amount of each nutrient component of the dish in real time according to the weight change of the dish detected by the weighing platform, and displays it through a display device.

7. The system according to any one of claims 1 to 5, characterized in that: The system further comprises: The dish recognition module is used to recognize the dish name according to the dish image, obtain the nutritional information of the dish according to the dish name, and send the dish name and the nutritional information to the display device for display.

8. The system according to any one of claims 1 to 5, characterized in that: The number of the weighing platforms is set to be multiple, each weighing platform is provided with a dish, each weighing platform is respectively configured with a display device, and each weighing platform is respectively provided with a group of the shooting devices to shoot the food picking area of ​​the weighing platform to obtain human behavior images and facial images.

9. The system according to any one of claims 1 to 5, characterized in that: The method of identifying the meal-picking behavior according to the human body coordinate frame, the position information of the key points, and the weight change of the dish comprises the following steps: Setting a meal-taking reference line on the processed human behavior image, wherein the meal-taking reference line is located between the edge line of the weighing platform and the edge line of the dining table; and a range defined between the meal-taking reference line and the bottom edge line of the human behavior image is used as a reference area; Determining whether a meal-taking behavior occurs within the human body coordinate frame according to the reference area; If the preset key point in the human body coordinate frame is located in the reference area, and the weight of the dish changes and lasts for a preset time, it is determined that the meal picking behavior occurs.

10. The system according to any one of claims 1 to 5, characterized in that: The identity recognition module obtains the identity information of the person who takes the meal according to the coordinate frame of the person who takes the meal and the face image, including the following steps: Input the face image into a face recognition model for processing, identify the face coordinate frame and obtain the identity information of the person; According to the coordinates, the coordinate frame of the human body that takes the meal is matched with the coordinate frame of the human face in the face image to obtain the identity information of the person who takes the meal.

Images