An intelligent dining nutrition management system

Abstract

The present invention relates to an intelligent dining nutrition management system, which belongs to the field of intelligent restaurant technology and solves the problem that the existing technology cannot meet the personalized needs of users for dietary nutrition management. The system of the present invention includes: a nutrition data module, which stores the recommended intake of the user's required nutritional information; a weighing platform; a shooting device, which shoots human behavior images and facial images; a behavior recognition module, which identifies the behavior of picking up meals based on the human coordinate frame and key points marked on the human behavior image and the weight change of the dishes; an identity recognition module, which is used to obtain the identity information of the person picking up meals; a nutrition management module, which retrieves the recommended intake of the nutritional information required by the person picking up meals based on the identity information of the person picking up meals; a display device, which displays the nutritional component information and weight change of the dishes, as well as the recommended intake of the nutritional information required by the person picking up meals. In the present invention, the person picking up meals can be accurately identified, thereby providing users with accurate personalized nutrition management services.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent restaurants, and in particular to an intelligent dining nutrition management system. Background Art

[0002] In hospitals and nursing homes, the nutritional and dietary health of inpatients needs to be managed. With the rapid development of the internet, the "intelligence + internet + product" model has been applied across various industries, leading to the emergence of smart restaurants. Smart restaurants utilize integrated management platforms to automate and intelligentize restaurant management, improving operational efficiency and service quality. However, existing smart restaurant management systems primarily focus on improving restaurant management efficiency. While some systems offer nutritional recommendations, they are unable to provide accurate, personalized nutritional management services and fail to meet the personalized needs of users.

[0003] At the same time, the existing smart restaurant system uses the facial recognition camera to capture facial images to determine the identity of the person picking up the meal, and uses the entry and exit of the face as the basis for determining the start and end time of picking up the meal. There are problems such as inaccurate recognition of the meal picking behavior due to multiple people picking up the meal, and the problem of misjudging the end of meal picking due to loss of facial signals. Therefore, it is impossible to accurately obtain information such as "who picked up the meal, what dishes were taken, and how much was taken", and thus it is impossible to accurately manage the user's dietary nutrition and health. Summary of the Invention

[0004] In view of the above analysis, an embodiment of the present invention aims to provide an intelligent dining nutrition management system to solve the problem that the existing technology cannot meet the user's personalized needs for dietary nutrition management.

[0005] In one aspect, an embodiment of the present invention provides an intelligent dining nutrition management system, the system comprising:

[0006] A nutrition data module stores the user's required nutrition information and recommended intake;

[0007] A weighing platform, used for placing dishes and detecting weight changes of the dishes;

[0008] A shooting device, used for shooting human behavior images and human face images;

[0009] a behavior recognition module that processes the human behavior image to identify a human coordinate frame and key points, and obtains position information of the key points; and then identifies the meal-taking behavior based on the human coordinate frame and the position information of the key points and the weight change of the dish; the key points include the key points of the human limb joints and the tableware;

[0010] an identity recognition module, which, when only one of the meal-picking behaviors is identified, obtains the identity information of the person who picks up the meal based on the coordinate frame of the person who takes the meal and the facial image;

[0011] A nutrition management module retrieves the recommended nutritional information required by the person picking up the meal based on his / her identity information;

[0012] The display device displays the nutritional information and weight changes of the dishes, as well as the recommended intake of nutritional information required by the diners.

[0013] Based on the further improvement of the above system, the system further includes:

[0014] Data storage module, used to store the user's personal meal collection records;

[0015] The personal meal pick-up record includes the user's ID number and the name of the dish, the weight of the meal, the time of completion of the meal pick-up, and the amount of nutrients taken each time.

[0016] Based on the further improvement of the above system, the nutrition management module determines the completion time of the meal picking by tracking the coordinate frame of the human body where the meal picking behavior occurs, and uses the weight change of the dish when the meal picking is completed as the meal picking weight, calculates the nutritional component consumption of the dish based on the meal picking weight, and then sends the nutritional component consumption of the dish to the personal meal picking record of the person who picks up the meal in the data storage module for storage.

[0017] Based on further improvements to the above system, when the meal is picked up, the nutrition management module also calculates the cumulative amount of nutrients taken of all dishes currently taken by the diner, and compares the cumulative amount of nutrients taken with the recommended intake. If the cumulative amount exceeds the recommended intake, a reminder message will be sent to the diner.

[0018] Based on the further improvement of the above system, the nutrition management module determines the completion time of the meal pickup by tracking the coordinate frame of the human body where the meal pickup behavior occurs, including the following steps:

[0019] The ID number of the person picking up the meal is obtained based on the identity information obtained, and the ID number is dynamically bound to the coordinate frame of the human body where the meal picking up occurs. Then, the coordinate frame of the human body is tracked based on the ID number to determine the completion time of the meal picking up.

[0020] Based on further improvements of the above system, the nutrition management module also calculates the current intake amount of each nutrient component of the dish in real time according to the weight change of the dish detected by the weighing platform, and displays it through a display device.

[0021] Based on the further improvement of the above system, the system further includes:

[0022] The dish recognition module is used to recognize the dish name based on the dish image, obtain the nutritional information of the dish based on the dish name, and send the dish name and the nutritional information to the display device for display.

[0023] Based on further improvements of the above system, the number of the weighing platforms is set to multiple, each weighing platform is provided with a dish, each weighing platform is respectively equipped with a display device, and each weighing platform is respectively provided with a group of the shooting devices to shoot the food picking area of ​​the weighing platform to obtain human behavior images and facial images.

[0024] Based on a further improvement of the above system, the method of identifying the meal-picking behavior based on the position information of the human coordinate frame and the key points and the weight change of the dish includes the following steps:

[0025] Setting a meal-taking reference line on the processed human behavior image, wherein the meal-taking reference line is located between the edge line of the weighing platform and the edge line of the dining table; and a range defined between the meal-taking reference line and the bottom edge line of the human behavior image is used as a reference area;

[0026] Determining whether a meal-taking action occurs within the human body coordinate frame according to the reference area;

[0027] If the preset key point in the human body coordinate frame is located in the reference area, and the weight of the dish changes and lasts for a preset time, it is determined that a meal-taking behavior occurs.

[0028] Based on the further improvement of the above system, the identity recognition module obtains the identity information of the person who picks up the food based on the coordinate frame of the person who takes the food and the facial image, including the following steps:

[0029] Inputting the facial image into a face recognition model for processing, identifying the facial coordinate frame and obtaining the identity information of the person;

[0030] The coordinate frame of the human body that takes the meal is matched with the coordinate frame of the human face in the face image according to the coordinates to obtain the identity information of the person who takes the meal.

[0031] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0032] 1. In the present invention, when recognizing meal-picking behavior based on human behavior images, the key points of human limb joints and tableware are all taken as key points to be identified, so that the meal-picking behavior can be accurately identified. Then, the meal-picking behavior and identity information are accurately associated based on the human coordinate frame marked on the face image and the human behavior image, and the identity information of the person picking up the meal is accurately identified, so that the recommended intake of the nutritional information required by the person picking up the meal can be displayed to the person picking up the meal in real time, providing users with accurate personalized nutrition management services to meet the user's personalized nutritional needs.

[0033] 2. In the present invention, by setting up a data storage module, the personal meal-taking records of the meal-takers can be accurately collected, which is beneficial for professionals to analyze the user's dietary preferences and provide personalized services.

[0034] 3. In the present invention, after identifying the meal-picking behavior, the completion time of this meal-picking behavior can be accurately determined by tracking the coordinate frame of the human body in the meal-picking behavior, which can avoid the problem of misjudging the end of meal-picking due to loss of facial signals, thereby facilitating accurate acquisition of the meal-picking weight and the nutritional content of the dishes.

[0035] 4. In the present invention, when the cumulative intake of nutrients exceeds the recommended intake, an alarm message is sent to the user, thereby realizing real-time nutritional intervention reminders.

[0036] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0038] Figure 1 This is a module diagram of an intelligent dining nutrition management system according to an embodiment of the present invention;

[0039] Figure 2 Schematic diagram of human limb joints and a human coordinate frame according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the positions of the meal pickup reference line and the screening line according to an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of an optimal recognition range defined in an embodiment of the present invention; DETAILED DESCRIPTION

[0042] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0043] The embodiment of the present invention discloses an intelligent dining nutrition management system. Figure 1 As shown in . The system includes:

[0044] A nutrition data module stores the user's required nutrition information and recommended intake;

[0045] A weighing platform, used for placing dishes and detecting weight changes of the dishes;

[0046] A shooting device, used for shooting human behavior images and human face images;

[0047] a behavior recognition module that processes the human behavior image to identify a human coordinate frame and key points, and obtains position information of the key points; and then identifies the meal-taking behavior based on the human coordinate frame and the position information of the key points and the weight change of the dish; the key points include the key points of the human limb joints and the tableware;

[0048] an identity recognition module, when only one of the meal-picking behaviors is identified, the identity recognition module obtains the identity information of the person who picks up the meal based on the coordinate frame of the person who takes the meal and the facial image;

[0049] A nutrition management module retrieves the recommended nutritional information required by the person picking up the meal based on his / her identity information;

[0050] The display device displays the nutritional information and weight changes of the dishes, as well as the recommended intake of nutritional information required by the diners.

[0051] During implementation, a container or space for placing food is provided on the weighing platform. For example, the weighing platform may be a buffet oven with a weighing function. The recommended nutritional intake required by the user is set by doctors or professionals in hospitals or nursing homes based on the user's personal situation.

[0052] Compared with the prior art, in the embodiment of the present invention, when identifying meal-picking behavior based on human behavior images, the key points of the human limb joints and tableware are all taken as key points that need to be identified, so that the meal-picking behavior can be accurately identified, and then the meal-picking behavior and identity information are accurately associated based on the human coordinate frame marked on the face image and the human behavior image, and the identity information of the person picking up the meal is accurately identified, so that the recommended intake of the nutritional information required by the person picking up the meal can be displayed to the person picking up the meal in real time, and accurate personalized nutrition management services can be provided to the user to meet the user's personalized nutritional meal needs.

[0053] In one embodiment, the system further comprises:

[0054] Data storage module, used to store the user's personal meal collection records;

[0055] The personal meal pick-up record includes the user's ID number and the name of the dish, the weight of the meal, the time of completion of the meal pick-up, and the amount of nutrients taken each time.

[0056] In the embodiment of the present invention, by providing a data storage module, the personal meal-taking record of the person who takes the meal can be accurately collected, which is helpful for professionals to analyze the user's dietary preferences and provide personalized services.

[0057] Specifically, the nutrition management module determines the completion time of the meal pickup by tracking the coordinate frame of the human body in which the meal pickup behavior occurs, and uses the weight change of the dish when the meal pickup is completed as the meal pickup weight, calculates the nutritional component consumption of the dish based on the meal pickup weight, and then sends the nutritional component consumption of the dish to the personal meal pickup record of the person who picks up the meal in the data storage module for storage.

[0058] Compared with the prior art, in the embodiment of the present invention, after identifying the meal-picking behavior, the meal-picking completion time can be accurately determined by tracking the coordinate frame of the human body in which the meal-picking behavior occurs, thereby avoiding the problem of misjudging the end of meal-picking due to loss of facial signals, thereby facilitating accurate acquisition of the meal-picking weight and the nutritional content of the dishes.

[0059] Specifically, when the meal is picked up, the nutrition management module also calculates the cumulative amount of nutrients taken of all dishes currently taken by the diner, and compares the cumulative amount of nutrients taken with the recommended intake. If the cumulative amount exceeds the recommended intake, a reminder message is sent to the diner.

[0060] In the embodiment of the present invention, when the cumulative intake of nutrients exceeds the recommended intake, an alarm message is sent to the user, thereby realizing real-time nutritional intervention reminders.

[0061] Specifically, the alarm information may be a text signal, a light signal or a sound signal, and may be issued through a display device, or a device such as an alarm light or an alarm.

[0062] Specifically, the nutrition management module determines the completion time of meal pickup by tracking the coordinate frame of the human body where the meal pickup occurs, including the following steps:

[0063] The ID number of the person picking up the meal is obtained based on the identity information obtained, and the ID number is dynamically bound to the coordinate frame of the human body where the meal picking up occurs. Then, the coordinate frame of the human body is tracked based on the ID number to determine the completion time of the meal picking up.

[0064] In this embodiment, each user is set with an ID number, and the ID number of the person who picks up the meal is obtained after obtaining the identity information of the person who picks up the meal. The ID number is continuously and dynamically bound to the coordinate frame of the human body where the meal picking behavior occurs, so that the meal picking behavior and the identity information of the person who picks up the meal are timely bound, so that the position of the person who picks up the meal and the entire meal picking process can be accurately tracked. When the meal picking behavior is no longer recognized in the human coordinate frame, the meal picking is completed, and this time is the meal picking completion time of this meal picking, and then the accurate meal picking weight of the person who picks up the meal is obtained according to the meal picking completion time.

[0065] Specifically, the nutrition management module also calculates the current intake amount of each nutrient component of the dish in real time based on the weight change of the dish detected by the weighing platform, and displays it through a display device.

[0066] In the embodiment of the present invention, the current amount of each nutrient component of the dish is displayed to the person picking up the meal in real time, so that the person picking up the meal can control the intake of each nutrient component and realize real-time reminder of dietary nutrition.

[0067] Specifically, the system further includes:

[0068] The dish recognition module is used to recognize the dish name based on the dish image, obtain the nutritional information of the dish based on the dish name, and send the dish name and the nutritional information to the display device for display.

[0069] During implementation, during the restaurant's dish preparation period, the system starts the serving mode. After the restaurant staff places the dish on the weighing table, the camera automatically takes an image of the dish. Then the dish recognition module uses the dish image to identify the dish name, obtains the nutritional information of the dish, and displays it through the display device. There is no need for staff to manually input dish information, thereby reducing the staff's labor intensity and improving work efficiency.

[0070] Furthermore, the dish recognition module processes the dish image using a dish recognition model to identify the dish name. Specifically, the dish recognition model may be a neural network model.

[0071] In a specific embodiment, the number of the weighing platforms is set to multiple, each weighing platform is provided with a dish, each weighing platform is respectively equipped with a display device, and each weighing platform is respectively provided with a group of the shooting devices to shoot the food picking area of ​​the weighing platform to obtain human behavior images and facial images.

[0072] Specifically, each set of the shooting devices includes a first camera and a second camera. The first camera is used to shoot human behavior images and / or food images, and the second camera is used to shoot human face images.

[0073] During implementation, the first camera is positioned above the weighing platform in the same group, and faces the weighing platform so that the weighing platform is within the camera's field of view. The second camera is positioned above the weighing platform in the same group, and faces the head of the person in front of the dining table, with the second camera's shooting center and the centerline of the weighing platform in the same group located on the same vertical plane.

[0074] In an embodiment of the present invention, multiple sets of weighing platforms, photographing devices and display devices are provided on the dining table, thereby enabling nutritional dietary management of a variety of dishes.

[0075] The first camera captures images of human behavior and dishes. Therefore, the person picking up the food, the weighing platform, and the dishes on it should be within the first camera's frame, ensuring that the person's behavior is captured in the first camera's image. The second camera captures facial images. It precisely monitors the customer's head area, helping to capture their facial features. Its center of gravity is aligned with the centerline of the weighing platform, ensuring consistent and accurate capture and minimizing errors caused by angular deviation.

[0076] Specifically, when the behavior recognition module identifies the behavior of picking up food based on human behavior images, it regards the human body and tableware as a whole. The key points of the human limb joints and the tableware are all key points that need to be identified. Then, the meal-picking behavior is comprehensively identified based on the position of the preset key points and the weight changes of the dishes, so that the meal-picking behavior can be identified more accurately and the misjudgment of the meal-picking behavior of multiple people can be reduced.

[0077] Among them, when identifying the meal-picking behavior, the behavior recognition module processes the human behavior image through the key point recognition model, identifies the human coordinate frame and key points, and obtains the position information of the key points.

[0078] Specifically, the key point model is a YOLO V-POSE model, such as the YOLO V7-POSE model or the YOLO V9-POSE model. The YOLO V-POSE model is an extension of the YOLO object detection framework for human pose estimation. It adds key point detection to object detection and can identify key points in an image, such as joints. The output of the YOLO V-POSE model is a set of coordinate points and target bounding boxes representing key points of objects in the image.

[0079] like Figure 2 As shown in , the model can detect the human body's limb joints, the key points of the tableware, and a human coordinate frame. During implementation, the 17 limb joints of the human body and the 3 key points on the tableware are used as targets for recognition.

[0080] In a specific embodiment, the YOLOV7-POSE model is used as a key point recognition model. The limb joint detection algorithm of the YOLOV7-POSE model is implemented based on the top-down method, that is, the target frame of each person is first detected, and then the posture of each frame is estimated and the result is output. The YOLOV7-POSE model accepts the input image and generates a feature map, which is then used to predict the position of each person's limb joints. It can estimate the posture of multiple people in real time and is suitable for scenarios where people pick up food.

[0081] First, the behavior recognition camera is used to capture human behavior images in the food pickup area, and the acquired behavior images are input into the YOLOV7-POSE model.

[0082] It should be noted that the key point model in the embodiment of the present invention is a trained model. Specifically, training the key point model includes the following steps:

[0083] Obtain several human behavior images in different food pickup scenarios;

[0084] Marking key points and human coordinate frames on each of the human behavior images; wherein the marked key points include human limb joints and tableware key points;

[0085] The labeled human behavior images are divided into training sets and test sets according to a preset ratio;

[0086] Training the key point model using the training set;

[0087] The trained key point model is verified through the test set to obtain the trained key point recognition model.

[0088] After the behavior recognition module processes the human behavior image through the key point recognition model, the behavior recognition module recognizes the meal-taking behavior according to the human coordinate frame and the position information of the key points and the weight change of the dish.

[0089] In a specific embodiment, the behavior recognition module identifies the meal-taking behavior based on the position information of the human coordinate frame and the key points and the weight change of the dish, including the following steps:

[0090] Setting a meal-taking reference line on the processed human behavior image, wherein the meal-taking reference line is located between the edge line of the weighing platform and the edge line of the dining table; and a range defined between the meal-taking reference line and the bottom edge line of the human behavior image is used as a reference area;

[0091] Determining whether a meal-taking action occurs within the human body coordinate frame according to the reference area;

[0092] If the preset key point in the human body coordinate frame is located in the reference area, and the weight of the dish changes and lasts for a preset time, it is determined that a meal-taking behavior occurs.

[0093] In the embodiment of the present invention, Figure 3 As shown in , a meal-taking reference line is drawn between the edge line of the weighing table and the edge line of the dining table, and some of the identifiable key points (key points of human limb joints and tableware) are selected as preset key points. When the preset key points exceed the meal-taking reference line and enter the reference area, and the weight of the dish changes and lasts for a preset time, for example, within 1 second, it is determined that a meal-taking behavior has occurred.

[0094] In this embodiment, the meal-picking behavior is determined to have occurred only after the weight of the dish has changed and lasted for a preset time, so as to avoid misidentification of the meal-picking behavior due to measurement errors.

[0095] More specifically, the dishes include dishes that require tableware to be served and dishes that do not require tableware to be served;

[0096] Wherein, when the dish is a dish that requires tableware to be taken, the hand joint points among the human limb joint points and the tableware key points are selected as the preset key points;

[0097] When the dish is a dish that does not require tableware to be taken, the hand joints among the human limb joints are selected as the preset key points.

[0098] For example, dishes that require tableware include: meat dishes, vegetarian dishes, soups, etc.; dishes that do not require tableware include fruits, drinks, etc.

[0099] At the same time, for dishes that require tableware to pick up, if the person picking up the food does not take a spoon, it will not be judged as a picking up behavior, which can avoid the problem of errors in the weight of the food due to the weight of the tableware.

[0100] Specifically, the edge line of the dining table is parallel to the edge line of the weighing platform, and the interval between the edge line of the weighing platform and the edge line of the dining table is greater than or equal to a preset distance.

[0101] In the embodiment of the present invention, considering that if the weighing platform is placed relatively close to the edge line of the dining table, the hand joints of people passing by this position are likely to exceed the meal-taking reference lines (the first meal-taking line and the second meal-taking line), thereby causing misjudgment of the meal-taking behavior, the weighing platform should not be placed too close to the edge line of the dining table to avoid misjudgment of the meal-taking behavior.

[0102] Preferably, before determining whether a meal-picking behavior occurs within the human coordinate frame according to the reference area, the behavior recognition module recognizes the meal-picking behavior according to the human coordinate frame, the key point position information, and the weight change of the dish, and further includes the following steps:

[0103] Setting a screening line on the human behavior image, wherein the screening line is located outside the dining table;

[0104] Determining whether a human coordinate frame in the human behavior image may be in a meal-taking behavior based on the screening line;

[0105] If the bottom of the human coordinate frame exceeds the screening line, it is determined that a meal-taking behavior may occur in the human coordinate frame, and then whether a meal-taking behavior occurs in the human coordinate frame is determined based on the reference area;

[0106] If the bottom of the human coordinate frame does not exceed the screening line, it is determined that the meal-taking behavior is impossible in the human coordinate frame, and then whether the meal-taking behavior occurs in the human coordinate frame is no longer determined based on the reference area.

[0107] In the embodiment of the present invention, by setting a screening line to remove the human coordinate frame where the human body is unlikely to take the food, the number of key point position judgments can be reduced and the recognition efficiency can be improved.

[0108] In one embodiment, the identity recognition module obtains the identity information of the person who picks up the food based on the coordinate frame of the person who picks up the food and the facial image, including the following steps:

[0109] Inputting the facial image into a face recognition model for processing, identifying the facial coordinate frame and obtaining the identity information of the person;

[0110] The coordinate frame of the human body that takes the meal is matched with the coordinate frame of the human face in the face image according to the coordinates to obtain the identity information of the person who takes the meal.

[0111] In an embodiment of the present invention, face recognition is performed using a face recognition model to obtain identity information of all persons appearing in the food pickup area, and then position matching is performed based on coordinates, so that the food pickup behavior can be matched with the identity information.

[0112] During implementation, the face recognition model may adopt a neural network model, such as a DeepFace model or a FaceNet model.

[0113] Specifically, the step of matching the coordinate frame of the human body that has taken the meal with the coordinate frame of the human face in the face image according to the coordinates includes the following steps:

[0114] Obtaining position coordinate information of a target weighing platform in the image captured by the first camera; wherein the target weighing platform is a weighing platform in the same group as the first camera;

[0115] Defining an optimal recognition range of the target weighing platform according to the position coordinate information of the target weighing platform, and extracting an image in the optimal recognition range as the human behavior image;

[0116] Align the vertical center line of the human behavior image with the vertical center line of the human face image.

[0117] In the embodiment of the present invention, there is no need to strictly regulate the relative position between the first camera and the second camera. The optimal recognition range is obtained by detecting the position coordinate information of the target weighing platform in the picture taken by the first camera, such as Figure 4 As shown in the figure, the image in the optimal recognition range is extracted as the human behavior image, and then the center line of the human behavior image is aligned with the center line of the face image along the Y axis, that is, the coordinate alignment of the two images is completed, which can ensure the accurate matching of the human coordinate frame and the face coordinate frame, and accurately obtain the identity information of the person who picks up the meal.

[0118] Among them, when the optimal recognition range of the target weighing platform is delineated according to the position coordinate information of the target weighing platform, the target weighing platform should be located on the center line of the optimal recognition range. Specifically, in the two-dimensional coordinate system of the shooting picture of the first camera, the bottom edge of the shooting picture is the X axis, the height of the shooting picture is H, and the coordinates of the four corners of the target weighing platform are defined as (x1, y1), (x2, y1), (x1, y2), (x2, y2), wherein; the coordinates of the four corners of the corresponding optimal recognition range are (x1-Δx, 0), (x2+Δx, 0), (x1-Δx, H), (x2+Δx, H), wherein Δx>0, Δx is the deviation value, specifically, Δx is a preset value, or Δx is the distance between the target weighing platform and the adjacent weighing platform.

[0119] It should be noted that, in actual applications, the shooting range of the second camera is smaller, and the shooting range of the first camera is larger. Taking the above situation into consideration, in an embodiment of the present invention, the optimal recognition range is defined in the shooting picture of the first camera to extract the human behavior image.

[0120] In addition, if the alignment of the images captured by the first camera and the second camera cannot be completed automatically, an alarm signal is issued to prompt the staff to adjust the installation position or installation angle of the two cameras.

[0121] It should be noted that in the embodiment of the present invention, the weighing platform, the first camera, the second camera and the display device are hardware devices of the system, and the behavior recognition module, the identity recognition module, the nutrition management module, the dish recognition module, etc. are software modules of the system, which can be installed in a computer system or an Internet platform.

[0122] The intelligent dining nutrition management system of the embodiment of the present invention is an intelligent solution that integrates biometric recognition software such as facial recognition and behavior recognition and intelligent equipment. It is an important component of the smart restaurant. Through intelligent equipment and artificial intelligence algorithms, it meets the user's personalized dietary nutrition needs and improves the efficiency of restaurant management.

[0123] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0124] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. An intelligent dining nutrition management system, characterized in that: The system comprises: A nutrition data module stores the user's required nutrition information and recommended intake; A weighing platform, used for placing dishes and detecting weight changes of the dishes; A shooting device, used for shooting human behavior images and human face images; a behavior recognition module that processes the human behavior image to identify a human coordinate frame and key points, and obtains position information of the key points; identifies the meal-taking behavior based on the human coordinate frame and the position information of the key points and the weight change of the dish; the key points include the key points of the human limb joints and the tableware; an identity recognition module, which, when only one of the meal-picking behaviors is identified, obtains the identity information of the person who picks up the meal based on the coordinate frame of the person who takes the meal and the facial image; A nutrition management module retrieves the recommended nutritional information required by the person picking up the meal based on his / her identity information; a display device for displaying nutritional information and weight changes of dishes, as well as the recommended intake of nutritional information required by the person picking up the meal; The method of identifying the meal-picking behavior based on the position information of the human body coordinate frame and the key points and the weight change of the dish includes the following steps: Setting a meal-taking reference line on the processed human behavior image, wherein the meal-taking reference line is located between the edge line of the weighing platform and the edge line of the dining table; and a range defined between the meal-taking reference line and the bottom edge line of the human behavior image is used as a reference area; Determining whether a meal-taking action occurs within the human body coordinate frame according to the reference area; If the preset key point in the human body coordinate frame is located in the reference area, and the weight of the dish changes and lasts for a preset time, it is determined that a meal-taking behavior occurs.

2. The system according to claim 1, wherein: The system further comprises: Data storage module, used to store the user's personal meal collection records; The personal meal pick-up record includes the user's ID number and the name of the dish, the weight of the meal, the time of completion of the meal pick-up, and the amount of nutrients taken each time.

3. The system according to claim 2, characterized in that The nutrition management module determines the completion time of the meal picking by tracking the coordinate frame of the human body in the meal picking behavior, and uses the weight change of the dish when the meal picking is completed as the meal picking weight, calculates the nutritional component consumption of the dish based on the meal picking weight, and then sends the nutritional component consumption of the dish to the personal meal picking record of the person who picks up the meal in the data storage module for storage.

4. The system according to claim 3, characterized in that When the meal is taken, the nutrition management module also calculates the cumulative amount of nutrients taken of all dishes currently taken by the diner, and compares the cumulative amount of nutrients taken with the recommended intake. If the cumulative amount exceeds the recommended intake, a reminder message is sent to the diner.

5. The system according to claim 3, wherein: The nutrition management module determines the completion time of meal pickup by tracking the coordinate frame of the human body where the meal pickup occurs, including the following steps: The ID number of the person picking up the meal is obtained based on the identity information obtained, and the ID number is dynamically bound to the coordinate frame of the human body where the meal picking up occurs. Then, the coordinate frame of the human body is tracked based on the ID number to determine the completion time of the meal picking up.

6. The system according to any one of claims 1 to 5, characterized in that The nutrition management module also calculates the current intake amount of each nutrient component of the dish in real time based on the weight change of the dish detected by the weighing platform, and displays it through a display device.

7. The system according to any one of claims 1 to 5, characterized in that The system further comprises: The dish recognition module is used to recognize the dish name based on the dish image, obtain the nutritional information of the dish based on the dish name, and send the dish name and the nutritional information to the display device for display.

8. The system according to any one of claims 1 to 5, characterized in that The number of the weighing platforms is set to be multiple, and a dish is set on each weighing platform. Each weighing platform is equipped with a display device, and each weighing platform is respectively provided with a group of the shooting devices to shoot the food picking area of ​​the weighing platform to obtain human behavior images and facial images.

9. The system according to any one of claims 1 to 5, characterized in that The dishes include dishes that require tableware to be served and dishes that do not require tableware to be served; Wherein, when the dish is a dish that requires tableware to be taken, the hand joint points among the human limb joint points and the tableware key points are selected as the preset key points; When the dish is a dish that does not require tableware, the hand joints among the human limb joints are selected as the preset key points; Setting a screening line on the human behavior image, wherein the screening line is located outside the dining table; Determining whether a human coordinate frame in the human behavior image may be in a meal-taking behavior based on the screening line; If the bottom of the human coordinate frame exceeds the screening line, it is determined that a meal-taking behavior may occur in the human coordinate frame, and then whether a meal-taking behavior occurs in the human coordinate frame is determined based on the reference area; If the bottom of the human coordinate frame does not exceed the screening line, it is determined that the meal-taking behavior is impossible in the human coordinate frame, and then whether the meal-taking behavior occurs in the human coordinate frame is no longer determined based on the reference area.

10. The system according to any one of claims 1 to 5, characterized in that The identity recognition module obtains the identity information of the person who takes the meal based on the coordinate frame of the person who takes the meal and the facial image, including the following steps: Inputting the facial image into a face recognition model for processing, identifying the facial coordinate frame and obtaining the identity information of the person; The coordinate frame of the human body that takes the meal is matched with the coordinate frame of the human face in the face image according to the coordinates to obtain the identity information of the person who takes the meal.

Images