A dish recommendation method and system

Abstract

The application provides a dish recommendation method and system, wherein the dish recommendation method comprises the following steps: obtaining face video data comprising at least one frame of image; performing face attribute recognition on at least part of the image comprising a target object in the face video data to obtain face expression data and facial posture data of the target object; generating a current-time ordering preference of the target object according to the face expression data and the facial posture data; when it is determined that there is historical ordering data of the target object, generating a historical ordering preference of the target object according to the historical ordering data; obtaining a recommended dish according to the historical ordering preference and the current-time ordering preference according to a preset rule, and displaying the recommended dish to the target object. The dish recommendation effect is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for recommending dishes. Background Technology

[0002] Traditional manual ordering is inefficient. Existing systems use facial recognition to record user preferences, creating a historical database. When a user orders again, facial recognition identifies them and automatically retrieves frequently ordered items. However, this process doesn't consider real-time user experience or changes in taste, resulting in poor food recommendations. Furthermore, while facial recognition can recommend similar dishes to users with similar facial features, similar facial features don't guarantee different tastes, limiting the practicality of such recommendations. Therefore, improving the effectiveness of food recommendations becomes crucial. Summary of the Invention

[0003] This invention provides a method and system for recommending dishes, which can improve the effectiveness of dish recommendations.

[0004] In a first aspect, embodiments of the present invention provide a method for recommending dishes, including:

[0005] Acquire facial video data including at least one frame of image;

[0006] Facial attribute recognition is performed on at least a portion of the images including the target object in the facial video data to obtain facial expression data and facial pose data of the target object;

[0007] The target object's current order preferences are generated based on the facial expression data and the facial posture data.

[0008] When it is determined that historical order data of the target object exists, the historical order preference of the target object is generated based on the historical order data.

[0009] According to preset rules, dishes to be recommended are obtained based on the historical order preferences and the current order preferences, and the dishes to be recommended are displayed to the target object.

[0010] In one possible implementation, generating the target object's current order preference based on the facial expression data and the facial pose data includes:

[0011] Based on the facial expression data, determine the expression code used to represent the target expression of the target object;

[0012] Determine the target time corresponding to the expression code;

[0013] The facial orientation data corresponding to the target time is determined from the facial pose data;

[0014] Determine the heatmap region of interest corresponding to the facial orientation data, and the first set of dishes corresponding to the heatmap region of interest;

[0015] The first set of dishes is taken as the current order preference of the target object.

[0016] In one possible implementation, generating the target object's historical order preferences based on the historical order data when it is determined that historical order data for the target object exists includes:

[0017] Based on the historical order data, determine the target object's historical order records;

[0018] The second set of dishes and the number of times each dish was ordered were determined from the historical order records.

[0019] Based on the second set of dishes and the number of orders, the historical order preferences of the target object are generated.

[0020] In one possible implementation, the step of obtaining recommended dishes based on the historical order preferences and the current order preferences according to preset rules, and displaying the recommended dishes to the target object, includes:

[0021] Based on the sales data of the dishes, the weights of the historical order preferences and the current order preferences are set respectively;

[0022] The weight of the dish to be recommended is determined based on the historical order preferences and the weight corresponding to the current order preferences.

[0023] The dishes to be recommended are displayed to the target object in descending order of weight.

[0024] In one possible implementation, performing facial attribute recognition on at least a portion of the image including the target object in the facial video data to obtain facial expression data and facial pose data of the target object includes:

[0025] Feature extraction is performed on at least a portion of the images including the target object in the facial video data using a deep neural network to determine the facial expression data of the target object;

[0026] Extract at least one key point of the target object from each frame of the at least partial image;

[0027] Based on the at least one key point, the facial pose data of the target object is determined.

[0028] In one possible implementation, after acquiring the face video data including at least one frame of image, the method further includes:

[0029] Face detection is performed on each frame of the at least some images, and images whose face clarity and face occlusion meet the preset quality are selected.

[0030] Images that meet the preset quality among at least a portion of the images are used as face images for face attribute recognition.

[0031] In one possible implementation, after acquiring the face video data including at least one frame of image, the method further includes:

[0032] If the target object is detected to have placed an order, the recommendation of the recommended dishes to the target object will be stopped.

[0033] Secondly, an embodiment of the present invention provides a menu recommendation system, comprising:

[0034] Image acquisition module, face attribute recognition module, and processing module;

[0035] The image acquisition module is used to acquire face video data including at least one frame of image;

[0036] The face attribute recognition module is used to perform face attribute recognition on at least a portion of the images including the target object in the face video data, obtain the face expression data and facial pose data of the target object, and send the face expression data and facial pose data to the processing module;

[0037] The processing module is used to generate the current order preference of the target object based on the facial expression data and the facial posture data. When it is determined that there is historical order data of the target object, it generates the historical order preference of the target object based on the historical order data. According to preset rules, it obtains the dishes to be recommended based on the current order preference and the historical order preference, and displays the dishes to be recommended to the target object.

[0038] In one possible implementation, the processing module is used to:

[0039] Based on the facial expression data, determine the expression code used to represent the target expression of the target object;

[0040] Determine the target time corresponding to the expression code;

[0041] The facial orientation data corresponding to the target time is determined from the facial pose data;

[0042] Determine the heatmap region of interest corresponding to the facial orientation data, and the first set of dishes corresponding to the heatmap region of interest;

[0043] The first set of dishes is taken as the current order preference of the target object.

[0044] In one possible implementation, the processing module is used to:

[0045] Based on the historical order data, determine the target object's historical order records;

[0046] The second set of dishes and the number of times each dish was ordered were determined from the historical order records.

[0047] Based on the second set of dishes and the number of orders, the historical order preferences of the target object are generated.

[0048] In one possible implementation, the processing module is used to:

[0049] Based on the sales data of the dishes, the weights of the historical order preferences and the current order preferences are set respectively;

[0050] The weight of the dish to be recommended is determined based on the historical order preferences and the weight corresponding to the current order preferences.

[0051] The dish recommendation system also includes a display module, which is used to display the dishes to be recommended to the target object in descending order of weight.

[0052] The beneficial effects of this invention are as follows:

[0053] This invention provides a method and system for recommending dishes. First, facial video data including at least one frame is acquired. Then, facial attribute recognition is performed on at least a portion of the images including the target object in the facial video data. For example, when the at least one frame of the facial video data includes multiple frames, facial attribute recognition can be performed on each frame including the target object in the multiple frames, or on a portion of the images including the target object in the multiple frames, thereby obtaining facial expression data and facial posture data of the target object. Then, the current order preference of the target object is generated based on the facial expression data and facial posture data. Since facial expression data and facial posture data can change in real time, the current order preference determined based on the real-time changing facial expression data and facial posture data of the target object is closer to the actual ordering needs of the target object. In addition, when it is determined that there is historical order data of the target object, the historical order preference of the target object can be generated based on the historical order data. That is, when the target object is a regular user, the corresponding historical order preference can be determined based on its historical order data. Then, according to preset rules, dishes to be recommended are obtained based on the target user's historical and current order preferences, and these dishes are displayed to the target user. In other words, dishes can be recommended to the target user by combining their historical and current order preferences. In this way, the dish recommendations take into account both the target user's current and historical order preferences, thereby improving the effectiveness of the dish recommendations. Attached Figure Description

[0054] Figure 1 A flowchart illustrating a method for recommending dishes according to an embodiment of the present invention;

[0055] Figure 2 for Figure 1 Flowchart of the method for step S103;

[0056] Figure 3 for Figure 1 Flowchart of step S104;

[0057] Figure 4 for Figure 1 Flowchart of the method for step S105;

[0058] Figure 5 for Figure 1 Flowchart of the method for step S102;

[0059] Figure 6 In order to be in Figure 1 Flowchart of one of the methods following step S101;

[0060] Figure 7This is an overall flowchart of one embodiment of a dish recommendation method provided by the present invention;

[0061] Figure 8 This is a structural block diagram of a dish recommendation system provided in an embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Furthermore, the embodiments and features in the embodiments of the present invention can be combined with each other without conflict. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms "comprising" or "including," or similar terms as used in this invention, mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects.

[0064] It should be noted that the dimensions and shapes of the figures in the accompanying drawings do not reflect actual proportions and are intended only to illustrate the content of the invention. Furthermore, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0065] In existing technologies, dishes are often recommended to users based on their historical ordering preferences. However, the entire recommendation process does not take into account the user's real-time feelings and changes in taste, resulting in poor dish recommendation effectiveness.

[0066] Therefore, embodiments of the present invention provide a method and system for recommending dishes to improve the effectiveness of dish recommendations.

[0067] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for recommending dishes, including:

[0068] S101: Acquire face video data including at least one frame of image;

[0069] In specific implementation, the face video data can be acquired through an image acquisition unit including a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS). The face video data can include one frame or multiple frames, which can be arranged in chronological order. In practical applications, the images included in the face video data can be set according to actual usage requirements, and no limitation is made here.

[0070] S102: Perform facial attribute recognition on at least a portion of the images including the target object in the facial video data to obtain facial expression data and facial pose data of the target object;

[0071] In the specific implementation process, after acquiring the facial video data, machine learning methods can be used to perform face detection on the first frame image of the facial video data, extracting the face positions and their occupied areas in the image. Based on the face positions and the areas occupied by each face position, at least one object is determined. When there are multiple objects, the object closest to the image acquisition unit used to acquire the facial video data can be selected as the target object, or the object with the largest face area can be selected as the target object. When there is only one object, that object is directly selected as the target object. Of course, the target object can also be set according to the actual application needs, which is not limited here. After determining the target object, facial attribute recognition can be performed on at least a portion of the image in the facial video data that includes the target object to determine the facial expression data and facial pose data of the target object. The facial expression data is used to represent the expression of the target object, such as happiness, anger, etc., and the facial pose data is used to represent the facial pose of the target object, such as the face facing the upper left corner of the screen, or the face facing the lower right corner of the screen, etc.

[0072] S103: Generate the target object's current order preference based on the facial expression data and the facial posture data;

[0073] In practical implementation, the target audience's current order preferences can be generated based on the facial expression data and facial posture data. Changes in facial expression data during the ordering process reveal the target audience's emotional state, while changes in facial posture data reveal changes in the objects of attention. Therefore, the order preferences generated from these data more closely match the target audience's current order situation. Furthermore, during the ordering process, facial posture data can be used to determine the text location the target audience is focusing on on the current order page, and the duration of their attention on dishes at that location. For example, a longer attention duration corresponds to a higher weight for dishes at that location, more closely aligning with the target audience's current order needs. In this way, combining changes in facial expression and facial posture data during the ordering process to determine the target audience's current order preferences improves the efficiency of current dish recommendations.

[0074] S104: When it is determined that historical order data of the target object exists, the historical order preference of the target object is generated based on the historical order data;

[0075] In the specific implementation process, when it is determined that the target object has historical order data, the target object's historical order preferences can be generated based on the historical order data. In this way, when the target object is a long-time user or customer, its historical order preferences can be generated based on the target object's historical order data, and corresponding guidance can be provided for the target object's menu recommendations through the historical order preferences.

[0076] S105: According to preset rules, obtain the dishes to be recommended based on the historical order preferences and the current order preferences, and display the dishes to be recommended to the target object.

[0077] In practical implementation, recommended dishes can be obtained according to preset rules based on historical order preferences and current order preferences. Alternatively, the historical and current order preferences can be combined to determine the recommended dishes. The preset rules can be rules pre-set according to actual application needs. For example, the weights of historical and current order preferences can be pre-set, such as setting the weight of current order preferences to 0.8 and the weight of historical order preferences to 0.2. Other rules can also be set, which are not limited here. By combining the target audience's historical and current order preferences, dish recommendations can be made to the target audience. In this way, the dish recommendations take into account both the target audience's current and historical order preferences, thereby improving the effectiveness of the dish recommendations. Furthermore, since the current order preferences of the target user can be generated based on the facial expression data and facial posture data, when it is determined that there is historical order data for the target user, the historical order preferences of the target user can be generated based on the historical order data. Based on the current order preferences and historical order preferences, recommended dishes can be generated and recommended to the target user. The entire dish recommendation process can be carried out without the user's awareness, which improves the user's ordering experience while ensuring the effectiveness of the dish recommendation.

[0078] In embodiments of the present invention, such as Figure 2 As shown, step S103: generating the target object's current order preference based on the facial expression data and the facial pose data, including:

[0079] S201: Determine the expression code for representing the target expression of the target object based on the facial expression data;

[0080] S202: Determine the target time corresponding to the expression code;

[0081] S203: Determine the facial orientation data corresponding to the target time from the facial pose data;

[0082] S204: Determine the heatmap region of interest corresponding to the facial orientation data, and the first set of dishes corresponding to the heatmap region of interest;

[0083] S205: Take the first set of dishes as the current order preference of the target object.

[0084] In the specific implementation process, steps S201 to S205 are implemented as follows:

[0085] First, facial expression detection can be used to record real-time facial expression data of the target object during the browsing of the order page. Based on this data, an expression code representing the target object's expression can be determined. This can be achieved by first identifying the expression code for each frame containing the target object in at least a portion of the images, and then filtering out the expression codes corresponding to the target expression. A pre-defined correspondence between facial expressions and expression codes can be established. For example, an angry expression could be pre-coded as 1, a normal expression as 2, a happy expression as 3, and a surprised expression as 4. Of course, the correspondence can be set according to the specific application, and this is not limited here.

[0086] After determining the expression code representing the target expression of the target object based on the facial expression data, the target time of the expression code is determined. For example, based on the facial expression data of object A, the times when the expression code corresponding to object A's happiness is 3 are determined to be times a, b, and c. Then, facial pose detection is used to determine the facial orientation data at the target time from the facial pose data. Then, the attention heatmap corresponding to the facial orientation data and the first set of dishes corresponding to the attention heatmap are determined. Taking the above example, it is determined from the facial pose data that at time a, object A's face is facing the upper left corner of the current order page p, and at time b, object A's face is facing the lower right corner of the current order page p. Accordingly, the attention heatmap i corresponding to the upper left corner of the current order page p and the dish h1 corresponding to attention heatmap i are determined, and the dish h2 corresponding to attention heatmap j corresponding to the lower right corner of the current order page p are determined. In this way, the first set of dishes corresponding to all attention heatmaps is obtained, including dishes h1 and h2. This system allows for pre-setting of dishes in different areas of each ordering page, enabling the selection of dishes corresponding to popular areas on the ordering page. The first set of dishes is then used as the target user's current order preference. By analyzing the target user's real-time facial expression and posture data during the ordering process, the current order preference is generated. This generated preference more closely matches the target user's actual ordering needs, thus improving the efficiency of dish recommendations. Furthermore, the entire ordering process can be completed without the target user's awareness, ensuring a positive user experience. In addition, in practical applications, facial expressions can be quantified through expression encoding, improving the speed of facial expression recognition.

[0087] In embodiments of the present invention, such as Figure 3As shown, step S104: When it is determined that historical order data of the target object exists, the historical order preference of the target object is generated based on the historical order data, including:

[0088] S301: Based on the historical order data, determine the historical order records of the target object;

[0089] S302: Determine the second set of dishes and the number of times each dish was ordered from the historical order records;

[0090] S303: Generate the target object's historical order preferences based on the second set of dishes and the number of orders.

[0091] In the specific implementation process, steps S301 to S303 are implemented as follows:

[0092] First, based on the historical order data, the target user's historical order records are determined. Then, a second set of dishes and the number of times each dish was ordered are determined from these historical order records. For example, if user B is detected as a returning user, their historical order data can be obtained to determine the corresponding historical order records. From these records, the corresponding set of dishes is determined, including dishes s1, s2, and s3, where dish s1 was ordered 1 time, dish s2 was ordered 3 times, and dish s3 was ordered 5 times. Next, based on the second set of dishes and the number of orders, the target user's historical order preferences are generated. Using the above example, by statistically analyzing the set of dishes and the number of orders for each dish in user B's historical order records, it can be determined that user B has a historical preference for dish s3. By generating historical order preferences based on the target user's historical order data, menu recommendations can be made to the target user based on their historical order preferences, thereby improving the effectiveness of dish recommendations.

[0093] In embodiments of the present invention, such as Figure 4 As shown, step S105: According to preset rules, obtain the dishes to be recommended based on the historical order preferences and the current order preferences, and display the dishes to be recommended to the target object, including:

[0094] S401: Based on the sales data of the dishes, set the weights of the historical order preferences and the current order preferences respectively;

[0095] S402: Determine the weight of the dish to be recommended based on the historical order preferences and the weight corresponding to the current order preferences;

[0096] S403: Display the dishes to be recommended to the target object in descending order of weight.

[0097] In the specific implementation process, steps S401 to S403 are implemented as follows:

[0098] First, based on the sales performance of the dishes, the weights of the historical order preferences and the current order preferences are set respectively. For example, if statistics show that newly launched menu items are more popular and sell better, the weight of the current order preference can be set to 1, and the weight of the historical order preference can be set to 0.5. Of course, the weights of the historical order preferences and the current order preferences can also be adjusted according to the actual sales performance of the dishes, which is not limited here. Then, based on the weights corresponding to the historical order preferences and the current order preferences, the weights of the dishes to be recommended are determined. Taking the current order preference weight as 1 and the historical order preference weight as 0.5 as an example, if the first menu set of the current order preference includes dishes c1 and c2, and the second menu set of the historical order preference includes dishes c1 and c3, then the weight of dish c1 is 1.5, the weight of dish c2 is 1, and the weight of dish c3 is 0.5. After determining the weights of the dishes to be recommended, they are displayed to the target audience in descending order of weight. Using the example above, c1, c2, and c3 are displayed to the target audience in descending order of weight. Because the recommendation of dishes can be based on a combination of historical order preferences and current order preferences according to sales data, the effectiveness of dish recommendations is improved.

[0099] In embodiments of the present invention, such as Figure 5 As shown, step S102: Perform facial attribute recognition on at least a portion of the images including the target object in the facial video data to obtain facial expression data and facial pose data of the target object, including:

[0100] S501: Extract features from at least a portion of the images including the target object in the face video data using a deep neural network to determine the facial expression data of the target object;

[0101] S502: Extract at least one key point of the target object from each frame of the at least partial image;

[0102] S503: Determine the facial pose data of the target object based on the at least one key point.

[0103] In the specific implementation process, steps S501 to S503 are implemented as follows:

[0104] First, a deep neural network is used to extract features from at least a portion of the images containing the target object in the facial video data to determine the facial expression data of the target object. The deep neural network can extract and classify features from at least a portion of the images, outputting a feature vector of size 4, with four outputs representing anger, normalcy, happiness, and surprise, respectively. This allows for the recognition of the target object's facial expression data in each frame of the at least a portion of the images, thus determining the target object's real-time facial expression data. Furthermore, facial keypoint extraction can be used to extract at least one keypoint of the target object from each frame of the at least a portion of the images. For example, from image p1, the left eye, right eye, nose, left corner of the mouth, and right corner of the mouth of the target object can be extracted. Then, based on the at least one keypoint, the facial pose data of the target object is determined. This can be achieved using matrix operations to calculate the facial pose, representing it as deflection angles in the x, y, and z directions. These three deflection angles are linearly amplified and converted into point coordinates in two-dimensional space, thereby determining the position of the target object's face facing the single-screen display, and subsequently determining the corresponding dish.

[0105] In embodiments of the present invention, such as Figure 6 As shown, after obtaining face video data including at least one frame in step S101, the method further includes:

[0106] S601: Perform face detection on each frame of the at least some images, and select images whose face clarity and face occlusion meet the preset quality.

[0107] S602: Select the images that meet the preset quality from at least a portion of the images as the images to be identified for facial attributes.

[0108] In the specific implementation process, steps S601 to S602 are implemented as follows:

[0109] First, face detection is performed on each frame of the at least partial image. Images with face clarity and occlusion levels that meet a preset quality are selected. This can be achieved using face detection technology, extracting the face position and size from each frame (i.e., cutting out the face portion from each frame), followed by face clarity and occlusion detection. Images with face clarity and occlusion levels that meet the preset quality are then selected from the cut-out faces. Alternatively, blurry or occluded images can be filtered out, resulting in images with high face clarity and no occlusion. These are then used as images that meet the preset conditions. These images that meet the preset quality are then used for face attribute recognition. Since face attribute recognition is based on images that meet the preset quality from the at least partial image containing the target object, the accuracy of face attribute recognition is ensured. For example, face clarity is divided into five levels from low to high: 0, 1, 2, 3, and 4. Correspondingly, a face clarity level of 4 represents the clearest image. The degree of facial occlusion can be divided into two categories: occluded and unoccluded. During the facial attribute recognition process, when the facial clarity is lower than 4 or the face is occluded, the frame image is automatically skipped and facial attribute recognition is performed on the next frame. As long as the facial clarity is 4 and there is no occlusion, facial attribute recognition is performed on the frame image, thereby improving the speed of facial attribute recognition while ensuring the accuracy of facial attribute recognition.

[0110] In the specific implementation process, after step S602: selecting images that meet the preset quality from at least a portion of the images as images to be identified by facial attributes, machine learning can be used to perform facial identity recognition on the images to be identified by facial attributes. First, a convolutional neural network is used to extract facial image features from a preset image set including multiple frames. For example, feature vectors of size 512 are extracted and stored in a face database. When an image to be identified by facial attributes is input, feature extraction is performed on the input image to obtain the corresponding feature vector, and Euclidean distance is calculated with each feature vector in the face database to obtain the best match for the current face to be identified, thereby realizing the identity recognition of the corresponding object. For example, given an input image for facial attribute recognition, a convolutional neural network is first used to extract features. The extracted features are then compared with pre-stored facial features in a face database using distance calculations, such as cosine distance, with a threshold of 0.95. If the similarity of the facial features is higher than this threshold, the target object corresponding to the current facial features is determined to be a returning user; otherwise, it is determined to be a new user. When the target object is a returning user, the historical order data of that user can be searched in the order information database to determine their historical ordering preferences. When the target object is a new user, after the current ordering process is completed, the user's current order data can be stored in the order information database and bound to the user's facial features for more accurate subsequent dish recommendations.

[0111] like Figure 7 The diagram shows one overall flowchart of a dish recommendation method provided by an embodiment of the present invention. The specific implementation of each step has been described in detail above and will not be repeated here. Since face recognition and facial attribute recognition can be performed after face detection filters out face images that meet preset quality, and when the target is a returning user, their historical ordering data can be determined, and their current ordering preferences can be determined in real time based on their facial expression data and facial posture data, the method combines the target user's historical ordering preferences and current ordering preferences to recommend dishes to the target user. In this way, the dish recommendation takes into account both the target user's current and historical ordering preferences, thereby improving the effectiveness of the dish recommendation.

[0112] In this embodiment of the invention, after step S101: acquiring face video data including at least one frame of image, the method further includes:

[0113] If the target object is detected to have placed an order, the recommendation of the recommended dishes to the target object will be stopped.

[0114] In the specific implementation process, after acquiring facial video data including at least one frame, if it is detected that the target object has completed placing an order, the recommendation of dishes for the target object is stopped. This can be achieved by detecting that the target object has pressed the "Place Order" button on the order menu, or by detecting that the target object's browsing action has ended, for example, if no facial image of the target object is acquired within a preset time, then the recommendation of dishes for the target object is stopped. This avoids invalid dish recommendations and improves the efficiency of dish recommendation. Of course, the determination of whether the target object has completed placing an order can also be based on actual application needs, and is not limited here.

[0115] In the specific implementation process, the images in the facial video data are processed by algorithms such as face detection, face recognition, and facial attribute recognition. These algorithms can exchange face bounding boxes, facial key points, and the output results of each attribute recognition algorithm. A preset face structure can be used to record and input the results of each algorithm, thereby enabling communication between them and improving the speed of menu recommendation. Furthermore, because the algorithms communicate with each other during the menu recommendation process, the computational load and parameter count are low, allowing the menu recommendation method in this embodiment to be applied to edge intelligent devices. Even for edge intelligent devices with low computing power, menu recommendations can still be performed quickly.

[0116] Based on the same inventive concept, such as Figure 8 As shown, this embodiment of the invention also provides a menu recommendation system, including:

[0117] Image acquisition module 10, face attribute recognition module 20, and processing module 30;

[0118] The image acquisition module 10 is used to acquire face video data including at least one frame of image;

[0119] The face attribute recognition module 20 is used to perform face attribute recognition on at least a portion of the images including the target object in the face video data, obtain the face expression data and facial pose data of the target object, and send the face expression data and facial pose data to the processing module;

[0120] The processing module 30 is used to generate the current order preference of the target object based on the facial expression data and the facial posture data. When it is determined that there is historical order data of the target object, it generates the historical order preference of the target object based on the historical order data. According to preset rules, it obtains the dishes to be recommended based on the current order preference and the historical order preference, and displays the dishes to be recommended to the target object.

[0121] In this embodiment of the invention, the processing module 30 is used for:

[0122] Based on the facial expression data, determine the expression code used to represent the target expression of the target object;

[0123] Determine the target time corresponding to the expression code;

[0124] The facial orientation data corresponding to the target time is determined from the facial pose data;

[0125] Determine the heatmap region of interest corresponding to the facial orientation data, and the first set of dishes corresponding to the heatmap region of interest;

[0126] The first set of dishes is taken as the current order preference of the target object.

[0127] In this embodiment of the invention, the processing module 30 is used for:

[0128] Based on the historical order data, determine the target object's historical order records;

[0129] The second set of dishes and the number of times each dish was ordered were determined from the historical order records.

[0130] Based on the second set of dishes and the number of orders, the historical order preferences of the target object are generated.

[0131] In this embodiment of the invention, the processing module 30 is used for:

[0132] Based on the sales data of the dishes, the weights of the historical order preferences and the current order preferences are set respectively;

[0133] The weight of the dish to be recommended is determined based on the historical order preferences and the weight corresponding to the current order preferences.

[0134] The menu recommendation system also includes a display module, which is used for:

[0135] The dishes to be recommended are displayed to the target object in descending order of weight.

[0136] In this embodiment of the invention, the face attribute recognition module 20 is used for:

[0137] Feature extraction is performed on at least a portion of the images including the target object in the facial video data using a deep neural network to determine the facial expression data of the target object;

[0138] Extract at least one key point of the target object from each frame of the at least partial image;

[0139] Based on the at least one key point, the facial pose data of the target object is determined.

[0140] In this embodiment of the invention, after the image acquisition module 10 acquires face video data including at least one frame of image, the device further includes a face detection module, which is used for:

[0141] Face detection is performed on each frame of the at least some images, and images whose face clarity and face occlusion meet the preset quality are selected.

[0142] Images that meet the preset quality among at least a portion of the images are used as images for face attribute recognition.

[0143] In this embodiment of the invention, after the image acquisition module 10 acquires face video data including at least one frame of image, the processing module 30 is further configured to:

[0144] If the target object is detected to have placed an order, the recommendation of the recommended dishes to the target object will be stopped.

[0145] In the specific implementation process, the processing module 30 can communicate with the image acquisition module 10, the face attribute recognition module 20, and the face detection module through the Transmission Control Protocol (TCP). The data can be encapsulated into a structure and transmitted between the server and the client using a socket tool, thereby ensuring the performance of the menu recommendation system.

[0146] Furthermore, the principle behind this dish recommendation system is similar to that of the dish recommendation method. Therefore, the implementation of this dish recommendation system can refer to the implementation of the aforementioned dish recommendation method, and the repetitive parts will not be repeated.

[0147] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0148] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for recommending dishes, characterized in that, include: Acquire facial video data including at least one frame of image; Facial attribute recognition is performed on at least a portion of the images including the target object in the facial video data to obtain facial expression data and facial pose data of the target object; The target object's current order preferences are generated based on the facial expression data and the facial posture data. When it is determined that historical order data of the target object exists, the historical order preference of the target object is generated based on the historical order data. According to preset rules, dishes to be recommended are obtained based on the historical order preferences and the current order preferences, and the dishes to be recommended are displayed to the target object; The step of generating the target object's current order preference based on the facial expression data and the facial pose data includes: Based on the facial expression data, determine the expression code used to represent the target expression of the target object; Determine the target time corresponding to the expression code; The facial orientation data corresponding to the target time is determined from the facial pose data; Determine the heatmap region of interest corresponding to the facial orientation data, and the first set of dishes corresponding to the heatmap region of interest; The first set of dishes is taken as the current order preference of the target object; The step of obtaining recommended dishes based on historical order preferences and current order preferences according to preset rules, and displaying the recommended dishes to the target object, includes: Based on the sales data of the dishes, the weights of the historical order preferences and the current order preferences are set respectively; The weight of the dish to be recommended is determined based on the historical order preferences and the weight corresponding to the current order preferences. The dishes to be recommended are displayed to the target object in descending order of weight.

2. The method as described in claim 1, characterized in that, When it is determined that historical order data for the target object exists, generating historical order preferences for the target object based on the historical order data includes: Based on the historical order data, determine the target object's historical order records; The second set of dishes and the number of times each dish was ordered were determined from the historical order records. Based on the second set of dishes and the number of orders, the historical order preferences of the target object are generated.

3. The method as described in claim 1, characterized in that, The step of performing facial attribute recognition on at least a portion of the images containing the target object in the facial video data to obtain facial expression data and facial pose data of the target object includes: Feature extraction is performed on at least a portion of the images including the target object in the facial video data using a deep neural network to determine the facial expression data of the target object; Extract at least one key point of the target object from each frame of the at least partial image; Based on the at least one key point, the facial pose data of the target object is determined.

4. The method according to any one of claims 1-3, characterized in that, After acquiring face video data including at least one frame, the method further includes: Face detection is performed on each frame of the at least some images, and images whose face clarity and face occlusion meet the preset quality are selected. Images that meet the preset quality among at least a portion of the images are used as images for face attribute recognition.

5. The method according to any one of claims 1-3, characterized in that, After acquiring face video data including at least one frame, the method further includes: If the target object is detected to have placed an order, the recommendation of the recommended dishes to the target object will be stopped.

6. A menu recommendation system, characterized in that, include: Image acquisition module, face attribute recognition module, and processing module; The image acquisition module is used to acquire face video data including at least one frame of image; The face attribute recognition module is used to perform face attribute recognition on at least a portion of the images including the target object in the face video data, obtain the face expression data and facial pose data of the target object, and send the face expression data and facial pose data to the processing module; The processing module is used to generate the current order preference of the target object based on the facial expression data and the facial posture data. When it is determined that there is historical order data of the target object, it generates the historical order preference of the target object based on the historical order data. According to preset rules, it obtains the dishes to be recommended based on the current order preference and the historical order preference, and displays the dishes to be recommended to the target object. The processing module is used for: Based on the facial expression data, determine the expression code used to represent the target expression of the target object; Determine the target time corresponding to the expression code; The facial orientation data corresponding to the target time is determined from the facial pose data; Determine the heatmap region of interest corresponding to the facial orientation data, and the first set of dishes corresponding to the heatmap region of interest; The first set of dishes is taken as the current order preference of the target object; The processing module is used for: Based on the sales data of the dishes, the weights of the historical order preferences and the current order preferences are set respectively; The weight of the dish to be recommended is determined based on the historical order preferences and the weight corresponding to the current order preferences. The menu recommendation system also includes a display module, which is used for: The dishes to be recommended are displayed to the target object in descending order of weight.

7. The system as described in claim 6, characterized in that, The processing module is used for: Based on the historical order data, determine the target object's historical order records; The second set of dishes and the number of times each dish was ordered were determined from the historical order records. Based on the second set of dishes and the number of orders, the historical order preferences of the target object are generated.

Images