Dish evaluation method, device and equipment and storage medium
The dish evaluation method uses image processing and modeling to infer user evaluations of dishes by analyzing dining behavior, addressing the lack of accurate evaluation methods in catering companies.
Patent Information
- Application Number
- CN202411569728.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Catering companies lack an effective method to accurately obtain users' evaluations of dishes, as current methods like on-site interviews and online platforms provide unreliable and insufficient data.
A dish evaluation method that involves acquiring multiple frames of user dining images, processing them through a character interaction model to determine dining action trajectories, and using a dish evaluation model to infer evaluation information based on dining behavior and historical data.
Enables catering companies to accurately obtain user evaluations of dishes by analyzing dining behavior, improving data authenticity and reliability.
Smart Images

Figure CN119091435A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a dish evaluation method, device, equipment and storage medium. Background Art
[0002] In the daily operation of catering enterprises, consumer satisfaction is very important, especially the degree of satisfaction with the dishes determines the probability of customers repurchasing. Currently, catering enterprises generally obtain data on consumer satisfaction through on-site interviews, online third-party platform evaluations, etc., and the authenticity and sample size of the data are difficult to meet business needs.
[0003] At present, there is still a lack of an effective method for catering companies to accurately obtain users' evaluations of dishes. Summary of the invention
[0004] In view of this, the purpose of this application is to provide a dish evaluation method, device, equipment and storage medium, which can effectively enable catering companies to accurately obtain users' evaluations of dishes.
[0005] In a first aspect, the present application provides a dish evaluation method, comprising: Acquire multiple frames of user dining images; Inputting multiple frames of the user dining images into a character interaction model to obtain the user's interaction action information output by the character interaction model; Based on the interactive action information, determining the dining action trajectory of the user; Determining the user's dining behavior information for the first dish based on the user's dining action trajectory; Based on the dining behavior information, evaluation information of the first dish is determined.
[0006] Optionally, the dining behavior information includes any one or more of the following: the number of times the first dish is picked up, the frequency of picking up the first dish, the number of times the first dish is scooped up, and the frequency of scooping up the first dish.
[0007] Optionally, determining the evaluation information of the first dish based on the dining behavior information includes: Performing weighted calculation based on the dining behavior information to obtain first weighted information; The first weighted information is input into a dish evaluation model to obtain evaluation information of the first dish output by the dish evaluation model; wherein the dish evaluation model is trained based on historical evaluation information and historical weighted information of the first dish, and the historical weighted information is obtained by weighted calculation of historical dining behavior information.
[0008] Optionally, acquiring multiple frames of user dining images includes: A plurality of frames of images of the user dining are acquired by a camera device, and the camera device is integrated into a lighting system.
[0009] Optionally, the method further includes: Determining remaining information of the first dish based on multiple frames of the user dining images; Based on the dining behavior information, determining the evaluation information of the first dish includes: Based on the dining behavior information and the remaining information of the dish, evaluation information of the dish is determined.
[0010] Optionally, the method further includes: Recognizing an identification object through a camera device to obtain chef identification information, wherein the identification object is set on the dinner plate on which the first dish is placed; The chef identification information is bound to the evaluation information of the first dish to determine the evaluation information of the chef.
[0011] Optionally, the interactive action information includes the interactive action between the user and the object, and the trajectory tracking information between two adjacent frames of the user dining images; Determining the dining action trajectory of the user based on the interactive action information includes: Performing normalization processing based on the trajectory tracking information to obtain normalized information between two adjacent frames of the user dining images; Inputting the normalized information into a trajectory consistency discrimination model to obtain a trajectory consistency discrimination result output by the trajectory consistency discrimination model, wherein the trajectory consistency discrimination result is used to indicate whether the action trajectories of the interactive action corresponding to two adjacent frames of the user dining images are consistent; The interactive actions with consistent action trajectories are merged to determine the dining action trajectory of the user.
[0012] In a second aspect, the present application provides a dish evaluation device, comprising: An acquisition unit, used for acquiring multiple frames of user dining images; An input-output unit, used for inputting a plurality of frames of the user dining images into a character interaction model, and obtaining the user's interaction action information output by the character interaction model; a determining unit, configured to determine a dining action trajectory of the user based on the interactive action information; The determining unit is further configured to determine the dining behavior information of the user with respect to the first dish based on the dining action trajectory of the user; The determining unit is further used to determine evaluation information of the first dish based on the dining behavior information.
[0013] In a third aspect, the present application provides a dish evaluation device, comprising: Memory for storing computer programs; A processor is used to execute the computer program stored in the memory to implement the steps of the dish evaluation method as described in the first aspect.
[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the dish evaluation method as described in the first aspect.
[0015] It can be seen that the present application provides a dish evaluation method, device, equipment and storage medium. In a dish evaluation method, multiple frames of user dining images are obtained; multiple frames of user dining images are input into a character interaction model to obtain the user's interactive action information output by the character interaction model; based on the interactive action information, the user's dining action trajectory is determined; based on the user's dining action trajectory, the user's dining behavior information for the first dish is determined; based on the dining behavior information, the evaluation information of the first dish is determined. The above method acquires the user's dining behavior information during the dining process by collecting and processing the user's dining images, thereby inferring the user's evaluation information on the dish, and enabling the catering company to accurately obtain the user's evaluation of the dish. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.
[0017] Figure 1 It is a flowchart of a dish evaluation method provided in an embodiment of the present application; Figure 2 is a schematic diagram of the installation position of a camera device provided in an embodiment of the present application; Figure 3 This is a flow chart of a character interaction model processing a user dining image provided by an embodiment of the present application Figure 4 It is a schematic diagram of a process for determining the remaining information provided by the present application; Figure 5 This is a schematic diagram of a dinner plate structure provided in an embodiment of the present application; Figure 6 is a schematic diagram of an identification object for identifying chef identification information provided by an embodiment of the present application; Figure 7is a flow chart of another dish evaluation method provided in an embodiment of the present application; Figure 8 It is a structural schematic diagram of a dish evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0019] To facilitate understanding of the technical solution provided by the present application, a dish evaluation method provided by the present application will be described below in conjunction with the accompanying drawings. Figure 1 , is a flow chart of a dish evaluation method provided in an embodiment of the present application, wherein the method includes S101-S105.
[0020] S101: Acquire multiple frames of user dining images.
[0021] In the embodiments of the present application, the specific method of acquiring multiple frames of user dining images is not limited. As a possible implementation method, acquiring multiple frames of user dining images includes: acquiring multiple frames of user dining images through a camera device, wherein the camera device is integrated into a lighting system.
[0022] It is understandable that if Figure 2 As shown, Figure 2 The figure is a schematic diagram of the installation position of a camera device provided in an embodiment of the present application. The camera device may include a camera. The present application may integrate the camera into the lighting system. Specifically, the camera and the light bulb may be integrated together to facilitate the collection of user dining images. The present application does not limit the acquisition time and frequency of obtaining multiple frames of user dining images. As an example, the present application may regularly collect photos within the dining table range from the beginning of the user's dining, collecting 3 to 5 photos per second. The present application does not collect videos to avoid infringing on user privacy.
[0023] The present application can collect the dining behavior of the dining guests on the dining table in real time through the camera device, thereby obtaining multiple frames of user dining images. In the embodiment of the present application, the time interval between two adjacent frames of user dining images in the multiple frames of user dining images is less than the preset time threshold, that is, the multiple frames of user dining images in the present application can be regarded as continuous, so as to have a better recognition effect. The size of the preset time threshold can be set according to actual needs, and the present application does not specifically limit it.
[0024] S102: Input multiple frames of user dining images into a character interaction model to obtain user interaction action information output by the character interaction model.
[0025] In the embodiment of the present application, after obtaining multiple frames of user dining images, the multiple frames of user dining images can be input into the character interaction model to obtain the user's interactive action information output by the character interaction model. The character interaction model can be obtained by training the machine learning model using user dining training data.
[0026] The present application does not limit the specific content of the interactive action information. As an example, the interactive action information may include the category of the interactive action between the user and the object, the trajectory tracking information between two adjacent frames of the user's dining images, etc. It is understandable that the interactive action between the user and the object may include, for example, a gripping action with chopsticks or a scooping action with a spoon, etc., and the trajectory tracking information may include the trajectory of the interactive frame corresponding to the two adjacent frames of the user's dining images.
[0027] This application does not limit the specific process of the character interaction model processing the user's dining image. As an example, Figure 3 As shown, Figure 3 A flow chart of a character interaction model for processing user dining images provided in an embodiment of the present application; the character interaction model uses a two-stage method to process the image to be detected, and the image to be detected is the user dining image recorded above. In the first stage, the hand / chopstick / spoon target detection module is used to locate the regional position frame of the human hand and the object (chopsticks, spoon). In the second stage, the intersection of the target object regional frame of the human hand and the spoon or chopsticks is calculated and screened, and the paired frames with the intersection of the hand and spoon, and the hand and chopsticks greater than 0 are retained. Then, the regional feature candidate area composed of the object frame of the hand and chopsticks or spoon is classified and identified twice by the action classification module to determine whether it is a hand-held chopstick gripping action and / or a hand-held spoon scooping action, and to identify whether the trajectory of the interactive action information between two adjacent frames of the user dining image is consistent, thereby determining the interactive action information. Figure 3 In the example, the hand / chopstick / spoon target detection module is a general target detection module, such as a target detection algorithm (YOLO, You Only Look Once) series algorithm, and the image feature extraction module is the backbone network of the general target detection series.
[0028] The present application does not limit the specific training method of the character interaction model. As an example, in an embodiment of the present application, the initial character interaction model can be trained by using the dining images of the user to be trained and the labels corresponding to the dining images of the user to be trained to obtain a character interaction model. The labels corresponding to the dining images of the user to be trained are used to identify the interactive action information corresponding to the dining images of the user to be trained.
[0029] S103: Determine the user's dining action trajectory based on the interactive action information.
[0030] The present application does not limit the specific method of determining the dining action trajectory of the user based on the interactive action information. As an example, the interactive action information includes the interactive action between the user and the object and the trajectory tracking information between two adjacent frames of the user's dining images. S103 determines the dining action trajectory of the user based on the interactive action information, including the following steps A1-A3: A1: Perform normalization processing based on the trajectory tracking information to obtain normalized information between two adjacent frames of user dining images.
[0031] It is understandable that the trajectory tracking information may include the trajectory of the interaction frame corresponding to two adjacent frames of the user dining image, such as the intersection-over-intersection ratio of the area detection frame composed of the person and the object (chopsticks or spoon) in the previous and next two frames: ; The intersection of the hand detection frame in the previous frame and the chopstick detection frame in the next frame: ; The intersection of the chopsticks in the previous frame and the hand detection frame in the next frame: ; The intersection-over-union ratio of the chopstick detection frame in the previous frame and the chopstick detection frame in the next frame: ; The intersection-over-union ratio of the hand detection frame in the previous frame and the hand detection frame in the next frame: ; The difference in the tilt angle of the chopsticks between the previous frame and the next frame: ; The distance difference between the chopstick head in the previous frame and the next frame: .
[0032] In the embodiment of the present application, the above data can be normalized to the interval [-1, 1], because the intersection and union ratio is between [0, 1], so here only needs to and Normalize, specifically, the following formula can be used to normalize and Normalize: , in is a constant, for example, ; It represents the normalized difference in the tilt angle of the chopsticks between the previous frame and the next frame; , in ; It represents the distance difference between the chopstick head in the previous frame and the next frame after normalization. represents the width of the user's dining image, and h represents the length of the user's dining image.
[0033] A2: Inputting the normalized information into a trajectory consistency discrimination model to obtain a trajectory consistency discrimination result output by the trajectory consistency discrimination model.
[0034] The normalized information can be By inputting the normalized information into the trajectory consistency discrimination model, the trajectory consistency discrimination result output by the trajectory consistency discrimination model can be obtained. The trajectory consistency discrimination result is used to indicate whether the action trajectories of the interactive actions corresponding to two adjacent frames of user dining images are consistent. It can be understood that the trajectory consistency discrimination result can indicate whether the action trajectories of the previous and next two frames of images belong to the same action of the same trajectory.
[0035] The present application does not limit the specific structure of the trajectory consistency discrimination model. As an example, the present application may adopt a multi-layer perceptron model, and the multi-layer perceptron model may have 4 layers. As an example, the present application may input the normalized information Z into the multi-layer perceptron model P=f(Z), and when the probability P>0.6, it is determined that the trajectory is consistent.
[0036] The present application does not limit the specific training method of the trajectory consistency discrimination model. As an example, in the embodiment of the present application, the training can be performed using data samples with consistent trajectories and inconsistent trajectories, and the number of data samples with consistent trajectories and inconsistent trajectories is 1:4. The data sample includes training normalization information and a label corresponding to the training normalization information. The label corresponding to the training normalization information is used to indicate whether the training normalization information is trajectory consistent or trajectory inconsistent.
[0037] A3: Merge the interactive actions with consistent action trajectories to determine the dining action trajectory of the user.
[0038] In the embodiment of the present application, the interactive actions of the same trajectory can be merged, that is, the actions of the same trajectory can be connected in series to determine the user's dining action trajectory.
[0039] It is understandable that since traditional motion tracking requires a sufficiently high frame rate and also requires the features of the tracked object to be relatively obvious, the present application can extract data from the character interaction actions by tracking the overall interaction actions of people and objects (people and spoons, people and chopsticks), and train a trajectory consistency discrimination model based on these data, so as to achieve tracking of user interaction actions under low frame rates and unclear features.
[0040] S104: Based on the user's dining action trajectory, determine the user's dining behavior information for the first dish.
[0041] In the embodiment of the present application, after obtaining the user's dining action trajectory, the user's dining behavior information for the first dish can be counted, and the dining behavior information includes any one or more of the following: the number of times the first dish is picked up, the frequency of picking up the first dish, the number of times the first dish is scooped up, and the frequency of scooping up the first dish. It can be understood that the first dish can be any dish, such as soup dishes, meat dishes, vegetable dishes, etc.
[0042] S105: Determine evaluation information of the first dish based on the dining behavior information.
[0043] In the embodiment of the present application, after the dining behavior information is determined, the evaluation information of the first dish can be determined based on the dining behavior information. It can be understood that the dining behavior information can reflect the user's acceptance of the dish, so that the satisfaction with the dish can be inferred, that is, the evaluation information is determined.
[0044] The above method acquires the dining behavior information of users during the dining process by collecting and processing the images of users for analysis, thereby inferring the evaluation information of users on dishes, and enabling catering enterprises to accurately obtain the evaluation of users on dishes.
[0045] The present application does not limit the specific manner of determining the evaluation information of the first dish based on the dining behavior information. As an example, S105 determines the evaluation information of the first dish based on the dining behavior information, including the following steps B1-B2: B1: Perform weighted calculation based on the dining behavior information to obtain first weighted information.
[0046] In the embodiment of the present application, the dining behavior information can be as follows: exists in the form of , The time period for picking up with chopsticks or scooping with a spoon. for The number of times food is picked up with chopsticks or scooped up with a spoon within a time period.
[0047] In the embodiment of the present application, the dining behavior information may be merged first to reduce the amount of data to be processed. As an example, the present application may merge the dining behavior information according to the following formula: ; in, It can represent the time length T of the merger, which is usually 60S. The i-th data representing the combined dining behavior information may indicate the number of times the food is picked up with chopsticks or scooped up with a spoon within a time interval of T. Indicates the number of times the food is picked up with chopsticks or scooped up with a spoon within a period of x.
[0048] Through the above processing of the input data, the result of the processing is ,in Both include the number of times the food is picked up with chopsticks or scooped up with a spoon within the time interval T.
[0049] This application can be used to Perform weighted calculation. Specifically, the following weighted formula can be used: ; In the formula, represents the weighted result of the Kth data, T represents the length of the merged time, and K is any natural number. Indicates the merged dining behavior information The Kth data in is the weighting parameter, generally It is understood that this application may The number and frequency weighting is performed at every T time interval.
[0050] It can be understood that, through the above weighted formula, the weighted first weighted information can be obtained: .
[0051] B2: Input the first weighted information into a dish evaluation model to obtain evaluation information of the first dish output by the dish evaluation model.
[0052] The dish evaluation model of the present application is trained based on the historical evaluation information and historical weighted information of the first dish, wherein the historical weighted information is obtained by weighted calculation of the historical dining behavior information. Thus, the dish evaluation model can obtain the evaluation information of the first dish corresponding to the first weighted information by inputting the first weighted information.
[0053] This application does not limit the specific processing process of the dish evaluation model on the first weighted information. As an example, this application assumes that the historical evaluation information of the first dish obeys a normal distribution. Then, this application can use the historical weighted information in the historical database to calculate the mean of the normal distribution. and variance , we can get the density function of the normal distribution , where i is the weighted value of the number and frequency of a dish to be rated, and the final rating result is .
[0054] Through the above-mentioned dish evaluation model, the evaluation information of the first dish can be obtained quickly and accurately, so that catering companies can accurately obtain users' evaluations of dishes.
[0055] As a possible implementation manner, a dish evaluation method provided in the present application also includes the following steps: determining the remaining information of the first dish based on the user dining image.
[0056] This application does not limit the specific method of determining the remaining information of the first dish based on the user's dining image, such as Figure 4 As shown, Figure 4 A schematic diagram of a process for determining remaining information provided in the present application. It is understandable that the remaining information in the present application can be divided into a finished state and an unfinished state according to the state of the dish.
[0057] This application collects the first dish finished pictures in the historical data and marks them into two categories: finished and unfinished, and uses these data to build an image feature retrieval library. After obtaining the user's dining image, the user's dining image will be feature extracted and then searched with the image feature retrieval library for matching judgment.
[0058] Then S105 determines the evaluation information of the first dish based on the dining behavior information, including: Based on the dining behavior information and the remaining information of the first dish, evaluation information of the first dish is determined.
[0059] In an embodiment of the present application, the evaluation information of a dish can be displayed in the form of a score. The present application does not limit the specific method of determining the evaluation information of a dish based on the dining behavior information and the remaining information of the dish. As an example, after determining the evaluation information of a dish based on the dining behavior information, the present application adds or subtracts points in combination with the remaining information of the dish. For example, the first dish whose remaining information is that the dish is in the finished state can be given a corresponding score increase, and the first dish whose remaining information is that the dish is in the unfinished state can be given a corresponding score subtraction.
[0060] This application can more accurately infer the user's evaluation information on the dishes through dining behavior information and the remaining information of the dishes, allowing catering companies to more accurately obtain the user's evaluation of the dishes.
[0061] As a possible implementation method, the present application provides a dish evaluation method, which also includes the following method: C1: Recognize an identification object through a camera device to obtain chef identification information, wherein the identification object is set on the plate on which the first dish is placed.
[0062] Figure 5 A schematic diagram of a dinner plate structure provided in an embodiment of the present application. The present application also provides a special dinner plate, with two sides or any side of the dinner plate ( Figure 5 (not shown) is provided with a placement area where an identification object for identifying chef identification information can be placed. Figure 6A schematic diagram of a marking object for identifying chef identification information provided in an embodiment of the present application. When the chef serves a dish, the marking object is placed in the placement area of the plate. The present application does not limit the specific shape of the marking object, and its shape can be changed according to actual needs. The camera device provided in the present application can obtain the chef identification information by identifying the marking object. It can be understood that the chef identification information is used to indicate which chef prepared the dish.
[0063] C2: Bind the chef identification information with the evaluation information of the first dish to determine the evaluation information of the chef.
[0064] In the embodiment of the present application, the chef identification information is bound to the evaluation information of the first dish, so that the evaluation information of the dish can be bound to the chef to obtain the evaluation information of the chef.
[0065] This application enables catering companies to accurately obtain evaluation information about chefs by binding the evaluation information of the first dish with the chef.
[0066] As a possible implementation method, before the present application inputs multiple frames of user dining images into the character interaction model, it may also include the following steps: filtering the multiple frames of user dining images to obtain multiple frames of filtered user dining images, and inputting the multiple frames of filtered user dining images into the character interaction model.
[0067] It is understandable that the present application can screen user dining images and select user dining images with more analysis value for subsequent processing, which can reduce the amount of data processing and improve data processing efficiency. The present application does not limit the specific method of screening. As an example, the present application can decide whether to upload based on the amount of information contained in each photo, and clean up user dining images with duplicate information.
[0068] Combine the following Figure 7 , introduce this application. Figure 7 A flow chart of another dish evaluation method provided for an embodiment of the present application. In the present application, multiple frames of continuous user dining images can be input, and artificial intelligence technology can be used to construct a character interaction model based on the input continuous images, using target detection, image recognition, and background detection modules to detect and identify customer gripping actions. At the same time, the relationship between each action can be used to connect actions with the same trajectory in series, so as to accurately judge the number or frequency of the customer's gripping / scooping. The dishes can be evaluated based on the number and frequency of the customer's gripping with chopsticks and scooping with a spoon, and combined with the final remaining number of dishes, to obtain evaluation information of the dishes. In addition, the present application also uses image recognition to identify which chef the dish belongs to, and binds the dish to the chef.
[0069] The above method collects and processes user dining images to obtain information about the user's dining behavior during the dining process, thereby inferring the user's evaluation information on the dishes, allowing catering companies to accurately obtain the user's evaluation of the dishes. In addition, the present application binds the evaluation information of the dishes with the chef, allowing catering companies to accurately obtain the evaluation information of the chef.
[0070] The following is an introduction to a dish evaluation device provided in an embodiment of the present application. The device described below and the dish evaluation method described above can be referred to in correspondence with each other. Figure 8 , Figure 8 This is a schematic diagram of the structure of a dish evaluation device provided in an embodiment of the present application, wherein the device includes an acquisition unit 801, an input-output unit 802, and a determination unit 803.
[0071] An acquisition unit 801 is used to acquire multiple frames of user dining images; An input-output unit 802 is used to input multiple frames of the user dining images into a character interaction model to obtain the user's interaction action information output by the character interaction model; A determination unit 803, configured to determine a dining action trajectory of the user based on the interaction action information; The determining unit 803 is further configured to determine the dining behavior information of the user with respect to the first dish based on the dining action trajectory of the user; The determining unit 803 is further configured to determine evaluation information of the first dish based on the dining behavior information.
[0072] As a possible implementation manner, the dining behavior information includes any one or more of the following: the number of times the first dish is picked up, the frequency of picking up the first dish, the number of times the first dish is scooped up, and the frequency of scooping up the first dish.
[0073] As a possible implementation manner, the determining unit 803 specifically includes: A weighted calculation subunit, configured to perform weighted calculation based on the dining behavior information to obtain first weighted information; An input-output subunit is used to input the first weighted information into a dish evaluation model to obtain evaluation information of the first dish output by the dish evaluation model; wherein the dish evaluation model is trained based on historical evaluation information and historical weighted information of the first dish, and the historical weighted information is obtained by weighted calculation of historical dining behavior information.
[0074] As a possible implementation manner, the acquisition unit 801 is specifically configured to acquire multiple frames of the user dining images through a camera device, and the camera device is integrated in a lighting system.
[0075] As a possible implementation manner, the device further includes: The determining unit 803 is further configured to determine remaining information of the first dish based on multiple frames of the user dining image; The determining unit 803 is specifically configured to determine the evaluation information of the dish based on the dining behavior information and the remaining information of the dish.
[0076] As a possible implementation manner, the device further includes: The acquisition unit 801 is further configured to identify an identification object through a camera device to acquire chef identification information, wherein the identification object is set on the dinner plate on which the first dish is placed.
[0077] A binding unit is used to bind the chef identification information with the evaluation information of the first dish to determine the evaluation information of the chef.
[0078] As a possible implementation, the interactive action information includes the interactive action between the user and the object, and the trajectory tracking information between two adjacent frames of the user dining images; The determining unit 803 specifically includes: A normalization subunit, configured to perform normalization processing based on the trajectory tracking information to obtain normalized information between two adjacent frames of the user dining images; An input-output subunit, used for inputting the normalized information into a trajectory consistency discrimination model to obtain a trajectory consistency discrimination result output by the trajectory consistency discrimination model, wherein the trajectory consistency discrimination result is used to indicate whether the action trajectories of the interactive action corresponding to two adjacent frames of the user dining images are consistent; The merging subunit is used to merge the interactive actions with consistent action trajectories to determine the dining action trajectory of the user.
[0079] It should be noted that the dish evaluation device provided in the embodiment of the present application has the technical effects of any of the above embodiments, and the embodiment of the present application will not be described in detail here.
[0080] The present application also provides a dish evaluation device, which may include a memory and a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the dish evaluation method as described in the above embodiment is implemented.
[0081] It should be noted that the dish evaluation device provided in the embodiment of the present application has the technical effects of any of the above embodiments, and the embodiment of the present application will not be described in detail here.
[0082] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0083] It should be noted that a computer-readable storage medium provided in an embodiment of the present application has the technical effects of any of the above embodiments, and the embodiments of the present application are not described in detail here.
[0084] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0085] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0086] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.
Claims
1. A dish evaluation method, characterized in that: include: Acquire multiple frames of user dining images; Inputting multiple frames of the user dining images into a character interaction model to obtain the user's interaction action information output by the character interaction model; Based on the interactive action information, determining the dining action trajectory of the user; Determining the user's dining behavior information for the first dish based on the user's dining action trajectory; Based on the dining behavior information, evaluation information of the first dish is determined.
2. The method according to claim 1, characterized in that The dining behavior information includes any one or more of the following: the number of times the first dish is picked up, the frequency of picking up the first dish, the number of times the first dish is scooped up, and the frequency of scooping up the first dish.
3. The method according to claim 2, characterized in that The determining, based on the dining behavior information, evaluation information of the first dish includes: Performing weighted calculation based on the dining behavior information to obtain first weighted information; The first weighted information is input into a dish evaluation model to obtain evaluation information of the first dish output by the dish evaluation model; wherein the dish evaluation model is trained based on historical evaluation information and historical weighted information of the first dish, and the historical weighted information is obtained by weighted calculation of historical dining behavior information.
4. The method according to claim 1, characterized in that: The step of acquiring multiple frames of user dining images includes: A plurality of frames of images of the user dining are acquired by a camera device, and the camera device is integrated into a lighting system.
5. The method according to claim 1, further comprising: Determining remaining information of the first dish based on multiple frames of the user dining images; Based on the dining behavior information, determining the evaluation information of the first dish includes: Based on the dining behavior information and the remaining information of the first dish, evaluation information of the first dish is determined.
6. The method according to claim 1, characterized in that The method further comprises: Recognizing an identification object through a camera device to obtain chef identification information, wherein the identification object is set on the dinner plate on which the first dish is placed; The chef identification information is bound to the evaluation information of the first dish to determine the evaluation information of the chef.
7. The method according to claim 1, characterized in that The interactive action information includes the interactive action between the user and the object, and the trajectory tracking information between two adjacent frames of the user dining images; Determining the dining action trajectory of the user based on the interactive action information includes: Performing normalization processing based on the trajectory tracking information to obtain normalized information between two adjacent frames of the user dining images; Inputting the normalized information into a trajectory consistency discrimination model to obtain a trajectory consistency discrimination result output by the trajectory consistency discrimination model, wherein the trajectory consistency discrimination result is used to indicate whether the action trajectories of the interactive action corresponding to two adjacent frames of the user dining images are consistent; The interactive actions with consistent action trajectories are merged to determine the dining action trajectory of the user.
8. A dish evaluation device, characterized in that: include: An acquisition unit, used for acquiring multiple frames of user dining images; An input-output unit, used for inputting a plurality of frames of the user dining images into a character interaction model, and obtaining the user's interaction action information output by the character interaction model; a determining unit, configured to determine a dining action trajectory of the user based on the interactive action information; The determining unit is further configured to determine the dining behavior information of the user with respect to the first dish based on the dining action trajectory of the user; The determining unit is further used to determine evaluation information of the first dish based on the dining behavior information.
9. A dish evaluation device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program stored in the memory to implement the steps of the dish evaluation method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the steps of the dish evaluation method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Intelligent dish evaluation system
CN109784787A
Customer satisfaction evaluation method and system based on surveillance camera shooting
CN109858949A
Method and system for individually adjusting rotating speed of intelligent rotating table
CN110363685A
Evaluation information generation method, device and equipment and storage medium
CN111738758A
Dining satisfaction evaluation method and device, and storage medium
CN112016350A
