Intelligent canteen dining management method and system based on artificial intelligence algorithm

Abstract

The application provides a kind of wisdom canteen dining management method and system based on artificial intelligence algorithm, first, the real-time video data set of canteen dining area is acquired, the real-time video data set includes multi-view continuous acquisition and image frame unit with timestamp mark, then dining behavior feature extraction is carried out on video data, get meal path feature and meal contact feature, simultaneously acquire meal payment state feature, analyze dining behavior feature set and payment state feature using pre-training model, generate evaluation results containing meal flow smoothness and meal selection preference, determine operation management adjustment direction according to evaluation results, such as window service efficiency adjustment and meal supply optimization direction, generate management execution instruction containing service window scheduling instruction and meal supplement instruction, and send to canteen management system to trigger operation adjustment, realize canteen intelligent management.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart canteens, in particular to a smart canteen dining management method and system based on an artificial intelligence algorithm. BACKGROUND

[0002] In the traditional canteen operation and management mode, the management of canteen dining is mainly dependent on manual observation and experience judgment. It is difficult for canteen staff to comprehensively and timely grasp the behavior dynamics of diners in the canteen, such as the dining path of diners, the interactive action with food, and other information. At the same time, in terms of obtaining the payment state of diners' food, although the existing payment system can record payment information, it lacks effective correlation analysis with dining behavior.

[0003] The existing canteen management mode cannot systematically extract and analyze the characteristics of dining behavior, and it is difficult to accurately assess the smoothness of the dining process and the food selection preference of diners. This leads to a lack of scientific basis for canteen operation and management, and it is impossible to timely adjust the window service efficiency and food supply according to the actual needs and behavior patterns of diners, which easily leads to problems such as low window service efficiency, unreasonable food supply, and affects the dining experience of diners and the overall operation efficiency of the canteen. SUMMARY

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present application, the embodiments of the present application provide a smart canteen dining management method based on an artificial intelligence algorithm, which comprises:

[0005] acquiring a set of real-time video data covering the dining area of the canteen, the set of real-time video data comprising a plurality of image frame units with timestamp labels collected continuously from multiple perspectives;

[0006] performing dining behavior feature extraction processing on the set of real-time video data to obtain a set of dining behavior features of diners in each image frame unit, the set of dining behavior features comprising dining path features and food contact features, the dining path features reflecting the movement trajectory information of diners from the entrance to the dining area through the dining window, and the food contact features reflecting the interactive action information of diners with food containers;

[0007] acquiring food payment state features of each diner, calling a pre-trained dining behavior analysis model to analyze the dining behavior features of the diners and the food payment state features, and generating a dining behavior evaluation result, the dining behavior evaluation result comprising a description of the smoothness of the dining process and a description of the food selection preference;

[0008] Determine the canteen operation management adjustment direction according to the dining behavior evaluation result, and the operation management adjustment direction includes a window service efficiency adjustment direction and a meal supply optimization direction.

[0009] Generate management execution instructions including service window scheduling instructions and meal replenishment instructions based on the operation management adjustment direction, and send the management execution instructions to the canteen management system to trigger operation adjustment operations.

[0010] For example, the meal payment state characteristics of each diner are obtained, including:

[0011] Obtain user interaction data, including user identity verification information and current meal image information;

[0012] Intelligently identify the current meal image information to generate a meal identification result containing meal type identification;

[0013] Based on the user identity verification information and the meal identification result, a preset consumption rule set is called to perform fee accounting processing to generate pending settlement fee information;

[0014] According to the user identity verification information and the pending settlement fee information, trigger the payment system to perform fee deduction operations to generate payment state information;

[0015] When detecting system communication interruption, store the user identity verification information, meal identification result and pending settlement fee information to the local storage module, and synchronize to the background server to complete the fee supplement deduction operation after communication recovery.

[0016] The user interaction data includes user identity verification information and current meal image information, including:

[0017] Obtain user face image data through a biological feature acquisition device, perform feature extraction processing on the face image data, and generate a user face feature vector;

[0018] Match the user face feature vector with a pre-stored canteen white list feature library, and generate user identity verification information after successful matching;

[0019] Or obtain user displayed payment QR code image data through a two-dimensional code scanning device, and generate user identity verification information by decoding the payment QR code image data;

[0020] Obtain scene image data containing multiple meal containers through a meal shooting device, perform image enhancement processing on the scene image data, and generate current meal image information without background interference.

[0021] The intelligent recognition processing is performed on the current dish image information to generate a dish recognition result containing a dish type identifier, and the dish recognition result contains a dish type identifier.

[0022] Target detection processing is performed on the current dish image information to locate the region boundary of each serving dish container in the image to generate a plurality of dish candidate regions.

[0023] Feature extraction processing is performed on each dish candidate region to extract color distribution features, texture density features and contour shape features in the region to generate a dish candidate feature set.

[0024] The dish candidate feature set is input into a pre-trained dish recognition model, and the dish recognition model contains a feature matching module and a dynamic learning module.

[0025] The feature matching module is used to perform similarity calculation processing on the dish candidate feature set and the standard dish feature library built in the model to filter out the standard dish feature with the highest similarity.

[0026] The dynamic learning module is used to perform incremental learning processing on the candidate features that are not matched to the standard dish features, update the standard dish feature library and generate a new dish type identifier.

[0027] Based on the matching result and the incremental learning result, a dish recognition result containing a plurality of dish type identifiers is generated.

[0028] In another aspect, the embodiment of the present application also provides a smart canteen dining management system based on an artificial intelligence algorithm, which comprises a processor and a machine-readable storage medium, the machine-readable storage medium is connected with the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to realize the above-mentioned method.

[0029] Based on the above aspects, the embodiment of the present application can comprehensively and accurately obtain the meal taking path features and dish contact features of the diners by acquiring a real-time video data set covering the canteen dining area and performing dining behavior feature extraction processing on the image frame units therein, and combining with the dish payment state features, using a pre-trained dining behavior analysis model to generate a dining behavior evaluation result containing a meal taking flow smoothness description and a dish selection preference description, determining the canteen operation management adjustment direction based on the evaluation result, and generating management execution instructions containing service window scheduling instructions and dish replenishment instructions, and sending them to the canteen management system to trigger operation adjustment operations, so as to realize intelligent and accurate management of the canteen dining situation, dynamically adjust the canteen operation management according to the actual behavior and needs of the diners, effectively improve the window service efficiency, optimize the dish supply, and improve the dining experience of the diners and the overall operation benefit of the canteen. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is the execution flow schematic diagram of the smart dining hall dining management method based on an artificial intelligence algorithm provided by an embodiment of the application.

[0031] Figure 2 is a schematic diagram of exemplary hardware and software components of the smart dining hall dining management system based on an artificial intelligence algorithm provided by an embodiment of the application. DETAILED DESCRIPTION

[0032] The application will be described in detail below with reference to the accompanying drawings of the specification, Figure 1 is a flow schematic diagram of the smart dining hall dining management method based on an artificial intelligence algorithm provided by an embodiment of the application, which will be described in detail below.

[0033] Step S110: Obtain a real-time video data set covering the dining area of the dining hall, which contains image frame units with timestamp labels collected continuously from multiple perspectives.

[0034] In this embodiment, in the dining hall scene, in order to comprehensively capture the dynamic information in the dining area, multiple monitoring cameras can be deployed at different key positions in the dining hall. The monitoring cameras are distributed at the entrance of the dining hall, various meal pickup windows, and different corners of the dining area, etc., to monitor the dining hall from different perspectives. Each camera continuously collects images at a set frame rate, and adds accurate timestamp labels to each image frame during the collection process. Thus, the image frames with timestamps from multiple cameras constitute a real-time video data set. For example, the camera at the entrance is mainly used to record the situation of diners entering the dining hall, the camera at the meal pickup window can clearly capture the interaction between the diners and the meal containers and the queuing situation, and the camera in the dining area can observe the state of the diners dining. The images collected by the cameras from different perspectives complement each other, so that the dining situation of the entire dining hall can be accurately recorded.

[0035] Step S120: Perform dining behavior feature extraction processing on the real-time video data set to obtain a dining behavior feature set of diners in each image frame unit, the dining behavior feature set including meal pickup path features and meal product contact features, the meal pickup path features reflecting the movement trajectory information of diners from the entrance to the meal pickup window to the dining area, and the meal product contact features reflecting the interaction action information between diners and meal containers.

[0036] In order to analyze the dining behavior of diners in depth, key behavior features need to be extracted from the real-time video data set. This step mainly involves the extraction of meal pickup path features and meal product contact features. The specific processing steps are as follows:

[0037] Step S121: frame processing is performed on the set of real-time video data to obtain a plurality of single-frame image units with continuous time sequence relationship.

[0038] Since the real-time video data exists in the form of continuous video stream, in order to facilitate subsequent processing, it needs to be split into single image frames. Frame processing is to separate each frame of image in the video stream according to the frame rate of the video. Each image frame has a timestamp added previously, and these image frames are arranged in time sequence to form a set of single-frame image units with continuous time sequence relationship. For example, if the frame rate of the video is several frames per second, then the video per second will be split into a corresponding number of single-frame images, and these images are arranged in time sequence in turn.

[0039] Step S122: target detection processing is performed on the single-frame image units to identify the position information of the diners and the position information of the food containers, the position information including two-dimensional plane coordinates.

[0040] After obtaining the single-frame image units, the diners and food containers in each image need to be positioned. Advanced target detection algorithms are used to identify the positions of the diners and food containers in the image, and their position information is represented as two-dimensional plane coordinates. In order to establish a two-dimensional plane coordinate system, a fixed corner of the canteen can be selected as the origin, and the horizontal and vertical directions as the coordinate axes. The target detection algorithm will identify the bounding box of the diners and food containers in the image, and then calculate the center position of the bounding box and convert it to two-dimensional plane coordinates. For example, for a diner, the target detection algorithm will frame the body contour in the image, and then calculate the center position of the contour and map it to the two-dimensional plane coordinate system to obtain the corresponding coordinate value. The same method is applied to the food containers to determine their positions in the two-dimensional plane.

[0041] Step S123: trajectory tracking processing is performed on the single-frame image units with continuous time sequence to generate a take-out path feature according to the position information changes of the diners in adjacent single-frame image units, the take-out path feature including a path length parameter and a set of path inflection point coordinates.

[0042] In order to generate the take-out path feature, trajectory tracking needs to be performed on the continuous single-frame image units. By comparing the position coordinate changes of the same diner in adjacent single-frame images, the moving direction and distance of the diner can be determined. The specific trajectory tracking process is as follows:

[0043] Step S1231: the position information of the same diner in adjacent single-frame image units is extracted, and the coordinate difference value of the position information is calculated as a displacement vector.

[0044] In the continuous single-frame images, for each diner, the position coordinates in the adjacent two frames can be extracted. Then, by calculating the difference between the two coordinates, a displacement vector is obtained. The displacement vector represents the moving direction and distance of the diner between the adjacent two frames. For example, the position coordinates of the diner in the t-th frame are (x_t, y_t), and the position coordinates of the diner in the t+1-th frame are (x_{t+1}, y_{t+1}), then the displacement vector can be calculated by (x_{t+1}-x_t, y_{t+1}-y_t).

[0045] Step S1232: Calculate the moving speed parameter between adjacent single-frame image units according to the displacement vector, the moving speed parameter being the ratio of the displacement vector length to the time interval.

[0046] After obtaining the displacement vector, the moving speed of the diner between the adjacent two frames needs to be calculated. The moving speed parameter can be calculated by the ratio of the length of the displacement vector to the time interval between the adjacent two frames. The length of the displacement vector represents the actual distance moved by the diner between the adjacent two frames, and the time interval reflects the time taken for the movement. By the above calculation method, the moving speed of the diner between each adjacent two frames can be obtained, and the speed parameter can reflect the fast or slow movement of the diner in different time periods.

[0047] Step S1233: Construct a moving trajectory analysis model of the diner based on the displacement vector and the moving speed parameter.

[0048] The displacement vector and the moving speed parameter are taken as inputs to construct a moving trajectory analysis model. The model can be realized by machine learning or deep learning method, which can predict and analyze the moving trajectory of the diner according to the displacement and speed information of the diner at different times. The model will consider the historical movement of the diner and the current movement state to more accurately describe the moving trajectory of the diner.

[0049] Step S1234: Calculate the path length parameter of the diner in the continuous single-frame image units by the moving trajectory analysis model, the path length parameter being the cumulative value of the lengths of the adjacent displacement vectors.

[0050] Using the moving trajectory analysis model, the path length of the diner from entering the canteen to reaching the dining area can be calculated. The path length parameter is obtained by sequentially accumulating the lengths of the displacement vectors between the adjacent two frames. The length of each displacement vector represents the actual distance moved by the diner between the adjacent two frames, and the total path length of the diner in the entire meal process can be obtained by accumulating these distances.

[0051] Step S1235: Detecting the turning points of the trajectory coordinate points output by the moving trajectory analysis model, identifying the coordinate points where the trajectory direction changes significantly as the path turning point coordinate set, and determining that the significant change is through the displacement vector direction angle exceeding a preset critical value.

[0052] After obtaining the moving trajectory of the diner, it is necessary to identify the turning points in the trajectory. The turning point refers to the position where the moving direction of the diner changes significantly. By analyzing the trajectory coordinate points output by the moving trajectory analysis model, the direction angle between adjacent displacement vectors is calculated. When the angle exceeds the preset critical value, it is considered that the point is the turning point of the trajectory. The coordinates of these turning points are recorded, and the path turning point coordinate set is obtained. The above turning point information can reflect the obstacles or decision points that the diner may encounter during the meal.

[0053] Step S124: Action recognition processing is performed on the single-frame image unit to analyze the spatial position relationship change between the diner and the food container, and food contact features are generated, including contact start time point, contact end time point and contact action type description.

[0054] In order to generate food contact features, action recognition processing needs to be performed on the single-frame image unit to analyze the spatial position relationship change between the diner and the food container. The specific steps are as follows:

[0055] Step S1241: Region segmentation processing is performed on the single-frame image unit to divide the diner's body region and the food container region.

[0056] In the single-frame image, the image segmentation algorithm is used to divide the diner's body and the food container into different regions. The image segmentation algorithm can be based on a deep learning model, which classifies the image at the pixel level to separate the diner's body and the food container from the background and form independent regions, thereby more accurately analyzing the spatial position relationship between the diner and the food container.

[0057] Step S1242: Calculate the spatial overlap area parameter of the diner's body region and the food container region.

[0058] After dividing the diner's body region and the food container region, the spatial overlap area between the diner's body region and the food container region is calculated. The spatial overlap area parameter can reflect the proximity between the diner and the food container. By comparing the spatial overlap area in different frames of images, it can be determined whether the diner has contacted the food container.

[0059] Step S1243: According to the trend of the spatial overlap area parameter changing with time, determine the contact start time point and the contact end time point of the diner and the food container.

[0060] By observing the trend of the spatial overlap area parameter over time, the contact start time point and the contact end time point of the diner and the food container can be determined. When the spatial overlap area changes from none to a certain threshold, it is considered that the contact starts, and the corresponding timestamp is the contact start time point. When the spatial overlap area changes from yes to none and is lower than a certain threshold, it is considered that the contact ends, and the corresponding timestamp is the contact end time point.

[0061] Step S1244: Perform action pattern recognition processing on the single-frame image units in the contact period to extract the motion direction sequence and the force change sequence of the diner's limbs.

[0062] After determining the contact period, perform action pattern recognition processing on the single-frame image units in the contact period. Use the action recognition algorithm to extract the motion direction sequence and the force change sequence of the diner's limbs. The motion direction sequence can reflect the moving direction of the diner's limbs during the contact process, and the force change sequence can reflect the force degree change of the diner when contacting the food container.

[0063] Step S1245: Match the motion direction sequence and the force change sequence with a preset action type library to generate a contact action type description, which includes a taking action, a putting action, and a searching action.

[0064] Match the extracted motion direction sequence and force change sequence with a preset action type library. The action type library contains various common contact action types of diners and food containers, such as taking action, putting action, and searching action, etc. By comparing the motion direction sequence and the force change sequence with the characteristics of each action in the action type library, the contact action type of the diner can be determined, and the corresponding contact action type description can be generated.

[0065] Step S125: Input the taking path feature and the food contact feature into a feature association module for timestamp alignment processing to obtain a dining behavior feature set with time synchronization.

[0066] After obtaining the taking path feature and the food contact feature, they need to be processed for timestamp alignment. The feature association module will match and align the two features according to the timestamp information contained in the taking path feature and the food contact feature. Through the above method, the taking path feature and the food contact feature can be ensured to be synchronized in time, forming a dining behavior feature set with time synchronization. The above dining behavior feature set can more comprehensively reflect the dining behavior of the diner.

[0067] Step S130: Obtain the meal payment state feature of each diner, call the pre-trained dining behavior analysis model to analyze the dining behavior feature set of the diner and the meal payment state feature, generate a dining behavior evaluation result, and the dining behavior evaluation result contains a smoothness description of the meal taking process and a meal selection preference description.

[0068] In order to comprehensively evaluate the dining behavior of the diner, it is necessary to obtain the meal payment state feature and analyze it in combination with the dining behavior feature set. The following are the specific processing steps:

[0069] Step S131: Obtain the meal payment state feature of each diner.

[0070] In order to obtain the meal payment state feature of the diner, the following steps are needed:

[0071] Step S1311: Obtain user interaction data, which includes user identity verification information and current meal image information.

[0072] User interaction data is obtained in various ways. On the one hand, through a biometric feature acquisition device such as a facial recognition camera, facial image data of the user is obtained. Feature extraction processing is performed on these facial image data to generate a user facial feature vector. Then the user facial feature vector is matched with the pre-stored canteen white list feature library, and user identity verification information is generated after successful matching. On the other hand, a two-dimensional code scanning device can also be used to obtain payment two-dimensional code image data displayed by the user, and the two-dimensional code image data is decoded to generate user identity verification information. At the same time, a meal taking device is used to obtain scene image data containing multiple meal containers, and image enhancement processing is performed on the scene image data to remove background interference and generate current meal image information.

[0073] Step S1312: Perform intelligent recognition processing on the current meal image information to generate a meal recognition result containing a meal type identifier.

[0074] The intelligent recognition processing of the current meal image information is as follows: first, the target detection processing is performed on the current meal image information, the region boundary of each meal container in the image is located, and a plurality of meal candidate regions are generated. Then, the feature extraction processing is performed on each meal candidate region, the color distribution feature, the texture density feature and the contour shape feature in the region are extracted, and a meal candidate feature set is generated. The meal candidate feature set is input into a pre-trained meal recognition model, and the meal recognition model includes a feature matching module and a dynamic learning module. The feature matching module performs similarity calculation processing on the meal candidate feature set and the standard meal feature library built in the model, and selects the standard meal feature with the highest similarity. For the candidate feature that does not match the standard meal feature, the dynamic learning module performs incremental learning processing, updates the standard meal feature library and generates a new meal type identifier. Finally, based on the matching result and the incremental learning result, a meal recognition result containing a plurality of meal type identifiers is generated.

[0075] Step S1313: Based on the user identity verification information and the meal recognition result, a preset consumption rule set is called to perform fee accounting processing, and settlement fee information is generated.

[0076] According to the user identity verification information and the meal recognition result, a preset consumption rule set is called to perform fee accounting. The preset consumption rule set contains the price information of different meals and various discount rules. By matching the meal type identifier in the meal recognition result with the price information in the consumption rule set, the total cost of the selected meal of the diner can be calculated, and settlement fee information is generated.

[0077] Step S1314: According to the user identity verification information and the settlement fee information, a payment system is triggered to perform a fee deduction operation, and payment state information is generated.

[0078] According to the user identity verification information and the settlement fee information, the payment system of the cafeteria is triggered to perform a fee deduction operation. The payment system will deduct the corresponding fee according to the payment method of the user (such as bank card, electronic wallet, etc.), and generate payment state information, such as payment success, payment failure, payment delay, etc.

[0079] Step S1315: When it is detected that the system communication is interrupted, the user identity verification information, meal recognition result and settlement fee information are stored in the local storage module, and after the communication is restored, the fee is supplemented and deducted by synchronizing to the background server.

[0080] In order to deal with the situation of system communication interruption, when the communication interruption is detected, the user identity authentication information, the meal identification result and the to-be-settled fee information can be stored into the local storage module. The local storage module can be a local server or a storage device of the canteen. When the communication is restored, the information is synchronized to the background server, the fee supplement operation is completed, and the accuracy of the fee settlement is ensured.

[0081] Step S132: calling the pre-trained dining behavior analysis model to analyze the dining behavior features of the diner and the meal payment state features, and generating a dining behavior evaluation result.

[0082] The dining behavior features of the diner and the meal payment state features are input into the pre-trained dining behavior analysis model for dining behavior analysis. The specific steps are as follows:

[0083] Step S1321: inputting the dining behavior features of the diner and the meal payment state features into the feature fusion layer of the dining behavior analysis model for spatio-temporal correlation modeling processing, and generating a fusion feature vector with behavior continuity constraint.

[0084] In the feature fusion layer of the dining behavior analysis model, the dining behavior features and the meal payment state features are processed by spatio-temporal correlation modeling. The feature fusion layer considers the correlation between these features in time and space, and fuses them together through certain algorithms to generate a fusion feature vector with behavior continuity constraint. The fusion feature vector contains the comprehensive information of the dining behavior and payment state of the diner.

[0085] Step S1322: performing take meal process analysis processing on the fusion feature vector through the flow evaluation module of the dining behavior analysis model, extracting the correlation index of the path length parameter and the moving speed parameter, and generating a take meal process smoothness description, the take meal process smoothness description containing the congestion node coordinates and the congestion duration.

[0086] The flow evaluation module of the dining behavior analysis model performs take meal process analysis processing on the fusion feature vector. The specific process is as follows:

[0087] Step S13221: extracting the path length parameter set, the moving speed parameter set and the associated meal payment state feature set from the fusion feature vector, the meal payment state feature set containing the payment timestamp, the payment success flag and the payment delay duration.

[0088] In the fusion feature vector, a path length parameter set, a moving speed parameter set, and a related meal payment state feature set are extracted. The path length parameter set records the path length information of the diners in the meal taking process, the moving speed parameter set reflects the moving speed of the diners, and the meal payment state feature set contains important information related to payment, such as payment timestamp, payment success flag, and payment delay duration.

[0089] Step S13222: Calculate the Pearson correlation coefficient of the path length parameter set and the moving speed parameter set to obtain a correlation index of path length and moving speed.

[0090] By calculating the Pearson correlation coefficient of the path length parameter set and the moving speed parameter set, the correlation between the path length and the moving speed is measured. The Pearson correlation coefficient can reflect the degree of linear correlation between the two parameters, and through the Pearson correlation coefficient, it can be judged whether there is some association between the meal taking path length and the moving speed of the diners.

[0091] Step S13223: Identify an abnormal node in the meal taking process according to the correlation index, the abnormal node being a path inflection point coordinate point with a correlation index lower than a preset threshold.

[0092] According to the calculated correlation index, an abnormal node in the meal taking process is identified. When the correlation index is lower than the preset threshold, the corresponding path inflection point coordinate point is considered to be an abnormal node. These abnormal nodes may indicate that the diners encountered obstacles or abnormal situations during the meal taking process.

[0093] Step S13224: Extract the timestamp information corresponding to the abnormal node, and combine the payment timestamp in the meal payment state feature set to determine whether the abnormal node is located in the payment window associated area.

[0094] The timestamp information corresponding to the abnormal node is extracted, and the payment timestamp in the meal payment state feature set is combined to determine whether the abnormal node is located in the payment window associated area. By comparing the timestamp and the position information of the abnormal node, it can be determined whether the abnormal situation is related to the payment process.

[0095] Step S13225: If the abnormal node is located in the payment window associated area, the congestion duration is determined in combination with the moving speed parameter change trend of the front and rear single frame image units and the payment delay duration; if the abnormal node is not in the payment related area, the congestion duration is determined only based on the moving speed parameter change trend.

[0096] When the abnormal node is located in the payment window associated area, in addition to considering the change trend of the mobile speed parameter of the diners in the front and rear single frame image units, the payment delay time length is also combined to determine the congestion duration. The change trend of the mobile speed parameter can reflect the moving state of the diners near the abnormal node. If the speed is significantly reduced or even stopped, it means that there may be congestion. The payment delay time length provides additional information from the perspective of the payment link. The payment delay may be an important factor leading to congestion. Specifically, the time length from the time point when the mobile speed starts to decrease before the abnormal node to the time point when the mobile speed returns to normal after the abnormal node is the preliminary congestion time reference. Then, the payment delay time length is adjusted. If the payment delay time length is long, it means that the congestion may be caused by the unsmooth payment process. Therefore, the payment delay time should be appropriately included in the calculation of the congestion duration. For example, if the payment delay is caused by payment system failure or diner operation error, etc., these factors need to be considered when determining the congestion duration, and the payment delay time period should be reasonably included to obtain the accurate congestion duration.

[0097] If the abnormal node is not located in the payment related area, then only the change trend of the mobile speed parameter is used to determine the congestion duration. The moving speed of the diners in the single frame image before and after the abnormal node is observed. When the speed is significantly lower than the normal moving speed, the congestion duration is the time period from the beginning of the speed reduction to the end of the speed returning to the normal level. The normal moving speed can be determined by analyzing a large amount of moving speed data of diners in non-congestion conditions to determine a reasonable range. For example, in normal conditions, the moving speed of diners in the canteen is relatively stable. When the moving speed suddenly decreases significantly near a certain abnormal node and returns to normal after a period of time, the time from the speed reduction to the speed returning to normal is the congestion duration of the abnormal node.

[0098] Step S13226: The path inflection point coordinates of the abnormal node are taken as the flow congestion node coordinates. The congestion duration, the flow congestion node coordinates and the associated payment state characteristics are recorded to generate the pickup flow smoothness description.

[0099] The coordinates of the path inflection points of the identified abnormal nodes are determined as the coordinates of the process congestion nodes. These coordinates accurately identify the specific locations where congestion occurs in the meal pickup process. The calculated congestion duration is then associated with the process congestion node coordinates and the associated payment status features. Associated payment status features include payment timestamps, payment success flags, and payment delay duration. This information can help further analyze the causes and impacts of congestion. For example, if the payment delay corresponding to a process congestion node coordinate is long, and the payment success flag indicates that the payment process is not smooth, then it can be inferred that the congestion may be related to the payment link. By integrating this information, a detailed description of the smoothness of the meal pickup process is generated. This description of the smoothness of the meal pickup process includes not only the location information of the congestion node, but also the duration of the congestion and payment status information that may be related to the congestion.

[0100] Step S1323: The preference recognition module of the dining behavior analysis model performs meal selection analysis on the fused feature vector, counts the occurrence frequencies of different contact action types on each meal container, and generates a meal selection preference description. The meal selection preference description includes meal categories whose contact frequency meets a preset threshold and meal categories whose contact frequency does not reach the preset threshold.

[0101] The preference recognition module of the dining behavior analysis model performs a meal selection analysis on the fused feature vector to understand the diners' preferences for different meals. The specific process is as follows:

[0102] Step S13231: extracting a contact action type description set, a corresponding food container identification set, and an associated food payment status feature set from the fused feature vector, wherein the food payment status feature set includes a payment success flag and payment timestamp information.

[0103] The contact action type description set, the corresponding food container identification set, and the associated food payment status feature set are extracted from the fused feature vector. The contact action type description set records the various contact actions between the diner and the food container, such as taking food, putting food down, and searching for food. The food container identification set clarifies the identity of each food container so that it can correspond to a specific food type. The payment success flag in the food payment status feature set can determine whether the diner ultimately selected the food, and the payment timestamp information can reflect the time when the diner made the selection.

[0104] Step S13232: Establish an association mapping table of contact action type, food container identifier and food payment status characteristics, wherein the association mapping table records the contact action type, occurrence timestamp and payment success flag corresponding to each food container identifier.

[0105] In order to better analyze the meal selection preference of the diners, an association mapping table is established. The mapping table associates the contact action type, meal container identifier and meal payment state feature. For each meal container identifier, the corresponding contact action type, the timestamp of the contact action occurrence and the payment success flag are recorded. Through the association mapping table, it can be determined that each meal container is contacted by the diners in what action at different time points, and whether it is finally selected for payment.

[0106] Step S13233: Time window division processing is performed on the association mapping table, and the number of occurrences of the contact action type of each meal container identifier when the payment success flag is true in each time window is counted.

[0107] The association mapping table is windowed according to time, and the entire dining time period is divided into a plurality of time windows. In each time window, the number of occurrences of different contact action types under the condition that the payment success flag is true for each meal container identifier is counted. For example, in a certain time window, for a specific meal container identifier, the number of occurrences of the meal taking action, the meal placing action and the searching action is counted, and thus the change of the contact behavior of the diners to different meals in different time periods can be analyzed.

[0108] Step S13234: The proportion of the number of occurrences of each contact action type on the same meal container identifier is calculated, and a contact action frequency distribution matrix is generated.

[0109] After counting the number of occurrences of each contact action type corresponding to each meal container identifier in each time window, the proportion of the number of occurrences of each contact action type on the same meal container identifier is calculated. These proportion data are arranged into a matrix, i.e. a contact action frequency distribution matrix. The contact action frequency distribution matrix can intuitively show the frequency distribution of different meal containers in different contact action types. For example, the rows of the contact action frequency distribution matrix can represent different meal container identifiers, the columns can represent different contact action types, and the elements in the contact action frequency distribution matrix are the proportion of the number of occurrences of the corresponding contact action type.

[0110] Step S13235: According to the contact action frequency distribution matrix, the contact action types with a number of occurrences that meet a preset threshold are screened, and the meal categories with a contact frequency that meets a preset threshold and the meal categories with a contact frequency that does not meet a preset threshold are determined in combination with the corresponding meal container identifiers to generate a meal selection preference description.

[0111] According to the contact action frequency distribution matrix, the contact action type whose occurrence frequency proportion meets the preset threshold is screened out. The preset threshold is set according to a large amount of historical data and actual experience, and is used to judge whether the contact of the diners on the meal is frequent enough to indicate the preference. For the meal container identifiers whose contact action type occurrence frequency proportion meets the preset threshold, the corresponding meal category is determined as the meal category whose contact frequency meets the preset threshold, and these meals are usually the preferred meals of the diners. For the meal container identifiers whose contact action type occurrence frequency proportion does not meet the preset threshold, the corresponding meal category is the meal category whose contact frequency does not meet the preset threshold, and it may be the meal that the diners are not interested in. The two types of meal category information are integrated together to generate the meal selection preference description.

[0112] Step S1324: The meal taking flow smoothness description and the meal selection preference description are integrated for information processing to generate a dining behavior evaluation result containing timestamp alignment.

[0113] In order to obtain a comprehensive dining behavior evaluation result, the meal taking flow smoothness description and the meal selection preference description need to be integrated for information. First, ensure that the timestamp information in the two descriptions is aligned, so as to ensure the consistency and accuracy of the data. For example, the timestamp of a congestion node recorded in the meal taking flow smoothness description corresponds to the contact of the diners on the meal at the same time point in the meal selection preference description. Then, the flow congestion node coordinates, congestion duration and associated payment state characteristics in the meal taking flow smoothness description are combined with the meal category whose contact frequency meets the preset threshold and the meal category whose contact frequency does not meet the preset threshold in the meal selection preference description. Through the above method, an evaluation result containing complete dining behavior information is generated, which can comprehensively reflect the behavior characteristics of the diners in the meal taking process and the preference tendency of the diners on the meal.

[0114] Step S140: According to the dining behavior evaluation result, the adjustment direction of the canteen operation management is determined, and the operation management adjustment direction includes the window service efficiency adjustment direction and the meal supply optimization direction.

[0115] According to the generated dining behavior evaluation result, the adjustment direction of the canteen operation management can be determined, which is specifically divided into the window service efficiency adjustment direction and the meal supply optimization direction, and the following is the detailed determination process:

[0116] Step S141: Analyzing the meal taking flow smoothness description in the dining behavior evaluation result, the flow congestion node coordinates and the congestion duration are extracted.

[0117] The smoothness of the pickup process in the evaluation result of the dining behavior is analyzed, and key information, i.e., the pickup process congestion node coordinates and the congestion duration, is extracted. The pickup process congestion node coordinates accurately indicate the specific location of the congestion in the pickup process, and the congestion duration reflects the severity and duration of the congestion. Thus, it can be determined which locations in the pickup process are prone to congestion and the impact range and time length of the congestion.

[0118] Step S142: Determine the corresponding pickup window position according to the process congestion node coordinates, and calculate the service pressure parameter of the pickup window according to the congestion duration.

[0119] According to the extracted process congestion node coordinates, the corresponding pickup window position is determined. Since the process congestion node is usually closely related to the service condition of the pickup window, the pickup window that may have service pressure can be found through the corresponding relationship of the coordinates. Then, the service pressure parameter of the pickup window is calculated according to the congestion duration. The longer the congestion duration, the lower the service efficiency of the pickup window and the greater the service pressure. The specific calculation method can consider the comprehensive relationship of the congestion duration and the number of diners served by the pickup window in a certain period of time, for example, taking the ratio of the congestion duration to the number of diners as a basic service pressure parameter measurement index, and adjusting it in combination with other related factors to obtain a more accurate service pressure parameter.

[0120] Step S143: Compare the service pressure parameter with a preset service pressure threshold to generate a window service efficiency adjustment direction, which includes the window position where the service pressure parameter meets the preset threshold and the window position where the service pressure parameter does not meet the preset threshold.

[0121] The calculated pickup window service pressure parameter is compared with a preset service pressure threshold. The preset service pressure threshold is set according to the operation experience and actual situation of the cafeteria, and is used to judge whether the service pressure of the pickup window is within a reasonable range. If the service pressure parameter meets the preset threshold, it means that the service pressure of the pickup window is large and may need to be adjusted to improve the service efficiency; if the service pressure parameter does not meet the preset threshold, it means that the service pressure of the pickup window is relatively small. According to the comparison result, the window position where the service pressure parameter meets the preset threshold and the window position where the service pressure parameter does not meet the preset threshold are recorded respectively to generate a window service efficiency adjustment direction.

[0122] Step S1431: Normalize the service pressure parameter to obtain a standardized pressure value in the range of 0-1.

[0123] In order to facilitate the comparison and analysis of the service pressure parameters of different pickup windows, the calculated service pressure parameters are normalized. Normalization can unify the service pressure parameters with different value ranges to the range of 0-1, so that the service pressures of each pickup window are comparable. The specific normalization method can select a suitable algorithm according to the value characteristics of the service pressure parameters, for example, using the maximum-minimum normalization method to map the minimum value of the service pressure parameter to 0 and the maximum value to 1, and other values are scaled in proportion, thereby obtaining the standardized pressure value.

[0124] Step S1432: determining the pickup window with the standardized pressure value meeting the preset pressure threshold as the window needing to adjust the service personnel, and the pickup window with the standardized pressure value not reaching the preset pressure threshold as the window needing to adjust the service personnel.

[0125] According to the standardized pressure value obtained after normalization, the preset pressure threshold is compared. The preset pressure threshold is a reasonable standard verified by practice, which is used to judge whether the service pressure of the pickup window is too high or too low. When the standardized pressure value meets the preset pressure threshold, it means that the service pressure of the pickup window is in the range that needs to be adjusted, and it is determined as the window needing to adjust the service personnel; when the standardized pressure value does not reach the preset pressure threshold, it can also be judged whether the service personnel needs to be adjusted according to the actual situation, because the too low service pressure may mean that the personnel configuration is excessive.

[0126] Step S1433: counting the number and position distribution information of the window needing to adjust the service personnel.

[0127] The pickup windows determined to need to adjust the service personnel are counted, and the number and position distribution information of these windows are recorded. The number information can help the canteen manager understand the scale that needs to be adjusted, and the position distribution information can provide specific direction for personnel deployment. For example, it is counted that how many windows needing to adjust the service personnel are in different areas of the canteen, so that personnel can be reassigned targetedly.

[0128] Step S1434: generating the personnel adjustment position in the service window scheduling instruction according to the position distribution information of the window needing to adjust the service personnel, the personnel adjustment position being a coordinate set of the window needing to adjust the service personnel.

[0129] According to the position distribution information of the window needing to adjust the service personnel, the personnel adjustment position in the service window scheduling instruction is generated. The coordinate information of these windows needing to adjust the service personnel is arranged into a set, and the set is the personnel adjustment position. By specifying the personnel adjustment position, the canteen management system can accurately guide the deployment of service personnel, ensure that personnel can arrive at the window needing to be adjusted in time, and improve service efficiency.

[0130] Step S1435: integrating the personnel adjustment positions to generate a window service efficiency adjustment direction.

[0131] The generated personnel adjustment position information is integrated to form a complete window service efficiency adjustment direction. The window service efficiency adjustment direction contains the position information of all windows that need to adjust the service personnel and the corresponding adjustment requirements. Through the above integration, the canteen manager can determine which windows need to increase or decrease service personnel and how to allocate personnel, thereby optimizing the window service efficiency.

[0132] Step S144: parsing the meal selection preference description in the dining behavior evaluation result to extract meal categories with contact frequency meeting a preset threshold and meal categories with contact frequency not meeting the preset threshold.

[0133] The meal selection preference description in the dining behavior evaluation result is parsed to extract meal categories with contact frequency meeting a preset threshold and meal categories with contact frequency not meeting the preset threshold. These two types of meal category information reflect the degree of preference of diners for different meals. Meal categories with contact frequency meeting the preset threshold are usually preferred by diners, while meal categories with contact frequency not meeting the preset threshold may not be popular with diners.

[0134] Step S145: statistics the consumption speed parameter of the meal categories with contact frequency meeting the preset threshold in the historical time period, and statistics the remaining amount parameter of the meal categories with contact frequency not meeting the preset threshold in the historical time period.

[0135] The relevant parameters of meal categories with contact frequency meeting the preset threshold and meal categories with contact frequency not meeting the preset threshold in the historical time period are respectively calculated. For meal categories with contact frequency meeting the preset threshold, the consumption speed parameter, i.e. the consumption quantity of the meal per unit time, is calculated. The consumption speed parameter can reflect the popularity and market demand of the meal. For meal categories with contact frequency not meeting the preset threshold, the remaining amount parameter, i.e. the quantity of the meal remaining after a certain period of time, is calculated. The remaining amount parameter can reflect whether the supply of the meal is excessive.

[0136] Step S146: generating a meal supply optimization direction according to the consumption speed parameter and the remaining amount parameter, the meal supply optimization direction containing meal categories that need to adjust the supply amount.

[0137] According to the consumption speed parameter and the remaining amount parameter calculated, a meal supply optimization direction is generated. The specific generation process is as follows:

[0138] Step S1461: Calculate the difference between the consumption speed parameter of the meal category with the contact frequency meeting the preset threshold and the historical average consumption speed, to obtain a consumption speed deviation value.

[0139] For the consumption speed parameter of the meal category with the contact frequency meeting the preset threshold, compare it with the average consumption speed of the meal in the historical time period, and calculate the difference between the two to obtain a consumption speed deviation value. The consumption speed deviation value can reflect the difference between the current consumption speed of the meal and the historical average. If the deviation value is positive, it means that the current consumption speed is higher than the historical average, and the supply amount may need to be increased; if the deviation value is negative, it means that the current consumption speed is lower than the historical average, and the supply amount needs to be appropriately reduced.

[0140] Step S1462: Calculate the difference between the remaining amount parameter of the meal category with the contact frequency not meeting the preset threshold and the historical average remaining amount, to obtain a remaining amount deviation value.

[0141] For the remaining amount parameter of the meal category with the contact frequency not meeting the preset threshold, compare it with the average remaining amount of the meal in the historical time period, and calculate the difference between the two to obtain a remaining amount deviation value. The remaining amount deviation value can reflect the difference between the current remaining amount of the meal and the historical average. If the deviation value is positive, it means that the current remaining amount is higher than the historical average, and the supply amount may need to be reduced; if the deviation value is negative, it means that the current remaining amount is lower than the historical average, and the supply amount needs to be appropriately increased.

[0142] Step S1463: Determine the meal category with the contact frequency meeting the preset threshold as the meal category that needs to adjust the supply amount, when the consumption speed deviation value meets the preset deviation threshold.

[0143] Compare the calculated consumption speed deviation value with the preset deviation threshold. The preset deviation threshold is set according to the operation experience and market demand of the canteen, and is used to judge whether the change of consumption speed needs to adjust the supply amount. When the consumption speed deviation value meets the preset deviation threshold, it means that the consumption speed of the meal has changed greatly, and the supply amount needs to be adjusted. The meal category is determined as the meal category that needs to adjust the supply amount.

[0144] Step S1464: Determine the meal category with the contact frequency not meeting the preset threshold as the meal category that needs to adjust the supply amount, when the remaining amount deviation value meets the preset deviation threshold.

[0145] Similarly, compare the remaining amount deviation value with the preset deviation threshold. When the remaining amount deviation value meets the preset deviation threshold, it means that the remaining amount of the meal has changed greatly, and the supply amount needs to be adjusted. The meal category is determined as the meal category that needs to adjust the supply amount.

[0146] Step S1465: Extract the identification information of the meal category whose supply amount needs to be adjusted.

[0147] The identification information of the meal category whose supply amount needs to be adjusted is extracted, which can uniquely identify each meal category. By extracting the identification information, the meal that needs to adjust the supply amount can be accurately managed and allocated.

[0148] Step S1466: Generate meal supply optimization direction as the meal category whose supply amount needs to be adjusted based on the identification information of the meal category whose supply amount needs to be adjusted.

[0149] After extracting the identification information of the meal category whose supply amount needs to be adjusted, the meal supply optimization direction can be generated based on it. These identification information is the key basis for accurately pointing to the meal whose supply amount needs to be adjusted. These identification information is integrated to form a clear and explicit meal list, and the meal category contained in the meal list is the meal category that needs to adjust the supply amount. The meal supply optimization direction is around these meal categories that need to be adjusted. For example, if the identification information shows that the supply amount of some stir-fried dishes, main dishes and soups needs to be adjusted, the meal supply optimization direction will clearly indicate that the supply amount of these stir-fried dishes, main dishes and soups needs to be adjusted. It may be to increase the supply amount to meet the needs of diners, or it may be to reduce the supply amount to avoid waste. Through the above way, the canteen can optimize the meal supply according to the actual dining behavior evaluation result, improve the utilization efficiency of resources and the satisfaction of diners.

[0150] Step S150: Generate management execution instructions containing service window scheduling instructions and meal replenishment instructions based on the operation management adjustment direction, and send the management execution instructions to the canteen management system to trigger operation adjustment operation.

[0151] According to the determined canteen operation management adjustment direction, specific management execution instructions need to be generated, which include service window scheduling instructions and meal replenishment instructions, and then these instructions are sent to the canteen management system to start operation adjustment operation.

[0152] Step S151: Generate service window scheduling instructions based on the window service efficiency adjustment direction.

[0153] The service window efficiency adjustment direction determines which pickup window needs to adjust the service personnel. The service window scheduling instruction is generated according to the service window efficiency adjustment direction. First, the position distribution information of the window that needs to adjust the service personnel and the personnel adjustment position set are referred to. For those windows that need to increase the number of service personnel, the instruction specifies the personnel to be allocated from which other windows or standby personnel reserves, the number of personnel to be allocated, and the expected arrival time, etc. For example, if a pickup window located in the corner of the cafeteria has a large service pressure and needs to increase the number of service personnel, the service window scheduling instruction may indicate that a certain number of service personnel are allocated from other relatively idle area windows to the window and are required to arrive within a certain time. For the window that needs to reduce the number of service personnel, the instruction specifies that these personnel are allocated to other windows that have needs or are arranged to perform other work tasks, and also specifies the allocation time and subsequent work arrangement.

[0154] Step S152: generating a meal supplement instruction based on the meal supply optimization direction.

[0155] The meal supply optimization direction determines the meal category that needs to adjust the supply amount. The meal supplement instruction is generated according to the meal supply optimization direction. For those meal categories that need to increase the supply amount, the meal supplement instruction specifies the specific identification of the meal, the supply amount to be increased, the expected supplement time, etc. For example, if a popular stir-fried dish needs to increase the supply amount, the instruction will inform the kitchen to prepare the corresponding ingredients, make a sufficient number of the stir-fried dish according to a certain recipe and process, and supplement it to the pickup window before the peak meal time. For the meal category that needs to reduce the supply amount, the instruction requires the kitchen to reduce the production amount of the meal to avoid waste due to excessive supply. At the same time, the inventory situation is also considered to reasonably arrange the consumption of inventory and the supplement of new meals.

[0156] Step S153: integrating the service window scheduling instruction and the meal supplement instruction to generate a management execution instruction.

[0157] The generated service window scheduling instruction and meal supplement instruction are integrated to form a complete management execution instruction. The management execution instruction contains detailed information of service window personnel allocation and meal supply adjustment. During the integration process, the coordination and consistency between the various parts of the management execution instruction can be ensured to avoid conflicts or contradictions. For example, the time of personnel allocation in the service window scheduling instruction cannot conflict with the time of meal arrival at the window in the meal supplement instruction to ensure the smooth progress of the entire operation adjustment process.

[0158] Step S154: sending the management execution instruction to the cafeteria management system to trigger the operation adjustment operation.

[0159] The integrated management execution instruction is sent to the canteen management system. After receiving the management execution instruction, the canteen management system can automatically trigger the corresponding operation adjustment operation according to the content of the management execution instruction. For the service window scheduling instruction, the system will convey the deployment information to the relevant service personnel, update the personnel's work arrangement and post information. The service personnel will arrive at the new work post within the specified time according to the requirements of the instruction and start the new work task. For the meal supplement instruction, the system will convey the meal preparation and supplement information to the kitchen staff, and the kitchen staff will prepare the ingredients, make the meals, and supplement the meals to the meal pickup window within the specified time. Through the above-mentioned manner, the canteen can timely adjust the operation management strategy according to the dining behavior evaluation result, and improve the service efficiency and the rationality of meal supply.

[0160] Regarding the construction and training of the artificial intelligence model, the dining behavior analysis model mainly consists of a feature fusion layer, a process evaluation module and a preference identification module. The feature fusion layer is responsible for the spatio-temporal correlation modeling processing of the input dining behavior feature set and meal payment state feature. It receives these two features as input, correlates them in the time and space dimensions, and generates a fusion feature vector with behavior continuity constraints through a series of processing operations (although it is a black box, it can be understood as a complex combination and transformation of features). The process evaluation module receives the fusion feature vector, analyzes and processes the meal pickup process, extracts the correlation index of the path length parameter and the moving speed parameter, and generates a meal pickup process smoothness description. The preference identification module also receives the fusion feature vector, analyzes and processes the meal selection, and counts the occurrence frequency of different contact action types on each meal container to generate a meal selection preference description. The three modules are connected in sequence, and the output of the feature fusion layer is used as the input of the process evaluation module and the preference identification module to complete the dining behavior analysis task together.

[0161] A large amount of historical dining behavior data can be collected, including the dining behavior feature set and the meal payment state feature. The dining behavior feature set covers the meal pickup path feature and the meal contact feature, the meal pickup path feature includes the path length parameter and the path inflection point coordinate set, and the meal contact feature includes the contact start time point, the contact end time point and the contact action type description. The meal payment state feature includes the payment timestamp, the payment success flag and the payment delay duration. At the same time, accurate meal pickup process smoothness description and meal selection preference description are labeled for these data as training labels. For example, for the meal pickup process smoothness description, the process congestion node coordinates and congestion duration are labeled; for the meal selection preference description, the meal categories with contact frequency meeting the preset threshold and the meal categories with contact frequency not meeting the preset threshold are labeled.

[0162] Then, the prepared training data is input into the dining behavior analysis model for training. First, parameters of the training are set, such as learning rate, iteration times, etc. The learning rate controls the step size of parameter update in the training process, and the iteration times represent the number of rounds of learning of the model on the training data. During the training process, the model will continuously adjust its parameters, so that the take-out process smoothness description and food selection preference description output by the model are as close as possible to the labeled tags. Specifically, for the feature fusion layer, the internal parameters can be adjusted to make the generated fusion feature vector better reflect the comprehensive information of dining behavior and payment status; for the process evaluation module and the preference identification module, their respective parameters will also be adjusted to improve the analysis accuracy of the take-out process and food selection. Through multiple iterations of training, the performance of the model is continuously optimized until the output result of the model reaches the satisfactory accuracy requirement.

[0163] Next, a part of the test data that does not participate in the training is used to evaluate the trained dining behavior analysis model. The error indicators between the take-out process smoothness description and food selection preference description output by the model and the labeled tags of the test data are calculated, such as accuracy, recall rate, etc. If the error indicators do not meet the expected requirements, it means that the performance of the model needs to be further optimized. At this time, analyze where the model makes mistakes, for example, is the analysis of some special take-out process situations inaccurate, or is the judgment of food selection preference wrong. According to the analysis result, adjust the training parameters or increase the diversity of the training data, and retrain until the performance of the model on the test data reaches a satisfactory effect.

[0164] During the entire data collection and processing process, some privacy-sensitive data are involved, such as user identity verification information (including facial image data, payment QR code image data, etc.). In order to protect these privacy-sensitive data, a series of privacy protection and leakage prevention technical means are adopted. In the data collection stage, the biometric feature collection device and the QR code scanning device will encrypt the collected data, converting the original image data into encrypted feature vectors. In this way, even if the data is intercepted during transmission, attackers cannot obtain valuable information. In the data storage stage, when storing user identity verification information, meal identification results, and pending settlement fee information, etc. in the local storage module or the background server, advanced encryption algorithms are used to encrypt the data for storage. Only authorized system components can decrypt and process these data. At the same time, a strict access control mechanism is set up to manage the access rights of the data. Only personnel and system modules with corresponding permissions can access these privacy-sensitive data. In the data transmission process, secure communication protocols such as SSL / TLS are used to ensure the integrity and confidentiality of the data during transmission, preventing data tampering and theft. Through these privacy protection and leakage prevention technical means, the privacy-sensitive data of users can be effectively protected, and the risks brought by data leakage can be avoided.

[0165] Figure 2 A schematic diagram of exemplary hardware and software components of the intelligent canteen dining management system 100 based on artificial intelligence algorithm that can implement the idea of the present application is shown. For example, the processor 120 can be used in the intelligent canteen dining management system 100 based on artificial intelligence algorithm and used to perform the functions in the present application.

[0166] The intelligent canteen dining management system 100 based on artificial intelligence algorithm can be a general server or a special-purpose server, both of which can be used to implement the intelligent canteen dining management method based on artificial intelligence algorithm of the present application. Although only one server is shown in the present application, for the sake of convenience, the functions described in the present application can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0167] For example, the intelligent canteen dining management system 100 based on artificial intelligence algorithm can include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and different forms of storage media 140, such as a disk, a ROM, or a RAM, or any combination thereof. Illustratively, the intelligent canteen dining management system 100 based on artificial intelligence algorithm can also include program instructions stored in a ROM, a RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application can be implemented according to these program instructions. The intelligent canteen dining management system 100 based on artificial intelligence algorithm also includes an I / O interface 150 between the computer and other input and output devices.

[0168] For ease of illustration, only one processor is described in the intelligent canteen dining management system 100 based on artificial intelligence algorithm. However, it should be noted that the intelligent canteen dining management system 100 based on artificial intelligence algorithm in the present application can also include multiple processors, so the steps described in the present application performed by one processor can also be jointly performed or individually performed by multiple processors. For example, if the processor of the intelligent canteen dining management system 100 based on artificial intelligence algorithm performs steps A and B, it should be understood that steps A and B can also be jointly performed by two different processors or individually performed in one processor. For example, a first processor performs step A, a second processor performs step B, or a first processor and a second processor jointly perform steps A and B.

[0169] In addition, the present application also provides a readable storage medium, wherein computer executable instructions are pre-set in the readable storage medium, and when a processor executes the computer executable instructions, the intelligent canteen dining management method based on artificial intelligence algorithm is realized.

[0170] It should be noted that, in order to simplify the description of the present application and to help understand one or more embodiments of the present application, in the foregoing description of the embodiments of the present application, various features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A smart canteen dining management method based on artificial intelligence algorithm, characterized in that: The method comprises: Acquire a real-time video data set covering a dining area of ​​a cafeteria, wherein the real-time video data set includes image frame units continuously acquired from multiple perspectives and marked with time stamps; Performing dining behavior feature extraction processing on the real-time video data set to obtain a dining behavior feature set of the diner in each image frame unit, the dining behavior feature set including a meal pickup path feature and a food contact feature. The meal pickup path feature reflects movement trajectory information of the diner from the entrance to the meal pickup window and then to the dining area, and the food contact feature reflects interaction information between the diner and the food container. Obtaining the meal payment status characteristics of each diner, calling a pre-trained dining behavior analysis model to perform dining behavior analysis on the diner's dining behavior feature set and the meal payment status characteristics, and generating a dining behavior evaluation result, which includes a description of the smoothness of the meal collection process and a description of meal selection preferences; Determining the direction of canteen operation and management adjustments based on the dining behavior evaluation results, wherein the operation and management adjustment directions include window service efficiency adjustment and food supply optimization; generating a management execution instruction including a service window scheduling instruction and a meal replenishment instruction based on the operation management adjustment direction, and sending the management execution instruction to the cafeteria management system to trigger an operation adjustment operation; The adjustment direction of the canteen operation and management is determined based on the dining behavior evaluation results. The adjustment direction of the operation and management includes the adjustment direction of window service efficiency and the optimization direction of food supply, including: Parsing the description of the smoothness of the meal collection process in the dining behavior evaluation result, and extracting the coordinates of the process congestion nodes and the congestion duration; Determine the corresponding pick-up window position according to the process congestion node coordinates, and calculate the service pressure parameter of the pick-up window according to the congestion duration; Comparing the service pressure parameter with a preset service pressure threshold to generate a window service efficiency adjustment direction, wherein the window service efficiency adjustment direction includes a window position where the service pressure parameter meets the preset threshold and a window position where the service pressure parameter does not meet the preset threshold; parsing the description of meal selection preferences in the dining behavior evaluation results, and extracting meal categories whose contact frequency meets a preset threshold and whose contact frequency does not meet the preset threshold; Counting the consumption rate parameters of the food categories whose contact frequency meets the preset threshold within the historical time period, and counting the remaining quantity parameters of the food categories whose contact frequency does not reach the preset threshold within the historical time period; A meal supply optimization direction is generated according to the consumption speed parameter and the remaining quantity parameter, where the meal supply optimization direction includes meal categories for which supply quantities need to be adjusted.

2. The intelligent canteen dining management method based on artificial intelligence algorithm according to claim 1 is characterized in that: The dining behavior feature extraction process is performed on the real-time video data set to obtain the diner's meal picking path feature and food contact feature in each image frame unit, including: Performing frame processing on the real-time video data set to obtain a plurality of single-frame image units having a continuous time series relationship; performing target detection processing on the single-frame image unit to identify location information of the diner and location information of the food container, wherein the location information includes two-dimensional plane coordinates; Performing trajectory tracking processing on single-frame image units of a continuous time series, generating meal-collecting path features based on changes in the diner's position information in adjacent single-frame image units, wherein the meal-collecting path features include a path length parameter and a set of path inflection point coordinates; performing action recognition processing on the single-frame image unit, analyzing the change in the spatial position relationship between the diner and the food container, and generating a food contact feature, wherein the food contact feature includes a contact start time point, a contact end time point, and a description of the contact action type; The meal-taking path feature and the meal-item contact feature are input into a feature association module for timestamp alignment processing to obtain a dining behavior feature set with time synchronization.

3. The intelligent canteen dining management method based on artificial intelligence algorithm according to claim 2 is characterized in that: The process of performing trajectory tracking on the single-frame image units of the continuous time series and generating meal-collecting path features based on changes in the position information of the diner in adjacent single-frame image units includes: Extracting the position information of the same diner in adjacent single-frame image units and calculating the coordinate difference of the position information as a displacement vector; Calculating a movement speed parameter between adjacent single-frame image units according to the displacement vector, wherein the movement speed parameter is a ratio of a displacement vector modulus to a time interval; Constructing a movement trajectory analysis model of the diner based on the displacement vector and the movement speed parameter; Calculating the path length parameter of the diner in the continuous single-frame image unit by the movement trajectory analysis model, wherein the path length parameter is the accumulated value of the modulus lengths of each adjacent displacement vector; The trajectory coordinate points output by the movement trajectory analysis model are subjected to inflection point detection processing, and coordinate points where the trajectory direction changes significantly are identified as a path inflection point coordinate set. The significant change is determined by the displacement vector direction angle exceeding a preset critical value.

4. The intelligent canteen dining management method based on artificial intelligence algorithm according to claim 2 is characterized in that: The performing motion recognition processing on the single-frame image unit, analyzing the change in the spatial position relationship between the diner and the food container, and generating food contact features includes: Performing region segmentation processing on the single-frame image unit to divide the diner's body area and the food container area; Calculating a spatial overlapping area parameter between the diner's limb area and the food container area; Determining a contact start time point and a contact end time point between the diner and the food container based on a temporal trend of the spatial overlap area parameter; Perform motion pattern recognition on the single-frame image units within the contact time period to extract the movement direction sequence and force change sequence of the diner's limbs; A preset action type library is matched according to the motion direction sequence and the force change sequence to generate a contact action type description, wherein the action type description includes a meal taking action, a meal putting action, and a rummaging action.

5. The intelligent canteen dining management method based on artificial intelligence algorithm according to claim 1 is characterized in that: The calling of the pre-trained dining behavior analysis model performs dining behavior analysis on the dining behavior feature set of the diner and the meal payment status feature to generate a dining behavior evaluation result, including: Inputting the dining behavior feature set of the diner and the meal payment status feature into the feature fusion layer of the dining behavior analysis model, performing spatiotemporal correlation modeling processing, and generating a fused feature vector with behavior continuity constraints; The process evaluation module of the dining behavior analysis model performs meal collection process analysis on the fused feature vector, extracts the correlation index between the path length parameter and the movement speed parameter, and generates a description of the smoothness of the meal collection process, wherein the description includes the coordinates of the process congestion node and the congestion duration; The preference recognition module of the dining behavior analysis model performs food selection analysis on the fused feature vector, counts the frequencies of different contact action types on each food container, and generates a food selection preference description, wherein the food selection preference description includes food categories whose contact frequencies meet a preset threshold and food categories whose contact frequencies do not meet the preset threshold; The description of the smoothness of the meal pickup process and the description of the meal selection preference are integrated to generate a dining behavior evaluation result including timestamp alignment.

6. The intelligent canteen dining management method based on artificial intelligence algorithm according to claim 5 is characterized in that: The process evaluation module of the dining behavior analysis model performs meal collection process analysis on the fused feature vector, extracts the correlation index between the path length parameter and the moving speed parameter, and generates a description of the smoothness of the meal collection process, including: Extracting a path length parameter set, a movement speed parameter set, and an associated meal payment status feature set from the fused feature vector, the meal payment status feature set comprising a payment timestamp, a payment success flag, and a payment delay duration; Calculating the Pearson correlation coefficient between the path length parameter set and the moving speed parameter set to obtain a correlation index between the path length and the moving speed; Identify abnormal nodes in the meal pickup process based on the correlation index, where the abnormal nodes are path inflection point coordinates where the correlation index is lower than a preset critical value; Extracting the timestamp information corresponding to the abnormal node, and combining it with the payment timestamp in the meal payment status feature set to determine whether the abnormal node is located in the payment window associated area; If the abnormal node is located in the payment window-related area, the congestion duration is determined based on the change trend of the movement speed parameter of the previous and next single-frame image units and the payment delay duration. If the abnormal node is not in the payment-related area, the congestion duration is determined based only on the change trend of the movement speed parameter. The path inflection point coordinates of the abnormal node are used as the process congestion node coordinates, the congestion duration is associated with the process congestion node coordinates and the associated payment status features, and a description of the smoothness of the meal pickup process is generated.

7. The intelligent canteen dining management method based on artificial intelligence algorithm according to claim 5 is characterized in that: The preference recognition module of the dining behavior analysis model performs food selection analysis on the fused feature vector, counts the occurrence frequencies of different contact action types on each food container, and generates a food selection preference description, including: Extracting a contact action type description set, a corresponding food container identification set, and an associated food payment status feature set from the fused feature vector, wherein the food payment status feature set includes a payment success flag and payment timestamp information; Establishing an association mapping table between contact action types, food container identifiers, and food payment status characteristics, wherein the association mapping table records the contact action type, occurrence timestamp, and payment success flag corresponding to each food container identifier; Divide the association mapping table into time windows, and count the number of times the contact action type of each food container identifier occurs when the payment success flag is true within each time window; Calculate the proportion of each contact action type occurring on the same food container logo to generate a contact action frequency distribution matrix; According to the contact action frequency distribution matrix, the contact action types whose occurrence ratio meets the preset threshold are screened, and the food categories whose contact frequency meets the preset threshold and the food categories whose contact frequency does not reach the preset threshold are determined in combination with the corresponding food container identification to generate a food selection preference description.

8. The intelligent canteen dining management method based on artificial intelligence algorithm according to claim 1 is characterized in that: The comparing the service pressure parameter with a preset service pressure threshold to generate a window service efficiency adjustment direction includes: Normalizing the service pressure parameter to obtain a standardized pressure value within the range of 0-1; Determine that the pick-up window whose standardized pressure value meets the preset pressure threshold is the window that needs to adjust the service staff, and the pick-up window whose standardized pressure value does not reach the preset pressure threshold is the window that needs to adjust the service staff; Counting the number and location distribution information of the windows where service personnel need to be adjusted; Generating a personnel adjustment position in a service window scheduling instruction according to the position distribution information of the windows where service personnel need to be adjusted, wherein the personnel adjustment position is a coordinate set of the windows where service personnel need to be adjusted; Integrate the personnel adjustment positions to generate the direction of window service efficiency adjustment; Furthermore, generating a meal supply optimization direction according to the consumption rate parameter and the remaining quantity parameter includes: Calculating the difference between the consumption speed parameter of the food category whose contact frequency meets the preset threshold and the historical average consumption speed to obtain a consumption speed deviation value; Calculating the difference between the remaining quantity parameter of the food category whose contact frequency does not reach the preset threshold and the historical average remaining quantity to obtain a remaining quantity deviation value; Determine the food category whose consumption speed deviation value meets the preset deviation threshold and whose contact frequency meets the preset threshold as the food category for which the supply quantity needs to be adjusted; Determine the food category whose remaining quantity deviation value meets the preset deviation threshold and whose contact frequency does not reach the preset threshold as the food category for which the supply quantity needs to be adjusted; Extracting identification information of the food category for which the supply quantity needs to be adjusted; The identification information of the meal category whose supply quantity needs to be adjusted is used as the meal category whose supply quantity needs to be adjusted, and a meal supply optimization direction is generated.

9. A smart canteen dining management system based on artificial intelligence algorithm, characterized by: It includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the smart canteen dining management method based on artificial intelligence algorithm as described in any one of claims 1 to 8.

Images