Intelligent dinner plate scheduling and pricing method, medium and equipment

Abstract

The invention discloses an intelligent dinner plate scheduling and pricing method, a medium and equipment, and the method comprises the steps: receiving multi-source sensing data which comprises the unique identification information of dinner plates, the image data of a meal taking region and the weight change data of the dinner plates, and carrying out the multi-modal fusion recognition, so as to generate binding pricing information; on the basis of the bound pricing information and historical consumption data, dynamic meal preparation suggestions and supplementary scheduling instructions are generated through a space-time prediction model, and then corresponding interfaces are called to execute resource allocation and collect dinner plate recovery state data; and finally, performing residue visual analysis on the dinner plate recovery state data to generate a waste report and outputting the waste report. According to the invention, closed-loop intelligent management from consumption perception and intelligent pricing to production resource dynamic prediction and dinner plate real-time scheduling is realized, the operation efficiency is improved, and resource waste is reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent management technology in the catering industry, specifically to an intelligent plate scheduling and pricing method, medium, and equipment. Background Technology

[0002] With the continuous improvement of the catering industry's level of intelligence, technologies such as self-service food collection and intelligent payment have been widely applied in various canteens and restaurants. Existing technologies mainly use visual recognition and weighing sensors to achieve food identification and automatic pricing, thereby improving payment efficiency and user experience. Meanwhile, to promote thrift and reduce waste, some solutions introduce image analysis technology in the dish return process to identify and statistically analyze food residue, providing feedback to users. However, these technologies primarily focus on recording and post-event analysis of individual consumption behavior; their core logic lies in achieving accurate pricing and waste statistics, essentially remaining a static and reactive management approach.

[0003] In actual canteen or restaurant operations, food pickup and consumption are closely linked to kitchen production and tray scheduling, forming a dynamic process. Existing solutions lack deep integration and proactive utilization of multi-source sensing data, failing to effectively link real-time front-end consumption dynamics with back-end production, food preparation, and tray circulation. For example, when frequent pickup of a particular dish leads to a rapid decrease in stock, the existing system cannot predict and trigger replenishment in the kitchen in a timely manner; similarly, when delayed tray collection leads to a shortage of available trays at the front-end, it cannot automatically generate scheduling instructions. This results in kitchen production often relying on experience-based estimations, easily leading to insufficient or excessive food preparation, while tray scheduling relies on manual inspections, resulting in low efficiency and delayed response. Furthermore, food waste analysis data has not been further transformed into optimization guidance for kitchen procurement and dish portion settings, making it difficult to achieve an overall improvement in resource utilization efficiency at the system level. Summary of the Invention

[0004] In view of the above problems, the present invention provides an intelligent plate scheduling and pricing method, medium and equipment, which realizes dynamic prediction and real-time allocation of restaurant operation resources by deeply integrating consumption data with production scheduling.

[0005] To achieve the above objectives, in a first aspect, this application provides an intelligent plate scheduling and pricing method, comprising: Receive multi-source sensing data including unique tray identification information, image data of the food pick-up area, and tray weight change data; Multimodal fusion and recognition are performed on multi-source sensing data to generate bound pricing information that includes the type of dishes taken, the portion size of the dishes, and the corresponding plate identification. Based on the binding pricing information and historical consumption data, dynamic meal preparation suggestions for kitchen production resources and replenishment scheduling instructions for front-of-house tray inventory are generated through a spatiotemporal prediction model. Based on dynamic meal preparation suggestions and replenishment scheduling instructions, the corresponding production control interface and logistics scheduling interface are invoked to execute resource allocation, and the meal tray recycling status data is collected in real time. Visual analysis of food waste status data is performed on the plates to generate a waste report that includes the type and amount of food waste. Output the waste report.

[0006] In some embodiments, multimodal fusion recognition is performed on multi-source sensing data to generate bound pricing information including the type of dish taken, the portion size, and the corresponding plate identifier, including: Based on the unique identifier of the plate, retrieve the set of candidate dish types associated with the unique identifier of the plate from the preset plate-dish association mapping table; The image data of the food pick-up area is subjected to food target detection and feature extraction to obtain image recognition results. The image recognition results include the types of food identified in the image and the corresponding image recognition confidence. Differential calculations are performed on the changes in plate weight to obtain the net increase in weight caused by the food retrieval operation; Based on the candidate dish type set, the image recognition results are filtered, and the dish types belonging to the candidate dish type set are determined as the initially identified dishes; The preliminary identification of dishes, image recognition confidence scores, and net weight gain data are input into a pre-defined fusion decision model. The fusion decision model outputs the final dish type determination result and dish portion determination result based on the relationship between image recognition confidence and preset confidence threshold, and the relationship between net weight gain data and preset standard weight range of dishes. The results of dish type determination and dish portion determination are linked and bound to the unique identification information of the plate, and the bound pricing information is calculated and generated based on the preset pricing rules.

[0007] In some embodiments, the fusion decision model outputs the final dish type determination result and dish portion determination result based on the relationship between image recognition confidence and a preset confidence threshold, and the relationship between net weight gain data and a preset standard weight range for dishes, including: Determine whether the image recognition confidence level is greater than or equal to the first confidence threshold; If the image recognition confidence level is greater than or equal to the first confidence threshold, the initially identified dish will be directly determined as the dish type determination result. If the image recognition confidence level is less than the first confidence threshold but greater than or equal to the second confidence threshold, then a weighted fusion decision process is executed, including: The weight of weight evidence is determined based on the range of standard weights of the dishes that the net weight gain data falls into, and the weight of image evidence is determined based on the confidence level of image recognition. Based on the weight of weighted evidence and the weight of image evidence, a weighted decision is made on the initially identified dishes, and the dish type determination result is output. If the image recognition confidence level is less than the second confidence level threshold, the manual review process is triggered, and the dish type confirmed by the manual review is taken as the dish type determination result. After determining the dish type, query the standard weight range of the dish corresponding to the dish type determination result; Compare the net weight gain data with the standard weight range of the dishes; If the net weight gain data is within the standard weight range of the dish, the median of the standard weight range or the preset standard portion will be used as the result of the dish portion determination. If the net weight gain exceeds the standard weight range of the dish, the net weight gain will be used directly as the result of the dish portion size determination.

[0008] In some embodiments, based on the bound pricing information and historical consumption data, a spatiotemporal prediction model is used to generate dynamic meal preparation suggestions for kitchen production resources and replenishment scheduling instructions for front-of-house tray inventory, including: Extract the real-time consumption quantity of each dish type within the current statistical period from the bound pricing information; Retrieve the historical consumption quantity sequence of each type of dish within a historical period that has the same time attribute as the current moment from historical consumption data; The real-time consumption quantity, historical consumption quantity sequence, and preset external environmental factor data are all input into the pre-trained spatiotemporal prediction model. The spatiotemporal prediction model predicts the demand for each type of dish within a preset time period based on real-time consumption, historical consumption sequences, and external environmental factors. The production gap for each dish type is calculated by comparing the demand forecast with the real-time inventory of each dish type in the kitchen. Based on the production gap and the preset single production batch capacity, dynamic meal preparation suggestions are generated, including the type of dishes to be produced, the quantity to be produced, and the suggested production start time. At the same time, based on the number of plates that have been picked up in the bundled pricing information and the preset front-desk plate inventory safety threshold, it is determined whether the front-desk plate inventory is lower than the front-desk plate inventory safety threshold. If it is determined that the number of plates at the front desk is lower than the safe threshold for plate inventory, a replenishment scheduling instruction is generated based on the quantity and location of cleanable plates in the plate recycling status data to guide the replenishment path and quantity of plates.

[0009] In some embodiments, the spatiotemporal prediction model predicts the demand for each type of dish within a preset future time period based on real-time consumption volume, historical consumption volume sequences, and external environmental factor data, including: Periodic features are extracted from historical consumption sequences to obtain time-series feature vectors that reflect the long-term trend and periodic pattern of food consumption. Short-term fluctuation features are extracted from the real-time consumption quantity to obtain a real-time feature vector reflecting the current consumption rate and the degree of deviation. Encode the external environmental factor data, which includes at least weather conditions, date type, and promotional activity information, and convert the encoded external environmental factor data into an environmental feature vector. The time-series feature vector, real-time feature vector, and environmental feature vector are concatenated to form a fused feature vector; The fused feature vector is input into the feature encoding layer in the spatiotemporal prediction model. The feature encoding layer performs nonlinear transformation and feature dimensionality reduction on the fused feature vector and outputs a deep feature representation. The deep feature representation is input into the sequence prediction layer in the spatiotemporal prediction model. The sequence prediction layer performs weighted aggregation of features at different time steps in the deep feature representation based on the attention mechanism, and outputs a sequence of demand quantities for multiple consecutive preset time periods in the future. Extract the predicted demand quantity for each of the future preset time periods from the demand quantity sequence of multiple consecutive preset time periods.

[0010] In some embodiments, based on dynamic meal preparation suggestions and replenishment scheduling instructions, the corresponding production control interface and logistics scheduling interface are invoked to execute resource allocation, and real-time data on the status of meal tray collection is collected, including: Analyze dynamic meal preparation suggestions to extract the types of dishes to be produced, the quantities to be produced, and the suggested production start time; Match the corresponding automated kitchen production equipment or manual workstation terminal according to the type of dish to be produced; By calling the production control interface associated with the automated production equipment or manual workstation in the back kitchen, the production instruction containing the quantity to be produced and the suggested production start time is sent to the automated production equipment or manual workstation in the back kitchen. Parse the supplementary scheduling instructions to extract the path and quantity of the trays to be replenished. Based on the guide tray replenishment route, match the corresponding logistics robot or material scheduling system; By calling the logistics scheduling interface associated with the logistics robot or material scheduling system, the delivery instructions containing the route and quantity of the replenished plates are sent to the logistics robot or material scheduling system. During the execution of production and delivery orders, the system continuously receives tray recycling status data, which includes at least the unique identifier of the recycled tray, the recycling timestamp, and the recycling location information.

[0011] In some embodiments, visual analysis of food waste is performed on the plate recycling status data to generate a waste report that includes the type and amount of food waste, including: Based on the recycling location information in the dish recycling status data, a high-resolution image acquisition device deployed at the corresponding recycling location is triggered to acquire an image of the surface residue on the recycled dish. Instance segmentation processing is performed on surface residue images to identify and segment residual food regions belonging to different categories in the images; Extract visual features from each area of ​​residual food and input these visual features into a pre-trained residue classification model; The food residue classification model is based on visual features and outputs the classification results of the food components for each area of ​​residual food. Based on the classification results of dish ingredients, residual food areas belonging to the same dish type are merged, and the pixel area ratio of the merged area is calculated. Based on the known plate specifications and pixel area ratio of the recycled plates, estimate the amount of waste corresponding to each type of dish. The system correlates the categorization results of food ingredients with the estimated amount of waste and adds a recycling timestamp to generate a structured waste report.

[0012] In some embodiments, the food residue classification model, based on visual features, outputs a classification result of the food components for each area of ​​residual food, including: Visual features are input into the feature embedding layer of the residue classification model. The feature embedding layer maps the visual features to a high-dimensional semantic space and generates region feature embedding vectors. The similarity between the region feature embedding vector and multiple dish prototype vectors in the predefined dish prototype vector library is calculated. Each dish prototype vector in the dish prototype vector library corresponds to a dish ingredient. Based on the similarity calculation results, select the top few dish prototype vectors with the highest similarity to the region feature embedding vector, and obtain the dish components corresponding to the top few dish prototype vectors as candidate classification results. The region feature embedding vector and the dish prototype vector corresponding to the first few dish prototype vectors are input into the context reasoning module in the waste classification model. The context reasoning module uses the relationship between the region feature embedding vector and the previous several dish prototype vectors, as well as the dish type determination result bound to the recycled plate in the food picking process, as the prior context to reorder and filter the candidate classification results based on their confidence. The dish ingredient with the highest confidence, processed by the context reasoning module, is output as the dish ingredient classification result.

[0013] In a second aspect, the present invention also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method described in the first aspect.

[0014] In a third aspect, the present invention also provides an electronic device including a memory and a processor, the memory being used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0015] Unlike existing technologies, the above-mentioned technical solution provides an intelligent plate scheduling and pricing method, medium, and device. It receives multi-source sensing data including unique plate identifiers, image data of the food collection area, and plate weight change data, and performs multimodal fusion recognition to generate bound pricing information. Based on this bound pricing information and historical consumption data, a spatiotemporal prediction model generates dynamic meal preparation suggestions and replenishment scheduling instructions, then calls corresponding interfaces to execute resource allocation and collect plate recycling status data. Finally, visual analysis of the plate recycling status data is performed to generate and output a waste report. This invention achieves closed-loop intelligent management from consumption perception and intelligent pricing to dynamic prediction of production resources and real-time plate scheduling, improving operational efficiency and reducing resource waste.

[0016] The above description of the invention is merely an overview of the technical solution of this application. In order to enable those skilled in the art to better understand the technical solution of this application and to implement it based on the description and drawings, and to make the above-mentioned objectives and other objectives, features and advantages of this application easier to understand, the following description is provided in conjunction with the specific embodiments and drawings of this application. Attached Figure Description

[0017] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of specific embodiments of the present invention and other related contents, and should not be considered as limitations on this application.

[0018] In the accompanying drawings of the instruction manual: Figure 1 This is a schematic diagram illustrating steps S101 to S106 of the method described in the specific implementation embodiment; Figure 2 This is a schematic diagram illustrating steps S201 to S207 of the method described in the specific implementation embodiment; Figure 3 This is a schematic diagram of the structure of the electronic device described in a specific embodiment.

[0019] The reference numerals used in the above figures are explained as follows: 1. Electronic equipment; 11. Memory; 12. Processor. Detailed Implementation

[0020] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.

[0021] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0022] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0023] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0024] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0025] Without further limitations, the use of terms such as “comprising,” “including,” “having,” or other similar open-ended expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0026] As understood in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceeding" are understood to exclude the stated number; expressions such as "above," "below," and "within" are understood to include the stated number. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times," unless otherwise explicitly specified.

[0027] In the description of the embodiments of this application, the space-related expressions used, such as "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "vertical," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential," indicate the orientation or positional relationship based on the orientation or positional relationship shown in the specific embodiments or drawings. They are only for the purpose of describing the specific embodiments of this application or for the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0028] Please see Figure 1 In a first aspect, this embodiment provides an intelligent plate scheduling and pricing method, including: S101, Receive multi-source sensing data including unique tray identification information, food pick-up area image data and tray weight change data; S102. Perform multimodal fusion recognition on multi-source sensing data to generate bound pricing information that includes the type of dishes taken, the portion size of the dishes, and the corresponding plate identification. S103. Based on the binding pricing information and historical consumption data, generate dynamic meal preparation suggestions for kitchen production resources and replenishment scheduling instructions for front-of-house tray inventory through a spatiotemporal prediction model. S104. Based on dynamic meal preparation suggestions and replenishment scheduling instructions, call the corresponding production control interface and logistics scheduling interface to execute resource allocation, and collect meal tray recycling status data in real time. S105. Perform visual analysis of food waste status data on plate recycling to generate a waste report that includes the type and amount of food waste. S106. Output the waste report.

[0029] In step S101, the unique identification information of the plate is automatically obtained by the corresponding reading and writing device through carriers such as RFID tags and QR codes attached to the plate when the plate enters the food collection area. This information is used to uniquely track the circulation status of the plate in the system. Image data of the food collection area is continuously captured by cameras deployed above the food collection counter, recording the user's food collection actions and visual information of the contents of the plate. The plate weight change data comes from the weighing sensor integrated under the food collection counter or a specific food collection position. By continuously monitoring the weight of the plate and calculating the weight difference between adjacent time points, the weight increase or decrease caused by food collection is reflected. This step integrates three different dimensions of perception data—identification, vision, and weight—to form the original information foundation for subsequent accurate identification and decision-making.

[0030] In step S102, multimodal fusion recognition refers to comprehensively utilizing visual features from the image data of the food collection area and physical change information from the plate weight change data. This is achieved through information complementarity and cross-validation to improve the accuracy of dish identification and portion determination. Specifically, target detection is performed on the image data to identify candidate dish types. Simultaneously, the net weight gain of the selected dish is inferred from the weight change data. These two types of information are then correlated and used to determine the final dish type and actual portion size. The bound pricing information is a structured data record that includes at least the successfully identified dish type, the portion size determined based on the weight data, the unique plate identifier strictly associated with it, and the consumption amount calculated according to pre-designed pricing rules. The bound pricing information generated in this step provides an accurate and real-time data source of consumption details for subsequent operational scheduling.

[0031] In step S103, the spatiotemporal prediction model is a machine learning or statistical model trained to learn the changing patterns of consumption data over time and in various contextual factors. It uses real-time consumption dynamics obtained from the aggregation of current pricing information, as well as long-term trends and periodic patterns inherent in historical consumption data, as primary inputs to predict the demand for each type of dish within a preset future time period. Furthermore, based on this prediction result, dynamic meal preparation suggestions are generated to guide the kitchen in planning the quantity and timing of dish production to address the predicted demand gap. Simultaneously, based on the number of already taken dishes counted in the pricing information, the inventory level of clean dishes at the front desk is assessed in real time. When the inventory falls below a preset safety threshold, a replenishment scheduling instruction is generated to guide the replenishment of dishes from the storage or cleaning area to the take-out area. This step transforms real-time consumption data into forward-looking production and material scheduling instructions through the prediction model.

[0032] In step S104, the production control interface is a software interface used for issuing instructions and communicating status with automated production equipment or production management terminals in the kitchen; the logistics scheduling interface is an interface used for task interaction with logistics robots, conveyor belt systems, or material management platforms within the restaurant. By calling the production control interface, production task parameters included in dynamic meal preparation suggestions can be sent to the corresponding equipment or workstations; by calling the logistics scheduling interface, supplementary scheduling instructions can be converted into specific material handling tasks. The execution of resource allocation drives relevant equipment or notifies relevant personnel to complete the above tasks. The tray recycling status data is collected in real time by sensors deployed at recycling points, and includes at least the unique identification information of the recycled tray, the recycling timestamp, and the recycling location information, thereby marking the endpoint of the tray circulation. It should be noted that the collection of tray recycling status data is a continuously running automated process independent of the execution of resource allocation instructions, and its data is updated in real time in the system and can be queried and accessed by other modules at any time.

[0033] In step S105, the visual analysis of food waste triggers high-definition image acquisition of the corresponding recycling point based on the plate recycling status data. Preferably, after acquiring the surface image of the recycled plate, computer vision technology is used to identify and distinguish different types of food residue areas in the image, and to estimate the proportion of each type of residue. It is important to note that the target category (i.e., food ingredient) for residue identification is consistent with the food type system defined in the food collection process. For example, the identified "braised pork ingredient" corresponds to the food type "braised pork." The food waste type refers to the food category to which the identified residue belongs, and the waste amount is a quantitative estimate of the corresponding category of residue. The resulting waste report is a structured data record. This step transforms plate recycling behavior into quantifiable waste data.

[0034] In step S106, outputting the waste report means distributing or presenting the structured data record through a preset channel, such as sending it to a management monitoring terminal, generating statistical reports, or associating it with the corresponding consumption records.

[0035] This embodiment achieves accurate pricing by fusing multi-source sensing data and combines real-time consumption information with historical data. It utilizes a spatiotemporal prediction model to generate forward-looking production and material scheduling instructions, driving resource allocation through a standard interface. Simultaneously, it collects and recovers data in a closed loop and analyzes waste. This method constructs an intelligent decision-making system driven by real-time data, capable of proactive prediction, dynamic scheduling, and closed-loop feedback. It achieves a leap from passive response to proactive planning, improving operational efficiency and resource utilization.

[0036] Please see Figure 2In some embodiments, multimodal fusion recognition is performed on multi-source sensing data to generate bound pricing information including the type of dish taken, the portion size, and the corresponding plate identifier, including: S201. Based on the unique identifier information of the plate, retrieve the set of candidate dish types associated with the unique identifier information of the plate from the preset plate-dish association mapping table; S202. Perform food target detection and feature extraction on the image data of the food pick-up area to obtain image recognition results. The image recognition results include the types of food identified in the image and the corresponding image recognition confidence. S203. Perform differential calculation on the plate weight change data to obtain the net weight increase data caused by the food taking operation; S204. Based on the candidate dish type set, filter the image recognition results and determine the dish types that belong to the candidate dish type set as the preliminary recognized dishes. S205. Input the preliminary identification of dishes, image recognition confidence level and net weight gain data into the preset fusion decision model; S206. The fusion decision model outputs the final dish type determination result and dish portion determination result based on the relationship between image recognition confidence and preset confidence threshold, and the relationship between net weight gain data and preset standard weight range of dishes. S207. Associate and bind the dish type determination result, dish portion determination result, and plate unique identifier information, and calculate and generate the bound pricing information based on the preset pricing rules.

[0037] In step S201, the preset plate-dish association mapping table is a data table pre-stored in the system, which establishes the correspondence between the unique identifier of the plate and the types of dishes that may be served on it. This plate-dish association mapping table can be constructed and updated based on the plate's design model, color, serving line or window, and statistical analysis results of historical consumption data. For example, plates of a specific color may be specifically used for vegetarian dishes, or a certain serving line may exclusively offer noodle dishes. By retrieving this mapping table to obtain a set of candidate dish types, a reasonable search range is defined for subsequent image recognition, effectively eliminating impossible dish options and improving recognition efficiency and targeting.

[0038] In step S202, food target detection and feature extraction are performed on the image data of the food pick-up area. This can be achieved using deep learning target detection algorithms based on convolutional neural networks, such as YOLO or SSD networks. These algorithms can perform end-to-end image processing, directly outputting the bounding boxes of each detected food target in the image, the type of food they belong to, and an image recognition confidence score that represents the reliability of the identification. This image recognition confidence score is typically a value between 0 and 1; a higher value indicates that the algorithm is more confident in identifying the type of food. This step realizes the transformation from the raw image to structured recognition information.

[0039] In step S203, differential calculations are performed on the plate weight change data to accurately separate the weight of the food added by a single food-taking operation. Specifically, readings from the weighing sensor are continuously collected, and a filtering algorithm (such as moving average filtering) is used to smooth out instantaneous fluctuations caused by slight plate shaking or sensor noise. When a sustained and significant weight change is detected and eventually tends to a new stable value, the stabilized weight value is recorded. The net weight increase caused by the food-taking operation is the difference between the current stabilized weight value and the previous stable weight value. This step accurately obtains the actual weight of the food taken by the user each time.

[0040] In step S204, the image recognition results are filtered based on the candidate dish type set, which is a verification process based on prior knowledge. The dish types output in the image recognition results are compared with the candidate dish type set retrieved from the plate-dish association mapping table. Only when the identified dish type belongs to the candidate set is it retained and determined as a preliminary identified dish; if the identified dish type is not in the candidate set, the image recognition result is considered unreliable or contains interference and is discarded. This step uses the constraints of the physical world (the fixed pairing relationship between plates and dishes) to correct errors that may occur in pure visual recognition.

[0041] In steps S205 and S206, a pre-defined fusion decision model is used to integrate visual recognition and weight information to make a final determination of the dish type and portion size. The standard weight range for each dish type is a pre-defined weight interval, which can be obtained by statistically analyzing a large number of standard portion weight samples of that dish, for example, by taking the average plus or minus twice the standard deviation as the range boundary. The fusion decision model is used to evaluate the confidence level of image recognition and determine whether the net weight gain falls within the standard weight range corresponding to the initially identified dish. The final dish type determination result is output by analyzing the consistency between these two sets of evidence. Preferably, for the portion size determination result, if the net weight gain is within the standard weight range, the standard portion value can be directly adopted or finely adjusted based on the weight data; if it exceeds the range, it may mean that the portion size is non-standard, and the net weight gain can be directly used as the portion size result. This step overcomes the limitations of a single sensor modality by fusing heterogeneous information, improving the fault tolerance and determination accuracy of the recognition system.

[0042] In step S207, associating the dish type determination result and the dish portion determination result with the unique plate identifier information means creating or updating a record in the database or memory data structure, using the unique plate identifier information as the index key. This record completely stores the dish type and portion size identified this time. Preset pricing rules define how to calculate the amount based on the dish type and portion size. For example, for dishes priced by weight, the amount equals the portion size multiplied by the unit price per unit weight; for dishes priced by serving, a fixed price is directly applied. The consumption amount is calculated according to this rule and encapsulated together with the dish type, portion size, and plate identifier to generate complete bound pricing information.

[0043] This embodiment introduces a plate-dish association mapping table to impose prior constraints on the recognition range. It also constructs a hierarchical multimodal fusion recognition process through deep learning-based target detection, weight difference calculation, and fusion decision model. First, it uses physical association relationships to filter out erroneous visual recognition, and then makes a decision by combining visual confidence and weight compliance as dual evidence. This effectively solves the recognition problems caused by similar appearance of dishes, changes in lighting, and overlapping placement in the canteen environment, as well as the weight measurement interference caused by non-standard food picking actions. Thus, it provides a highly reliable and accurate basis for dish recognition and pricing for the intelligent plate system.

[0044] In some embodiments, the fusion decision model outputs the final dish type determination result and dish portion determination result based on the relationship between image recognition confidence and a preset confidence threshold, and the relationship between net weight gain data and a preset standard weight range for dishes, including: Determine whether the image recognition confidence level is greater than or equal to the first confidence threshold; If the image recognition confidence level is greater than or equal to the first confidence threshold, the initially identified dish will be directly determined as the dish type determination result. If the image recognition confidence level is less than the first confidence threshold but greater than or equal to the second confidence threshold, then a weighted fusion decision process is executed, including: The weight of weight evidence is determined based on the range of standard weights of the dishes that the net weight gain data falls into, and the weight of image evidence is determined based on the confidence level of image recognition. Based on the weight of weighted evidence and the weight of image evidence, a weighted decision is made on the initially identified dishes, and the dish type determination result is output. If the image recognition confidence level is less than the second confidence level threshold, the manual review process is triggered, and the dish type confirmed by the manual review is taken as the dish type determination result. After determining the dish type, query the standard weight range of the dish corresponding to the dish type determination result; Compare the net weight gain data with the standard weight range of the dishes; If the net weight gain data is within the standard weight range of the dish, the median of the standard weight range or the preset standard portion will be used as the result of the dish portion determination. If the net weight gain exceeds the standard weight range of the dish, the net weight gain will be used directly as the result of the dish portion size determination.

[0045] In this embodiment, the first confidence threshold and the second confidence threshold are two predefined values, with the first confidence threshold being higher than the second confidence threshold. These two thresholds are set based on the performance analysis of the image recognition model on historical verification datasets. For example, by analyzing the correlation curve between the model's output confidence and the actual recognition accuracy, an inflection point value that can distinguish between high-reliability recognition, medium-reliability recognition, and low-reliability recognition states is selected as the threshold. By setting these two thresholds, the reliability of the image recognition results is divided into three levels: high, medium, and low, and differentiated decision-making paths are adopted accordingly.

[0046] When the image recognition confidence level is greater than or equal to the first confidence threshold, it indicates that the visual recognition result has high reliability. Therefore, the preliminary identification of the dish is directly adopted as the final dish type determination result, making the decision-making process efficient and direct.

[0047] If the image recognition confidence level falls between the first and second confidence thresholds, the process proceeds to the weighted fusion decision-making stage. In this process, the weight evidence weight is determined based on the proximity of the net weight gain data to the center value of its corresponding standard weight range for the dish; the closer the net weight gain data is to the center of the range, the higher the weight. The image evidence weight is obtained from the image recognition confidence level through a linear or non-linear mapping; the higher the confidence level, the greater the weight. After obtaining the two evidence weights, a weighted decision is made based on these weights for the initially identified dish, for example, by calculating a weighted comprehensive score, outputting the final dish type determination result. This process introduces physical evidence, such as weight, to supplement and correct for insufficient visual evidence.

[0048] When the image recognition confidence level is less than the second confidence level threshold, it indicates that the current visual recognition result is extremely unreliable. The manual review process is automatically triggered, and an information package containing the image of the suspicious plate, the preliminary recognition result, and the net weight gain data is pushed to the preset manual review terminal. After review by the staff, the type of dish is determined, and the result of manual confirmation is sent back as the dish type determination result, thus providing a final guarantee when the full automation fails.

[0049] After determining the dish type, the standard weight range corresponding to that dish type is queried to determine the portion size. The median of the standard weight range is the arithmetic mean of the upper and lower limits of that range, and the preset standard portion size is the standard weight per serving defined by the restaurant for that dish.

[0050] The net weight gain data is compared with this range. If the net weight gain data is within this range, the portion size is considered to meet the standard, and the median of the standard weight range of the dish or the preset standard portion size is used as the result of the portion size judgment. If the net weight gain data exceeds this range, the portion size is considered to be abnormal. In this case, the net weight gain data is directly used as the result of the portion size judgment to reflect the actual portion size.

[0051] This embodiment constructs a hierarchical and progressive intelligent decision-making framework by setting multiple confidence thresholds: in the high-confidence range, visual results are trusted to ensure efficiency; in the medium-confidence range, weighted evidence is introduced for weighted fusion to improve accuracy; and in the low-confidence range, manual review is enabled to ensure reliability. Simultaneously, the component judgment logic takes into account the differences between standard and actual operations. This embodiment enables the fusion decision model to flexibly adapt to input data of varying quality, maximizing automation while effectively controlling the risk of misidentification and improving recognition and decision-making capabilities.

[0052] In some embodiments, based on the bound pricing information and historical consumption data, a spatiotemporal prediction model is used to generate dynamic meal preparation suggestions for kitchen production resources and replenishment scheduling instructions for front-of-house tray inventory, including: Extract the real-time consumption quantity of each dish type within the current statistical period from the bound pricing information; Retrieve the historical consumption quantity sequence of each type of dish within a historical period that has the same time attribute as the current moment from historical consumption data; The real-time consumption quantity, historical consumption quantity sequence, and preset external environmental factor data are all input into the pre-trained spatiotemporal prediction model. The spatiotemporal prediction model predicts the demand for each type of dish within a preset time period based on real-time consumption, historical consumption sequences, and external environmental factors. The production gap for each dish type is calculated by comparing the demand forecast with the real-time inventory of each dish type in the kitchen. Based on the production gap and the preset single production batch capacity, dynamic meal preparation suggestions are generated, including the type of dishes to be produced, the quantity to be produced, and the suggested production start time. At the same time, based on the number of plates that have been picked up in the bundled pricing information and the preset front-desk plate inventory safety threshold, it is determined whether the front-desk plate inventory is lower than the front-desk plate inventory safety threshold. If it is determined that the number of plates at the front desk is lower than the safe threshold for plate inventory, a replenishment scheduling instruction is generated based on the quantity and location of cleanable plates in the plate recycling status data to guide the replenishment path and quantity of plates.

[0053] In this embodiment, the current statistical period is a preset time window used to aggregate real-time consumption data. The real-time consumption quantity is extracted from the bound pricing information; that is, all successfully generated bound pricing information records within this period are grouped according to dish type and their portion size determination results are accumulated to obtain the total consumption of each dish in the current time period.

[0054] Historical consumption data stores the consumption quantities by dish type for each statistical period in the past. Retrieving historical periods with the same time attributes as the current moment means finding those periods in history that have the same day of the week, the same meal time (breakfast, lunch, or dinner), and similar time periods as the current moment, and extracting the historical consumption quantities of each dish type within these periods, arranging them in chronological order to form a historical consumption quantity sequence.

[0055] The pre-set external environmental factor data includes environmental information that may affect canteen customer flow and consumption preferences, such as weather conditions, temperature, date type (weekday, weekend, holiday), and whether there are special events or activities. This data is obtained through access to meteorological service interfaces, institutional scheduling systems, or manually set by administrators, and is encoded into numerical or categorical features that the model can process.

[0056] Pre-trained spatiotemporal prediction models are machine learning models capable of learning complex temporal dependencies and external factors in time series data, such as recurrent neural networks or attention-based time series models. The training process for a spatiotemporal prediction model includes: collecting historical consumption quantity sequences for each statistical period within a historical timeframe, external environmental factor data, and actual consumption quantities after a predetermined future timeframe as the true value label, forming a training sample set; concatenating or fusing the historical consumption quantity sequences and external environmental factor data to form the input feature vector for each training sample; using this training sample set, with the input feature vector as the model input and the corresponding future actual consumption quantity as the supervision label, calculating the error between the model's predicted value and the true label using a loss function commonly used in time series prediction (such as mean squared error); iteratively adjusting the model's internal weight parameters through backpropagation and a gradient descent optimizer (such as the Adam optimizer) to continuously minimize the prediction error until the model's prediction performance on the validation set reaches a predetermined standard, thus completing the training and enabling the model to learn the inherent patterns of future demand inference from historical sequences and external factors. The trained model receives real-time consumption data, historical consumption data sequences, and current external environmental factor data, and outputs a predicted demand for each type of dish within a preset future time period.

[0057] The predicted demand is compared with the real-time inventory of each dish type obtained by the kitchen through the inventory management system. The production gap is calculated by subtracting the real-time inventory from the predicted demand. A positive result indicates production demand, while a negative or zero result indicates sufficient inventory. The preset single production batch capacity refers to the number of standard dishes that the kitchen's cooking equipment can produce in one operation. Based on the production gap and the single production batch capacity, the production quantity in the dynamic meal preparation suggestion is determined as the minimum value of the production gap rounded up to an integer multiple of the single production batch capacity. The suggested production start time is calculated backwards from the cooking time required for each dish to ensure that production is completed before the predicted demand arrives.

[0058] Simultaneously, based on the number of plates already taken as recorded in the pricing information, the number of plates currently being used or taken but not yet used can be estimated. Combined with the initial total number of plates at the front desk, the front desk plate inventory can be determined. This estimation is essentially a dynamic calculation process: a pool of usable plates is maintained in real time, equal to the initial total number of plates at the front desk, plus the number of plates that have been collected and cleaned from the plate collection status data, and then the number of plates taken but not yet collected as recorded in the pricing information is subtracted. The preset front desk plate inventory safety threshold is an empirical value used to identify the inventory level at which a replenishment alert needs to be triggered. If the front desk plate inventory is determined to be below this safety threshold, a replenishment scheduling instruction is generated based on the number of plates marked as clean and usable in the plate collection status data and their physical location (e.g., dishwashing area, storage area). This instruction needs to calculate the number of plates that need to be replenished and plan a delivery route from the plate surplus area to the insufficient inventory area based on location information.

[0059] This embodiment establishes a complete decision-making chain, from real-time consumption data aggregation, historical pattern retrieval, and integration of environmental factors to predictive model calculations, and finally to generating specific instructions based on inventory and capacity constraints. This enables precise and forward-looking scheduling of kitchen production and front-of-house material supply. This embodiment transforms restaurant operations from vague management relying on human experience to scientific decision-making based on data and algorithms. It effectively mitigates supply and demand fluctuations, reduces food waste caused by inaccurate food preparation and service interruptions due to plate shortages, and significantly improves restaurant operational efficiency and resource utilization.

[0060] In some embodiments, the spatiotemporal prediction model predicts the demand for each type of dish within a preset future time period based on real-time consumption volume, historical consumption volume sequences, and external environmental factor data, including: Periodic features are extracted from historical consumption sequences to obtain time-series feature vectors that reflect the long-term trend and periodic pattern of food consumption. Short-term fluctuation features are extracted from the real-time consumption quantity to obtain a real-time feature vector reflecting the current consumption rate and the degree of deviation. Encode the external environmental factor data, which includes at least weather conditions, date type, and promotional activity information, and convert the encoded external environmental factor data into an environmental feature vector. The time-series feature vector, real-time feature vector, and environmental feature vector are concatenated to form a fused feature vector; The fused feature vector is input into the feature encoding layer in the spatiotemporal prediction model. The feature encoding layer performs nonlinear transformation and feature dimensionality reduction on the fused feature vector and outputs a deep feature representation. The deep feature representation is input into the sequence prediction layer in the spatiotemporal prediction model. The sequence prediction layer performs weighted aggregation of features at different time steps in the deep feature representation based on the attention mechanism, and outputs a sequence of demand quantities for multiple consecutive preset time periods in the future. Extract the predicted demand quantity for each of the future preset time periods from the demand quantity sequence of multiple consecutive preset time periods.

[0061] In this embodiment, periodic features are extracted from the historical consumption sequence to quantify the inherent long-term trends and repetitive patterns in food consumption. Preferably, time series analysis methods can be applied to the historical consumption sequence, such as calculating its moving average at different time scales (e.g., daily, weekly) to capture trend components, extracting the amplitude and phase corresponding to the main periodic frequencies through Fourier transform to characterize the periodicity, or fitting a seasonal autoregressive model and obtaining its coefficients. Combining these calculated feature values ​​constitutes a time-series feature vector reflecting long-term patterns.

[0062] Extracting short-term fluctuation characteristics from real-time consumption focuses on consumption dynamics within the most recent time window. By analyzing a short series comprising a few recent periods, including the current statistical period, the mean of this short series can be calculated to reflect the current consumption rate, its variance to measure the degree of fluctuation, and the difference or ratio between the mean of this short series and the corresponding historical average consumption for the same period can be calculated to quantify the degree of deviation from the normal level. Combining these short-term statistics yields a real-time feature vector representing the immediate state.

[0063] External environmental factor data includes at least weather conditions, date types, and promotional activity information. This data is encoded; for example, weather conditions (e.g., sunny, rainy, cloudy, snowy) are converted into a multi-dimensional sparse vector using one-hot encoding; date types (e.g., day of the week, holiday status) are encoded using sine / cosine cyclic encoding to preserve their cyclical nature; and promotional activity information is represented by binary variables (0 for none, 1 for present) or specific discount rates. The encoded data from each category is then concatenated or further transformed using a fully connected layer to convert it into a unified-dimensional environmental feature vector.

[0064] The time-series feature vector, real-time feature vector, and environmental feature vector are concatenated along the feature dimension to form a fused feature vector that includes long-term patterns, short-term dynamics, and external condition information.

[0065] In spatiotemporal prediction models, the feature encoding layer typically consists of one or more fully connected neural networks, each containing linear transformations and nonlinear activation functions (such as ReLU). After the fused feature vectors are input to the feature encoding layer, they undergo nonlinear transformations and potential dimensionality reduction operations. The original features are mapped to a more representative and discriminative high- or low-dimensional semantic space, and the output is a deep feature representation, achieving high-order fusion and abstraction of multi-source heterogeneous features.

[0066] The sequence prediction layer in the spatiotemporal prediction model is built on an attention mechanism, enabling the model to dynamically evaluate the importance of information at each historical time step in the deep feature representation when predicting future demand. Preferably, by calculating the similarity between a set of query vectors (usually related to the current prediction target) and a set of key vectors (derived from the deep feature representation), the similarity scores are normalized into weights. These weights are then used to perform a weighted summation of the corresponding value vectors (also derived from the deep feature representation), thereby generating a context vector focusing on key historical information. Based on this context vector, the sequence prediction layer decodes and outputs a sequence of demand quantities for multiple consecutive preset time periods in the future.

[0067] Extracting the predicted demand quantity corresponding to the future preset time period from the sequence of demand quantities for multiple consecutive preset time periods in the future refers to selecting the value at the corresponding time index position from the sequence output by the model as the final prediction result based on the specific future time period of interest in the actual scheduling needs.

[0068] This embodiment employs systematic feature engineering to separate long-term periodic and short-term volatility features from historical sequences and integrates them with structured environmental features, providing the model with comprehensive and high-quality information input. Furthermore, deep feature learning is performed through a feature encoding layer, and an attention mechanism in the sequence prediction layer is used to achieve adaptive historical information focusing, constructing a well-structured and functionally defined deep learning prediction architecture. This embodiment significantly enhances the spatiotemporal prediction model's understanding and generalization ability of complex catering consumption time-series patterns, enabling it to predict future demand more accurately and robustly, thus providing solid and reliable data support for subsequent dynamic scheduling decisions.

[0069] In some embodiments, based on dynamic meal preparation suggestions and replenishment scheduling instructions, the corresponding production control interface and logistics scheduling interface are invoked to execute resource allocation, and real-time data on the status of meal tray collection is collected, including: Analyze dynamic meal preparation suggestions to extract the types of dishes to be produced, the quantities to be produced, and the suggested production start time; Match the corresponding automated kitchen production equipment or manual workstation terminal according to the type of dish to be produced; By calling the production control interface associated with the automated production equipment or manual workstation in the back kitchen, the production instruction containing the quantity to be produced and the suggested production start time is sent to the automated production equipment or manual workstation in the back kitchen. Parse the supplementary scheduling instructions to extract the path and quantity of the trays to be replenished. Based on the guide tray replenishment route, match the corresponding logistics robot or material scheduling system; By calling the logistics scheduling interface associated with the logistics robot or material scheduling system, the delivery instructions containing the route and quantity of the replenished plates are sent to the logistics robot or material scheduling system. During the execution of production and delivery orders, the system continuously receives tray recycling status data, which includes at least the unique identifier of the recycled tray, the recycling timestamp, and the recycling location information.

[0070] In this embodiment, dynamic meal preparation suggestions and replenishment scheduling instructions are typically encapsulated and transmitted in structured data formats such as JSON or XML. The parsing process is implemented by calling the corresponding data parsing library (such as a JSON parser) to extract predefined key field values ​​from the instruction data packet, thereby obtaining specific parameters such as the type of dish to be produced, the quantity to be produced, the suggested production start time, and the path and quantity of guide plate replenishment.

[0071] Preferably, the kitchen's automated production equipment or manual workstation terminals are matched based on a preset mapping table between dish types and execution terminals. This mapping table records the responsible cooking equipment (such as a smart wok or steamer with a specific number) or the identifier of the manual workstation terminal responsible for preparing each dish type. By querying this table, the abstract dish type is converted into the physical or logical terminal address of a specific executable production task.

[0072] The production control interface is a pre-agreed communication contract with the kitchen equipment or workstation terminal software system, such as a RESTful API based on the HTTP protocol. When this interface is called, the extracted parameters, such as the quantity to be produced and the suggested production start time, are encapsulated into a specific request message (such as the payload of an HTTP POST request) according to the interface specification and sent to the network address of the target device or terminal. After receiving the message, the device or terminal parses and executes the production instruction.

[0073] The logistics scheduling interface is a pre-agreed task delivery interface with the logistics robot control system or material scheduling system. When this interface is called, the path and quantity parameters for replenishing the trays are encapsulated in a task format recognizable by the target system (such as a JSON object containing a sequence of target location coordinates and the quantity of items to be transported), and sent through this interface. The logistics system then generates a specific sequence of movement and grasping instructions based on this.

[0074] The continuous reception of tray return status data is achieved by listening to message streams automatically published by sensors (such as RFID readers) deployed at the return points when a tray is detected being placed in the bin. Each published message contains at least the unique identifier of the identified tray, a timestamp of the detection, and the return point number. The system receives and parses these messages, converting them into structured tray return status data records, where the return location information is mapped from the return point number.

[0075] This embodiment constructs a reliable bridge connecting the intelligent scheduling decision-making layer and the physical device execution layer through standardized data parsing, configuration mapping-based terminal addressing, and unified interface calls. It transforms abstract scheduling suggestions into specific instructions that the devices can understand and execute, ensuring the accurate implementation of decisions. At the same time, by establishing a real-time, automated data collection channel for dish recycling, it provides a crucial feedback information flow for the entire system, enabling the complete business process from dish collection and scheduling to recycling to form a closed loop, laying a data foundation for subsequent waste analysis and operational optimization.

[0076] In some embodiments, visual analysis of food waste is performed on the plate recycling status data to generate a waste report that includes the type and amount of food waste, including: Based on the recycling location information in the dish recycling status data, a high-resolution image acquisition device deployed at the corresponding recycling location is triggered to acquire an image of the surface residue on the recycled dish. Instance segmentation processing is performed on surface residue images to identify and segment residual food regions belonging to different categories in the images; Extract visual features from each area of ​​residual food and input these visual features into a pre-trained residue classification model; The food residue classification model is based on visual features and outputs the classification results of the food components for each area of ​​residual food. Based on the classification results of dish ingredients, residual food areas belonging to the same dish type are merged, and the pixel area ratio of the merged area is calculated. Based on the known plate specifications and pixel area ratio of the recycled plates, estimate the amount of waste corresponding to each type of dish. The system correlates the categorization results of food ingredients with the estimated amount of waste and adds a recycling timestamp to generate a structured waste report.

[0077] In this embodiment, a high-resolution image acquisition device deployed at the recycling location, such as an industrial camera with a fixed focal length, captures an image of the surface residue on the recycled tray after receiving a trigger signal. The trigger signal is obtained by parsing the recycling location information from the tray recycling status data, ensuring that image acquisition and tray placement are synchronized in time and space.

[0078] For instance segmentation processing of surface residue images, a deep learning-based instance segmentation model, such as Mask R-CNN, is preferred. This model can output a pixel-level mask for each detected food residue region in the image, along with a preliminary class confidence score, thereby distinguishing different food residues at the pixel level and forming independent food residue regions.

[0079] Visual features of each remaining food area are extracted. Typically, the deep feature map extracted by the backbone network of an instance segmentation model (such as ResNet) at the corresponding image location of that area is converted into a fixed-dimensional feature vector through pooling operations (such as global average pooling). This visual feature vector encodes high-level semantic information such as texture, color, and shape of the area.

[0080] A pre-trained food waste classification model is a machine learning model that takes a visual feature vector as input and outputs the food component category, such as a multilayer perceptron or support vector machine. The training process involves: collecting a large dataset of labeled image patches of food waste regions, each patch corresponding to a known food component (e.g., rice, vegetables, meat); extracting visual feature vectors from each patch and using the corresponding food component as a supervision label; using this dataset, with the visual feature vectors as input and the food component label as output, iteratively adjusting the model parameters using optimization algorithms (e.g., cross-entropy loss and stochastic gradient descent) to enable the model to identify food components from visual features. The trained food waste classification model receives the extracted visual feature vectors and outputs the food component classification result for that food waste region.

[0081] Based on the food composition classification results, the pixel masks corresponding to multiple residual food areas identified by the model as belonging to the same food type are logically ORed and merged to form a single mask area. The pixel area ratio of this merged area is calculated, which is the ratio of its total number of pixels to the total number of pixels in the entire plate image area (which can be predetermined through image segmentation).

[0082] Based on the known specifications of the recycled plates, such as their actual diameter and depth, the amount of waste is estimated by combining the pixel area ratio. Preferably, the conversion relationship between image pixel size and actual size is obtained through camera calibration, and the pixel area is converted into the actual projected area; then, based on the typical accumulation characteristics of this type of food residue or a preset volume-area empirical coefficient, the projected area is estimated as an approximate volume or weight, which is taken as the amount of waste corresponding to this type of food.

[0083] The waste report is generated by associating the food ingredient classification results with the estimated waste amount, organizing them in key-value pairs, and adding the recycling timestamp obtained from the plate recycling status data, all together to form a structured data record. This report can be directly used for database storage or visualization.

[0084] This embodiment constructs a complete automated analysis pipeline for food waste by using triggered image acquisition, deep learning instance segmentation, visual feature-based waste classification, and geometric estimation from pixels to physical quantities. It transforms the difficult-to-quantify post-meal waste phenomenon into structured data that is accurate to the type of food ingredients and estimated weight, realizing an objective, precise, and efficient measurement of food waste. This provides an indispensable data foundation for subsequent in-depth analysis of the causes of waste, optimization of food portion sizes, and formulation of waste reduction strategies.

[0085] In some embodiments, the food residue classification model, based on visual features, outputs a classification result of the food components for each area of ​​residual food, including: Visual features are input into the feature embedding layer of the residue classification model. The feature embedding layer maps the visual features to a high-dimensional semantic space and generates region feature embedding vectors. The similarity between the region feature embedding vector and multiple dish prototype vectors in the predefined dish prototype vector library is calculated. Each dish prototype vector in the dish prototype vector library corresponds to a dish ingredient. Based on the similarity calculation results, select the top few dish prototype vectors with the highest similarity to the region feature embedding vector, and obtain the dish components corresponding to the top few dish prototype vectors as candidate classification results. The region feature embedding vector and the dish prototype vector corresponding to the first few dish prototype vectors are input into the context reasoning module in the waste classification model. The context reasoning module uses the relationship between the region feature embedding vector and the previous several dish prototype vectors, as well as the dish type determination result bound to the recycled plate in the food picking process, as the prior context to reorder and filter the candidate classification results based on their confidence. The dish ingredient with the highest confidence, processed by the context reasoning module, is output as the dish ingredient classification result.

[0086] In this embodiment, the feature embedding layer of the residue classification model typically consists of one or more fully connected neural network layers, each containing a linear transformation and a nonlinear activation function. After the visual feature vector is input to the feature embedding layer, it is transformed into a high-dimensional semantic space through the nonlinear mapping of this layer. The optimization objective of this space is to make the feature representations of samples of the same class close to each other and the feature representations of samples of different classes far apart. The output is the region feature embedding vector of this region.

[0087] The predefined recipe prototype vector library is constructed synchronously during the training phase of the waste classification model. During training, for each recipe component, visual features of the residual food region corresponding to all training samples are collected; these visual features are input into the feature embedding layer currently being trained to obtain the corresponding region feature embedding vector; the mean or center point of the region feature embedding vectors of all samples of the same recipe component is calculated, and the center point vector is defined as the recipe prototype vector of that recipe component; the set of prototype vectors of all recipe components constitutes the recipe prototype vector library.

[0088] Preferably, a cosine similarity metric is used to calculate the similarity between the region feature embedding vector and multiple dish prototype vectors in the dish prototype vector library. The cosine similarity score between the region feature embedding vector and each dish prototype vector in the library is calculated; a higher score indicates that the two are more similar in semantic space.

[0089] Based on the similarity calculation results, select several dish prototype vectors with the highest similarity scores to the region feature embedding vectors. The number of selected dish prototype vectors is a preset positive integer. The dish components corresponding to these selected dish prototype vectors are listed as candidate classification results.

[0090] The context reasoning module receives the region feature embedding vector, the selected dish prototype vector, and the dish type determination result bound to the returned plate during the food collection process, serving as prior context. This prior context is obtained by querying the system database based on the unique identifier of the plate. The context reasoning module performs inference based on these inputs. For example, the context reasoning module can be a small neural network that learns the correlation strength between the region feature embedding vector and each candidate prototype vector under a given prior context. The context reasoning module outputs an adjusted confidence score for the candidate classification results, achieving confidence re-ranking. Preferably, the context reasoning module can reduce the confidence of candidates with weak logical correlation to the prior context (i.e., the user's already collected food records) while increasing the confidence of candidates with strong correlation. Finally, the module selects the dish component with the highest adjusted confidence.

[0091] This embodiment enhances the semantic meaning of visual features through a feature embedding layer and constructs an interpretable food prototype vector library as a classification benchmark. Furthermore, it introduces dining history as a strong prior context, and uses a contextual reasoning module to perform contextual calibration and optimization on the preliminary classification results based on similarity. This embodiment fully leverages the strong prior association between leftover food in a dining setting and the food the user has taken, effectively overcoming the problem of decreased classification performance when relying solely on visual similarity due to degraded residue morphology and mixed types. It improves the accuracy and robustness of residue component identification, ensuring the generation of highly reliable waste reports.

[0092] In a second aspect, this embodiment also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the method described in the first aspect.

[0093] The computer program involved in this embodiment can be stored in a computer device readable storage medium, which includes, but is not limited to, disks, magnetic tapes, magnetic cards, floppy disks, flash memory, optical disks, optical cards, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM), etc. It also includes other biological, physical, or chemical structures capable of performing similar or equivalent functions to the storage media listed above, such as DNA, RNA, proteins, and other units with information storage capabilities. In specific embodiments, the storage medium involved can be one of the above-mentioned media types or a combination of the above media types. In different embodiments, the computer program involved in the embodiment can be centrally stored in a single medium or distributed across multiple media. The memory containing the computer device readable storage medium can be non-volatile memory or random access memory. These computer device readable storage media can be built into the device or connected to the device involved in the embodiment as an external device or part of an external device. In some embodiments, the memory having a computer device readable storage medium is deployed locally; in other embodiments, the memory may be deployed remotely from the processor, for example, as a network-attached memory accessed via RF circuitry or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), or a suitable combination thereof, as long as computer device access to the memory is enabled. Furthermore, the computer program involved in the embodiments may be stored in plaintext / ciphertext form, or it may be designed as training data, integrated and recombined through model training and implicitly stored in the parameter states of a deep neural network or other machine learning model.

[0094] Please see Figure 3 In a third aspect, this embodiment also provides an electronic device 1, including a memory 11 and a processor 12, wherein the memory 11 is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor 12 to implement the method described in the first aspect.

[0095] The processor described in this embodiment can be implemented by hardware, firmware, software, or a combination thereof. It can be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor. It also includes other physical, biological, or chemical structures that can implement the same or equivalent functions as the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of this application, or any combination of the steps mentioned therein.

[0096] By adopting the above technical solutions, this invention differs from existing technologies and has the following beneficial effects: By receiving and integrating multi-source sensory data such as plate markings, visual images, and weight changes, and using multimodal fusion recognition technology to generate bound pricing information including dish type, dish portion, and corresponding plate markings, the accuracy and automation of dish identification and pricing in complex environments are achieved; Based on the bound pricing information and historical consumption data, a spatiotemporal prediction model is used to proactively predict demand and generate dynamic meal preparation suggestions and replenishment scheduling instructions, transforming real-time consumption data into proactive decisions that drive kitchen production and front-end material scheduling, realizing a shift from passive response to proactive planning; By calling standardized production control interfaces and logistics scheduling interfaces to execute resource allocation and collecting plate recycling status data in real time, the accurate implementation of intelligent decisions and closed-loop data collection of business processes are ensured; Further, visual analysis of the recycling status data is performed on the residue, and wasteful dish components are identified through instance segmentation and a classification model based on prototype learning and contextual reasoning. Combined with geometric estimation, the amount of waste is quantified, and a structured waste report is generated, achieving an objective and precise measurement of food waste. This invention provides a full-link intelligent scheduling method that integrates precise sensing, intelligent pricing, dynamic predictive scheduling, closed-loop feedback, and waste quantification analysis, significantly improving the automation level, resource utilization efficiency, and refined management capabilities of catering operations.

[0097] Finally, it should be noted that although the above embodiments have been described in the text and drawings of this application, this should not limit the scope of patent protection of this application. Any technical solutions that are based on the essential concept of this application and utilize the content described in the text and drawings of this application, resulting in equivalent structural or procedural substitutions or modifications, as well as the direct or indirect application of the technical solutions of the above embodiments to other related technical fields, are all included within the scope of patent protection of this application.

Claims

1. A smart dish scheduling and pricing method, characterized in that, include: Receive multi-source sensing data including unique tray identification information, image data of the food pick-up area, and tray weight change data; The multi-source sensing data is fused and identified in a multimodal manner to generate bound pricing information that includes the type of dish taken, the portion size of the dish, and the corresponding plate identifier; Based on the bound pricing information and historical consumption data, dynamic meal preparation suggestions for kitchen production resources and replenishment scheduling instructions for front-of-house tray inventory are generated through a spatiotemporal prediction model. Based on the dynamic meal preparation suggestions and the supplementary scheduling instructions, the corresponding production control interface and logistics scheduling interface are invoked to execute resource allocation, and the meal tray recycling status data is collected in real time. Visual analysis of food waste is performed on the data on the status of plate recycling to generate a waste report that includes the type and amount of food waste. Output the waste report.

2. The intelligent plate scheduling and pricing method according to claim 1, characterized in that, The multi-source sensing data is fused and identified using a multimodal method to generate bound pricing information that includes the type of dish already taken, the portion size, and the corresponding plate identifier, including: Based on the unique identifier of the plate, a set of candidate dish types associated with the unique identifier of the plate is retrieved from a preset plate-dish association mapping table; The image data of the food pick-up area is subjected to food target detection and feature extraction to obtain image recognition results. The image recognition results include the types of dishes identified in the image and the corresponding image recognition confidence scores. The difference calculation is performed on the plate weight change data to obtain the net weight increase data caused by the food taking operation; Based on the set of candidate dish types, the image recognition results are filtered, and the dish types belonging to the set of candidate dish types are determined as the initially identified dishes; The preliminary identified dishes, the image recognition confidence level, and the net weight gain data are input into a preset fusion decision model; The fusion decision model outputs the final dish type determination result and dish portion determination result based on the relationship between the image recognition confidence level and the preset confidence threshold, and the relationship between the net weight gain data and the preset standard weight range of the dish. The dish type determination result, the dish portion determination result, and the unique identifier of the plate are associated and bound together, and the bound pricing information is calculated and generated based on the preset pricing rules.

3. The intelligent plate scheduling and pricing method according to claim 2, characterized in that, The fusion decision model, based on the relationship between the image recognition confidence level and a preset confidence threshold, and the relationship between the net weight gain data and a preset standard weight range for dishes, outputs the final dish type determination result and dish portion determination result, including: Determine whether the image recognition confidence level is greater than or equal to the first confidence threshold; If the image recognition confidence level is greater than or equal to the first confidence level threshold, then the initially identified dish is directly determined as the dish type determination result; If the image recognition confidence level is less than the first confidence threshold and greater than or equal to the second confidence threshold, then a weighted fusion decision process is performed, including: The weight of the weight evidence is determined based on the net weight gain data falling within the standard weight range of the dish, and the weight of the image evidence is determined based on the image recognition confidence level. Based on the weight of the weight evidence and the weight of the image evidence, a weighted decision is made on the initially identified dish, and the dish type determination result is output. If the image recognition confidence level is less than the second confidence level threshold, a manual review process is triggered, and the dish type confirmed by the manual review is taken as the dish type determination result. After determining the dish type, query the standard weight range of the dish corresponding to the dish type determination result; Compare the net weight gain data with the standard weight range of the dish; If the net weight gain data is within the standard weight range of the dish, then the median of the standard weight range of the dish or the preset standard portion is taken as the result of the portion determination of the dish. If the net weight gain exceeds the standard weight range of the dish, the net weight gain will be used directly as the result of determining the portion size of the dish.

4. The intelligent plate scheduling and pricing method according to claim 1, characterized in that, Based on the aforementioned binding pricing information and historical consumption data, a spatiotemporal prediction model is used to generate dynamic meal preparation suggestions for kitchen production resources and replenishment scheduling instructions for front-of-house tray inventory, including: Extract the real-time consumption quantity of each dish type within the current statistical period from the bound pricing information; From the historical consumption data, retrieve the historical consumption quantity sequence of each type of dish within a historical period that has the same time attribute as the current moment; The real-time consumption quantity, the historical consumption quantity sequence, and the preset external environmental factor data are input together into the pre-trained spatiotemporal prediction model. The spatiotemporal prediction model predicts the demand for each type of dish within a preset future time period based on the real-time consumption quantity, the historical consumption quantity sequence, and the external environmental factor data. The predicted demand quantity is compared with the real-time inventory quantity of each type of dish in the kitchen to calculate the production gap quantity of each type of dish. Based on the production gap and the preset single production batch capacity, a dynamic meal preparation suggestion is generated, which includes the type of dish to be produced, the quantity to be produced, and the suggested production start time. At the same time, based on the number of plates that have been picked up in the bound pricing information and the preset front-desk plate inventory safety threshold, it is determined whether the front-desk plate inventory is lower than the front-desk plate inventory safety threshold. If it is determined that the number of plates at the front desk is lower than the safe threshold for the number of plates at the front desk, then a replenishment scheduling instruction is generated based on the number and location of cleanable plates in the plate recycling status data to guide the replenishment path and quantity of plates.

5. The intelligent plate scheduling and pricing method according to claim 4, characterized in that, The spatiotemporal prediction model, based on the real-time consumption quantity, the historical consumption quantity sequence, and the external environmental factor data, predicts the demand for each type of dish within a preset future time period, including: Periodic features are extracted from the historical consumption sequence to obtain a time-series feature vector that reflects the long-term trend and periodic pattern of food consumption; Short-term fluctuation features are extracted from the real-time consumption quantity to obtain a real-time feature vector reflecting the current consumption rate and the degree of deviation. The external environmental factor data is encoded, and the external environmental factor data includes at least weather conditions, date type, and promotional activity information. The encoded external environmental factor data is then converted into an environmental feature vector. The time-series feature vector, the real-time feature vector, and the environmental feature vector are concatenated to form a fused feature vector; The fused feature vector is input into the feature encoding layer in the spatiotemporal prediction model. The feature encoding layer performs nonlinear transformation and feature dimensionality reduction on the fused feature vector and outputs a deep feature representation. The deep feature representation is input into the sequence prediction layer in the spatiotemporal prediction model. The sequence prediction layer performs weighted aggregation of features at different time steps in the deep feature representation based on an attention mechanism, and outputs a sequence of demand quantities for multiple consecutive preset time periods in the future. Extract the predicted demand quantity corresponding to the future preset time period from the demand quantity sequence of the multiple consecutive preset time periods in the future.

6. The intelligent plate scheduling and pricing method according to claim 4, characterized in that, Based on the dynamic meal preparation suggestions and the replenishment scheduling instructions, the corresponding production control interface and logistics scheduling interface are invoked to execute resource allocation, and real-time data on the status of dish return is collected, including: The dynamic meal preparation suggestions are analyzed to extract the types of dishes to be produced, the quantities to be produced, and the suggested production start time. Based on the type of dish to be produced, match the corresponding automated kitchen production equipment or manual workstation terminal; By calling the production control interface associated with the automated production equipment in the back kitchen or the manual workstation terminal, a production instruction containing the quantity to be produced and the suggested production start time is sent to the automated production equipment in the back kitchen or the manual workstation terminal. Parse the supplementary scheduling instruction and extract the guide plate replenishment path and quantity; Based on the indicated tray replenishment path, match the corresponding logistics robot or material scheduling system; By calling the logistics scheduling interface associated with the logistics robot or the material scheduling system, a delivery instruction containing the guide plate replenishment route and quantity is sent to the logistics robot or the material scheduling system. During the execution of the production and delivery instructions, the tray recycling status data is continuously received. The tray recycling status data includes at least the unique identifier of the recycled tray, the recycling timestamp, and the recycling location information.

7. The intelligent plate scheduling and pricing method according to claim 6, characterized in that, The data on the status of dish recycling is subjected to visual analysis of food waste to generate a waste report that includes the type and amount of food waste, including: Based on the recycling location information in the plate recycling status data, a high-resolution image acquisition device deployed at the corresponding recycling location is triggered to acquire an image of the surface residue on the recycled plate; The surface residue image is segmented to identify and segment the residual food areas belonging to different categories in the image; Visual features of each of the remaining food areas are extracted and input into a pre-trained residue classification model; The residue classification model outputs the food composition classification results for each of the remaining food areas based on the visual features. Based on the classification results of the dish ingredients, the residual food areas belonging to the same dish type are merged, and the pixel area ratio of the merged area is calculated. Based on the known plate specifications of the recycled plates and the pixel area ratio, the amount of waste corresponding to each type of dish is estimated; The waste report is generated by associating the food ingredient classification results with the estimated waste amount and attaching the recycling timestamp.

8. The intelligent plate scheduling and pricing method according to claim 7, characterized in that, The food residue classification model, based on the visual features, outputs a food component classification result for each of the remaining food areas, including: The visual features are input into the feature embedding layer of the residue classification model, and the feature embedding layer maps the visual features to a high-dimensional semantic space to generate a region feature embedding vector. The similarity between the region feature embedding vector and multiple dish prototype vectors in a predefined dish prototype vector library is calculated, where each dish prototype vector in the dish prototype vector library corresponds to a dish ingredient. Based on the similarity calculation results, select the top few dish prototype vectors with the highest similarity to the region feature embedding vector, and obtain the dish ingredients corresponding to the top few dish prototype vectors as candidate classification results; The region feature embedding vector and the dish prototype vector corresponding to the first few dish prototype vectors are input into the context reasoning module of the residue classification model. The context reasoning module uses the association between the region feature embedding vector and the first few dish prototype vectors, as well as the dish type determination result bound to the recycled plate in the food collection process, as a prior context to reorder and filter the candidate classification results based on their confidence. The dish ingredient with the highest confidence after being processed by the context reasoning module is output as the dish ingredient classification result.

9. A computer-readable storage medium storing computer program instructions thereon, characterized in that, The computer program instructions, when executed by a processor, implement the method as described in any one of claims 1 to 8.

10. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1 to 8.

Images