Reinforcement learning-based joint decision method for catering production and dish replenishment frequency

Abstract

This invention discloses a joint decision-making method for food preparation output and replenishment frequency based on reinforcement learning, relating to the field of artificial intelligence technology. Within each decision cycle, the following steps are executed sequentially for each food category in the canteen according to a preset priority order: Step 1, read the operational indicators and global remaining resource indicators of the current food category from the canteen monitoring system, determine the current dominant phase based on the phase confidence distribution, and generate a phase-aware state token; Step 2, trim candidate joint decision actions to the physically feasible domain limited by the global remaining resources; Step 3, extract demand scenario samples from the demand scenario library corresponding to each implicit operational phase based on the phase confidence distribution, perform forward simulation, and issue the final joint decision action to the corresponding stall for execution. This invention effectively solves key problems in existing technologies such as rigid phase division, high online trial-and-error costs, neglect of demand randomness in constraint processing, and resource conflicts among multiple food categories.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a joint decision-making method for meal preparation output and replenishment frequency based on reinforcement learning. Background Technology

[0002] In existing technologies, canteen operation and management largely rely on human experience for decision-making. The head chef or catering manager determines the daily production plan for each meal based on rough statistics of historical diners and personal judgment. Replenishment is then arranged by on-site staff who visually assess remaining food and immediately notify the kitchen. The main drawback of this approach is that decision-making heavily depends on individual experience, lacking a systematic modeling of the dynamic evolution of dining demand over time, making it difficult to flexibly adjust strategies between different operational phases. For example, while the pre-peak dining surge and the post-peak decline may show similar numerical levels in some observed indicators, their corresponding demand trends are diametrically opposed. Decision-making methods based on static thresholds or fixed rules cannot distinguish these dynamic differences.

[0003] With the widespread adoption of data acquisition technologies, some studies have attempted to introduce statistical forecasting methods into canteen operation decisions. Time series models, such as autoregressive moving average models and exponential smoothing methods, have been used to predict the number of diners in future periods, while linear programming or integer programming methods have been used to optimize the allocation of output given demand forecasts. However, these two-stage "predict first, optimize later" methods separate demand forecasting from decision optimization into independent stages, where prediction errors are amplified rather than absorbed during the optimization phase. Furthermore, linear programming methods typically assume a linear relationship between decision variables and constraints, while in actual canteen operations, the rate of food inventory depletion is affected by the nonlinear interactions of multiple factors such as queue length, diners' preferences, and the deterioration of food appearance. The linear assumption is insufficient to characterize this complex dynamic. Summary of the Invention

[0004] The purpose of this invention is to provide a joint decision-making method for meal preparation output and replenishment frequency based on reinforcement learning, which effectively solves key problems in the prior art such as rigid stage division, high cost of online trial and error, neglect of demand randomness in constraint processing, and conflict of resources among multiple categories.

[0005] To address the aforementioned technical problems, this invention provides a joint decision-making method for meal preparation output and replenishment frequency based on reinforcement learning. Within each decision cycle, the following steps are executed sequentially for each dish category in the canteen according to a preset priority order. After completing all steps for each dish category, the resources occupied by the final joint decision action are deducted from the global remaining resources. Subsequent dish categories continue execution using the deducted global remaining resources as constraints. Step 1: Read the operational indicators and global remaining resource indicators of the current dish category from the canteen monitoring system, combine them into an observation vector, input the observation vector into the Hidden Markov Operational Phase Recognition Model for forward recursive update to obtain the phase confidence distribution, determine the current dominant phase based on the phase confidence distribution and generate a phase perception state token. Step 2: Construct a phase conditional encoding sequence based on tokens from multiple recently completed decision cycles. Append the target reward token and the current phase perception state token to the end of the phase conditional encoding sequence and input it into the decision Transformer model. Generate candidate joint decision actions through a causal self-attention mechanism and trim the candidate joint decision actions to the physical feasible domain limited by the global remaining resources. Step 3: Based on the phase confidence distribution, extract demand scenario samples from the demand scenario library corresponding to each implicit operational phase for forward simulation, and count the frequency of food shortage default and the frequency of waiting default. When both default frequencies meet the upper limit of the preset permitted frequency, output the final joint decision action. When any default frequency exceeds the upper limit of the preset permitted frequency, adjust and re-verify, and send the final joint decision action to the corresponding stall for execution.

[0006] Furthermore, the Hidden Markov Operating Phase Recognition Model includes four implicit operating phases: the cooling phase, the warming phase, the peak phase, and the falling phase. It also includes a phase transition probability table between the four implicit operating phases and an observation emission probability table for each implicit operating phase.

[0007] Furthermore, the operational metrics include the current remaining amount of food, real-time food preparation rate, current number of people in the queue, number of unfulfilled reservations, and cumulative food keeping time; the global remaining resource metrics include the available capacity of the kitchen, the amount of raw materials available, and the total number of service units that can be allocated; the operational metrics and the global remaining resource metrics, a total of 8 metrics, are combined into an observation vector.

[0008] Furthermore, the forward recursive update in step 1 is performed as follows: using the forward confidence information retained at the end of the previous decision cycle and the observation vector at the current moment, forward recursive updates are performed on the confidence values ​​of each implicit operational phase, and all updated confidence values ​​are normalized so that their sum is 1, thus obtaining the phase confidence distribution at the current moment; the current dominant phase is the implicit operational phase with the largest confidence value in the phase confidence distribution; the phase perception state token is formed by concatenating the preset phase embedding vector corresponding to the current dominant phase and the observation vector along the feature dimension.

[0009] Furthermore, the phase condition encoding sequence in step 2 is constructed as follows: trace back along the time axis to the most recent N consecutive completed decision cycles, extract the phase perception state tokens generated, the action tokens executed, and the reward tokens acquired in each completed decision cycle, and connect the three tokens one after the other in a fixed order of reward token first, phase perception state token in the middle, and action token last, to form a phase condition encoding sequence with a length of 3N tokens.

[0010] Furthermore, the target reward token is preset by the operations manager based on the daily food shortage tolerance level and waste control level; after the decision Transformer model encodes the superimposed position of each token in the complete input sequence, it is processed layer by layer through multiple causal self-attention blocks. In each causal self-attention block, each token only performs attention association calculation on itself and all tokens in the sequence that are located before it; the output representation of the phase perception state token at the current moment at the corresponding position in the complete sequence is taken, projected to the action space through a linear mapping layer, and candidate joint decision actions are generated.

[0011] Furthermore, the candidate joint decision-making action includes four components: the amount of food preparation increase or decrease, the batch value of replenishment, the interval value of replenishment, and the stall manpower switching instruction. The way to trim it to the physical feasible domain is as follows: trim the amount of food preparation increase or decrease to no more than the current global remaining kitchen available capacity and no more than the upper limit that the current global remaining available raw materials can support; trim the interval value of replenishment to no less than the single batch preparation time of the current dish category; and trim the stall manpower switching instruction to no more than the current global remaining total number of deployable service units.

[0012] Furthermore, in step 3, the method for extracting demand scenario samples from the demand scenario library corresponding to each implicit operating phase is as follows: multiply the confidence value of each implicit operating phase in the phase confidence distribution by the preset total number of samples M and round down to obtain the initial quota of each implicit operating phase. The difference between M and the sum of all initial quotas is added to the quota of the corresponding implicit operating phase one by one in descending order of confidence value until the total quota is exactly M. Then, random sampling with replacement is performed from the demand scenario library corresponding to each implicit operating phase according to their respective final quotas to obtain M demand scenario samples.

[0013] Furthermore, the forward simulation is executed as follows: For each of the M demand scenario samples, the following simulation process is performed one by one. Taking the current remaining amount of dishes as the starting point of the simulation, the simulated supply side is updated according to the increase or decrease in food preparation and the batch value of replenishment in the candidate joint decision actions after pruning. The simulated demand side is consumed according to the number of simulated customers arriving and the number of simulated dishes ordered in the corresponding time step in the current simulated demand scenario sample. The process is advanced step by step until a complete decision cycle is covered. The simulated remaining amount of dishes and the simulated queuing waiting time are recorded for each simulated time step. The food shortage default frequency is the proportion of the number of demand scenario samples with at least one time step where the simulated remaining amount of dishes is lower than the preset food shortage threshold to M. The waiting default frequency is the proportion of the number of demand scenario samples with at least one time step where the simulated queuing waiting time exceeds the preset waiting tolerance limit to M.

[0014] Furthermore, the execution method for adjustment and re-verification in step 3 is as follows: when the frequency of food shortage default exceeds the upper limit of the corresponding preset allowable frequency, the amount of food prepared is increased by 1 preset food preparation step unit and the food replenishment time interval is shortened by 1 preset interval step unit; when the frequency of waiting default exceeds the upper limit of the corresponding preset allowable frequency, the stall manpower switching instruction is adjusted to dispatch 1 service unit; after each adjustment, the trimming operation in step 2 is re-executed to ensure that each component is still within the physical feasible domain limited by the current global remaining resources, and then the forward simulation and default frequency statistics are re-executed; when the number of consecutive adjustments reaches the preset maximum number of iterations and any default frequency still exceeds the upper limit of the corresponding preset allowable frequency, the emergency joint decision action that minimizes the sum of the food shortage default frequency and the waiting default frequency is selected from the preset emergency action set as the final joint decision action for the current dish category.

[0015] The reinforcement learning-based joint decision-making method for meal preparation output and replenishment frequency of the present invention has the following beneficial effects: First, this invention introduces a Hidden Markov Model (HMM) to identify the phased characteristics of canteen operations in real time. Based on multi-dimensional observation indicators, it can automatically infer the current operational phase and output a phase confidence distribution, enabling subsequent decision-making and safety verification processes to perceive dynamic changes in the operational rhythm. Compared to existing technologies that divide operational phases into fixed time windows or rely on manual experience, this invention's phase identification mechanism has adaptive capabilities. Even when the actual dining rhythm deviates from the historical average pattern, it can still accurately capture the essential characteristics of the current operational state, avoiding the decision-making inaccuracies of fixed rules on atypical operating days.

[0016] Secondly, this invention employs a Decision Transformer as the core decision generation model, transforming the reinforcement learning problem into a conditional generation problem based on historical decision sequences. It utilizes a large amount of historical operational logs accumulated in the cafeteria for offline training, eliminating the need for online trial-and-error exploration in a real operational environment. This feature fundamentally eliminates the risk of food waste and service interruptions caused by exploratory actions in traditional online reinforcement learning methods, making reinforcement learning technology practically deployable in scenarios like cafeterias where trial-and-error costs are high and irreversible. Simultaneously, the causal self-attention mechanism ensures strict consistency in the information flow direction during the training and inference phases, preventing future information leaks from interfering with decision quality.

[0017] Third, the opportunity-constrained safety verification layer designed in this invention uses a Monte Carlo sampling method weighted by phase confidence distribution to extract demand scenarios proportionally from each phase scenario library for forward simulation. This enables a probabilistic assessment of the default risk of candidate decision actions on food shortage and waiting time constraints. Compared with deterministic constraint processing methods, this invention allows for the acceptance of low-probability default events within a preset probability tolerance range. While ensuring the bottom line of service safety, it significantly improves resource utilization efficiency and reduces food waste caused by overly conservative decision-making. The iterative adjustment mechanism, combined with the maximum iteration limit and the fallback design of the emergency action set, provides the ability to gradually improve the default level while ensuring the finite-time termination of the decision-making process under extreme conditions. Attached Figure Description

[0018] Figure 1 A comparison diagram of the demand arrival probability density distribution for four operating phases provided in this embodiment of the invention; Figure 2 This is a comparison chart of phase confidence distribution weighted sampling quota allocation provided in an embodiment of the present invention; Figure 3 The causal attention mask matrix diagram of the Decision Transformer provided in the embodiments of the present invention; Figure 4 This is a diagram showing the attention weight distribution of each historical token in the current phase-aware state token pair sequence, provided in an embodiment of the present invention. Detailed Implementation

[0019] The reinforcement learning-based joint decision-making method for meal preparation output and replenishment frequency executes the following steps sequentially for each dish category in the canteen according to a preset priority order within each decision cycle. After completing all steps for each dish category, the resources occupied by the final joint decision action are deducted from the global remaining resources. Subsequent dish categories continue to execute with the deducted global remaining resources as constraints. Step 1: Read the operational indicators and global remaining resource indicators of the current dish category from the canteen monitoring system, combine them into an observation vector, input the observation vector into the Hidden Markov Operational Phase Recognition Model for forward recursive update to obtain the phase confidence distribution, determine the current dominant phase based on the phase confidence distribution and generate a phase perception state token. Step 2: Construct a phase conditional encoding sequence based on tokens from multiple recently completed decision cycles. Append the target reward token and the current phase perception state token to the end of the phase conditional encoding sequence and input it into the decision Transformer model. Generate candidate joint decision actions through a causal self-attention mechanism and trim the candidate joint decision actions to the physical feasible domain limited by the global remaining resources. Step 3: Based on the phase confidence distribution, extract demand scenario samples from the demand scenario library corresponding to each implicit operational phase for forward simulation, and count the frequency of food shortage default and the frequency of waiting default. When both default frequencies meet the upper limit of the preset permitted frequency, output the final joint decision action. When any default frequency exceeds the upper limit of the preset permitted frequency, adjust and re-verify, and send the final joint decision action to the corresponding stall for execution.

[0020] In the daily operation of a canteen, decisions regarding food preparation volume and replenishment frequency directly impact the continuity of food supply and the effectiveness of controlling food waste. Traditional methods rely on the head chef's personal experience for judgment, lacking real-time awareness of changes in operational status. This often leads to food shortages during peak periods and significant waste during off-peak periods. This method treats the canteen operation process as a dynamic system with phased state transitions. By identifying the current operational stage online and combining it with a sequential decision-making model to generate joint actions for food preparation and replenishment, it achieves refined control of supply and demand balance.

[0021] The entire method executes decisions on each category of dishes in the canteen according to a preset priority order within each decision cycle. The priority is determined based on a comprehensive ranking of each dish category's average daily sales, gross profit contribution, and susceptibility to shortages. Categories with high sales volume and high complaint rates regarding shortages are prioritized and receive resource allocation first. After completing all decision steps for each dish category, the kitchen capacity, raw material usage, and service unit quantity occupied by its final joint decision action are deducted from the global remaining resources. This ensures that subsequent dish categories continue to make decisions within the actual available resource boundaries, thereby avoiding duplicate occupation of shared resources among multiple dish categories. The length of the decision cycle can be set according to the actual operational rhythm of the canteen. In a typical implementation, it is set to 15 minutes, meaning that a new round of decision-making is executed for each dish category every 15 minutes. In scenarios with large fluctuations in customer flow, the decision cycle can be shortened to 10 minutes or 5 minutes to improve response sensitivity.

[0022] The specific implementation process of online phase identification and phase-aware state token generation in step 1 is explained below.

[0023] At the start of each decision cycle, the system first collects operational metrics corresponding to the current dish category from the cafeteria monitoring system. These operational metrics include five items: current remaining quantity of the dish, real-time serving rate, current queue size, number of unfulfilled reservations, and cumulative heat preservation time. Current remaining quantity refers to the number of available servings of that dish category remaining in the heat preservation station or preparation container at the current moment, obtained directly from weighing sensors or the kitchen inventory management system. For example, at a certain moment, the current remaining quantity of braised pork is 47 servings. Real-time serving rate indicates the actual sales speed of that dish category within the most recent statistical window, measured in servings per minute. For example, if 12 servings were sold in the last 5 minutes, the real-time serving rate is 2.4 servings per minute. Current queue size refers to the number of customers waiting to pick up their food at the corresponding food stall, obtained through visual headcount counting devices or a queuing system. The number of unfulfilled reservations refers to the number of orders containing this dish category that have been placed through online reservation channels but have not yet been picked up at the restaurant. This number is pushed to the decision-making system in real time by the reservation system. The cumulative heat preservation time of the dishes refers to the continuous heat preservation time of the earliest batch of dishes in the current heat preservation station from when they were taken out of the pot to the current moment, recorded in minutes. This indicator reflects the degree of quality degradation of the dishes and the urgency of them being about to be discarded.

[0024] Meanwhile, the system also reads global remaining resource indicators, which include three items: available kitchen capacity, available raw materials, and the total number of deployable service units. Available kitchen capacity is measured by the maximum number of new portions the kitchen can prepare within the current decision-making cycle. Its value is constrained by factors such as the number of stoves, cooking equipment occupancy, and the number of dishes currently being prepared. For example, at a certain moment, the global remaining available kitchen capacity might be 120 portions. Available raw materials refer to the maximum number of portions that can be supported by the raw materials currently available in the warehouse and preparation area. For example, the current available raw material quantity might be 200 portions. The total number of deployable service units refers to the number of idle service personnel not yet assigned to various stalls. Each service unit represents one person who can perform food preparation and serving work at a stall. After previous dish categories have already allocated and occupied some resources, the global remaining resource indicators read for subsequent dish categories are the deducted remaining values, not the initial total.

[0025] The five operational metrics and three global remaining resource metrics are arranged and concatenated in a fixed order to form an observation vector containing eight components. Before concatenation, each metric is normalized to eliminate dimensional differences. One feasible normalization method is to subtract the mean of each metric over the past 30 business days and then divide by the standard deviation to obtain a standardized value with zero mean and unit variance. Another option is to scale each metric by dividing it by a preset physical upper bound, such as dividing the current number of people in the queue by the maximum designed queue capacity of the stall, so that the values ​​of each component fall within the range of 0 to 1. The choice of normalization method does not affect the execution logic of subsequent steps, and implementers can determine it according to the characteristics of the data.

[0026] Next, the observation vectors are input into the Hidden Markov Model (HMM) for operational phase identification. The reason for using an HMM to identify operational phases is that the cafeteria's operation during a business period exhibits clear phased evolutionary characteristics: initially, customer flow is sparse, representing a slow phase; then, customer flow gradually increases, entering a warming phase; at peak times, customer flow is dense and food preparation pressure is at its highest; finally, customer flow declines, entering a closing phase. The transitions between these four phases exhibit Markov properties, meaning that the current phase depends primarily on the previous phase, rather than on more distant historical data. Furthermore, the system cannot directly observe which phase it is currently in; it can only infer it through indirect indicators such as remaining food, food preparation rate, and queue length. This precisely corresponds to the modeling paradigm in HMMs where the implicit state cannot be directly observed but must be inferred through the observation sequence.

[0027] The Hidden Markov Model for Operational Phase Recognition includes four implicit operational phases: a cooling phase, a warming phase, a peak phase, and a declining phase. The model incorporates two sets of parameters: one is a phase transition probability table between the four implicit operational phases, recording the probability of transitioning from one implicit operational phase to another. For example, the probability of transitioning from a warming phase to a peak phase is 0.35, the probability of remaining in a peak phase is 0.55, and the probability of transitioning from a peak phase to a declining phase is 0.40, etc. The entire transition probability table is a 4x4 matrix, with the sum of elements in each row being 1. The other set is an observation emission probability table corresponding to each implicit operational phase, describing the conditional probability distribution of each component of the observation vector under a given implicit operational phase. In one implementation, the observed emission probability of each implicit operational phase is modeled using a multidimensional Gaussian distribution. Each implicit operational phase corresponds to a set of mean vectors and covariance matrices. For example, in the peak phase, the mean vector has a higher mean for the real-time food preparation rate component and a lower mean for the current food quantity component, while the opposite is true for the slow phase. Both sets of parameters are obtained offline by performing an expectation-maximization algorithm on at least 60 days of operational records. After estimation, the results are embedded in the model for online inference. In another optional implementation, the observed emission probability can also be modeled using a Gaussian mixture distribution to adapt to scenarios where the observed data exhibits a multi-peak distribution. The Gaussian mixture distribution corresponding to each implicit operational phase contains two to three mixture components.

[0028] refer to Figure 1 The horizontal axis represents the number of diners arriving per unit time step, measured in people per minute, ranging from 0 to 65; the vertical axis represents the probability density value. The figure shows four probability density curves, corresponding to the four hidden operational phases defined by the Hidden Markov Operational Phase Recognition Model. Idle Phase The demand distribution follows the mean Standard deviation The Gaussian distribution of the probability density curve has the highest peak value, concentrated in the low-value region at the left end of the horizontal axis, and is plotted as a solid line, reflecting the characteristics of sparse dining traffic and narrow fluctuation range during off-peak hours. Warming phase. The demand distribution parameters are , The curve, drawn as a dashed line, shifts its peak position to the right, representing the transitional state where the flow of people gradually increases before reaching its extreme value as the peak dining period approaches. Peak Phase Using the mean Standard deviation The Gaussian distribution, plotted as a dashed line, shows the peak of the curve to the right of the horizontal axis and the widest distribution width, reflecting the operational characteristics of high demand and significant random fluctuations during peak periods. Decline phase. The distribution parameters are , Plotted as a dotted line, the peak value lies between the warming phase and the idle phase, depicting the transition process after the peak dining period when the flow of people gradually decreases but remains above the idle level. The peak value of each curve is marked with a circle, and an arrowed leader points to the corresponding mean value label. to The degree of separation of the four curves along the horizontal axis visually demonstrates the significant differences in demand patterns between different operational phases. This difference forms the technical basis for establishing scenario libraries for each phase and weighting sampling according to the confidence distribution of each phase in the opportunity-constrained safety verification step. If demand samples from the four phases are mixed for sampling, too many low-demand samples will be introduced in the peak-dominated scenario, leading to an overly optimistic risk assessment; while too many high-demand samples will be introduced in the idle-dominated scenario, leading to a overly conservative decision-making process. Neither approach can accurately assess the constrained default probability of candidate joint decision actions.

[0029] The online inference process employs a forward recursive approach. Specifically, at the start of the first decision cycle, the system uses a preset initial phase distribution as the initial forward confidence information. For example, the initial confidence level is set to 0.70 for the slow phase, 0.20 for the warming phase, 0.05 for the peak phase, and 0.05 for the declining phase. This setting reflects the prior knowledge that business is likely to start in a slow phase. At the start of each subsequent decision cycle, the system updates its data using the forward confidence information retained at the end of the previous decision cycle and the current observation vector. The update process consists of two steps: Step 1 is the prediction step, where the confidence values ​​of each implicit operating phase retained at the end of the previous decision cycle are multiplied by the transition probabilities of the corresponding rows in the phase transition probability table. For each target implicit operating phase, the product results from the four source implicit operating phases are summed to obtain the prediction confidence of each implicit operating phase at the current time. Step 2 is the update step, where the observation vector at the current time is substituted into the observation emission probability distribution corresponding to each implicit operating phase to calculate the likelihood value. Then, the prediction confidence of each implicit operating phase is multiplied by the corresponding likelihood value. Finally, the product results of the four implicit operating phases are divided by the sum of the four product results to make the sum of the four normalized confidence values ​​equal to 1, thus obtaining the phase confidence distribution at the current time.

[0030] Let's take a specific numerical process as an example. Assume the forward confidence information retained at the end of the previous decision cycle is: 0.10 for the cool phase, 0.60 for the warming phase, 0.25 for the peak phase, and 0.05 for the declining phase. After the prediction step, the prediction confidence of each implicit operational phase is obtained by weighted summation based on the phase transition probability table. Assume the calculated results are: 0.06 for the cool phase, 0.38 for the warming phase, 0.47 for the peak phase, and 0.09 for the declining phase. The likelihood values ​​obtained from the observation vector collected at the current moment, after calculation using the emission probability distributions of each observation, are: 0.02 for the cool phase, 0.15 for the warming phase, 0.72 for the peak phase, and 0.08 for the declining phase. Multiplying each predicted confidence level by its corresponding likelihood value yields the following: Cool phase (0.0012), Warming phase (0.0570), Peak phase (0.3384), and Declining phase (0.0072), with a sum of 0.4038. After normalization, the phase confidence distribution at the current moment is approximately: Cool phase (approximately 0.003), Warming phase (approximately 0.141), Peak phase (approximately 0.838), and Declining phase (approximately 0.018). This distribution clearly indicates that the current moment is in the Peak phase with a relatively high confidence level.

[0031] The implicit operating phase with the highest confidence value from the phase confidence distribution is selected as the current dominant phase. In the numerical example above, the peak phase has the highest confidence value of 0.838 among the four implicit operating phases, therefore the peak phase is the current dominant phase. In extreme cases, if two implicit operating phases have equal and simultaneously maximum confidence values, the phase is selected in a fixed priority order: peak phase over warming phase, warming phase over declining phase, and declining phase over sluggish phase. This is because, in cases of ambiguity with comparable confidence values, prioritizing the phase with greater demand pressure can reduce the risk of food shortages.

[0032] After determining the current dominant phase, the system retrieves the preset phase embedding vector corresponding to the current dominant phase. The preset phase embedding vector is a fixed-dimensional real-valued vector, which pre-assigns a numerical representation in the vector space that distinguishes each implicit operational phase. In one implementation, the dimension of the phase embedding vector is set to 16. The embedding vectors of the four implicit operational phases are obtained by random initialization and optimization along with the model parameters during the training phase of the decision Transformer model. After training, they are fixed as constant vectors. In another optional implementation, a one-hot encoding method can also be used, where the first component of the 4-dimensional vector for the cool phase is 1 and the rest are 0, the second component for the warming phase is 1 and the rest are 0, and so on. Although this method has a lower dimension, it can still provide clear phase identification information for the subsequent decision Transformer model.

[0033] The preset phase embedding vector corresponding to the current dominant phase state is concatenated with the observation vector along the feature dimension to form the phase-aware state token for the current moment. Taking an embedding vector dimension of 16 and an observation vector dimension of 8 as an example, the concatenated phase-aware state token is a 24-dimensional real vector, where the first 16 components carry phase identification information and the last 8 components carry the operational status data for the current moment. The advantage of using concatenation instead of addition is that it maintains the integrity of both phase information and operational indicator information, avoiding information confusion caused by dimension alignment and numerical superposition. The phase confidence distribution for the current moment is retained as independent data for use by the opportunity constraint safety layer in step 3, and does not participate in the formation of the phase-aware state token.

[0034] refer to Figure 2 The horizontal axis lists four operating phases. to The vertical axis represents the number of sampling quotas extracted from each phase scenario library, with a total of [number missing]. The figure presents a sample of demand scenarios. Four parallel bar charts illustrate the differences in quota allocation under four typical confidence distribution scenarios, with each group distinguished by different fill textures. The peak-dominant scenario corresponds to the following phase confidence distribution: At this point, the peak phase state 168 sampling quotas were obtained, and the heating phase was improved. 29 were obtained, in the decay phase. 3 were obtained, and the idle phase was... Zero samples were obtained; the vast majority of samples came from the peak scenario library to accurately assess default risk under high load. The confidence distribution for the idle-dominated scenario is as follows: The idle phase receives 120 quotas, the warming phase receives 60, and the other two phases each receive 10. The sampling focus shifts to the low-demand range. The warming-dominated and decay-dominated scenarios are respectively... and Quota allocation is performed based on the confidence distribution, with each region concentrating its majority samples in the scene library corresponding to the dominant phase. Quota calculation follows the formula... ,in For the first The confidence level of each phase state. This represents the total number of samples. If the sum of the quotas after rounding is less than [a certain value], then [the remaining number is not specified]. If the remainders are not equal, then the remainders will be filled in order of confidence level from highest to lowest. Figure 2 The formula box in the upper left corner indicates this quota allocation rule. By comparing the four sets of bar charts, it can be seen that the quota allocation strictly follows the phase confidence distribution of the current period and is dynamically adjusted to ensure that the sampling ratio of each phase scenario in the Monte Carlo simulation is consistent with the uncertainty structure of the actual operating state, avoiding risk misjudgment caused by fixed proportional sampling under extreme phase distributions.

[0035] The specific implementation process of constructing the phase conditional encoding sequence and generating candidate joint decision actions using the decision Transformer model in step 2 is explained below.

[0036] The core idea of ​​Step 2 is to organize the decision-making experience accumulated during cafeteria operations in a sequential manner, and then use a Decision Transformer model to extract the best action that matches the current operational state and expected goals. Unlike traditional reinforcement learning methods, the Decision Transformer model does not perform iterative optimization through value functions or policy gradients. Instead, it transforms the reinforcement learning problem into a conditional sequence generation problem. That is, given a historical reward-state-action sequence and a desired target reward level, the model directly predicts the action to be taken. This sequence modeling approach is particularly suitable for cafeteria meal preparation decision-making scenarios because cafeteria operations do not allow for arbitrary trial and error exploration of different meal preparation strategies online, while historical business records accumulate a wealth of decision-making experience that can be learned from.

[0037] The construction process of the phase conditional encoding sequence is as follows. The system traces back along the time axis to the most recent N consecutive completed decision cycles. The value of N determines the length of the historical context that the model can refer to. In a typical implementation, N is set to 20, meaning that the model refers to the decision history of the most recent 20 decision cycles, approximately 5 hours (calculated with 15 minutes as one decision cycle). This length is sufficient to cover the complete operational cycle evolution from the start of the meal to the peak and then to the decline. In another implementation, if the decision cycle is shorter, such as 5 minutes, N can be increased accordingly to 40 to 60 to cover a considerable time span. For each completed decision cycle, the system extracts three types of tokens: the acquired reward token, the generated phase awareness state token, and the executed action token.

[0038] The reward token is a vector obtained by quantifying and encoding the operational effectiveness of each completed decision cycle. In one implementation, at the end of each decision cycle, the system evaluates the operational effectiveness from four dimensions: the shortage loss index records the cumulative number of time steps during which shortages occurred; the waste loss index records the number of dishes that had to be discarded due to exceeding the heat preservation time limit at the end of the cycle; the waiting loss index records the average waiting time of queuing customers during the cycle; and the manpower switching cost index records whether manpower allocation changes occurred during the cycle and the number of times. The values ​​of the above four dimensions are weighted and summed according to their respective preset coefficients to obtain a scalar form of comprehensive operational benefit value. In one optional setting, 3.0 points are deducted for each time step of shortage loss, 1.5 points are deducted for each discarded dish, 2.0 points are deducted for each minute increase in average waiting time, and 0.5 points are deducted for each manpower switch, with a base score of 100 points. The higher the comprehensive operational benefit value, the better the operational effectiveness of the decision cycle. The overall operating revenue value is projected through a linear mapping layer into a vector with the same dimension as the phase-aware state token, thus obtaining the reward token. The parameters of the linear mapping layer are optimized during the training phase of the decision Transformer model.

[0039] An action token is a vector encoded with the joint decision-making actions actually executed in each completed decision cycle. The joint decision-making action contains four components: the preparation increase / decrease amount, representing the increase or decrease in the number of preparations compared to the base preparation rate within the decision cycle (positive values ​​indicate increased production, negative values ​​indicate decreased production); the replenishment batch value, representing the number of portions of food replenished to the warming station each time; the replenishment time interval value, representing the time interval between two consecutive replenishment operations, in minutes; and the stall manpower switching instruction, representing the increase or decrease in the number of service personnel at the corresponding stall within the decision cycle (positive values ​​indicate increased deployment, 0 values ​​indicate no change). These four components are concatenated into a 4-dimensional vector, which is then projected through another linear mapping layer to form a vector with the same dimension as the phase-aware state token, thus obtaining the action token.

[0040] The three tokens are organized in a fixed order: within each decision cycle, the reward token is placed first, the phase-aware state token is placed in the middle, and the action token is placed last. This arrangement is designed so that placing the reward token first ensures that the state and action tokens in subsequent attention calculations can be directly associated with the reward information of that cycle, thus enabling the perception of the state-action correspondence under different reward levels when generating actions. The three tokens from N completed decision cycles are then concatenated end-to-end to form a token of length [length missing]. A phase conditional encoding sequence of tokens, wherein N represents the number of decision cycles for backtracking. When N is 20, the phase conditional encoding sequence contains a total of 60 tokens.

[0041] At the end of the phase condition encoding sequence, the system sequentially adds two new elements: the target reward token for the current decision cycle and the phase awareness state token for the current moment. The target reward token is a key conditional input guiding the decision-making Transformer model to generate actions, and its function is similar to conveying to the model "what level of operational performance I hope to achieve next." The target reward token is preset by the operations manager based on the daily shortage tolerance level and waste control level. In one implementation, the shortage tolerance level and waste control level are each divided into three levels: strict, moderate, and lenient. The shortage tolerance level is also divided into three levels: strict, moderate, and lenient. These two levels combine to form nine preset configurations. Each configuration corresponds to a pre-defined target comprehensive operational benefit value. For example, strict shortage tolerance and strict waste control correspond to a target value of 92 points, and moderate shortage tolerance and moderate waste control correspond to a target value of 85 points. The target comprehensive operational benefit value is projected through the same linear mapping layer as the reward token to obtain the target reward token. The reward tokens during the training phase and the target reward tokens during the online inference phase share the same linear mapping layer and the same scoring scale system, ensuring that the model can correctly understand the operational performance level represented by the target reward tokens during inference.

[0042] After adding the target reward token and the current phase-aware state token, the length of the complete sequence becomes There are 62 tokens. When N is 20, the complete sequence contains 62 tokens. The structure of the complete sequence is as follows: the reward token for the first decision cycle, the phase-aware state token for the first decision cycle, the action token for the first decision cycle, the reward token for the second decision cycle, and so on up to the action token for the Nth decision cycle, the target reward token for the current decision cycle, and the phase-aware state token for the current moment. The model needs to predict the corresponding action, i.e., the candidate joint decision action, at the position of the phase-aware state token at the current moment.

[0043] The appended complete sequence is then input into a pre-trained decision Transformer model. The network structure of the decision Transformer model consists of a token embedding layer, a position encoding layer, multiple layers of causal self-attention blocks, and an output mapping layer, in sequence.

[0044] The function of the token embedding layer is to unify three different types of tokens into a single representation space. Since the reward token, phase-aware state token, and action token originate from different linear mapping layers, and their numerical ranges and semantic meanings differ, the token embedding layer applies type-specific linear transformations to each of the three tokens, unifying the dimension of the transformed embedding vectors. Dimension. In one implementation method Take 128, of which The dimension of the embedded space.

[0045] The positional encoding layer overlays positional information onto each token in the complete sequence. Since the Transformer architecture itself lacks the ability to perceive the sequential order of elements in a sequence, the role of positional encoding is to enable the model to distinguish tokens at different time steps. In one implementation, a learnable positional encoding method is used, that is, a learnable position is assigned to each position in the complete sequence. The dimensional vector is optimized along with the remaining model parameters during training. Reward tokens, phase-aware state tokens, and action tokens within the same decision cycle share the same time-step position encoding, but each is additionally superimposed with its own type position encoding to distinguish token types. Alternatively, a sine-cosine position encoding scheme can be used, where fixed position encoding values ​​are calculated using sine and cosine functions based on the token's absolute position in the sequence and the component index in the embedding dimension.

[0046] refer to Figure 4 The horizontal axis lists all 11 token positions in the sequence. The vertical axis represents attention weights. The fill depth of each bar chart is differentiated by token type: reward tokens. Status token filled with dark gray Use medium gray fill, action token Light gray fill is used. From the weight distribution, the target reward token... and the state token of the most recent cycle The highest attention weights were obtained, all around 0.200, indicating that the decision-making Transformer relies most heavily on the target return level set by the manager and the operational status information of the most recent historical period when generating the current action. (Action token for Period 3) The weight of the state token in period 3 is 0.120, and the weight of the state token in period 3 is 0.150. Together, they account for a significant proportion, reflecting the model's strong focus on recent decision-making experience. As the historical period is traced back further back, the weights of each token gradually decrease: the weight of the state token in period 2... A state token with a weight of 0.090 and a period of 1. The weight is reduced to 0.065, and the reward tokens for cycle 1 are... The weight is only 0.020. This time decay characteristic is consistent with the actual logic of restaurant operation scenarios, that is, the more operational data is closer to the current moment, the higher its reference value for real-time decision-making. Figure 4 The formula box in the upper right corner provides the expression for calculating the attention weights. ,in and Positions query vector and position The key vector, For embedded dimensions, For the number of attention heads, the range of summation in the denominator. This reflects the constraint effect of causal masking, ensuring that normalization is performed only at causally reachable locations.

[0047] The complete sequence after position encoding enters a multi-layered causal self-attention block for layer-by-layer processing. In one implementation, the number of causal self-attention blocks is set to 6 layers, each containing a multi-head attention sublayer and a feedforward network sublayer, with 8 attention heads. The core computational process in each causal self-attention block is a causal self-attention mechanism: for a position in the sequence... The token, whose attention calculation range is limited to position 1 to position 2. The model uses a causal mask to ensure that each token can only be associated with itself and all tokens preceding it in the sequence. Tokens following it are masked by the causal mask and cannot participate in the computation. This constraint aims to ensure that when the model generates the action at the current moment, it infers based solely on past rewards, states, and actions, without leaking future information, thus guaranteeing that the model's behavior during online inference is consistent with that during training.

[0048] Inside each attention head, position The token embedding is transformed into a query vector through query transformation, key transformation, and value transformation. Key vector Sum value vector ,in For position The query vector, For position The key vector, For position The value vectors, all three vectors have dimensions of 1. ,in The number of heads to focus on. Position. Position Attention score through and The inner product is calculated, then divided by... Scaling is performed to prevent the gradient from vanishing due to excessively large inner product values, where Dimensions for each attention head. When At this point, the causal mask sets the corresponding attention score to negative infinity, causing it to approach 0 in subsequent normalization, thus masking information about future positions. For positions 1 to 1... Normalize all scaled attention scores to sum them to 1, obtaining the attention weight distribution, and then convert the value vectors at each position. The position is obtained by weighting and summing the values ​​according to their corresponding attention weights. The output representation under the current attention head. The output representations of multiple attention heads are concatenated and then linearly transformed back to... The data is processed in a 3D space, then undergoes residual connections and layer normalization before entering the feedforward sub-layer. The feedforward sub-layer consists of two linear transformations sandwiching an activation function layer; in one implementation, the activation function uses Gaussian error linear units. The output of the feedforward sub-layer is also processed through residual connections and layer normalization before becoming the final output of the current causal self-attention block.

[0049] Because the complete sequence contains the embedding vector information of the current dominant phase (embedded in the phase-aware state tokens of each decision cycle), the causal self-attention mechanism, when calculating attention weights, can generate higher attention scores for the current phase-aware state token and tokens carrying similar phase embedding vectors in the historical sequence. This allows for greater utilization of action and reward information related to the current phase evolution trend in previous decision cycles during information aggregation. For example, when currently in a peak phase, the model tends to obtain more reference information from decision cycles in the historical sequence that are also in peak phases, because the operating conditions of these historical cycles are most similar to the current one. This phase-aware information utilization mechanism is not achieved through additional explicit rules, but rather naturally arises from the similarity of the inner product of phase embedding vectors in attention calculation.

[0050] After layer-by-layer processing of all causal self-attention blocks, the output representation of the current phase-aware state token position (i.e., the last position of the sequence) in the complete sequence is taken. The output representation encapsulates comprehensive information from historical decision-making experience, current operational status, and target return level. This output representation is then input into a linear mapping layer and projected onto the action space. The linear mapping layer will... The input is mapped to a 4-dimensional output, with four output components corresponding to the candidate joint decision action: the amount of food prepared (increase or decrease), the batch size for replenishing food, the time interval for replenishing food, and the stall manpower switching instruction. In one embodiment, the output range for the amount of food prepared (increase or decrease) is -30 to +50 units, the output range for the batch size for replenishing food is 5 to 40 units, the output range for the time interval for replenishing food is 3 to 30 minutes, and the output range for the stall manpower switching instruction is 0 to 3 service units. Each component is limited to the above ranges by applying a truncation operation at the output of the linear mapping layer.

[0051] refer to Figure 3 The matrix is The square matrix, with the key positions marked on the horizontal axis. The vertical axis indicates the query location. The corresponding token sequence consists of the reward-state-action triplet from three historical decision-making cycles, plus the target reward token and phase-aware state token for the current cycle, in the following order: ,common 1 token, of which This represents the number of historical cycles. Elements in the lower triangular region (including the diagonal) of the matrix have a value of 1, while elements in the upper triangular region have a value of 0. A value of 1 indicates a position... The query token can be used to monitor location. The key tokens have a value of 0, indicating that attention is blocked. This lower triangular structure implements causal constraints, meaning that each token can only access information about itself and tokens at previous positions when calculating attention weights, strictly prohibiting access to information about future time steps. Three key tokens are marked with dashed boxes in the matrix. The diagonal blocks correspond to the internal token groups of Period 1, Period 2, and Period 3, respectively, labeled "Period 1," "Period 2," and "Period 3," facilitating the identification of the position interval of each historical decision period in the sequence. The causal mask design ensures that the information flow direction of the decision Transformer is completely consistent during the training and inference phases: during training, a negative infinite bias is applied to the attention score through the mask matrix to make the softmax output of the upper triangular position approach zero; during inference, the current phase perceives the state token. Located at the end of the sequence, it can only access all tokens to its left. This mechanism prevents the model from fitting the current action with future actions or future reward information during offline training, thus avoiding the problem of inconsistent distribution between training and inference.

[0052] Training of the decision-making Transformer model is completed offline. Training data comes from complete operational records for at least 90 business days. Each record contains the observation vector for all decision cycles of that business day, the actual joint decision actions executed, and the actual comprehensive operational revenue value obtained. During training, the operational records for each business day are organized into sequence samples according to the aforementioned token construction rules. An autoregressive approach is used for training; that is, the mean squared error loss between the model's predicted action and the actual executed action is calculated at each action token position, and the model parameters are updated through backpropagation. In one implementation, the AdamW optimizer is used, with an initial learning rate set to... The initial learning rate represents the initial step size for parameter updates. The training epochs are set to 200, with each epoch iterating through all training samples once. After training, the model parameters are fixed and deployed to an online decision-making system for real-time inference.

[0053] After generating candidate joint decision actions, the system performs physical feasible region pruning on each component. The purpose of pruning is to ensure that the output of the decision Transformer model does not exceed the actual carrying capacity of the current global remaining resources. The trimming rules are as follows: The increase / decrease in food preparation quantity is trimmed to no more than the current global remaining available kitchen capacity and the upper limit supported by the current global remaining available raw materials. The smaller of these two values ​​is taken as the upper bound for the increase / decrease in food preparation quantity. If a candidate value exceeds this upper bound, it is replaced with the upper bound value. The replenishment interval value is trimmed to no less than the pre-preparation time for a single batch of the current dish category. The pre-preparation time for a single batch refers to the shortest time required from the start of preparation to when the dish is ready to be placed on the warming platform. For example, the pre-preparation time for a single batch of braised dishes is approximately 8 minutes, while for stir-fried dishes it is approximately 4 minutes. If a candidate replenishment interval value is less than the pre-preparation time for a single batch, it is replaced with the pre-preparation time for a single batch to avoid the system's required replenishment frequency exceeding the actual preparation speed of the kitchen. The stall manpower switching instruction is trimmed to no more than the current global remaining total number of deployable service units. If a candidate value exceeds this total, it is replaced with the current global remaining total number of deployable service units. After being pruned, the candidate joint decision actions are guaranteed to be physically executable under the current global remaining resource conditions, and then proceed to the opportunity constraint safety layer in step 3 for probabilistic safety verification.

[0054] In an optional enhanced implementation, the cutting operation can be further incorporated into the food heat preservation time constraint: if the current cumulative heat preservation time of the food is close to the preset maximum heat preservation time limit (e.g., less than 10 minutes away from the maximum heat preservation time limit), the lower bound of the food preparation increase or decrease will be raised to encourage the system to prioritize the consumption of existing inventory rather than continue to increase production, thereby reducing the risk of waste due to heat preservation time exceeding the limit.

[0055] The specific implementation process of performing phase-aware probabilistic safety verification on the pruned candidate joint decision actions through the chance constraint safety layer in step 3 and outputting the final joint decision action is explained below.

[0056] While the trimmed candidate joint decision actions output in Step 2 satisfy physical resource constraints such as kitchen capacity, raw material supply, and manpower allocation, they have not yet undergone a risk assessment of future demand fluctuations. Customer arrival numbers and menu preferences fluctuate randomly within each decision cycle, and even under the same operational phase, actual demand curves can differ significantly across business days. Executing only the deterministic actions output by the decision Transformer model could lead to food shortages in high-demand scenarios or waste due to excessive food preparation and prolonged warming in low-demand scenarios. The opportunity constraint safety layer serves to conduct Monte Carlo simulations of candidate actions across multiple possible demand scenarios, assessing whether the probability of these actions causing food shortages or long queues in the face of demand uncertainty is within acceptable limits. Only actions that pass the probability safety check are allowed to be executed.

[0057] The opportunity-constrained safety layer pre-configures independent demand scenario libraries for four implicit operational phases. The reason for building separate libraries for each phase is that customer arrival patterns and ordering behaviors differ fundamentally across operational phases: in the slow phase, customer arrivals are sparse and orders are concentrated on a few main dishes; in the peak phase, customer arrivals are dense and orders are spread across multiple dish categories; and the warming and declining phases each exhibit transitional demand characteristics. If demand data from all phases were mixed in the same scenario library, extracting low-demand scenarios from the slow phase during a peak phase would lead to overly optimistic risk assessments, while extracting high-demand scenarios from the peak phase during a slow phase would lead to overly conservative decisions. Separate phase-based libraries ensure that the extracted demand scenarios are statistically consistent with the current operational status.

[0058] Each demand scenario library is constructed based on actual demand records from at least 60 business days belonging to the corresponding operational phase decision cycle. Taking the peak phase demand scenario library as an example, the system filters all decision cycles identified as peak phases by the Hidden Markov Model of Operational Phase Recognition from historical operational records, extracts the actual number of customers arriving and the actual number of dishes ordered at each time step within each decision cycle, and stores the complete demand sequence of each decision cycle as a demand scenario sample in the peak phase demand scenario library. In a typical implementation, each decision cycle is divided into 1-minute time steps, and a 15-minute decision cycle corresponds to 15 time steps. Therefore, each demand scenario sample is a sequence containing 15 time steps, where each time step records the number of customers arriving within that minute and the number of dishes ordered for the current dish category. After accumulating over 60 business days, the demand scenario library for each implicit operational phase typically contains hundreds to thousands of demand scenario samples. As the number of business days continues to increase, the demand scenario database can be updated periodically. For example, every 30 business days, new demand records can be added to the database while the earliest batch of samples can be removed to maintain the timeliness of the scenario database in relation to the current business situation. In an optional implementation, a fixed capacity limit, such as 2,000 entries, can be set for the demand scenario database for each implicit operational phase. When the total number of new samples exceeds the limit, the earliest samples added to the database are removed in chronological order.

[0059] At the start of step 3 in each decision cycle, the system first reads the phase confidence distribution for the current moment, which was retained in step 1. The phase confidence distribution contains four confidence values, corresponding to the cooling phase, warming phase, peak phase, and decline phase, respectively. Based on the phase confidence distribution, the system proportionally samples demand scenarios from four demand scenario databases, with the total sample size being a preset limit. One, of which This is the preset total number of samples. The value of needs to be balanced between computational cost and statistical precision: Too small a value will result in a large variance in the estimated frequency of defaults, which may miss the true risks. If the value is too large, it will increase the computation time for each verification. In one implementation method... Set to 200, which can be reduced to 100 in scenarios with higher real-time requirements, and increased to 500 in scenarios with extremely strict security requirements.

[0060] The allocation process for sampling quotas is as follows: The confidence values ​​for each of the four implicit operational phases are multiplied by... Rounded down, we get 4 initial quotas. Taking the numerical example given in step 1, the phase confidence distribution is: cool phase 0.003, warming phase 0.141, peak phase 0.838, and falling phase 0.018. The total number of samples... The initial quotas were 200. heating phase Peak phase Falling phase ,in This indicates a floor operation. The sum of the four initial quotas is 198, which is... The difference is 2. The difference is then added in descending order of confidence level: the peak phase has the highest confidence level, so its quota is increased from 167 to 168; the heating phase has the second highest confidence level, so its quota is increased from 28 to 29. After the addition, the sum of the four quotas is exactly 200. This method of rounding down and then adding the remainder ensures that the total sample size is strictly equal to... At the same time, the quotas obtained by each implicit operating phase should be as proportional as possible to their confidence level values.

[0061] According to their respective final quotas, random sampling with replacement is performed from the four demand scenario databases. Sampling with replacement means that the same demand scenario sample may be selected repeatedly in the same sampling, which is especially necessary when the sample size in some demand scenario databases is small, because sampling without replacement is limited when the quota is close to the database capacity. After sampling, the sampling results of the four demand scenario databases are merged to obtain... Sample demand scenarios.

[0062] Next Each demand scenario sample is subjected to forward simulation. The purpose of forward simulation is to virtually execute a complete decision-making cycle of supply and demand process under each possible demand scenario, according to the pruned candidate joint decision actions, and record the time-step changes of food availability and waiting time, so as to evaluate whether the candidate actions will lead to food shortages or excessively long queues under the demand scenario.

[0063] against For each of the sample demand scenarios, the specific execution process of the forward simulation is as follows. The current remaining quantity of dishes is used as the initial inventory value for the simulation, denoted as... ,in To simulate the number of remaining dishes at the start time, this value is directly taken from the current remaining dishes read from the cafeteria monitoring system in step 1. Assume one complete decision cycle includes... One simulated time step, of which This represents the total number of time steps within the decision cycle. When the decision cycle is 15 minutes and each time step is 1 minute... It equals 15.

[0064] In the In each of the simulated time steps, among which The current simulation time step number, with values ​​ranging from 1 to... The simulation process is divided into two stages: supply-side update and demand-side consumption. In the supply-side update stage, the system determines whether the current time step is a replenishment point. Based on the replenishment interval values ​​in the trimmed candidate joint decision actions, the system determines the replenishment rhythm. If the current time step number is an integer multiple of the replenishment interval value, the system replenishes the inventory with the number of portions corresponding to the replenishment batch value in the trimmed candidate joint decision actions, while simultaneously adjusting the replenishment batch value based on the increase or decrease in food preparation. Specifically, the actual replenished portion number equals the replenishment batch value plus the increase or decrease in food preparation divided by the total number of replenishments within one decision cycle, rounded down. This allocation method evenly distributes the increase or decrease in food preparation across each replenishment. If the current time step is not a replenishment point, the supply-side update is not executed. In the demand-side consumption stage, the system reads the first... For each time step, the number of simulated customers arriving and the number of simulated dishes ordered are recorded. The simulated dish order quantity is then deducted from the current inventory. If the inventory drops below 0 after the deduction, the inventory is set to 0, and the excess is recorded as the unmet demand for that time step.

[0065] Two metrics are recorded after each simulation time step. The first metric is the simulated food inventory, which is the current inventory value after supply-side updates and demand-side consumption are completed. The second metric is the simulated queuing time, calculated as follows: the number of simulated customers arriving at the current time step divided by the actual number of service personnel corresponding to the stall manpower switching instructions in the trimmed candidate joint decision actions, and the service capacity per minute of each service personnel. The service capacity per minute of each service personnel is a preset constant, which is set to 3 people per minute in one implementation. If the current inventory is 0, resulting in the inability to serve food, an additional preset out-of-item waiting penalty time is added to the simulated queuing time, which is set to 5 minutes in one implementation, to reflect the extra waiting time customers need to wait for replenishment or to turn to other stalls when out of food.

[0066] Progressing step by step until the first Each time step is completed, forming a simulated food quantity trajectory corresponding to the current demand scenario sample (including...). Records of remaining food items at each time step and simulated queue waiting time trajectories (including...) (Records of queuing wait times at each time step).

[0067] For all The simulation results of each demand scenario sample are iterated and statistically analyzed to calculate two default frequencies. The statistical rule for the food shortage default frequency is as follows: if the simulated food quantity trajectory of a certain demand scenario sample shows that the simulated food quantity is lower than the preset food shortage threshold at at least one time step, then the demand scenario sample is marked as a food shortage default sample; the total number of all food shortage default samples is divided by... The frequency of defaults due to vegetable shortages is recorded as follows: ,in The frequency of food shortage defaults is defined as follows: In one implementation, the preset food shortage threshold is set to 5 units. This means that when inventory falls below 5 units, it is considered a near-disruption situation, rather than waiting until inventory reaches zero to determine default. This early warning design provides a buffer time for emergency replenishment in the kitchen. The statistical rules for waiting default frequency are similar: if at least one time step in the simulated queuing time trajectory of a certain demand scenario sample exceeds the preset waiting tolerance limit, then that demand scenario sample is marked as a waiting default sample; the total number of waiting default samples is divided by... The frequency of pending defaults is obtained and denoted as . ,in To account for the frequency of defaults. In one implementation, the preset waiting tolerance limit is set at 4 minutes; waiting times exceeding this limit will significantly reduce customer satisfaction.

[0068] Let's take a specific set of numerical values ​​as an example to illustrate the calculation of default frequency. Assume... With a target of 200, after forward simulation of all 200 demand scenario samples, 23 demand scenario samples showed that at least one time step in the simulated food quantity trajectory had a food quantity of less than 5 servings. Therefore, the frequency of food shortage defaults is [not specified]. for If, in the simulated queuing waiting time trajectory of 11 demand scenarios, there is at least one time step where the waiting time exceeds 4 minutes, then the waiting default frequency is... for .

[0069] The system compares the two default frequencies with their respective preset upper bounds for permitted frequencies. The preset upper bound for the vegetable shortage default frequency is denoted as... The upper bound of the preset license frequency corresponding to the waiting default frequency is denoted as ,in This is the upper bound of the preset permitted frequency for the frequency of food shortage defaults. A preset upper bound for the permitted frequency to wait for the default frequency. In one implementation... Set to 0.10. A value of 0.08 means that, in all simulated demand scenarios, a maximum of 10% of scenarios are allowed to experience the risk of food shortages, and a maximum of 8% of scenarios are allowed to experience excessively long queues. Operations managers can set different upper limits for the permitted frequency based on the importance and customer sensitivity of different dish categories; for example, setting stricter limits for signature dishes. Values ​​such as 0.05.

[0070] In the numerical example above, the frequency of default due to lack of vegetables (0.115) exceeds the upper limit of the preset permitted frequency (0.10), while the frequency of default due to waiting (0.055) does not exceed the upper limit of the preset permitted frequency (0.08). Because one default frequency exceeds the corresponding upper limit of the preset permitted frequency, the pruned candidate joint decision action fails the probabilistic safety check and needs to enter the adjustment stage.

[0071] The adjustment and re-verification process is as follows. The system determines the adjustment direction based on the default type exceeding the upper limit of the preset permissible frequency. When the default frequency of food shortage exceeds the corresponding upper limit of the preset permissible frequency, it indicates that the candidate action is too conservative on the supply side, requiring increased food preparation and faster replenishment. Therefore, the food preparation increase / decrease is increased by 1 pre-prepared food preparation step unit, and the replenishment interval is shortened by 1 preset interval step unit. In one implementation, the pre-prepared food preparation step unit is set to 5 portions, and the preset interval step unit is set to 1 minute. Increasing the food preparation increase / decrease can increase the actual number of portions replenished each time, thereby increasing the inventory buffer. Shortening the replenishment interval can increase the replenishment frequency, thereby allowing the inventory to be replenished more promptly. The combined effect of the two can effectively reduce the risk of food shortage. When the default frequency of waiting exceeds the corresponding upper limit of the preset permissible frequency, it indicates that the service capacity of the stall is insufficient to cope with the arrival speed of customers. Therefore, the stall manpower switching instruction is adjusted to add 1 service unit. Each additional service unit means that one more service person is on the stall to perform food preparation and serving work, directly improving the stall's instantaneous service throughput capacity. When the frequency of food shortage default and the frequency of waiting default both exceed their respective preset permitted frequency upper limits, the above two types of adjustments will be implemented simultaneously.

[0072] After each adjustment, the system re-executes the pruning operation in step 2 for the adjusted candidate joint decision-making actions. The adjusted food preparation increase / decrease, replenishment interval values, and stall manpower switching instructions are compared with the current global remaining kitchen capacity, the current global remaining raw material availability, the single-batch preparation lead time for the current dish category, and the current global remaining total number of deployable service units. This ensures that each adjusted portion remains within the physically feasible domain limited by the current global remaining resources. The necessity of this re-pruning lies in the fact that increasing the food preparation increase / decrease or adding service units during the adjustment process may cause the portion values ​​to exceed the remaining resource boundaries; direct simulation without pruning would yield unexecutable false results. After re-pruning, the adjusted candidate joint decision-making actions are re-executed using the exact same process described above. The time-step forward simulation and default frequency statistics for each demand scenario sample are performed. If the re-statistical default frequencies of food shortage and waiting fall back to within their respective preset permissible frequency upper bounds, the adjusted candidate joint decision action is taken as the final joint decision action for the current food category. If any default frequency still exceeds the corresponding preset permissible frequency upper bound, the next adjustment is performed according to the above rules.

[0073] In the numerical example above, the first adjustment increased the amount of food prepared by 5 portions and shortened the time interval for replenishing food by 1 minute. After re-trimming and re-simulating, the new frequency of food shortage default was 0.078, which has fallen back to within the upper limit of the preset permissible frequency of 0.10. The waiting default frequency was 0.061, which is still within the upper limit of the preset permissible frequency of 0.08. Both default frequencies meet the requirements. Therefore, the candidate joint decision action after the first adjustment is determined as the final joint decision action for the current dish category.

[0074] However, adjustments cannot always bring the default frequency back to within the permissible range. When at extreme peaks and global resources are nearly exhausted, repeated increases in food preparation may reach the upper limit of available kitchen capacity or raw material availability, preventing further supply increases. Similarly, repeated increases in service units may reach the upper limit of the total number of deployable service units, hindering further service capacity expansion. In such cases, the default frequency may never fall below the permissible upper limit. To prevent the system from entering an infinite adjustment loop, a preset maximum number of iterations is set as a termination condition. In one implementation, the preset maximum number of iterations is set to 10. If, after 10 consecutive adjustments, any default frequency still exceeds the corresponding preset permissible upper limit, the system terminates the adjustment loop and enters an emergency fallback process.

[0075] The emergency backup process is executed as follows: From a pre-set emergency action set, the emergency joint decision action that minimizes the sum of the frequency of food shortage defaults and the frequency of waiting defaults is selected as the final joint decision action for the current food category. The pre-set emergency action set is a set of conservative joint decision action schemes jointly configured by the operations manager and the kitchen supervisor during the system initialization phase. Each emergency joint decision action includes a fixed combination of four components: food preparation increase / decrease, replenishment batch size, replenishment interval, and stall manpower switching instructions. In one implementation, the pre-set emergency action set includes five emergency joint decision actions, each corresponding to a different level of emergency response. For example, the first action corresponds to a mild emergency, i.e., a slight increase in production to accelerate replenishment; the fifth action corresponds to an extreme emergency, i.e., replenishing food at maximum capacity and with the shortest interval, and allocating all remaining available service units to the current stall. The selection process is as follows: For each emergency joint decision action in the pre-set emergency action set, physical feasibility domain pruning is first performed, and then the same... For each demand scenario sample, a forward simulation was performed to count the frequency of defaults related to food shortages and waiting periods. The sum of these two default frequencies was calculated, and the emergency joint decision action with the smallest sum of the two default frequencies was taken as the final joint decision action for the current food category. The consideration of selecting the option with the smallest sum of the two default frequencies, rather than focusing solely on the frequency of a single default, is that in extreme scenarios, it may not be possible to reduce both default frequencies to within the permissible upper bound simultaneously. In such cases, choosing the option with the least overall default severity is a pragmatic strategy to minimize losses.

[0076] In an alternative implementation, the criterion for selecting emergency joint decision-making actions can also be changed to a weighted sum method. This involves multiplying the frequency of food shortage defaults by a food shortage severity coefficient, adding the frequency of waiting defaults by a waiting severity coefficient, and then taking the solution with the minimum weighted sum. The food shortage severity coefficient and the waiting severity coefficient are set by the operations manager based on the operational priority of specific dish categories. For example, for high-margin signature dishes, the food shortage severity coefficient is set to 2.0 while the waiting severity coefficient is set to 1.0, indicating that it's better to let customers wait a little longer than to run out of stock. For standardized, fast-serving regular dishes, both coefficients can be set to the same value.

[0077] After determining the final joint decision action for the current dish category, the system issues it to the corresponding stall for execution. The issued information includes four specific numerical values: the preparation increase / decrease quantity indicator (the number of portions the kitchen will increase or decrease in production compared to the baseline preparation rate within the current decision cycle), the replenishment batch value (the number of portions replenished to the warming station each time), the replenishment interval value (the number of minutes that should be allowed between two consecutive replenishment operations), and the stall manpower switching instruction (the number of service units that should be added to the current stall). Upon receiving these instructions, the stall service terminal notifies the kitchen preparation staff and stall service personnel to adjust their work rhythm according to the instructions via a display screen or voice prompts.

[0078] After the distribution is completed, the system deducts the kitchen capacity, raw material consumption, and number of service units required for the final joint decision action of the current dish category from the global remaining resources. The specific deduction rules are as follows: the kitchen capacity corresponding to the increase or decrease in food preparation is subtracted from the global remaining available kitchen capacity; the raw material consumption required for the increase or decrease in food preparation and the replenishment batch value is subtracted from the global remaining available raw material quantity; and the number of service units added in the stall manpower switching instruction is subtracted from the global remaining total number of deployable service units. The deducted global remaining resources serve as the global remaining resource indicator read when the next dish category executes steps 1 to 3, thus forming a serial resource reduction allocation chain among multiple dish categories, ensuring that the decision results of each dish category do not conflict under the constraint of the total global resources. Dish categories with higher priority receive sufficient resource allocation first, while dish categories with lower priority strive to optimize their supply and demand balance within the remaining resources. Once the final joint decision-making actions for all dish categories have been determined and issued, the decision-making process for the current decision-making cycle ends, and the system waits for the arrival of the next decision-making cycle before re-executing steps 1 to 3.

[0079] The present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A joint decision-making method for meal preparation output and replenishment frequency based on reinforcement learning, characterized in that, Within each decision cycle, the following steps are executed sequentially for each dish category in the cafeteria according to a preset priority order. After all steps for each dish category are completed, the resources used for the final joint decision action are deducted from the global remaining resources. Subsequent dish categories will continue to execute with the deducted global remaining resources as constraints: Step 1: Read the operational indicators and global remaining resource indicators of the current dish category from the canteen monitoring system, combine them into an observation vector, input the observation vector into the Hidden Markov Operational Phase Recognition Model for forward recursive update to obtain the phase confidence distribution, determine the current dominant phase based on the phase confidence distribution and generate a phase perception state token. Step 2: Construct a phase conditional encoding sequence based on tokens from multiple recently completed decision cycles. Append the target reward token and the current phase perception state token to the end of the phase conditional encoding sequence and input it into the decision Transformer model. Generate candidate joint decision actions through a causal self-attention mechanism and trim the candidate joint decision actions to the physical feasible domain limited by the global remaining resources. Step 3: Based on the phase confidence distribution, extract demand scenario samples from the demand scenario library corresponding to each implicit operational phase for forward simulation, and count the frequency of food shortage default and the frequency of waiting default. When both default frequencies meet the upper limit of the preset permitted frequency, output the final joint decision action. When any default frequency exceeds the upper limit of the preset permitted frequency, adjust and re-verify, and send the final joint decision action to the corresponding stall for execution.

2. The method according to claim 1, characterized in that, The Hidden Markov Operating Phase Recognition Model contains four implicit operating phases: the cooling phase, the warming phase, the peak phase, and the falling phase. It includes a phase transition probability table between the four implicit operating phases and an observation emission probability table for each implicit operating phase.

3. The method according to claim 1, characterized in that, Operational metrics include current food inventory, real-time food preparation rate, current queue size, number of unfulfilled reservations, and cumulative food heat preservation time; global remaining resource metrics include available kitchen capacity, available raw materials, and total number of deployable service units; the operational metrics and global remaining resource metrics together form an observation vector of 8 metrics.

4. The method according to claim 1, characterized in that, The forward recursive update in step 1 is performed as follows: using the forward confidence information retained at the end of the previous decision cycle and the observation vector at the current time, the forward recursive update is performed on the confidence values ​​of each implicit operational phase, and all the updated confidence values ​​are normalized so that their sum is 1, thus obtaining the phase confidence distribution at the current time. The current dominant phase is the implicit operational phase with the highest confidence value in the phase confidence distribution; the phase perception state token is formed by splicing the preset phase embedding vector and the observation vector corresponding to the current dominant phase along the feature dimension.

5. The method according to claim 1, characterized in that, The construction method of the phase condition coding sequence in step 2 is as follows: trace back along the time axis to the most recent N consecutive completed decision cycles, extract the phase perception state token, the executed action token, and the acquired reward token generated in each completed decision cycle, and connect the three tokens one after the other in a fixed order of reward token first, phase perception state token in the middle, and action token last, to form a phase condition coding sequence with a length of 3N tokens.

6. The method according to claim 5, characterized in that, The target reward token is preset by the operations manager based on the daily food shortage tolerance level and waste control level; the decision Transformer model encodes the superimposed position of each token in the complete input sequence, and then processes it layer by layer through multiple causal self-attention blocks. In each causal self-attention block, each token only performs attention association calculation on itself and all tokens in the sequence that are located before it; the output representation of the phase perception state token at the current moment at the corresponding position in the complete sequence is taken, and then projected onto the action space through a linear mapping layer to generate candidate joint decision actions.

7. The method according to claim 1, characterized in that, The candidate joint decision-making action includes four components: the amount of food preparation to increase or decrease, the batch size of food replenishment, the time interval for food replenishment, and the stall manpower switching instruction. The way to trim it to the physical feasible domain is as follows: trim the amount of food preparation to increase or decrease to no more than the current global remaining available kitchen capacity and no more than the upper limit that the current global remaining available raw materials can support; trim the time interval for food replenishment to no less than the pre-preparation time of a single batch for the current dish category; and trim the stall manpower switching instruction to no more than the current global remaining total number of deployable service units.

8. The method according to claim 1, characterized in that, In step 3, the method for extracting demand scenario samples from the demand scenario library corresponding to each implicit operating phase is as follows: multiply the confidence value of each implicit operating phase in the phase confidence distribution by the preset total number of samples M and round down to obtain the initial quota of each implicit operating phase. The difference between M and the sum of all initial quotas is added to the quota of the corresponding implicit operating phase one by one in descending order of confidence value until the total quota is exactly M. Then, random sampling with replacement is performed from the demand scenario library corresponding to each implicit operating phase according to their respective final quotas to obtain M demand scenario samples.

9. The method according to claim 8, characterized in that, The forward simulation is executed as follows: For each of the M demand scenario samples, the following simulation process is performed one by one. Taking the current remaining amount of dishes as the starting point of the simulation, the simulated supply side is updated according to the increase or decrease in food preparation and the batch value of replenishment in the candidate joint decision actions after pruning. The simulated demand side is consumed according to the number of simulated customers arriving and the number of simulated dishes ordered in the corresponding time step of the current simulated demand scenario sample. The process is advanced step by step until a complete decision cycle is covered. The simulated remaining amount of dishes and the simulated queuing waiting time are recorded for each simulated time step. The food shortage default frequency is the proportion of the number of demand scenario samples with at least one time step where the simulated remaining amount of dishes is lower than the preset food shortage threshold to M. The waiting default frequency is the proportion of the number of demand scenario samples with at least one time step where the simulated queuing waiting time exceeds the preset waiting tolerance limit to M.

10. The method according to claim 1, characterized in that, The execution method adjusted and re-verified in step 3 is as follows: when the frequency of food shortage default exceeds the upper limit of the corresponding preset allowable frequency, the amount of food prepared will be increased by 1 preset food preparation step unit and the food replenishment time interval will be shortened by 1 preset interval step unit; when the frequency of waiting default exceeds the upper limit of the corresponding preset allowable frequency, the stall manpower switching instruction will be adjusted to dispatch 1 additional service unit. After each adjustment, the pruning operation in step 2 is re-executed to ensure that each component is still within the physical feasible domain limited by the current global remaining resources. Then, the forward simulation and default frequency statistics are re-executed. If any default frequency still exceeds the upper bound of the corresponding preset permitted frequency when the number of consecutive adjustments reaches the preset maximum number of iterations, then the emergency joint decision action that minimizes the sum of the default frequency of food shortage and the default frequency of waiting is selected from the preset emergency action set as the final joint decision action for the current food category.

Images