Evaluation method and evaluation system for ordering suggestion model of catering chain store

Abstract

An evaluation method and an evaluation system for a food and beverage chain store ordering suggestion model relate to the technical field of data processing, introduce an out-of-stock risk rate and an overstock risk rate as core evaluation indexes, and improve the evaluation of a machine learning regression task from a pure technology level to a business influence level. And the actual pain point of the catering industry is directly associated. Meanwhile, different ordering strategy types are divided according to ordering habits of different stores, and differentiated weight parameters are distributed through strategy classification. According to the dynamic adjustment mechanism, the evaluation result can adapt to diversified business scenes, and the limitation of one-time cutting in the prior art is effectively avoided. According to the evaluation method and the evaluation system, the effect of the existing ordering suggestion model can be accurately evaluated, and the evaluation method and the evaluation system have better practical value.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to an evaluation method and system for an order suggestion model for restaurant chain stores. Background Technology

[0002] In recent years, with the development of the catering industry, more and more catering chain brands have emerged, and each brand has opened many stores. In the daily operation and management process, these catering chain stores frequently place orders for raw materials through the ordering system provided by the chain brand. In order to facilitate the daily ordering of stores, the chain brand provides a replenishment suggestion function. Based on the calculation of purchase, sales and inventory data and the prediction of future sales, it provides suggestions on the order quantity (boxes, kilograms, etc.) of each material when the store places an order, thereby improving the efficiency of store ordering and achieving more accurate ordering needs.

[0003] Typically, order quantity recommendations involve building machine learning or deep learning models to predict the required order quantities. Once this predictive model is trained and applied to specific business operations, its effectiveness needs continuous evaluation. Material order quantity prediction is a typical regression task in machine learning, and common evaluation methods include MAE, RMSE, R2, and many other metrics for measuring prediction error. However, these methods are general and technically oriented, failing to consider any business attributes and thus unable to provide sufficient guidance for optimization in complex and diverse business scenarios. The following problems may arise in restaurant store ordering: 1. Over-ordering leads to inventory backlog and risk, impacting store finances; 2. Under-ordering leads to inventory shortages and risk of stockouts, impacting product sales; 3. Different stores have different ordering habits—some are conservative due to inventory capacity or financial constraints, while others are more aggressive. Evaluation methods for these different ordering strategies need to be differentiated. Therefore, there is an urgent need to develop a dedicated order recommendation evaluation method for restaurant chain stores to address the practical problems faced by restaurant stores. Summary of the Invention

[0004] The purpose of this invention is to provide an evaluation method and system for order suggestion models for catering chain stores. It comprehensively considers factors such as the business attributes of the catering industry and the different strategies of stores, and can achieve accurate evaluation of the effectiveness of existing order suggestion models.

[0005] The embodiments of the present invention are implemented as follows: An evaluation method for an order recommendation model for restaurant chain stores, comprising: P1. Obtain the set of model-suggested order quantities S1 and the set of actual order quantities S2 for multiple materials for the target store within the predetermined time period; P2. Based on S1 and S2, calculate the out-of-stock risk rate, backlog risk rate, and complete consistency rate. P3. Based on the historical ordering behavior data of the target store, determine its ordering strategy type; P4. Based on the ordering strategy type, assign corresponding weight parameters to the stockout risk rate, backlog risk rate, and complete consistency rate. P5. Using the weighted out-of-stock risk rate, backlog risk rate, and perfect consistency rate, the order matching rate of the target store is calculated.

[0006] Furthermore, in other preferred embodiments of the present invention, the formula for calculating the order matching rate is as follows: , In the formula, MR To improve order matching rate, R stockout To account for the risk of stockouts, α Weighting for stockout risk; R overstock For backlog risk rate, β Weighting for backlog risk; P consistency For perfect consistency, γ Weighted by the perfect consistency rate; α+β+γ= 1.

[0007] Furthermore, in other preferred embodiments of the present invention, the formula for calculating the stockout risk rate is as follows: , In the formula, M Indicates the quantity of materials. f j This indicates the model's representation of materials. j Recommended order quantity y j Indicates the store's use of materials j The actual order quantity, 1•() represents the indicator function, 1 is satisfied if the condition is met, and 0 is not satisfied.

[0008] Furthermore, in other preferred embodiments of the present invention, the formula for calculating the backlog risk rate is: .

[0009] Furthermore, in other preferred embodiments of the present invention, the formula for calculating the perfect consistency rate is: .

[0010] Furthermore, in other preferred embodiments of the present invention, the ordering strategy types include a conservative strategy, an aggressive strategy, and a balanced strategy; for the conservative strategy, α>β For aggressive strategies, α< β For the balancing strategy, α = β .

[0011] Furthermore, in other preferred embodiments of the present invention, the weight allocation for the conservative strategy is as follows: α ∈[0.4,0.5], β ∈[0.1,0.2], γ ∈[0.3,0.5]; for an aggressive strategy, the weight allocation is as follows: α ∈[0.1,0.2], β ∈[0.4,0.5], γ ∈[0.3,0.5]; For the balancing strategy, the weight allocation is as follows: α ∈[0.3,0.4], β ∈[0.3,0.4], γ ∈[0.4,0.5].

[0012] Furthermore, in other preferred embodiments of the present invention, the weight allocation for the conservative strategy is as follows: α =0.5, β= 0.2, γ= 0.3; For aggressive strategies, the weight allocation is... α =0.2, β= 0.5, γ= 0.4; For the balancing strategy, the weight allocation is... α =0.3, β= 0.3, γ= 0.4.

[0013] Furthermore, in other preferred embodiments of the present invention, the order matching rate is graded based on the comparison between the calculated value and the theoretical maximum value of the order matching rate. When the calculated value reaches 80% to 100% of the theoretical maximum value, it is rated as excellent; when the calculated value reaches 70% to 80% of the theoretical maximum value, it is rated as good; and when the calculated value reaches less than 70% of the theoretical maximum value, it is rated as average.

[0014] An evaluation system for an order recommendation model for restaurant chain stores, comprising: The data acquisition module is configured to acquire the model-suggested order quantity set S1 and the actual order quantity set S2 for the target store. The indicator calculation module is configured to calculate the stockout risk rate, backlog risk rate, and complete consistency rate based on S1 and S2. The strategy classification module is configured to determine the ordering strategy type for target stores based on historical ordering behavior data. The weight allocation module is configured to assign weight parameters to the stockout risk rate, backlog risk rate, and perfect consistency rate based on the ordering strategy type. The matching rate calculation module is configured to calculate the order matching rate using weighted metrics to evaluate model performance.

[0015] The beneficial effects of the embodiments of the present invention are: This invention provides an evaluation method and system for order suggestion models used in restaurant chains. It introduces stockout risk rate and backlog risk rate as core evaluation indicators, elevating the evaluation of machine learning regression tasks from a purely technical level to a business impact level. This directly addresses the actual pain points of the catering industry. Furthermore, it categorizes different ordering strategies based on the ordering habits of different stores, assigning differentiated weight parameters through strategy classification. This dynamic adjustment mechanism allows the evaluation results to adapt to diverse business scenarios, effectively avoiding the limitations of a "one-size-fits-all" approach in existing technologies. This evaluation method and system can accurately evaluate the effectiveness of existing order suggestion models, demonstrating significant practical value. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the division of different raw material sets in a store as provided in Embodiment 1 of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example

[0019] An evaluation method for an order recommendation model for restaurant chain stores, comprising: P1. Obtain the set of model-suggested order quantities S1 and the set of actual order quantities S2 for multiple materials for the target store within the predetermined time period.

[0020] Specifically, please refer to Figure 1 As shown, with Figure 1 The gray rectangular area represents the set of available raw materials for a store. The green ellipse represents the set of raw materials (S1) suggested by the ordering model to the store, and the blue ellipse represents the set of raw materials actually ordered by the store (S2). The intersection of the two ellipses represents the set of raw materials where the suggested ordering model and the store's actual order overlap. This intersection can be divided into three parts: A (suggested order quantity > actual store order quantity), B (suggested order quantity = actual store order quantity), and C (suggested order quantity < actual store order quantity). Furthermore, excluding the intersection of S1 and S2, the remaining portion of S1 represents the set of raw materials (D) suggested for ordering but not actually ordered by the store, and the remaining portion of S2 represents the set of raw materials (E) ordered by the store but not recommended by the model. In addition, outside the areas of S1 and S2, there is a set of raw materials (F) that the model neither recommends for ordering nor that the store actually orders. Metrics for measuring stockout risk, overstock risk, and accuracy will be defined from this diagram.

[0021] P2. Based on S1 and S2, calculate the out-of-stock risk rate, backlog risk rate, and complete consistency rate.

[0022] The stockout risk rate assesses the risk of a store losing a sales opportunity when the model's suggested order quantity is less than the store's actual order quantity. It measures the probability that the suggested order quantity will be less than the store's actual order quantity, and its calculation formula is as follows: , In the formula, M Indicates the quantity of materials. f j This indicates the model's representation of materials. j Recommended order quantity y j Indicates the store's use of materials j The actual order quantity, 1•() represents the indicator function, 1 is satisfied if the condition is met, and 0 is not satisfied.

[0023] This formula can also be expressed as dividing the quantity of materials in raw material set C by the quantity of materials in raw material set S2.

[0024] The backlog risk assessment measures the risk of inventory buildup in stores when the model's suggested order quantity exceeds the actual store order quantity. It measures the probability of this happening, and its calculation formula is as follows: .

[0025] This formula can also be expressed as dividing the quantity of materials in raw material set A∪D by the quantity of materials in raw material set S1∪S2.

[0026] The perfect match rate measures the percentage of materials whose suggested order quantity matches the actual order quantity at the store. The formula is: .

[0027] This formula can also be expressed as dividing the quantity of materials in raw material set B∪F by the quantity of materials in raw material set S1∪S2∪F.

[0028] P3. Based on the historical ordering behavior data of the target store, determine its ordering strategy type.

[0029] P4. Based on the ordering strategy type, assign corresponding weight parameters to the stockout risk rate, backlog risk rate, and complete consistency rate.

[0030] In the daily ordering process of stores, ordering strategies are based on the habits of the customers who place orders that day. Some are conservative, estimating that they will have enough and never order more, while others are more aggressive, trying to stock up as much as possible. Different ordering habits will affect the evaluation of the suggested ordering model. Therefore, it is necessary to classify the stores according to their daily behavior. In this embodiment, the store's ordering strategies are divided into three categories: conservative strategy, aggressive strategy, and balanced strategy, and then different weights are assigned to the three different strategies.

[0031] Typically, stores employing a conservative strategy tend to order less. In this case, if the suggested order quantity from the recommendation model is still less than the store's actual order quantity, a higher penalty should be applied to the stockout risk rate. This means assigning a relatively high weight to the stockout risk rate and a lower weight to the corresponding overstock risk rate. This is reflected in the weighting of the stockout risk rate. α Weight greater than backlog risk rate β .

[0032] Stores employing aggressive strategies tend to overorder. In such cases, if the suggested order quantity from the recommendation model exceeds the store's actual order quantity, a higher penalty should be applied to the backlog risk rate. This means assigning a relatively high weight to the backlog risk rate and a lower weight to the stockout risk rate. This is reflected in the weighting of the backlog risk rate. β Weight greater than the out-of-stock risk rate α .

[0033] Stores employing a balanced strategy fall between conservative and aggressive strategies, maintaining a balance in the allocation of parameters for the two risks. This allows the weighting of the out-of-stock risk rate (α) to be equal to or nearly equal to the weighting of the backlog risk rate (β).

[0034] Furthermore, for the conservative strategy, the weight allocation is as follows:α ∈[0.4,0.5], β ∈[0.1,0.2], γ ∈[0.3,0.5]; for an aggressive strategy, the weight allocation is as follows: α ∈[0.1,0.2], β ∈[0.4,0.5], γ ∈[0.3,0.5]; For the balancing strategy, the weight allocation is as follows: α ∈[0.3,0.4], β ∈[0.3,0.4], γ ∈[0.4,0.5].

[0035] Optionally, for a conservative strategy, the weight allocation is as follows: α =0.5, β= 0.2, γ= 0.3; For aggressive strategies, the weight allocation is... α =0.2, β= 0.5, γ= 0.4; For the balancing strategy, the weight allocation is... α =0.3, β= 0.3, γ= 0.4.

[0036] P5. Using the weighted out-of-stock risk rate, backlog risk rate, and perfect consistency rate, the order matching rate of the target store is calculated.

[0037] The formula for calculating the order matching rate is as follows: , In the formula, MR To improve order matching rate, R stockout To account for the risk of stockouts, α Weighting for stockout risk; R overstock For backlog risk rate, β Weighting for backlog risk; P consistency For perfect consistency, γ Weighted by the perfect consistency rate; α+β+γ= 1.

[0038] Furthermore, the order matching rate is graded based on the comparison between the calculated value and the theoretical maximum value. When the calculated value reaches 80% to 100% of the theoretical maximum value, it is rated as excellent; when the calculated value reaches 70% to 80% of the theoretical maximum value, it is rated as good; and when the calculated value reaches less than 70% of the theoretical maximum value, it is rated as average.

[0039] Specifically, this embodiment provides a feasible parameter setting method, as shown in Table 1.

[0040] Table 1. Range of order matching rate values ​​under different strategies Strategy type Weight configuration Theoretical maximum value Theoretical minimum value Range of values Conservative strategy =0.5,=0.2,=0.3 0.3 -0.7 [-0.7, 0.3] Balanced strategy =0.3,=0.3,=0.4 0.4 -0.6 [-0.6, 0.4] radical strategy =0.2,=0.5,=0.3 0.3 -0.7 [-0.7, 0.3] The evaluation levels for different strategy types are shown in Tables 2-4.

[0041] Table 2. Performance Level of Conservative Strategy performance level Range of values Percentage range Business meaning excellent [0.24, 0.3] 80%-100% of the theoretical maximum value The algorithm performs well, has high prediction accuracy, and low business risk. good [0.21, 0.24) 70%-80% of the theoretical maximum value The algorithm performed well, its predictions were generally accurate, and business risks were manageable. generally [-0.7, 0.21) less than 70% of the theoretical maximum value The algorithm's performance was mediocre, with some prediction bias, and needs improvement. Table 3. Performance Level of the Balancing Strategy performance level Range of values Percentage range Business meaning excellent [0.32, 0.4] 80%-100% of the theoretical maximum value The algorithm performs well, has high prediction accuracy, and low business risk. good [0.28, 0.32) 70%-80% of the theoretical maximum value The algorithm performed well, its predictions were generally accurate, and business risks were manageable. generally [-0.6, 0.28) less than 70% of the theoretical maximum value The algorithm's performance was mediocre, with some prediction bias, and needs improvement. Table 4. Performance Level of Aggressive Strategy performance level Value range Percentage range Business meaning excellent [0.24, 0.3] 80%-100% of the theoretical maximum value The algorithm performs well, has high prediction accuracy, and low business risk. good [0.21, 0.24) 70%-80% of the theoretical maximum value The algorithm performed well, its predictions were generally accurate, and business risks were manageable. generally [-0.7, 0.21) less than 70% of the theoretical maximum value The algorithm's performance was mediocre, with some prediction bias, and needs improvement. Furthermore, this embodiment also provides an evaluation system for an order suggestion model for restaurant chain stores, which includes: The data acquisition module is configured to acquire the model-suggested order quantity set S1 and the actual order quantity set S2 for the target store. The indicator calculation module is configured to calculate the stockout risk rate, backlog risk rate, and complete consistency rate based on S1 and S2. The strategy classification module is configured to determine the ordering strategy type for target stores based on historical ordering behavior data. The weight allocation module is configured to assign weight parameters to the stockout risk rate, backlog risk rate, and perfect consistency rate based on the ordering strategy type. The matching rate calculation module is configured to calculate the order matching rate using weighted metrics to evaluate model performance.

[0042] This evaluation system can be used to perform the evaluation method of this embodiment for accurate evaluation of existing order recommendation models.

[0043] Application examples Using the evaluation method provided in Example 1, the existing ordering suggestion model was evaluated. From the existing stores, 20 stores each of the conservative, balanced, and aggressive strategies were selected based on their historical ordering data, and weights were configured according to Table 1. The order matching rate for stores with different strategies was calculated, and the results are shown in Table 5.

[0044] Table 5. Calculation results of order matching rate under different strategies Strategy type Average order matching rate Model performance evaluation conservative strategy 0.214 good Balanced strategy 0.369 excellent radical strategy 0.202 generally As shown in Table 5, the order recommendation model achieves a good order matching rate when forecasting orders for stores using a balanced strategy, with the model performance evaluation reaching an excellent level. However, for stores adopting conservative and aggressive strategies, the model fails to fully understand different ordering habits and cannot meet different ordering needs, resulting in a poor order matching rate and a performance evaluation of only good or average. This indicates that the model needs further optimization and upgrading to adapt to the ordering needs of stores using both conservative and aggressive strategies.

[0045] In summary, this invention provides an evaluation method and system for an order suggestion model used in restaurant chains. It introduces stockout risk rate and backlog risk rate as core evaluation indicators, elevating the evaluation of machine learning regression tasks from a purely technical level to a business impact level. This directly addresses the actual pain points of the catering industry. Furthermore, it categorizes different ordering strategies based on the ordering habits of different stores, assigning differentiated weight parameters through strategy classification. This dynamic adjustment mechanism allows the evaluation results to adapt to diverse business scenarios, effectively avoiding the limitations of a "one-size-fits-all" approach in existing technologies. This evaluation method and system can accurately evaluate the effectiveness of existing order suggestion models and has significant practical value.

[0046] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An evaluation method for an order recommendation model for restaurant chain stores, characterized in that, include: Obtain the set of model-suggested order quantities S1 and the set of actual order quantities S2 for multiple materials for the target store within a predetermined time period; Based on S1 and S2, calculate the out-of-stock risk rate, backlog risk rate, and complete consistency rate; Based on the historical ordering behavior data of the target stores, determine their ordering strategy type; Based on the ordering strategy type, assign corresponding weight parameters to the stockout risk rate, backlog risk rate, and complete consistency rate; The order matching rate of the target store is calculated using the weighted out-of-stock risk rate, the backlog risk rate, and the perfect match rate.

2. The evaluation method according to claim 1, characterized in that, The formula for calculating the order matching rate is: ， In the formula, MR To improve order matching rate, R stockout To account for the risk of stockouts, α Weighting for stockout risk; R overstock For backlog risk rate, β Weighting for backlog risk; P consistency For perfect consistency, γ Weighted by the perfect consistency rate; α+β+γ= 1.

3. The evaluation method according to claim 2, characterized in that, The formula for calculating the stockout risk rate is as follows: ， In the formula, M Indicates the quantity of materials. f j This indicates the model's representation of the material. j Recommended order quantity y j Indicates the store's use of materials j The actual order quantity, 1•() represents the indicator function, 1 is satisfied if the condition is met, and 0 is not satisfied if the condition is not met.

4. The evaluation method according to claim 3, characterized in that, The formula for calculating the backlog risk rate is: 。 5. The evaluation method according to claim 4, characterized in that, The formula for calculating the perfect consistency rate is: 。 6. The evaluation method according to claim 5, characterized in that, The ordering strategy types include conservative, aggressive, and balanced strategies; for the conservative strategy... α>β Regarding the aforementioned radical strategy, α < β For the aforementioned balancing strategy, α = β .

7. The evaluation method according to claim 6, characterized in that, For the conservative strategy, the weight allocation is as follows: α ∈[0.4,0.5], β ∈[0.1,0.2], γ ∈[0.3,0.5]; for the aggressive strategy, its weight allocation is as follows: α ∈[0.1,0.2], β ∈[0.4,0.5], γ ∈[0.3,0.5]; for the balancing strategy, its weight allocation is as follows: α ∈[0.3,0.4], β ∈[0.3,0.4], γ ∈[0.4,0.5].

8. The evaluation method according to claim 7, characterized in that, For the conservative strategy, the weight allocation is as follows: α =0.5, β= 0.2, γ= 0.3; For the aforementioned aggressive strategy, the weight allocation is as follows: α =0.2, β= 0.5, γ= 0.4; For the aforementioned balancing strategy, the weight allocation is as follows: α =0.3, β= 0.3, γ= 0.

4.

9. The evaluation method according to claim 8, characterized in that, The order matching rate is graded based on the comparison between the calculated value and the theoretical maximum value. When the calculated value reaches 80% to 100% of the theoretical maximum value, it is rated as excellent; when the calculated value reaches 70% to 80% of the theoretical maximum value, it is rated as good; and when the calculated value reaches less than 70% of the theoretical maximum value, it is rated as average.

10. An evaluation system for an order suggestion model for restaurant chain stores, characterized in that, include: The data acquisition module is configured to acquire the model-suggested order quantity set S1 and the actual order quantity set S2 for the target store. The indicator calculation module is configured to calculate the stockout risk rate, backlog risk rate, and complete consistency rate based on S1 and S2. The strategy classification module is configured to determine the ordering strategy type of the target store based on historical ordering behavior data; The weight allocation module is configured to assign weight parameters to the stockout risk rate, backlog risk rate, and complete consistency rate based on the ordering strategy type. The matching rate calculation module is configured to calculate the order matching rate using weighted metrics to evaluate model performance.

Images