Single-period end-to-end inventory control method considering substitutability and products

Abstract

This invention provides a single-cycle end-to-end inventory control method and product considering substitutability, relating to the field of intelligent decision-making technology. In this embodiment, the method includes: determining the observed features in the current cycle; inputting the features into an end-to-end inventory decision model to obtain an inventory control strategy, wherein the inventory control strategy is represented as a multi-dimensional vector, representing the purchase quantity of each level of resource. In this embodiment, the input to the end-to-end inventory decision model is the observable features, and the output is the inventory control strategy. Compared to the traditional predict-then-optimize framework, the end-to-end inventory decision model is easier to deploy in actual production environments, simpler to use, and more efficient in decision-making. Furthermore, the end-to-end inventory decision model proposed in this embodiment is highly efficient to train and can learn relevant inventory control strategies.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent decision-making technology, and in particular to a single-cycle end-to-end inventory control method and product that takes into account substitutability. Background Technology

[0002] In a system considering substitutable inventory (hereinafter referred to as substitutable inventory), demand can be met by resources of different grades (types). The goal of the substitutable inventory control problem (hereinafter referred to as the substitutable inventory problem) is to control the inventory of different grades of resources to better match demand, thereby minimizing the total cost of the system. Substitutable inventory is widely used in many production or service systems. Companies can flexibly use their various resources to meet random customer demand at different costs, thereby reducing enterprise costs and increasing profits. For example, a recycling remanufacturer can remanufacture different grades of old products to meet customer demand. Due to long delivery times and short order time windows, the remanufacturer needs to purchase recycled materials from recyclers in advance before the actual demand is revealed. After receiving customer orders, the remanufacturer will remanufacture the recycled materials. Typically, recycled materials are divided into different grades, with different purchase prices and remanufacturing costs. Higher-grade recycled materials have higher purchase costs and lower remanufacturing costs, while lower-grade recycled materials have the opposite. How to balance these two costs is crucial for remanufacturing companies. Cloud computing companies provide remote computing services to their customers. Before customer needs are revealed, companies need to configure the computing environment in advance, such as high-performance computing platforms and specific software services. Furthermore, customer needs can be met through different configurations. For example, computing requirements can be met by configuring different types of CPU servers. Similar setups can be found in examples such as airline cabin allocation and electric vehicle charging services.

[0003] Managing inventory levels for various resources effectively is a challenging issue. Different resources have different procurement costs (incurred when purchasing resources) and service costs (incurred when meeting customer demand). Over-allocation of resources results in high procurement costs; under-allocation leads to high stockout costs due to unmet demand. When demand is relatively stable, allocating resources with the lowest total cost (procurement cost plus service cost) is appropriate, as this minimizes overall costs. However, when demand is less stable, it's necessary to allocate resources with lower procurement costs (potentially higher service costs). This is because, when demand is high, these resources reduce the probability of stockouts; and when demand is low, their sunk costs are also lower.

[0004] Due to the uncertainty of demand, most inventory management methods adopt a framework called "predict first, optimize later" to address this challenge. In this framework, decision-makers first train a predictive model to estimate stochastic demand, and then solve the corresponding inventory optimization problem based on the estimated demand to obtain the allocation decision. Inevitably, when estimating the parameters of the predictive model in the first step, it is necessary to assume that the unknown demand follows a specific distribution (such as a normal distribution) or a functional form (such as a linear function). If the distribution learned by the predictive model matches the potential real distribution, then the optimal inventory control scheme can be obtained by solving the optimization problem. However, in many related application areas, the demand distribution is not explicitly known, and pre-determining the form of a distribution will bring certain problems: (1) It requires strong expert knowledge. The demand for different products is affected by different factors. For example, the demand for used mobile phones is affected by factors such as customer evaluation, whether a new model is released, and price, while the demand for cloud computing shows strong cyclical demand. It can be seen that determining the form of demand for different products or the distribution it follows requires relatively rich industry experience. (2) It introduces model selection bias. Due to the high uncertainty and unknowability of demand, even the most experienced experts cannot completely determine the correct demand form. Inappropriate demand forms can introduce model selection bias, which can be further amplified through the optimization step, leading to suboptimal decision-making. (3) Inefficient decision-making. The decision-making framework of prediction before optimization requires calling the optimization model solver to solve the corresponding model in the optimization step. This usually does not meet the needs of scenarios with high timeliness requirements, such as real-time inventory scheduling, and greatly affects decision-making efficiency.

[0005] Therefore, in order to address the challenges of complex inventory decisions, high timeliness requirements, and model selection bias introduced by the traditional forecast-then-optimize decision-making framework, a new single-cycle inventory control method is urgently needed. Summary of the Invention

[0006] This invention provides a single-cycle end-to-end inventory control method and product that takes into account substitutability, in order to at least partially solve the problems existing in related technologies.

[0007] A first aspect of this invention provides a single-cycle end-to-end inventory control method that considers substitutability, the method comprising:

[0008] The observed features in the current period are determined, and the features are input into the end-to-end inventory decision model to obtain the inventory control strategy. The inventory control strategy is represented as a multi-dimensional vector, which represents the purchase quantity of each level of resource.

[0009] The training samples of the end-to-end inventory decision model include: sample features and corresponding historical real demand. The training process of the end-to-end inventory decision model is unlabeled learning. The output of the end-to-end inventory decision model is an inventory control strategy A = h(x) that minimizes the empirical cost. The minimized empirical cost is expressed as:

[0010] in, It is a neural network model h in a given dataset H n The experience cost below, H n =[(x1,d1),…,(x n ,d n )], It is the collection of all neural network models, x i Let d represent the features of the i-th sample. i This represents the historical actual demand for the i-th sample.

[0011] Optionally, the input to the neural network model is x = (x1, ..., x...). i ,…,x d In the case of ), the input of the first hidden layer is passed through The calculation shows that the output of the first hidden layer is b. 1j =g(a 1j ), where the function g(·) is the activation function; for the hidden layer l, its input is The output is b lj =g(a lj The output of the neural network model is obtained through... calculate.

[0012] Optionally, the training objective of the neural network model is to find a set of parameters:

[0013]

[0014] Minimize experience costs;

[0015] Where n and m are set according to the number of neurons between different layers;

[0016] The neural network model is trained by repeatedly adjusting the weights Wi using the training dataset.

[0017] Optionally, the gradient is calculated. The weights are updated in the following ways:

[0018]

[0019] Where η is the learning rate.

[0020] Alternatively, the gradient can be calculated using the chain rule:

[0021]

[0022] in, Where c i It is the unit procurement cost of resources, φ i ,λ i It is the optimal solution to the following optimization problem:

[0023] max∑ i (φ i -λ i A i +μd;

[0024]

[0025] μ≤w;

[0026]

[0027]

[0028] Among them, A i This represents the quantity of resources purchased at level i; p represents the unit selling price of the product, and r i w represents the unit service cost of resource level i, and w represents the unit penalty cost for unmet demand; s i This indicates the unit processing cost of resource level i;

[0029] According to the backpropagation calculation formula of neural networks, Among them, b lk This represents the output of the k-th neuron in the hidden layer l of the neural network.

[0030] Optionally, the neural network model adopts a five-layer network architecture, with each layer having 11, 5, 5, 5, and 3 neurons, and uses the ReLU activation function.

[0031] Optionally, determining the features observed in the current period includes: taking the features observed in the current period and a preset number of previous historical periods as the features observed in the current period.

[0032] A second aspect of the present invention provides a single-cycle end-to-end inventory control device that takes into account substitutability, the device comprising:

[0033] The strategy generation module is used to determine the features observed in the current period, input the features into the end-to-end inventory decision model, and obtain the inventory control strategy. The inventory control strategy is represented as a multi-dimensional vector, representing the purchase quantity of each level of resource.

[0034] The training samples of the end-to-end inventory decision model include: sample features and corresponding historical real demand. The training process of the end-to-end inventory decision model is unlabeled learning. The output of the end-to-end inventory decision model is an inventory control strategy A = h(x) that minimizes the empirical cost. The minimized empirical cost is expressed as:

[0035] in, It is a neural network model h in a given dataset H n The experience cost below, H n =[(x1,d1),…,(x n ,d n )], It is the collection of all neural network models, x i Let d represent the features of the i-th sample. i This represents the historical actual demand for the i-th sample.

[0036] Optionally, the input to the neural network model is x = (x1, ..., ... i ,…, d In the case of ), the input of the first hidden layer is passed through The calculation shows that the output of the first hidden layer is b. 1j =(a 1j ), where the function g(·) is the activation function; for the hidden layer l, its input is The output is b lj =(a lj The output of the neural network model is obtained through... calculate.

[0037] Optionally, the training objective of the neural network model is to find a set of parameters:

[0038]

[0039] Minimize experience costs;

[0040] Where n and m are set according to the number of neurons between different layers;

[0041] The neural network model is trained by repeatedly adjusting the weights Wi using the training dataset.

[0042] Optionally, the gradient is calculated. The weights are updated in the following ways:

[0043]

[0044] Where η is the learning rate.

[0045] Alternatively, the gradient can be calculated using the chain rule:

[0046]

[0047] in, Where c i It is the unit procurement cost of resources, φ i ,λ i It is the optimal solution to the following optimization problem:

[0048] max∑ i (φ i -λ i A i +μd;

[0049]

[0050] μ≤w;

[0051]

[0052]

[0053] Among them, A i This represents the quantity of resources purchased at level i; p represents the unit selling price of the product, and r i w represents the unit service cost of resource level i, and w represents the unit penalty cost for unmet demand; s i This indicates the unit processing cost of resource level i;

[0054] According to the backpropagation calculation formula of neural networks, Among them, b lk This represents the output of the k-th neuron in the hidden layer l of the neural network.

[0055] Optionally, the neural network model adopts a five-layer network architecture, with each layer having 11, 5, 5, 5, and 3 neurons, and uses the ReLU activation function.

[0056] Optionally, determining the features observed in the current period includes: taking the features observed in the current period and a preset number of previous historical periods as the features observed in the current period.

[0057] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executed, implements the steps of the single-cycle end-to-end inventory control method considering substitutability as described in the first aspect of the present invention.

[0058] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the single-cycle end-to-end inventory control method considering substitutability as described in the first aspect of the present invention.

[0059] In this embodiment of the invention, the input to the end-to-end inventory decision model is observable features, and the output is an inventory control strategy. Compared to the traditional predict-then-optimize framework, the end-to-end inventory decision model is easier to deploy in real-world production environments, simpler to use, and more efficient in decision-making. Furthermore, the input end-to-end inventory decision model proposed in this embodiment of the invention is highly efficient to train and can also learn relevant inventory control strategies. Attached Figure Description

[0060] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart of a single-cycle end-to-end inventory control method considering substitutability according to an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of the neural network model structure involved in a single-cycle end-to-end inventory control method that considers substitutability according to an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of an example decision process for a single-cycle end-to-end inventory control method that considers substitutability, according to an embodiment of the present invention. Detailed Implementation

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] To address the problems of the "predict first, optimize later" framework in the background art, this invention proposes an end-to-end neural network decision-making framework that can directly output inventory control strategies for inventory control based on input features. The end-to-end neural network decision-making framework proposed in this invention does not require pre-determining the demand form or its distribution; it is trained directly from historical data, reducing reliance on expert knowledge, avoiding model selection bias, and eliminating the need for optimization solvers. This allows it to effectively handle various situations encountered in real-world decision-making, significantly compensating for the shortcomings of the predict-then-optimize decision-making framework.

[0066] Specifically, this invention establishes an end-to-end neural network decision-making framework to solve the single-cycle recycled inventory control problem considering substitutability. The input to the neural network model is observable features, and the output is the inventory control strategy. Compared to traditional predict-then-optimize frameworks, the end-to-end decision-making framework is easier to deploy in real-world production environments, simpler to use, and more efficient in decision-making.

[0067] This invention also proposes a corresponding algorithm for training an end-to-end neural network decision-making model. Since the optimal inventory control strategy is unknown during model training, the training of this model falls under the category of unsupervised learning. To address the unlabeled training problem, this invention redesigns the loss function and provides a relevant algorithm to calculate the gradient of the loss function, enabling the model to be trained.

[0068] In this embodiment of the invention, a theoretical proof of the effectiveness of the loss function design is also provided, offering a theoretical guarantee for training the end-to-end inventory control decision model. More specifically, using the loss function given in this embodiment as the training objective, when the neural network model minimizes this loss function, its output decision is the optimal inventory control decision.

[0069] Numerical experiments are also provided in this embodiment of the invention, showing that model decision-making based on an end-to-end framework results in lower remanufacturing costs than model decision-making based on a prediction-then-optimization framework. Furthermore, analysis shows that the end-to-end decision-making model is highly efficient to train and can learn relevant inventory control strategies.

[0070] Reference Figure 1 The figure shows a flowchart of a single-cycle end-to-end inventory control method considering substitutability according to an embodiment of the present invention. As shown in the figure, the single-cycle end-to-end inventory control method considering substitutability provided by the embodiment of the present invention includes:

[0071] S101, determine the features observed in the current period, input the features into the end-to-end inventory decision model to obtain the inventory control strategy, the inventory control strategy is represented as a multi-dimensional vector, representing the purchase quantity of each level of resource.

[0072] The training samples of the end-to-end inventory decision model include: sample features and corresponding historical real demand. The training process of the end-to-end inventory decision model is unlabeled learning. The output of the end-to-end inventory decision model is an inventory control strategy A = h(x) that minimizes the empirical cost. The minimized empirical cost is expressed as:

[0073] in, It is a neural network model, where h is in a given dataset H. n The experience cost below, H n=[(x1,d1),…,(x n ,d n )], It is the collection of all neural network models, x i Let d represent the features of the i-th sample. i This represents the historical actual demand for the i-th sample.

[0074] The parameters and decision variable symbols involved in the model proposed in the embodiments of this invention are listed below:

[0075] Table 1 Parameters

[0076]

[0077]

[0078] Table 2 Decision Variables

[0079]

[0080] In this embodiment of the invention, a recyclable inventory control model considering substitutability is first established. Consider a company providing products or services (hereinafter collectively referred to as services) to customers at a price p, where services can be satisfied by resources of different levels, assumed to be level i∈I. Due to the long resource procurement cycle and short service time window, the company needs to stockpile various resource inventories in advance before customer demand is clearly defined. After demand is met, the company allocates inventory resources according to demand to satisfy customer needs. At the end of the cycle, considering the timeliness of resource present value, excess resources are directly disposed of; therefore, this model is a single-cycle decision-making model. Different levels of resources have different unit procurement costs (c1,…,c…). I Unit service cost (r1,…,r) I ) and unit processing cost (s1,…,s I Procurement costs are the costs incurred by a company in purchasing resources. When demand is met, the company allocates its resource inventory to satisfy that demand, incurring service costs. When demand is not met, corresponding penalty costs are incurred. These penalty costs can be explained by lost profits, reduced corporate credit, or higher costs incurred from temporarily using additional resources. Because the net present value of resources decreases rapidly, excess inventory is disposed of immediately at the end of the cycle, incurring disposal costs. Disposal costs reflect whether the company can sell the resources at a lower price or have a third-party company handle the disposal. Therefore, disposal costs can be positive or negative. At the beginning of the decision-making cycle, when demand is not met, the decision-maker's goal is to minimize their expected total cost. Therefore, the following two-stage stochastic programming model can be established:

[0081]

[0082]

[0083] in:

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] The objective function (1) for the first stage is the total cost, which includes the purchase cost of different levels of resource inventory plus the expected cost after the demand in the second stage is met. Φ(A,d) represents the minimum total cost of the enterprise in the second stage given the inventory A and the demand d in the first stage. The objective function (3) includes the revenue from selling services (first item), the service cost of different levels of resources (second item), the stockout cost of unmet demand (third item), and the disposal cost of surplus resources at the end of the period (fourth item). Constraint (4) indicates that the consumption of resource of level i is less than the inventory, and constraint (5) indicates that the quantity of consumed resources plus the quantity of unmet demand equals the total demand. Constraint (6) indicates that the consumption of resource of level i plus the disposal quantity equals the purchase quantity. Constraints (2) and (7) are domain constraints.

[0090] In practice, although the actual customer demand is unknown when decision-makers make resource procurement decisions in the first stage, they can predict demand based on some observable characteristics (referred to in some studies as covariates, attributes, side information, etc.). For example, products with good customer reviews are expected to have stronger future demand, and the demand for some electronic products exhibits strong seasonality. Given an observable feature x, the conditional distribution of demand is obtained. The above-mentioned inventory control model for recycled goods considering substitutability can be remodeled into the following conditional optimization model after observing feature x:

[0091]

[0092] In order to obtain the optimal solution of the optimization model (8), the decision-maker must know the distribution of conditional requirements. The specific form of this. That is, given a feature x, the decision-maker first needs to infer the conditional distribution of demand. The result is then substituted into the optimization model for solution. Traditional research typically employs a more intuitive framework of prediction followed by optimization to address this problem. Decision-makers assume that demand follows certain specific distributions (such as normal distribution, Poisson distribution, etc.), and these distributions are controlled by parameters θ(x). Determining the parameters θ(x) suffices to determine the form of the distribution. The problem of determining the form of the distribution then becomes finding the relationship between θ(x) and x. For example, it can be assumed that demand follows a normal distribution. And θ(x) = (μ(x), σ(x)). Given historical data, the relationship between parameters θ(x) and x can be estimated using carefully designed statistical methods. However, this predict-then-optimize framework has its problems. First, the pre-assumption of the model's distribution requires substantial prior information, necessitating experts highly familiar with the field to accurately capture the relationship between features and demand. Second, the actual demand distribution typically does not belong to any specific distribution, the impact of features on demand varies, and features can influence each other. Using a specific model to fit the actual demand distribution introduces model selection bias. Research shows that this bias is further amplified during the optimization phase, especially in multi-product inventory decision-making scenarios considering substitutability. Third, in practical applications, this type of decision-making framework requires continuous calls to the solver to solve the optimization model during the optimization phase, making it difficult to meet the needs of decision-making systems with high timeliness requirements.

[0093] To address the three issues mentioned above, this invention proposes an end-to-end neural network-based framework for solving feature-based inventory control considering substitutability. The goal of this invention is to train a neural network model that, for a given feature x, directly outputs an inventory control decision A. This type of model adopts an end-to-end architecture, avoiding the need to pre-assume demand distribution during the solution process. Furthermore, its solution efficiency is significantly higher than that of optimization solvers.

[0094] In this embodiment of the invention, the objective of the end-to-end inventory control model is to determine the purchase quantity A of the observed characteristic x-level recycled goods. Therefore, the neural network model structure is designed as follows: Figure 2 As shown.

[0095] Assume the historical training data samples are: H n =[(x1,d1),…,(x n ,d n )]; where x i Let d represent the features of the i-th sample. iLet represent the historical real demand of the i-th sample. In the neural network model structure of end-to-end decision-making, for a given feature x, the output of the neural network model is the resource procurement decision A = □(x). For most traditional neural network learning problems, such as image classification or demand prediction, the labels are known. The end-to-end model training in this embodiment of the invention belongs to the category of unlabeled learning. The historical real demand d cannot directly guide the training of the neural network model. The model output is the resource procurement decision A, and the real demand d is not the true target of the model output. The true target of the model output is to minimize the inventory procurement decision of the optimization problem (8), but since the demand distribution form is unknown, the optimal procurement decision is not observable, so it cannot explicitly guide the training of the neural network. In order to solve this problem, this embodiment of the invention proposes to train the neural network to minimize the experience cost. That is, for a given historical data H n To find the neural network model that minimizes the total empirical cost:

[0096]

[0097] Among them, the definition It is a neural network model h in a given dataset H n The experience cost (or experience risk) is as follows. It is a collection of all neural network models.

[0098] In this embodiment of the invention, when the input of the neural network is x = (x1, ..., ... i ,…, d When ), the input of the first hidden layer can be obtained through Calculation. The output of the first hidden layer is b. 1j =(a 1j The function g(·) is called the activation function, which greatly increases the nonlinear fitting ability of the neural network. The most commonly used activation function is ReLU:

[0099] g(x) = max(0,x);

[0100] And sigmoid:

[0101]

[0102] For hidden layer l, its input is The output is b lj =(a lj Finally, the output of the neural network can be obtained through... The goal of a neural network model is to find a set of parameters:

[0103]

[0104] Minimize empirical risk (9), where n and m are set according to the number of neurons between different layers. The neural network model is trained by repeatedly using the training dataset to adjust the weights Wi. During training, the model adjusts the weights based on the difference between the predicted results and the actual results of the input data. This process continues until the error reaches a pre-set threshold or the maximum number of training iterations is reached. Generally, neural networks adjust parameter weights using the backpropagation algorithm, and the adjustment of weights is also called parameter optimization. Finally, through training, the neural network model learns how to predict the output results from the input data. During training, the stochastic gradient descent algorithm or other optimization algorithms (such as Adam) are generally used to adjust the weights to minimize empirical risk. Therefore, the key step in training a neural network model is to calculate the gradient to update the parameter matrix. That is, to calculate the gradient. And update the numbers in the following ways:

[0105]

[0106] Where η is the learning rate. For the learning rate between the last hidden layer and the output layer... (l+1)ki , noticed w (l+1)ki First it affects the output value A, then it affects the empirical risk. Its gradient can be calculated using the chain rule:

[0107]

[0108] However, for most optimization problems, empirical risk... It is non-convex and discontinuous; therefore, calculation Regarding A i Calculating the gradient is a significant challenge. This invention proposes a gradient calculation method to address this problem:

[0109] Theorem 1: Where c i It is the unit procurement cost of resources, φ i ,λ i It is the optimal solution to the following optimization problem:

[0110]

[0111]

[0112] μ≤w (12)

[0113]

[0114]

[0115] According to Theorem 1, the parameters for updating the neural network can be calculated, thereby training the neural network.

[0116] In this embodiment of the invention, related numerical experiments were also conducted to test the decision-making performance of the end-to-end model. This embodiment of the invention uses historical real-world demand data to verify the effectiveness of the end-to-end model and compares its performance with two models based on a prediction-then-optimization framework.

[0117] In this embodiment of the invention, a mobile phone remanufacturing problem is considered. The remanufacturer purchases recycled used mobile phones from intermediaries for remanufacturing, and its decision-making process is as follows: Figure 3 As shown.

[0118] This invention tested three decision-making models. The first model is the end-to-end decision-making model (E2E) proposed in this invention. The input of this model is the observed features, and the output is a multi-dimensional vector representing the procurement quantity of each level of resource. The second model is a Monte Carlo simulation model (MCM). This model uses a prediction-then-optimization framework. First, a neural network model is trained to predict demand. Then, the demand samples are resampled based on the prediction residuals. Finally, the demand samples are input into the optimization model to obtain the optimal resource procurement decision. More specifically, it is assumed that the demand follows the form d = f(x) + ε, where ε is a normal distribution with a mean of 0. First, the form of f(x) is fitted using a neural network, and then ε is estimated based on the residuals of the predicted values. In the model testing step, the neural network outputs the predicted value of the demand based on the feature x, and then generates 10 demand samples based on the Monte Carlo simulation sampling of ε. These samples are substituted into the model to solve for the optimal resource procurement decision. The third model is a point estimation model (PEM). The difference between the PEM model and the MCM model is that the PEM model does not have a Monte Carlo sampling step. The PEM model directly substitutes the output predicted demand values ​​into the optimization model for solution. The PEM model also belongs to the decision framework of prediction before optimization. PEM is a relatively easy-to-implement model, and due to its simplicity and intuitiveness, it is commonly used in research and practice. This embodiment of the invention uses PEM as a benchmark model where prediction and optimization are completely independent.

[0119] In this embodiment of the invention, the neural network model adopts a five-layer network architecture. The number of neurons per layer in the E2E model is [11, 5, 5, 5, 3], while the number of neurons per layer in the MCM and PEM models are [11, 5, 5, 5, 1], respectively. All neural networks use the ReLU activation function, where...

[0120] ReLU(x) = max(0,x). The model training learning rate was 0.1, with 200 training epochs, and the parameter optimizer was adam. All models were implemented using Python 3.9. The optimization problem was solved using Gurobi 10.0.0 with default settings. The neural network model was implemented using the PyTorch 1.13.1 framework. Numerical experiments were conducted on a computer running Windows 10, equipped with an Intel Core i7 3.40GHz processor and 48GB of RAM.

[0121] In this embodiment of the invention, the model is tested on a real mobile phone production dataset, which consists of mobile phone production data. The original data comes from the National Bureau of Statistics of China. Since many years do not include production data for January and February, after cleaning and processing, 210 months of mobile phone production data from 2001 to March to December 2021 are retained. In this embodiment, the data is numbered from 0 to 209 according to time sequence, with the vertical axis representing production demand.

[0122] In this embodiment of the invention, the dataset contains 170 randomly selected training data sets (approximately 81% of the total data) and 40 randomly selected test data sets (approximately 19% of the total data). After the model is trained on the training set, the effectiveness of the model's decisions is verified using the test set. The metric is the total cost incurred by the decision (procurement cost + remanufacturing cost + stockout cost + processing cost - sales revenue). The dataset is randomly divided 10 times to eliminate the influence of randomness on the experimental results. The model's input feature is an 11-dimensional variable. The first dimension is the year feature, which is an integer variable from 0 to 20. The next ten dimensions are month features using one-hot encoding. For example, [1,0,0,1,0,0,0,0,0,0,0] represents the month of May 2002.

[0123] Set the model parameters as follows: Unit purchase cost: c = [20, 30, 40], Unit remanufacturing cost:

[0124] r = [45, 30, 15], unit processing cost: s = [0, 0, 0], unit selling price: p = 120, unit stockout penalty cost: w = αp, with the ratio α varying within the range of [0.5, 3]. Higher α values ​​represent higher stockout penalty costs. To reflect real-world conditions, most test cases have an α value greater than 1.

[0125] In this embodiment of the invention, the empirical error of the E2E model is measured by formula (9), rather than the traditional mean squared error (MSE). In this embodiment, the E2E model training converges very quickly. When the number of training epochs is 10, the training error and test error are -423579 and -522861, respectively. When the number of training epochs is 100, the training error and test error of the model basically converge, and the model can be considered to have been trained successfully. During the training of the MCM / PEM model, the empirical error is measured by the mean squared error. When the number of training epochs is 100, the error still fluctuates, only stabilizing when the number of training epochs is greater than or equal to 150. This indicates that compared with the MCM / PEM model, the E2E model requires fewer training epochs and converges faster.

[0126] In this embodiment of the invention, test results show that the total inventory level output by the E2E model increases with increasing demand, generally slightly higher than the actual demand. Simultaneously, as the α value increases from 0.5 to 3, the total inventory level also increases. This means that as the α value increases, the unit stockout cost rises, and the E2E model learns to increase inventory levels to reduce stockout risk. The MCM model shows a similar trend, but its sensitivity to the α value is lower than that of the E2E model. The PEM model's decision largely ignores the impact of the α value. When t≥150, the actual demand shows a downward trend, but none of the neural network models learned this change; their predicted values ​​increased over time. This is because the yearly and monthly features contain too little information, causing the neural network to fail to learn the subtle changes in demand over time. Using time series as features to test the model can effectively solve this problem. Specifically, features observed in the current period and a preset number of previous historical periods can be used as features observed in the current period.

[0127] In this embodiment of the invention, the remanufacturing costs generated by the three model decisions are also compared. The cost of the PEM model increases rapidly with the increase of the α value. Because the PEM model does not consider information from downstream optimization tasks, that is, PEM only considers how to more accurately predict demand, both overly high and underlying predictions will result in symmetrical losses. However, as the α value increases, for remanufacturing companies, the unit stockout cost is much higher than the unit inventory cost. Companies are more willing to maintain a certain inventory of recycled materials to prevent stockouts. Therefore, overly high demand predictions will not cause a rapid increase in company costs, while underlying demand predictions will cause a significant increase in stockout costs, but the PEM model does not capture this kind of information. Therefore, if the prediction model does not utilize information from the downstream optimization model, and decision-makers make decisions directly based on the prediction values, undesirable results are likely to occur. The MCM model performs better than the PEM model, and its cost does not increase rapidly with the increase of the α value. This is because the MCM model extracts more information about demand from historical data. The MCM model learns the mean and variance of demand from historical data. When making decisions, the MCM model generates samples using Monte Carlo simulations and then makes optimal decisions based on these samples. In solving the optimization model, higher α values ​​are captured by the samples generated by the Monte Carlo simulation method, allowing the model to make the decision that minimizes remanufacturing costs. This method significantly improves upon the shortcomings of the PEM model, but two problems remain: first, model training requires assuming a distribution of demand (or residuals) and then learning the parameters of that distribution; second, decision-making requires Monte Carlo simulations and then building an optimization model based on the samples to solve the problem. When the number of samples is large, the difficulty of solving the model increases, affecting the timeliness of decision-making.

[0128] The E2E model is the best-performing model. Its total cost increases slowly with increasing α, and the increase is slightly smaller than that of the MCM model. The E2E model fully utilizes information from the optimization model during training, so its decisions adjust with changes in α, preventing a rapid increase in total cost. Furthermore, this end-to-end model architecture does not require excessive assumptions about demand distribution, making it easier to deploy in practice than the MCM model. In addition, when making decisions, the E2E model does not need to call an optimization solver to solve the optimal inventory purchase decision for recycled materials; it only needs to obtain the decision result through the forward propagation of the neural network. When the optimization model is complex, its decision-making efficiency is much higher than that of the MCM model. Overall, compared to the MCM model, the E2E model can increase profits by 5%-10%; compared to the PEM model, when α=3, the E2E model can increase profits by 182%.

[0129] As the α value increases, the procurement costs of both the E2E and MCM models increase, which can be explained by safety stock. Higher safety stock is an effective way to cope with higher unit stockout costs. With increasing α, both the E2E and MCM models purchase more recycled materials to prevent stockouts from increasing costs. Stockout costs increase sharply in the PEM model, while they only increase slightly in the E2E and MCM models. Remanufacturing costs in the PEM model are relatively stable, while remanufacturing costs in the E2E and MCM models increase slightly with increasing α. The same trend is reflected in remanufacturing revenue. This can be explained by the quantity demanded: as α increases, inventory levels rise in both the E2E and MCM models, the quantity demanded increases, and therefore both remanufacturing costs and revenues increase.

[0130] In this embodiment of the invention, the impact of time series features was also tested. Specifically, the demand from the past 11 periods (to correspond to the 11 features of year and month) was used as the feature input model. That is, when predicting the demand for the 12th period, the input features are the actual demand for the 1st to 11th periods.

[0131] Time series features contain more information, resulting in better fitting performance for neural network models. For different grades of recyclables, the E2E model tends to purchase more Grade 3 recyclables. This is because Grade 3 recyclables have the lowest total remanufacturing cost (purchase cost plus remanufacturing cost). Therefore, if demand is known in advance, the optimal purchasing decision is to purchase Grade 3 recyclables in quantities equivalent to demand. The second most purchased recyclable by the E2E model is Grade 1 recyclables. Although this grade has the highest total remanufacturing cost, its purchase cost is the lowest. When demand is lower than the purchase quantity, Grade 1 recyclables incur a loss of only 20%, while Grade 3 recyclables incur a loss of 40. Therefore, Grade 1 recyclables can be used to mitigate the risk of a mismatch between purchase quantity and demand. The decision results of the MCM model are similar to those of the E2E model.

[0132] Meanwhile, when features contain more information, the E2E model can learn some management insights. Intuitively, increasing the inventory level (purchase quantity) of Tier 1 recycled materials is a better strategy when demand becomes more volatile. Although its total remanufacturing cost is 10 higher than that of Tier 3 recycled materials, its scrap loss is 20 lower when the inventory level (purchase quantity) is higher than that of Tier 3 recycled materials. Therefore, increasing the inventory level of Tier 1 recycled materials can better cope with large fluctuations in demand; that is, excessively high or low demand will not cause costs to increase rapidly. In the time range of 125-175, demand fluctuates greatly, and it can be seen that the E2E model increases the inventory level of Tier 1 recycled materials during this period. At the same time, the MCM model does not learn a good strategy to cope with the increase in demand volatility. This is because the MCM neural network model does not consider the information of the optimization model during the training process, but introduces the information of the optimization model through subsequent simulation sampling steps. Therefore, its learning effect is not as good as that of the E2E model.

[0133] Based on the above numerical experiments, the following conclusions can be drawn:

[0134] The E2E model proposed in this invention outperforms the PEM and MCM models. During training, the E2E model can utilize information from downstream recycled goods inventory control issues to guide model training, thereby outputting a better inventory control strategy. The advantage of the E2E model increases with the value of α.

[0135] The E2E model proposed in this invention is highly efficient to train, requiring only 100 training epochs. After training, the decision-making process no longer depends on solvers (such as Gurobi, Cplex, etc.), which can significantly improve decision-making efficiency in practical applications.

[0136] The E2E model proposed in this invention can learn some inventory control strategies, such as increasing the total inventory level to reduce the risk of stockouts when stockout costs rise, and increasing the corresponding level of inventory to cope with the risk when demand fluctuations increase.

[0137] Based on the same inventive concept, embodiments of the present invention also provide a single-cycle end-to-end inventory control device that considers substitutability, the device comprising:

[0138] The strategy generation module is used to determine the features observed in the current period, input the features into the end-to-end inventory decision model, and obtain the inventory control strategy. The inventory control strategy is represented as a multi-dimensional vector, representing the purchase quantity of each level of resource.

[0139] The training samples of the end-to-end inventory decision model include: sample features and corresponding historical real demand. The training process of the end-to-end inventory decision model is unlabeled learning. The output of the end-to-end inventory decision model is an inventory control strategy A = h(x) that minimizes the empirical cost. The minimized empirical cost is expressed as:

[0140] in, It is a neural network model h in a given dataset H n The experience cost below, H n =[(x1,d1),…,(x n ,d n )], It is the collection of all neural network models, x i Let d represent the features of the i-th sample. i This represents the historical actual demand for the i-th sample.

[0141] Optionally, the input to the neural network model is x = (x1, ..., x...). i ,…,x d In the case of ), the input of the first hidden layer is passed through The calculation shows that the output of the first hidden layer is b. 1j =g(a 1j ), where the function g(·) is the activation function; for the hidden layer l, its input is The output is b lj =g(a lj The output of the neural network model is obtained through... calculate.

[0142] Optionally, the training objective of the neural network model is to find a set of parameters:

[0143]

[0144] Minimize experience costs;

[0145] Where n and m are set according to the number of neurons between different layers;

[0146] The neural network model is trained by repeatedly adjusting the weights Wi using the training dataset.

[0147] Optionally, the gradient is calculated. The weights are updated in the following ways:

[0148]

[0149] Where η is the learning rate.

[0150] Alternatively, the gradient can be calculated using the chain rule:

[0151]

[0152] in, Where c k It is the unit procurement cost of resources, φ i ,λ i It is the optimal solution to the following optimization problem:

[0153] max∑ i (φ i -λ i A i +μd;

[0154]

[0155] μ≤w;

[0156]

[0157]

[0158] Among them, A i This represents the quantity of resources purchased at level i; p represents the unit selling price of the product, and r i w represents the unit service cost of resource level i, and w represents the unit penalty cost for unmet demand; s i This indicates the unit processing cost of resource level i;

[0159] According to the backpropagation calculation formula of neural networks, Among them, b lk This represents the output of the k-th neuron in the hidden layer l of the neural network.

[0160] Optionally, the neural network model adopts a five-layer network architecture, with each layer having 11, 5, 5, 5, and 3 neurons, and uses the ReLU activation function.

[0161] Optionally, determining the features observed in the current period includes: taking the features observed in the current period and a preset number of previous historical periods as the features observed in the current period.

[0162] Based on the same inventive concept, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the single-cycle end-to-end inventory control method considering substitutability described in any of the above embodiments.

[0163] Based on the same inventive concept, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the single-cycle end-to-end inventory control method considering substitutability described in any of the above embodiments.

[0164] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0165] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable single-cycle end-to-end inventory control terminal device considering alternatives to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable single-cycle end-to-end inventory control terminal device considering alternatives, generate instructions for implementing the process... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0167] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable, alternative-considering, single-cycle, end-to-end inventory control terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable terminal device that considers alternative single-cycle end-to-end inventory control, causing a series of operational steps to be executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0169] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0170] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0171] The foregoing has provided a detailed description of a single-cycle end-to-end inventory control method and product that considers substitutability. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A single-cycle end-to-end inventory control method considering substitutability, characterized in that, The method includes: The observed features in the current period are determined, and the features are input into the end-to-end inventory decision model to obtain the inventory control strategy. The inventory control strategy is represented as a multi-dimensional vector, which represents the purchase quantity of each level of resource. The training samples for the end-to-end inventory decision model include: sample features and corresponding historical real demand. The optimal inventory control strategy is unknown during the training process. The training process is unlabeled learning. The output of the end-to-end inventory decision model is the inventory control strategy that minimizes empirical costs. The minimized empirical cost is expressed as: ； in, It is a neural network model In a given dataset The experience cost of the downfall, , It is the collection of all neural network models. Indicates the first Features of each sample Indicates the first The historical real needs of each sample; Calculate the gradient using the chain rule: ； in, ,in It is the unit procurement cost of resources. It is the optimal solution to the following optimization problem: ； ； ； ； ； in, Indicates the level as The quantity of resources procured; This indicates the unit price of the product. Indicates level Resource unit service cost This indicates the penalty cost for units that fail to meet demand; Indicates level Resource unit processing cost; The gradient is calculated using the backpropagation formula of the neural network. = ,in, For the hidden layer of a neural network The output of the kth neuron.

2. The single-cycle end-to-end inventory control method considering substitutability according to claim 1, characterized in that, The input to the neural network model is In this case, the input of the first hidden layer is passed through The calculation shows that the output of the first hidden layer is... , where the function ( ) is the activation function; for hidden layers Its input is The output is The output of the neural network model is obtained through... calculate.

3. The single-cycle end-to-end inventory control method considering substitutability according to claim 2, characterized in that, The training objective of the neural network model is to find a set of parameters: Minimize experience costs; in The settings are based on the number of neurons between different layers; The neural network model adjusts its weights by repeatedly using the training dataset. To conduct training.

4. The single-cycle end-to-end inventory control method considering substitutability according to claim 3, characterized in that, Calculate gradient The weights are updated in the following ways: ； in This is the learning rate.

5. The single-cycle end-to-end inventory control method considering substitutability according to claim 2, characterized in that, The neural network model adopts a five-layer network architecture, with each layer containing 11, 5, 5, 5, and 3 neurons, and uses the ReLU activation function.

6. The single-cycle end-to-end inventory control method considering substitutability according to claim 1, characterized in that, Determine the features observed in the current period, including: taking the features observed in the current period and a preset number of previous historical periods as the features observed in the current period.

7. A single-cycle end-to-end inventory control device considering substitutability, characterized in that, The device includes: The strategy generation module is used to determine the features observed in the current period, input the features into the end-to-end inventory decision model, and obtain the inventory control strategy. The inventory control strategy is represented as a multi-dimensional vector, representing the purchase quantity of each level of resource. The training samples for the end-to-end inventory decision model include: sample features and corresponding historical real demand. The optimal inventory control strategy is unknown during the training process. The training process is unlabeled learning. The output of the end-to-end inventory decision model is the inventory control strategy that minimizes empirical costs. The minimized empirical cost is expressed as: ； in, It is a neural network model In a given dataset The experience cost of the downfall, , It is the collection of all neural network models. Indicates the first Features of each sample Indicates the first The historical real needs of each sample; Calculate the gradient using the chain rule: ； in, ,in It is the unit procurement cost of resources. It is the optimal solution to the following optimization problem: ； ； ； ； ； in, Indicates the level as The quantity of resources procured; This indicates the unit price of the product. Indicates level Resource unit service cost This indicates the penalty cost for units that fail to meet demand; Indicates level Resource unit processing cost; The gradient is calculated using the backpropagation formula of the neural network. = ,in, For the hidden layer of a neural network The output of the kth neuron.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the single-cycle end-to-end inventory control method that takes into account substitutability as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the single-cycle end-to-end inventory control method that takes into account substitutability as described in any one of claims 1-6.

Images