Perishable product pricing and joint ordering method considering time reference price effect

Abstract

The invention discloses a perishable product pricing and joint ordering method based on multi-thread asynchronous deep reinforcement learning and considering a time reference price effect, and belongs to the technical field of perishable product supply chain front-end retail management. According to the method, a dynamic planning model of joint pricing and ordering considering a time reference price effect is formulated, the proposed method can select an effective and reasonable pricing and ordering strategy in interaction with a plurality of environments with uncertain requirements, and then on the basis of the strategy, a dynamic pricing and ordering scheme in an optimal limited period T is given; according to the invention, psychological expected prices of customers for perishable products in the hot sales season can be quantified; the relationship among the retailer pricing, the ordering scheme and the customer demand is balanced; and reference can be provided for front-end enterprises of the perishable product supply chain in the market development stage.

Description

Technical Field

[0001] The present invention relates to the field of front-end retailer operations management in perishable goods supply chains, focusing on how retailers can meet customer demand influenced by the time reference effect by setting dynamic replenishment and pricing strategies, optimize inventory management to reduce perishable goods losses, and improve operational efficiency and sales profits. Specifically, it relates to a perishable goods pricing and joint ordering modeling and multi-threaded asynchronous deep reinforcement learning method that considers the time reference price effect. Background Art

[0002] The perishable goods retail industry holds a significant market share within the commodity retail sector, particularly in the food and agricultural product sectors. Perishables fall into two categories: one is items whose physical state or quality deteriorates over time, such as vegetables, fish, and dairy products; the other is items whose market demand fluctuates rapidly around events, such as turkey and mooncakes, which are popular during Thanksgiving in Western countries and Mid-Autumn Festival in China. In real life, retailers primarily selling perishables in the second category often implement pre-holiday stocking, continuous sales, post-holiday clearance sales, or seasonal sales within a limited planning period around holidays. This allows them to better respond to time-sensitive demand fluctuations, monitor customer needs and feedback, and manage costs and profits.

[0003] Objectively, based on frequent transactions and past shopping experiences, customers develop a pattern of reference price expectations based on previously known or encountered prices. This provides them with an internal standard for evaluating specific sales prices. Subjectively, temporal scarcity also significantly influences how customers value items on shelves. For example, retailers often concentrate production and sales of mooncakes around holidays. This temporal concentration makes mooncakes relatively scarce for a short period of time, thereby inflating customer valuations of these products. Once a certain point in time has passed (such as before a holiday or other important deadline), customers experience less time pressure and their price expectations adjust accordingly, anticipating a price drop and feeling less urgency to purchase. Therefore, it is important not only to consider past prices but also to characterize the influence of time on customers' subjective valuations. The temporal reference price effect combines the influence of temporal scarcity and reference price effects, considering the temporal element in customers' price perception and decision-making processes for goods or services.

[0004] Retailers can use web-based technologies to collect valuable information and adjust decision-making factors, especially sales prices, in real time to attract more customers and achieve higher profitability. Customers can use web search technology to quickly access product price information and form internal reference prices.

[0005] Furthermore, assimilation-contrast theory suggests that customer reactions depend on whether the price is within an acceptable range, and certain factors (e.g., competition level, brand loyalty, and customer learning) may influence the relative width of this acceptable range for customers. Selling price changes within this range will not have a significant impact on customers' purchasing decisions, and the actual selling price can be considered similar to the customer's psychological price expectations. When price changes exceed this acceptable range, a reference price effect may occur, that is, it may have a significant impact on customers' brand selection or purchasing behavior through loss or gain effects, thereby affecting purchasing demand. Therefore, retailers should consider the time reference price effect with a threshold, which helps retailers set a reasonable selling price threshold so that it is not sufficient to influence customer purchasing behavior under the same price discount.

[0006] The emergence of information technologies such as RFID tags has enabled retailers to accurately track product status, enabling them to make timely and dynamic pricing and inventory decisions. However, considering pricing and inventory control independently may lead to poor performance of the entire system. Joint dynamic pricing and ordering enable retailers to respond to market changes more quickly. It can effectively balance supply and demand and significantly improve profitability. Deep reinforcement learning methods (DRL) mainly use neural networks as function approximators to approximate the Q-value function, value function or strategy in reinforcement learning. The application of DRL in perishable inventory management is mainly reflected in the optimization of dynamic pricing and inventory control strategies. In practical applications, it is necessary to have a special definition of action-state-reward and model the problem as a Markov decision process in order to design a corresponding algorithm framework. It can help companies achieve optimal inventory management in complex and changing supply chain environments.

[0007] Currently, there is little research on solving the joint pricing and ordering problem of perishable agricultural products with time scarcity characteristics. The dynamic programming solution is computationally expensive, and a more efficient solution method is needed. Summary of the Invention

[0008] The present invention solves the problem of periodically reviewing pricing and ordering a single product in an offline channel within a limited planning period T. The single product has a time scarcity characteristic, and the time reference price effect affects customers' willingness to purchase the product. The specific problem is described as follows:

[0009] At the beginning of each period t, the retailer observes the time reference price B of the customer group t and the initial inventory level x t , and use this to determine the order quantity u t and sales price p t When the current time reference price B t and sales price p tWhen the affected demand appears, the retailer obtains profit Q after completing the sales behavior t The unsold inventory has a perishable rate q during storage t , so there is a rotten inventory f that needs to be processed t Finally, the remaining inventory at the end of the period is used as the initial inventory level x for the next period. t+1 . According to the order quantity t Incur ordering costs, combined with the inventory of rotten goods f t Handling costs are incurred, while inventory holding costs and backorder costs are based on the final inventory level at the end of the period x t+1 Finally, the reference price B for the next period t+1 Update according to the psychological laws of customers. Figure 1 Schematic diagram showing the interaction between retailers and customer groups.

[0010] This invention aims to overcome the aforementioned shortcomings of existing technologies by incorporating a time-referenced price effect framework to quantify customers' perceived price and establish a dynamic programming model for pricing and co-ordering within a retailer's finite timeframe. This method provides a perishable product pricing and co-ordering modeling and multi-threaded asynchronous deep reinforcement learning method that considers the time-referenced price effect. This method can determine the retailer's optimal selling price and order quantity plan under dynamic conditions, thereby maximizing total sales profit within a finite timeframe, where costs include inventory holding costs, stockout costs, disposal costs, and ordering costs.

[0011] The proposed method for perishable goods pricing and joint ordering modeling, combined with multi-threaded asynchronous deep reinforcement learning, that considers the time reference price effect is scientifically sound, computationally fast, easy to use, and highly versatile. It not only reduces waste and excess inventory for businesses, but also manages the risks associated with price fluctuations. It provides data-driven decision support and helps formulate more effective business strategies, demonstrating its practical application.

[0012] To achieve the above objectives and solve the problem of pricing and joint ordering of perishable goods considering the time reference price effect, the present invention provides a multi-threaded asynchronous deep reinforcement learning technology, comprising the following steps:

[0013] Step 1: Establish a time reference price effect framework with a non-zero threshold

[0014] When customers receive information about the difference between the sales price and the time-reference price, there's often an insensitive zone. If the information falls within this insensitive zone, the customer's psychological mechanism isn't activated. When the price difference exceeds the profit or loss threshold—the boundary of the insensitive zone—a corresponding profit or loss effect occurs. Therefore, market demand is not only determined by traditional supply and demand theory but also influenced by customer psychological effects. In step one, the time-reference price effect is used to describe customer psychological effects. For time-reference prices with non-zero thresholds, a three-stage piecewise function is typically established:

[0015]

[0016] In the formula, τ(≥0) is the profit threshold, ρ(<0) is the loss threshold, and the customer group’s sensitivity to the difference between the time reference price and the sales price is mainly reflected in R′(B t -p t ,τ,ρ), if η G >η L When η G <η L When η G =η L When , the customer has no preference. t is the retailer’s selling price in period t, B t is the time reference price in the customer’s mind, which combines the time scarcity theory with the reference price effect. Its functional form is:

[0017] B t = ωr t +(1-ω)U t (2)

[0018] Where r t Indicates the customer's psychological expected price, U t ω represents the customer's perceived price of time, while ω indicates the customer group's sensitivity to psychological price expectations. The smaller ω is, the more likely the customer group is to use time-perceived price to assess their mental reference price for time. Conversely, the smaller ω is, the more likely the customer group is to underestimate the impact of time scarcity.

[0019] General t It is only affected by the price of the previous period and is not affected by time factors. Considering the r of the previous period t , using exponential smoothing to calculate the psychological expected price r t+1 It is expressed in the form of:

[0020] r t+1 =θr t +(1-θ)p t (3)

[0021] Among them, θ is the memory factor. The larger θ is, the more customers value the psychological expected price.

[0022] Retailers supply perishable goods with festive significance within a limited planning period. These perishables hold significant appeal for customers both before and after the event and are essential for daily life. During the period leading up to the holiday, demand for these perishables is high. Customers, unwilling to bear the risk of shortages, overvalue these potentially scarce perishables and are more willing to pay a higher price. However, after the holiday, as time passes, consumer enthusiasm gradually decreases, and customer desire for these perishables gradually declines. The impact of the time scarcity of these perishables on customers' subjective price valuations gradually decreases and stabilizes. The formula for customers' time-perceived price valuations, driven by time scarcity, is as follows:

[0023] U t+1 = p t β(t) (4)

[0024] Among them, β(t)>0 is the valuation coefficient, which represents the customer's perception coefficient of price at the time level, U t+1 Indicates that customers rely on p t The perceived price at the time given in period t+1 represents the psychological premium that customers pay to cope with the risk of possible shortages of goods, or the conservative valuation that customers are more willing to bear after the consumption enthusiasm gradually cools down.

[0025] This paper assumes that demand for perishable goods is directly influenced by price. Customers factor in time scarcity and make psychological valuations of past prices. This constant comparison with changing current prices causes demand to fluctuate over a finite planning horizon. The effect of time scarcity on demand at a specific point in time can be understood as time indirectly influencing demand changes through price comparisons. Therefore, the valuation coefficient β(t) is assumed to follow a quadratic polynomial function over a finite planning horizon, as follows:

[0026] β(t)=a(t-t0) 2 +c (5)

[0027] Where t0>0 indicates the time when the valuation coefficient is the largest, which may be close to a point around the holidays. a<0 indicates that the valuation coefficient increases first and then decreases over time.

[0028] Step 2: Build a dynamic pricing and joint ordering model with perishable rate distribution and separate the value function

[0029] In step 2, the perishability rate q t It usually refers to the decay loss rate of goods during storage, transportation and sales, which can be expressed as:

[0030]

[0031] q t Obeys the Beta distribution with parameters ξ=(λ,ν) and mean The variance is The perishability rates in each period are independent and identically distributed.

[0032] In step 2, the total profit function of the dynamic pricing and joint ordering model within a limited period T is as follows:

[0033]

[0034] Among them, Q t (p t ,u t ) indicates that the retailer sets the price p in period t t , order u t Single-period profit income; Q T (p T ,u T )

[0035] The retailer sets the price p in the last period. T , order u T single-period profit income.

[0036] Q t (p t ,u t )= p t S t (x t ,u t ,p t ,B t )-c t u t -A t (x t ,u t ,p t ,B t ,q t ,w t )-K t (x t ,u t ,p t ,B t ,q t ) (8)

[0037] Then there is a unit ordering cost c t ( <p min ). Sales volume is S t (x t ,u t ,p t ,Bt )=min{x t +u t ,D(p t ,B t )}.

[0039] Demand t It is about the sales price p t and the time reference price effect R(B t -p t ,τ,ρ).

[0040] Add a random variable σ t To express the deviation between actual demand and expected demand, reflecting demand uncertainty, and obeying normal distribution, where E[σ t ]=0. Therefore, the actual demand function in period t is:

[0041] D(p t ,B t )=d(p t )+R(B t -p t ,τ,ρ)+ σ t (9)

[0042] The inventory cost is K t =hz + -vz - , where h(>0) is the unit holding cost and v(>0) is the unit stockout cost. + =max{z,0} and z - =min{z,0} is the function form for calculating the actual inventory level at the end of the period. In this paper, z=x t +u t -S t (x t ,u t ,p t ,B t ). Of course, due to the occurrence of decay, the perishability rate q also needs to be considered t Due to the impact of , retailers need to clean up the rotten goods for the sake of cleaning the environment and ensure that the treatment of rotten goods meets the relevant requirements. Then there is a unit cost of disposing perishable goods w t ( <c t ).

[0043] Therefore, the expected inventory costs and perishable disposal costs per period are:

[0044] K t (x t ,u t ,p t ,B t,q t )=h(x t +u t -S t (x t ,u t ,p t ,B t )) + ×(1-q t )-v(x t +u t -S t (x t ,u t ,p t ,B t )) - (10)

[0046] A t (x t ,u t ,p t ,B t ,q t ,w t )=w t ×(x t +u t -S t (x t ,u t ,p t ,B t )) + ×q t (11)

[0047] The state of the dynamic pricing and joint ordering dynamic programming model is: the time reference price B of the customer group t and the initial inventory level x t , the action is the retailer’s selling price p t and order quantity u t , where p t ∈

[0048] [p min ,p max ],u t ∈[u min ,u max ]. min ,p max Represent the sales price p t The minimum and maximum values ​​are usually approximated to two decimal places, u min ,u max Represents the order quantity u t The minimum and maximum values ​​are generally integers. Each period’s decision variables are randomly sampled from the Beta action distribution given by the action network.

[0049] The shift equation is as follows:

[0050] x t+1 =[x t +u t -S t (x t ,u t ,p t ,B t )] + ×(1-q t ) (12)

[0051] When the scale of the decision stage is large, the more uncertainty the environment brings to the model, the higher the computational complexity index, and the dynamic programming method used for accurate solution is not applicable. This paper adopts the value function approximation optimization method to solve the dynamic pricing and joint ordering model. Maximizing the total profit function is the global optimization goal, and maximizing the value function is one of the means to achieve this global goal. The global optimum is achieved by finding the optimal control strategy at each time step. The value function V(x t ,B t ) represents the optimal total benefit from the beginning to the end of stage t, and the value function needs to satisfy the Bellman recursive equation:

[0052]

[0053] The state transition equations are Equations 3, 4, and 12. γ∈[0,1] is the discount factor, which determines the rate at which the agent's emphasis on future rewards decays. A higher discount factor indicates that the agent places greater emphasis on future rewards and tends to make decisions that maximize long-term benefits. Conversely, a lower discount factor places greater emphasis on immediate rewards, potentially leading to more short-sighted decisions. Since the time period is finite, γ is set to 1.

[0054] Step 3: Use multi-threaded asynchronous deep reinforcement learning technology to solve the dynamic programming model.

[0055] Step three specifically includes the following steps:

[0056] (a) Initialize the action network parameters π θ , initial state s o ,This state includes the inventory state and the customer time reference price state, setting the total number of iterations and the trajectory batch N;

[0057] (b) The cycle length of each trajectory is T. At each stage, the initial state is input into the action network, and the output is the four-parameter list [α1, α2, β1, β2] required to form the Beta distribution. The order quantity action distribution of each period follows u t~beta[α1,β1], the sales price action distribution follows p t ~beta[α2,β2];

[0058] (c) randomly sampling actions based on the above action distribution, where the actions include the retailer's pricing and order quantity for each period, and obtaining the corresponding action sampling probability;

[0059] (d) Calculate the immediate reward for this period based on the single-period profit equation in step 2 and update the end-of-period state, that is, update the beginning-of-period state for the next period, and repeat this process until the end of the planning period;

[0060] (e) In the process of updating the action network parameters, the trajectory set τ obtained after interacting with the environment is used n , according to the policy gradient theorem, the gradient of the value function is calculated. Because of the maximization problem, the action loss is the negative of the gradient.

[0061]

[0062]

[0063] (f) The above steps are the process of interaction between a local working network and the environment. We need to generate multiple working networks and a global network with the same network framework to meet the basis of multi-threaded asynchronous learning and improve the exploration ability of the method; that is, use a multi-core processor to generate working networks in parallel, and repeat steps (a)-(e) for each working network;

[0064] (g) When the training process reaches the set update step, the total reward values ​​of each working network are compared, and the maximum total profit function value is extracted from the working network group. The corresponding working network parameters are updated to the parameter set of the global network;

[0065] (h) Before the working network interacts with the environment, it first copies the parameter set from the global network, then continues training, and repeats step (g) when the update condition is met. Finally, after the number of iterations is reached, the optimal global model after convergence is obtained.

[0066] Step 4: Select the parameters of the last generation of global action network after convergence for testing, define the best action as the mean of the action distribution obtained by the network, and analyze the dynamic pricing and ordering plan of the retailer within a limited period T.

[0067] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows:

[0068] The present invention uses the above-mentioned modeling method and multi-threaded asynchronous deep reinforcement learning technical solution to develop a dynamic programming model for joint pricing and ordering that takes into account the time reference price effect. The proposed method can select effective and reasonable pricing and ordering strategies in multiple interactive environments with uncertain demand, and then, based on this strategy, provide an optimal dynamic pricing and ordering plan within a limited period T. The present invention can quantify customers' psychological expected prices for perishables; balance the relationship between retailers' pricing and ordering plans and customer demand; and provide a reference for front-end enterprises in the perishable supply chain during the market development stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a schematic diagram of the interaction between retailers and customer groups in the present invention.

[0070] Figure 2 It is a flow chart of the joint pricing and ordering method of the present invention.

[0071] Figure 3 It is a working network framework diagram of the present invention.

[0072] Figure 4 It is a diagram of the interaction between the work of the present invention and the global network.

[0073] Figure 5 It is the network weight training convergence diagram of the present invention.

[0074] Figure 6 It is a comparative diagram of the example effects of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0076] Example:

[0077] Many agricultural products are the main components of traditional family banquets. As the Spring Festival approaches, although most agricultural products have increased their prices, they are still very popular and in short supply. Chinese cabbage, as a typical example of such agricultural products, is a main ingredient in traditional Chinese cuisine. This embodiment uses the data from November 2020 to February 2021 in the data set of Problem C of the 2023 "Higher Education Cup" Mathematics Competition for data processing and screening, and uses the logarithmic decay model method to fit the relationship between Chinese cabbage price and demand. The formula is as follows:

[0078] f(p)=-11.30×In(p-0.98)+32.18 (16)

[0079] Assuming that customers only consider the judgment of products from a time scale, we can fit the functional expression of the valuation coefficient β(t). Then, during the four months of the peak season, we use a second-order polynomial fitting model to obtain the formula for β(t):

[0080]

[0081] In order to conduct numerical experiments, this embodiment is carried out in the same operating environment. The operating environment parameters are shown in Table 1:

[0082] Table 1 Operating environment parameters

[0083]

[0084] At the same time, to solve this problem, a series of hyperparameters need to be set. These hyperparameters are selected based on previous research related to reference price effects in the marketing and operations management literature, which ensures that the environment in each experiment has a feasible and common area.

[0085] Table 2 Method related hyperparameter settings

[0086]

[0087] like Figure 3 ,4, a perishable goods pricing and joint ordering modeling and multi-threaded asynchronous deep reinforcement learning method considering the time reference price effect is presented, the method includes the following steps in sequence:

[0088] 1) Obtain the above environmental information and build a deep action neural network with three fully connected layers as the middle layer and one fully connected layer as the output layer. The middle layer uses the tanh activation function, and the dropout function is used to prevent overfitting. The output layer uses the softplus activation function. The optimizer is set to Adam gradient descent optimizer, with an initial learning rate of 0.0001, which decreases adaptively with the number of iterations.

[0089] 2) Initialize 6 action networks with the same structure, one as the global network and the others as working networks.

[0090] 3a) Based on the above hyperparameter settings, establish a time reference price effect framework with a non-zero threshold:

[0091]

[0092] Among them, B t is the time reference price in the customer’s mind, which combines the time scarcity theory with the reference price effect. Its functional form is:

[0093] B t = ωr t+(1-ω)U t (19)

[0094] Based on the function fitted from the data set, define the actual demand function for each period t

[0095] D(p t ,B t )=d(p t )+R(B t -p t ,τ,ρ)+ σ t (20).

[0096] 3b) Then, mathematical derivation is performed to establish the objective function within a finite period T as follows:

[0097]

[0098] Among them, Q t (p t ,u t ) indicates that the retailer sets the price p in period t t , order u t Single-period profit income; Q T (p T ,u T ) indicates that the retailer sets the price p in the last period T , order u T Finally, the value function V(x t ,B t ), represents the optimal total benefit from the beginning to the end of stage t, and the value function needs to satisfy the Bellman recursive equation:

[0099]

[0100] The state transition equations are Equations 3, 4, and 12. γ = 1. The above formulas and indicators are all described in detail in the Summary of the Invention and will not be repeated here.

[0101] 4) The action network acts as an agent. When the current state is input, it outputs two parameters corresponding to the two action beta distributions, thereby determining the action distribution of sales price and order quantity. It also follows the randomness strategy and randomly extracts action values ​​to obtain the logarithmic probability of the extracted action.

[0102] 5) The agent takes actions, interacts with the environment, obtains the single-stage profit value as an immediate reward, and the state of the next stage, and repeats this process until the end of the finite T period, forming a trajectory.

[0103] 6) The agent generates 10 trajectories at the same time to form a trajectory pool. According to the policy gradient formula (14), it calculates the loss of the action network, calculates the network weight gradient, backpropagates, and updates the network weight.

[0104] 7) During the iteration process, all five working networks will repeat steps 4-6 until the update step is reached. The total reward values ​​of each working network are compared, and the global reward value is updated preferentially. The parameter set of the global network is updated to the best working network parameter set. After that, the weights of all working networks are downloaded from the updated global network parameters and retrained until the update conditions are met.

[0105] 8) After the above training process reaches the maximum number of iterations, the training is stopped and the last generation of action network model after convergence is taken for testing. During the test, the mode of the action distribution is taken for each action.

[0106] In order to demonstrate the optimization and reproducibility of the method, an experimental design was conducted with a maximum number of iterations of 2000 and different perishable rate distribution lists (random number seeds are 50, 100, 150, 200), for the combinations of action network hidden layers (2, 3, 4, 5), number of action network nodes (64, 128, 256), initial learning rate of action network (0.1, 0.01, 0.001, 0.0001), number of environments (5, 10, 15) and update steps (100, 200, 300, 400). The experiments were repeated many times in the hope of obtaining the parameter combination with the optimal results.

[0107] like Figure 5 As shown in the figure, the optimal convergence curve is obtained when the action network has 3 hidden layers, the number of action network nodes is 128, the initial learning rate of the action network is 0.0001, the number of environments is 5, and the update step is 200. Compared with the convergence result of a single training network (yellow line), the effect is improved to a certain extent. During the training process, the model is expected to reach convergence at around 1000, and the average total profit function that the trained policy network can provide is around 7600, which is consistent with reality.

[0108] The present invention selects the parameters of the last generation of action network after convergence for testing, and forms a new trajectory based on 120 stages of environmental interaction. It is compared with the model test scheme trained in an ideal state (the pricing scheme is consistent with the scheme proposed by the network model, but the order quantity is equal to the expected demand), the fixed pricing and dynamic ordering scheme, and has certain significant effects.

[0109] like Figure 6As shown, the solution tested using this method achieved higher average profit per period than the single-environment, manually fixed-pricing method, and was closer to the ideal. Furthermore, compared to the same order level, the average inventory level was lower than that of other methods. While maintaining a higher sales price and time-based price, it still maintained relatively good sales volume, though still somewhat below the ideal. The results showed an increase of approximately 15% over the original solution (profit value: 5840). This was within 120 days; the improvement would be even more significant over a period of one year or longer.

[0110] In order to show the results of the plan for reference by decision makers, Table 3 shows that 10 groups of states are randomly generated according to the Monte Carlo simulation method, and then the optimal action plan generated by the model based on these states.

[0111] Table 3 Dynamic pricing and ordering scheme

[0112] Inventory Status Time reference price status Average selling price Average order quantity 0 1.0 4.960 36 1 1.5 4.968 33 2 2.0 4.972 34 3 2.5 4.977 34 4 3.0 4.974 35 5 3.5 4.967 36 6 4.0 4.942 31 7 4.5 4.970 36 8 5.0 4.961 32 … … … …

[0113] Although the decision variables change slightly, the differences between the plans are not significant due to the limited range of the decision variables. However, the longer the planning period, the better the model training results. Over time, this leads to greater profit differences, a result of the dynamic interaction between the action network and the environment. As can be seen from the above examples, the present invention utilizes the aforementioned specific method to effectively quantify customers' psychologically acceptable price expectations for perishables, improve retailers' profits within a limited planning period, maintain safety stock levels, and balance retailer pricing with ordering plans and customer demand. This provides a reference for front-end companies in the perishable supply chain during market development.

Claims

1. A method for pricing and joint ordering of perishable goods considering the time reference price effect, characterized in that: The method is used to solve the problem of regular review of pricing and ordering of a single product in an offline channel within a limited planning period T, wherein the single product has time scarcity and perishable characteristics, and the sales behavior to the customer group is affected by the time reference price effect; the method specifically comprises the following steps: Step 1: Establish a time reference price effect framework with a non-zero threshold; Step 2: Establish a dynamic pricing and joint ordering model with perishable rate distribution and separate the value function; Step 3: Solve the dynamic programming model using a learning method based on multi-threaded asynchronous deep reinforcement; Step 4: Select the parameters of the last generation of global action network after convergence for testing, define the best action as the mode of the action distribution obtained by the network, and analyze the dynamic pricing and ordering plan of the retailer in a limited period.

2. The perishable goods pricing and joint ordering method considering the time reference price effect as claimed in claim 1, wherein in step 1, the time reference price effect is used to describe the customer psychological effect, and for the time reference price with a non-zero threshold, a three-stage piecewise function is established: In the formula, τ is the profit threshold; ρ is the loss threshold; p t is the retailer’s sales price in period t; B t It is the time reference price in the minds of customers, combining the time scarcity theory with the reference price effect, and its function form is: B t =ωr t +(1-ω)U t (2) In the formula, r t Indicates the customer's psychological expected price, U t represents the customer's time-perceived price, and ω represents the customer group's sensitivity to the psychological expected price; Consider the previous period r t , using exponential smoothing to calculate the psychological expected price r t+1 It is expressed in the form of: in, is the memory factor, The larger it is, the more customers value the psychological expected price; The customer's time-perceived price valuation due to time scarcity is as follows: U t+1 = p t β(t) (4) Among them, β(t)>0 is the valuation coefficient, which represents the customer's perception coefficient of price at the time level, U t+1 Indicates that customers rely on p t The time-perceived price given in period t+1; The valuation coefficient β(t) obeys a quadratic polynomial function in the finite planning period, and the formula is as follows: β(t)=a(t-t0) 2 +c (5) Among them, t0>0 indicates the moment when the valuation coefficient is the largest; a<0 indicates that the valuation coefficient increases first and then decreases with the change of time.

3. The method for pricing and joint ordering perishable goods considering the time reference price effect as claimed in claim 1, wherein in step 2, the perishable rate q t Refers to the decay and loss rate of goods during storage, transportation and sales, which is expressed as: q t It follows a Beta distribution with parameter ξ=(λ,ν) and mean The variance is The perishability rate in each period is independent and identically distributed; In step 2, within the limited period T, the total profit function of the dynamic pricing and joint ordering model is as follows: in, Q t (p t ,u t ) indicates that the retailer sets the price p in period t t , order u t Single-period profit income; Q T (p T ,u T ) indicates that the retailer sets the price p in the last period T , order u T single-period profit income; Q t (p t ,u t )=p t S t (x t ,u t ,p t ,B t )-c t u t -A t (x t ,u t ,p t ,B t ,q t ,w t )-K t (x t ,u t ,p t ,B t ,q t )(8) Then there is a unit ordering cost c t , c t <p min ; Sales volume is S t (x t ,u t ,p t ,B t )=min{x t +u t ,D(p t ,B t )}; Based on demand t About the sales price p t and the time reference price effect R(B t -p t ,τ,ρ) non-negative function; add a random variable σ t To express the deviation between actual demand and expected demand, reflecting the uncertainty of demand, it obeys the normal distribution, where E[σ t ]=0; then, the actual demand function in period t is: D(p t ,B t )d(p t )+R(B t -p t ,τ,ρ)+ σ t (9) The inventory cost is K t =hz + -vz - , where h is the unit holding cost, v is the unit shortage cost; z + = max{z,0} and z - =min{z,0} is the function form for calculating the actual inventory level at the end of the period; z=x t +u t -S t (x t ,u t ,p t ,B t ) Based on the occurrence of decay and the cleaning of decayed products, and considering the perishability rate q t The impact of the unit disposal cost of perishable goods is w t , w t <c t ; Then, the expected inventory cost and disposal cost of perishable goods in each period are: K t (x t ,u t ,p t ,B t ,q t )=h(x t +u t -S t (x t ,u t ,p t ,B t )) + ×(1-q t )-v(x t +u t -S t (x t ,u t ,p t ,B t )) - (10) A t (x t ,u t ,p t ,B t ,q t ,w t )=w t ×(x t +u t -S t (x t ,u t ,p t ,B t )) + ×q t (11) The state of the dynamic pricing and joint ordering model is: the time reference price B of the customer group t and the initial inventory level x t , the action is the retailer’s selling price p t , and the order quantity u t , where p t ∈[p min ,p max ],u t ∈[u min ,u max ]; p min ,p max Represent the sales price p t The minimum and maximum values ​​of u min ,u max Respectively represent the order quantity u t Minimum and maximum values, each period decision obeys Beta distribution; the inventory state transfer equation is as follows: x t+1 =[x t +u t -S t (x t ,u t ,p t ,B t )] + ×(1-q t ) (12) The value function V(x t ,B t ) represents the optimal total benefit from the beginning to the end of stage t, and the value function needs to satisfy the Bellman recursive equation: Among them, the state transfer equations are Equation 3, Equation 4, and Equation 12; γ∈[0, 1] is the discount factor, which determines the decay rate of the agent's emphasis on future rewards.

4. The method for pricing and joint ordering perishable goods considering the time reference price effect as claimed in claim 1, wherein step 3 specifically comprises the following steps: (a) Initialize action network parameters π θ , initial state s o , initial state s o Including inventory status and customer time reference price status values, setting the total number of iterations, and the track batch N; (b) The cycle length of each trajectory is T. At each stage, the initial state is input into the action network, and the output is the four-parameter list [α1, α2, β1, β2] required to form the Beta distribution, forming a fixed action distribution; (c) randomly sampling actions according to the action distribution in (b), where the actions include the retailer's pricing and order quantity for each period, and obtaining the corresponding action sampling probability; (d) According to the single-period profit equation in step 2, calculate the immediate reward of the corresponding stage and update the end-of-period state, that is, update the beginning-of-period state of the next period, and repeat this process until the end of the planning period; (e) In the process of updating the action network parameters, the trajectory set τ obtained after interacting with the environment is used n , the gradient of the value function is calculated according to the policy gradient theorem; the gradient update formula is as follows: (f) using a multi-core processor to generate working networks in parallel, and repeating steps (a)-(e) for each working network; (g) When the training process reaches the set update step, the total profit function values ​​of each working network are compared, and the maximum total profit function value is extracted from the working network group, and the corresponding working network parameters are updated to the parameter set of the global network; (h) Before the working network interacts with the environment, it is necessary to copy the parameter set from the global network, and then continue training. When the update condition is met, step (g) is repeated. Finally, after the number of iterations is reached, the optimal global model after convergence is obtained.

Images