Supply chain dynamic resource allocation method and device and computing equipment

Abstract

The invention discloses a supply chain dynamic resource allocation method, which comprises the following steps: acquiring original data in real time, including supply chain core data and external environment data; each node of the supply chain is used as a participant, an average field game model is constructed based on the original data, and the nodes comprise suppliers, warehouses and logistics providers; solving the average field game model by using a Hamiltonian-Jacobian-Bellman equation to obtain an initial optimal strategy of each participant, and further obtaining initial strategy probability distribution; performing iterative optimization on the initial strategy probability distribution by using a deep reinforcement learning algorithm to obtain optimized strategy probability distribution, the optimized strategy probability distribution representing the probability distribution of the strategy selected by each participant in the input state, the input state comprises a group state, an inventory level of each warehouse, an average daily demand of each warehouse and a social media public opinion popularity score; and determining and executing the optimal strategy of each participant in the input state according to the optimization strategy probability distribution. Based on this, a dynamic environment can be effectively handled, and dynamic optimization of supply chain resource allocation is realized.

Description

Technical Field

[0001] This invention relates to the field of supply chain technology, and in particular to a method, device and computing equipment for dynamic resource allocation in a supply chain. Background Technology

[0002] In today's complex ecosystem of globalized supply chains, businesses face the challenges of tens of thousands of orders daily, multi-level supplier collaboration, and real-time market fluctuations. Traditional supply chain management systems, like static maps, cannot cope with dynamically changing market environments. For example, when a supplier is unable to fulfill orders due to capacity constraints, the system cannot predict the response strategies of other suppliers; when sudden promotions cause a surge in demand, the historical data models relied upon by traditional methods cannot be adjusted in time, often leading to inventory shortages or backlogs.

[0003] Existing supply chain management solutions suffer from the following shortcomings: First, traditional linear programming and mixed-integer programming algorithms experience exponentially increasing computational complexity when dealing with large-scale supply chains (e.g., those with more than 1000 nodes), leading to decision-making delays of hours or even days. Second, these systems lack the ability to model the game-theoretic relationships among multiple stakeholders in the supply chain, treating suppliers, logistics providers, and customers as isolated units and failing to simulate the real dynamics of close interactions between market participants. Furthermore, traditional models rely on static historical data, making it impossible to capture sudden changes caused by real-time demand fluctuations, weather events, or social media sentiment. These shortcomings prevent enterprises from achieving true dynamic resource optimization, limiting them to a reactive, responsive approach.

[0004] Therefore, a dynamic resource allocation method for the supply chain is needed to solve the problems existing in the above-mentioned technical solutions. Summary of the Invention

[0005] To address this, the present invention provides a method and apparatus for dynamic resource allocation in the supply chain, in order to solve or at least alleviate the problems mentioned above.

[0006] According to one aspect of the present invention, a method for dynamic resource allocation in a supply chain is provided, executed in a computing device, comprising: acquiring raw data in real time, the raw data including core supply chain data and external environment data, the core supply chain data including daily order volume of the supply chain, inventory level of each warehouse, transportation delay data for each product, and capacity of each supplier, the external environment data including social media sentiment data; constructing an average-field game model based on the raw data, with each node of the supply chain as a participant, the nodes including suppliers, warehouses, and logistics providers; solving the average-field game model using the Hamilton-Jacobi-Bellman equation to obtain the initial optimal strategy for each participant, and obtaining an initial... An initial strategy probability distribution is generated, representing the probability distribution of each participant choosing a strategy under the group state. A deep reinforcement learning algorithm is used to iteratively optimize the initial strategy probability distribution to obtain an optimized strategy probability distribution, representing the probability distribution of each participant choosing a strategy under the input state. The input state includes the group state, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment score. Based on the optimized strategy probability distribution, the optimal strategy for each participant under the input state is determined and executed. The optimal strategy for each participant includes the optimal order allocation quantity for each supplier, the optimal replenishment quantity for each warehouse, and the optimal transportation quantity for each logistics provider.

[0007] Optionally, in the supply chain dynamic resource allocation method according to the present invention, each node of the supply chain is taken as a participant, and an average-field game model is constructed based on the original data, including: obtaining the average inventory state based on the inventory level of each warehouse, and obtaining the average transportation delay state based on the transportation delay data of each product; obtaining the group state of the supply chain based on the average inventory state and the average transportation delay state; constructing an objective function based on the group state of the supply chain with the goal of minimizing the expected utility of each participant, and obtaining the average-field game model based on the objective function, wherein the expected utility of the participant is related to the participant's strategy and the group state.

[0008] Optionally, in the supply chain dynamic resource allocation method according to the present invention, a deep reinforcement learning algorithm is used to iteratively optimize the initial strategy probability distribution to obtain an optimized strategy probability distribution. This includes: taking the group state of the supply chain, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment score as input states, and taking the initial optimal strategy of each participant as an action; constructing a reward function between the initial optimal strategy of each participant and the group state; and iteratively optimizing the initial strategy probability distribution by maximizing the cumulative reward based on the input states, the actions, and the reward function to obtain the optimized strategy probability distribution.

[0009] Optionally, in the supply chain dynamic resource allocation method according to the present invention, the initial policy probability distribution is iteratively optimized by maximizing the cumulative reward based on the input state, the action, and the reward function, including: using a near-end policy optimization algorithm to adjust the parameters of the policy network by maximizing the cumulative reward based on the input state, the action, and the reward function, so as to iteratively optimize the initial policy probability distribution.

[0010] Optionally, the supply chain dynamic resource allocation method according to the present invention further includes: extracting feature data from the raw data; wherein the feature data includes demand volatility, inventory health, and social media sentiment score, and the inventory health is related to the maximum inventory capacity of each warehouse and the average daily demand of each warehouse.

[0011] Optionally, in the supply chain dynamic resource allocation method according to the present invention, the raw data further includes the total daily warehouse demand, the maximum inventory capacity of each warehouse, and the average daily demand of each warehouse; extracting feature data from the raw data includes: calculating the demand volatility based on the total daily warehouse demand; calculating the inventory health based on the maximum inventory capacity of each warehouse and the average daily demand of each warehouse; and analyzing the social media sentiment data using an NLP model to obtain various keywords related to demand, and calculating the popularity score of each keyword to obtain the social media sentiment popularity score.

[0012] Optionally, in the supply chain dynamic resource allocation method according to the present invention, determining and executing the optimal strategy for each participant under the input state based on the optimization strategy probability distribution includes: pushing the optimization strategy probability distribution to the ERP system; determining and executing the optimal strategy for each participant under the input state through the ERP system based on the optimization strategy probability distribution; and collecting execution data.

[0013] Optionally, in the supply chain dynamic resource allocation method according to the present invention, obtaining the initial strategy probability distribution based on the initial optimal strategy of each participant includes: obtaining the initial strategy probability distribution based on the initial optimal strategy of each participant through the Fermi-Dirac distribution.

[0014] According to one aspect of the present invention, a supply chain dynamic resource allocation device is provided, deployed in a computing device, the device comprising: The acquisition module is suitable for acquiring raw data in real time. The raw data includes core supply chain data and external environment data. The core supply chain data includes the daily order volume of the supply chain, the inventory level of each warehouse, the transportation delay data of each product, and the production capacity of each supplier. The external environment data includes social media sentiment data. The module is designed to include each node in the supply chain as a participant and to build an average-field game model based on the raw data. The nodes include suppliers, warehouses, and logistics providers. The solution module is adapted to solve the mean-field game model using the Hamilton-Jacobi-Bellman equation to obtain the initial optimal strategy of each participant, and to obtain the initial strategy probability distribution based on the initial optimal strategy of each participant. The initial strategy probability distribution is used to represent the probability distribution of each participant choosing a strategy in the group state. The optimization module is adapted to use a deep reinforcement learning algorithm to iteratively optimize the initial policy probability distribution to obtain an optimized policy probability distribution. The optimized policy probability distribution is used to represent the probability distribution of each participant choosing a policy under the input state, wherein the input state includes the group state, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment heat score. The execution module is adapted to determine and execute the optimal strategy for each participant under the input state based on the probability distribution of the optimization strategy, wherein the optimal strategy for each participant includes the optimal order allocation quantity for each supplier, the optimal replenishment quantity for each warehouse, and the optimal transportation quantity for each logistics provider.

[0015] According to one aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the supply chain dynamic resource allocation method as described above.

[0016] According to one aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method as described above.

[0017] According to one aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the supply chain dynamic resource allocation method as described above.

[0018] According to the technical solution of this invention, a method for dynamic resource allocation in a supply chain is provided. First, an average-field game model is constructed, treating each node in the supply chain (including suppliers, warehouses, and logistics providers) as participants. This model simulates the real dynamics of close relationships between participants. Second, the Hamilton-Jacobi-Bellman equation is used to solve the average-field game model to obtain the initial optimal strategy for each participant, significantly reducing computational complexity. Furthermore, an initial strategy probability distribution is obtained based on each participant's initial optimal strategy. This initial strategy probability distribution is then iteratively optimized using a deep reinforcement learning algorithm to obtain an optimized strategy probability distribution. Based on this optimized strategy probability distribution, the optimal strategy for each participant under the input state can be determined. Therefore, strategies can be adjusted according to the dynamic changes in the group state of the supply chain, the inventory levels and daily demand of each warehouse, and the popularity of social media sentiment. This effectively addresses dynamic environments and achieves dynamic optimization of supply chain resource allocation.

[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0020] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of the invention will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout the invention, the same reference numerals generally refer to the same parts or elements.

[0021] Figure 1 A schematic diagram of a computing device 100 provided according to an embodiment of the present invention is shown; Figure 2 A flowchart illustrating a supply chain dynamic resource allocation method 200 provided according to an embodiment of the present invention is shown. Figure 3 A schematic diagram of a supply chain dynamic resource allocation device 300 provided according to an embodiment of the present invention is shown. Detailed Implementation

[0022] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0023] To address the problems of high computational complexity, inability to simulate the real dynamics of close relationships between market participants, and inability to cope with dynamic environments in existing supply chain management solutions, this invention innovatively integrates Mean Field Game (MFG) with Artificial Intelligence (AI) to propose a dynamic resource allocation method for the supply chain. First, a mean field game model is constructed by treating each node in the supply chain (including suppliers, warehouses, and logistics providers) as a participant, which can simulate the real dynamics of close relationships between participants. Second, the Hamilton-Jacobi-Bellman equation is used to solve the mean field game model, which can significantly reduce computational complexity. Furthermore, a deep reinforcement learning algorithm is used to iteratively optimize the initial strategy probability distribution, which can effectively cope with dynamic environments and achieve dynamic optimization of supply chain resource allocation.

[0024] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] Figure 1 A schematic diagram of a computing device 100 according to an embodiment of the present invention is shown. Figure 1 As shown, in a basic configuration, computing device 100 includes at least one processing unit 102 and system memory 104. According to one aspect, depending on the configuration and type of the computing device, the processing unit 102 may be implemented as a processor. System memory 104 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 104 includes an operating system 105.

[0026] According to one aspect, operating system 105 is, for example, suitable for controlling the operation of computing device 100. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 1 The basic configuration is illustrated by the components within the dashed lines. According to one aspect, the computing device 100 has additional features or functions. For example, according to one aspect, the computing device 100 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 1The middle part is shown by removable storage device 109 and non-removable storage device 110.

[0027] As stated above, according to one aspect, program module 103 is stored in system memory 104. According to one aspect, program module 103 may include one or more applications. The present invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc.

[0028] According to one aspect, program module 103 may include a plurality of program instructions suitable for executing the supply chain dynamic resource allocation method 200 of the present invention, such that computing device 100 is configured to execute the supply chain dynamic resource allocation method 200 of the present invention.

[0029] According to one aspect, program module 103 may include supply chain dynamic resource allocation device 300, which may be configured to perform the supply chain dynamic resource allocation method 200 of the present invention.

[0030] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 1 Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 100. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0031] According to one aspect, computing device 100 may also have one or more input devices 112, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 114, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 100 may include one or more communication connections 116 that allow communication with other computing devices 118. Examples of suitable communication connections 116 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0032] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program module 103). System memory 104, removable storage device 109, and non-removable storage device 110 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing device 100. According to one aspect, any such computer storage medium can be part of computing device 100. Computer storage media does not include carrier waves or other transmitted data signals.

[0033] According to one aspect, the communication medium is implemented by computer-readable instructions, data structures, program modules 103, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, the communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0034] In an embodiment of the present invention, a computing device 100 is configured to execute the supply chain dynamic resource allocation method 200 of the present invention. The computing device 100 includes one or more processors and one or more readable storage media storing program instructions that, when configured to be executed by one or more processors, cause the computing device to execute the supply chain dynamic resource allocation method 200 of the present invention.

[0035] Figure 2 A schematic flowchart of a supply chain dynamic resource allocation method 200 provided according to an embodiment of the present invention is shown. The supply chain dynamic resource allocation method 200 can be executed in a computing device (such as the aforementioned computing device 100).

[0036] In an embodiment of the present invention, the computing device 100 for executing the supply chain dynamic resource allocation method 200 of the present invention may be a terminal or a server.

[0037] like Figure 2 As shown, the supply chain dynamic resource allocation method 200 includes the following steps 210-250.

[0038] Step 210: The computing device 100 can acquire raw data in real time, including core supply chain data and external environment data.

[0039] Specifically, core supply chain data includes the daily order volume of the supply chain. Inventory levels in each warehouse Shipping delay data for each product Production capacity of each supplier External environment data includes social media sentiment data. Here, subscripts... They represent the first The warehouse, the The first product One supplier; symbol Indicates the first sky.

[0040] In some embodiments, the computing device 100 can collect core supply chain data in real time from the supply chain. In some embodiments, the computing device 100 can obtain external environment data by calling an API interface. The external environment data may include daily weather data. Economic indicators And social media sentiment data.

[0041] In some embodiments, the computing device 100 may acquire historical data in batches every day, such as demand data for each historical day (e.g., each day in the past 30 days) (e.g., total warehouse demand for each historical day, average daily demand for each warehouse) and supplier fulfillment rate data, which may also be used as raw data.

[0042] In some embodiments, the computing device 100 can also preprocess the raw data. Specifically, for missing data (e.g., transportation status), K-nearest neighbor (KNN) interpolation can be used to fill in the missing data. For example, for each missing value, K=5 neighbors with similar geographical locations and transportation types to the target route can be found among all transportation routes, their similarity weights can be calculated, and a weighted average can be performed to fill in the missing value.

[0043] Step 220: The computing device 100 can use each node in the supply chain (including suppliers, warehouses, and logistics providers) as participants to construct an average field game model based on the raw data (preprocessed raw data).

[0044] Based on this, by constructing an average field game model, the traditional multi-person interaction model can be simplified into an interaction model between the average state of individuals and groups, and the real dynamics of close connections between participants in the supply chain can be simulated, as well as real market competition and cooperation behavior.

[0045] Step 230: The computing device 100 can use the Hamilton-Jacobi-Bellman equation to solve the average-field game model to obtain the initial optimal strategy for each participant. Furthermore, the initial strategy probability distribution can be obtained based on the initial optimal strategy of each participant.

[0046] In this embodiment of the invention, the initial policy probability distribution is used to represent (describe) the probability distribution of each participant choosing a policy under the group state, so that each participant can adjust the policy according to the changes in the group state.

[0047] It should be noted that using the Hamilton-Jacobi-Bellman equation to solve the mean-field game model can significantly reduce computational complexity.

[0048] In some embodiments, the initial policy probability distribution can be obtained based on the initial optimal policy of each participant using the Fermi-Dirac distribution.

[0049] Step 240: The computing device 100 can use a deep reinforcement learning (DRL) algorithm to iteratively optimize the initial policy probability distribution to obtain an optimized policy probability distribution.

[0050] Based on this, mean-field game theory models can be integrated with deep reinforcement learning algorithms to achieve dynamic optimization of supply chain resource allocation. It should be noted that using deep reinforcement learning algorithms to iteratively optimize the initial policy probability distribution can overcome the limitations of mean-field game theory models in modeling complex nonlinear relationships, while ensuring the real-time adaptability and global optimality of the policy.

[0051] In this embodiment of the invention, the optimized strategy probability distribution represents the probability distribution of each participant selecting a strategy (executing a strategy action) under the input state, so that each participant can select the optimal strategy under the input state according to the optimized strategy probability distribution. The input state may include, but is not limited to, the group state, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment score. Based on this, the invention can adjust the strategy according to the dynamic changes in the group state of the supply chain, the inventory level and average daily demand of each warehouse, and the social media sentiment, thereby effectively responding to dynamic environments and achieving dynamic optimization of supply chain resource allocation.

[0052] In this embodiment of the invention, the strategy may specifically include order allocation quantity (corresponding to the supplier), replenishment quantity (corresponding to the warehouse), and transportation quantity (corresponding to the logistics provider).

[0053] Step 250: The computing device 100 can determine and execute the optimal strategy for each participant under the input state based on the probability distribution of the optimization strategy. The optimal strategy for each participant includes the optimal order allocation quantity for each supplier, the optimal replenishment quantity for each warehouse, and the optimal transportation quantity for each logistics provider.

[0054] In some embodiments, the computing device 100 is communicatively connected to the ERP system. In step 250, the computing device 100 can push the probability distribution of the optimization strategy (via an API interface) to the ERP system. Then, the ERP system can determine and execute the optimal strategy for each participant under the input state based on the probability distribution of the optimization strategy. At the same time, the ERP system can collect execution data (including actual transportation costs, order completion rates, etc.) and feed it back to the deep reinforcement learning algorithm to form a closed loop of "optimization-execution-feedback".

[0055] Furthermore, in some embodiments, statistical control charts (such as Shewhart charts) can be used to detect anomalies in real-time data such as inventory levels and to issue risk warnings. When an anomaly value is too large, an alternative strategy can be triggered, i.e., the strategy closest to the optimal strategy can be executed.

[0056] In some embodiments, the raw data also includes total daily warehouse demand (i.e., total demand across all warehouses in the supply chain), maximum inventory capacity for each warehouse, and average daily demand for each warehouse.

[0057] After acquiring (and preprocessing) the raw data in step 210, the computing device 100 can further extract feature data from the raw data (the preprocessed raw data). Feature data may include, for example, demand volatility, inventory health, and social media sentiment scores, but the present invention is not limited thereto. It should be noted that inventory health is related to the maximum inventory capacity of each warehouse and the average daily demand of each warehouse.

[0058] Specifically, when extracting feature data from the raw data, the computing device 100 can first calculate the demand volatility based on the total daily warehouse demand. For example, the demand volatility over the past 30 days can be calculated using the following formula: (1) In the formula, Indicates the first Total warehouse demand for the day This represents the average daily warehouse demand over the past 30 days. It's important to note that demand volatility measures market uncertainty; higher volatility indicates a greater need to reserve safety stock to mitigate risk.

[0059] Secondly, the computing device 100 can calculate inventory health based on the maximum inventory capacity of each warehouse and the average daily demand of each warehouse. The specific formula is as follows: (2) In the formula, Indicates the first The maximum inventory capacity of each warehouse Indicates the first The average daily demand of each warehouse. Indicates the first The warehouses have sufficient inventory. Indicates the first The warehouses are short of stock.

[0060] Furthermore, the computing device 100 can utilize NLP models (such as the BERT model) to analyze social media sentiment data, obtain various keywords related to demand (such as keywords like "promotion" and "out of stock"), and then calculate the popularity score of each keyword. This allows us to obtain a social media sentiment score, thereby quantifying the potential impact of external events on demand.

[0061] In some embodiments, to eliminate the dimensional differences between different feature data, all feature data can be standardized so that the mean of the feature data is 0 and the variance is 1, thus obtaining standardized features.

[0062] In some embodiments, the specific method for constructing the average field game model based on the original data in step 220 is as follows: First, the average inventory state can be obtained based on the inventory level of each warehouse. The average state of transportation delay is obtained based on the transportation delay data for each product. The specific formula is shown below.

[0063] (3) In the formula, , They represent the first Inventory levels in warehouses, the first The data shows the transportation delays for this product; N and M represent the total number of warehouses and transportation routes, respectively.

[0064] Furthermore, it can be based on the average inventory status and average state of transportation delay The group state of the supply chain is obtained. Specifically, the group state of the supply chain can be represented as follows: It can reflect the overall operational status of the supply chain.

[0065] Subsequently, based on the group status of the supply chain, each participant (the first...) can be... The objective function is constructed to minimize the expected utility (based on strategy and group state) of each participant. Based on this objective function, an average-field game model can be obtained. The objective function is as follows: (4) In the formula, and These are parameters that are artificially set to quantify the relative importance of costs and benefits. Includes transportation costs and inventory holding costs ,in, For unit transportation cost, Cost of holding inventory per unit. Calculated based on order profit.

[0066] It should be understood that the participants' expected utility and their strategies... and the collective state of the supply chain related.

[0067] In some embodiments, in step 230, the computing device 100 may use the Hamilton-Jacobi-Bellman equation to solve the objective function of the average-field game model, as shown in the following equation, to obtain the initial optimal strategy for each participant (the first optimal strategy). The initial optimal strategy for each participant ).

[0068] (5) in, Indicates immediate cost. Used to describe dynamic changes in inventory. represents the state transition coefficient. It is the value function.

[0069] In some embodiments, in step 230, the computing device 100 can determine the initial optimal strategy for each participant using the Fermi-Dirac distribution. By performing a probabilistic extension, the initial policy probability distribution is obtained. As shown in the following formula: (6) In the formula, For risk aversion parameters, used to control the conservatism of the strategy, where, when At that time, participants completely avoid risk and only choose the strategy with the lowest cost; when At that time, participants randomly select strategies, ignoring cost differences. Additionally, due to the group state of the supply chain... It will change dynamically over time, and the dynamic nature of the aforementioned initial strategy probability distribution allows participants to adjust their strategies based on the group state of the supply chain. By adjusting strategies in response to changes in the supply chain, each participant in the supply chain can be linked to changes in the group's state.

[0070] In the aforementioned steps, the initial policy probability distribution is obtained through the Fermi-Dirac distribution. Cost sensitivity and risk aversion parameters have been considered. Therefore, in some embodiments, in step 240, when the computing device 100 iteratively optimizes the initial policy probability distribution using a deep reinforcement learning algorithm, it can use the initial policy probability distribution obtained in the aforementioned steps. As the initial strategy for deep reinforcement learning algorithms This is used to initialize the policy network of the deep reinforcement learning algorithm. Based on this, the training time of the policy network can be significantly shortened. Representing the input state (state space) (The state in the middle).

[0071] In some embodiments, when iteratively optimizing the initial policy probability distribution using a deep reinforcement learning algorithm in step 240, the group state of the supply chain can be determined. Inventory levels in each warehouse Average daily demand for each warehouse Social media sentiment score As input state And, the initial optimal strategy for each participant. As an action, it should be understood that the group state of the supply chain and the average inventory state... and average state of transportation delay This is relevant. Furthermore, it allows us to construct (define) the initial optimal strategy (action) for each participant. With group status Reward function between This allows for the quantification of the policy's optimization effect through a reward function. Then, based on the input state, actions (each participant's initial optimal policy), and the reward function, the initial policy probability distribution can be iteratively optimized by maximizing the cumulative reward to obtain the optimized policy probability distribution. Specifically, by taking actions based on the input state and obtaining rewards based on the reward function, the initial policy probability distribution is iteratively optimized by maximizing the cumulative reward to obtain the optimized policy probability distribution.

[0072] In some embodiments, the reward function design incorporates the objective function of the mean-field game model. That is, the reward function between each participant's initial optimal strategy and the group state can be constructed based on the objective function of the mean-field game model. The specific reward function is shown in the following equation: (7) In the formula, and These are the weights for costs and benefits, respectively.

[0073] In this embodiment, a deep reinforcement learning algorithm is used to maximize the cumulative reward based on the input state, action (each participant's initial optimal policy), and reward function. This allows for continuous adjustment of the policy network parameters, thereby enabling the initial policy probability distribution to be optimized. The process evolves towards a better outcome, iteratively optimizing the initial policy probability distribution to ultimately obtain an optimized policy probability distribution. For example, when the average state of transportation delay... As the speed increases, deep reinforcement learning algorithms dynamically increase the probability of selecting high-reliability transportation routes while reducing the frequency of using high-cost transportation routes, thus achieving a balance between cost and efficiency.

[0074] In some embodiments, the Proximal Policy Optimization (PPO) algorithm can be used to adjust the parameters of the policy network by maximizing the cumulative reward based on the input state, action, and reward function, as shown in the following equation, in order to iteratively optimize the initial policy probability distribution.

[0075] (8) In the formula, For learning rate, This is the dominant function.

[0076] It should be noted that the probability distribution of the optimization strategy in the embodiments of the present invention is obtained based on the latest input state (including the group state of the supply chain, the inventory level and average daily demand of each warehouse, and the social media sentiment score).

[0077] In this embodiment of the invention, new raw data (such as transportation disruption data caused by sudden weather) can be acquired in real time. Based on this new raw data, the overall state of the supply chain is updated, and the initial strategy probability distribution is updated. This, in turn, updates the optimized strategy probability distribution, ensuring that the final optimized strategy probability distribution is always based on the latest supply chain operating status. For example, when social media sentiment... During periods of sudden surge (such as during the "618" promotional event), the deep reinforcement learning algorithm of this invention will rapidly correct the policy network through backpropagation to increase replenishment in high-demand areas, while dynamically allocating more transportation resources to alleviate logistics pressure.

[0078] In some embodiments, after each iterative optimization of the initial policy probability distribution using a deep reinforcement learning algorithm, the group state predicted based on the mean-field game model can be calculated. Strategy distortion rate relative to the actual state of the supply chain (e.g., based on the latest inventory levels and shipping delay data). If the distortion rate exceeds a threshold (e.g., 5%), policy retraining is triggered, updating the policy network parameters based on the latest data. This ensures that the probability distribution of the optimized policy obtained in this invention always remains consistent with the dynamics of the real supply chain, while combining the theoretical framework of the game theory model with the adaptive capabilities of deep reinforcement learning algorithms, achieving a leap from static optimization to dynamic optimization.

[0079] Figure 3 A schematic diagram of a supply chain dynamic resource allocation device 300 according to an embodiment of the present invention is shown. The supply chain dynamic resource allocation device 300 can be deployed in a computing device 100, and the supply chain dynamic resource allocation device 300 is configured to execute the supply chain dynamic resource allocation method 200 of the present invention.

[0080] like Figure 3 As shown, in an embodiment of the present invention, the supply chain dynamic resource allocation device 300 includes an acquisition module 310, a construction module 320, a solution module 330, an optimization module 340, and an execution module 350 that are sequentially and communicatively connected.

[0081] Among them, the acquisition module 310 can acquire raw data in real time. The raw data includes core supply chain data and external environment data. The core supply chain data includes the daily order volume of the supply chain, the inventory level of each warehouse, the transportation delay data of each product, and the production capacity of each supplier. The external environment data includes social media sentiment data.

[0082] Module 320 can treat each node in the supply chain as a participant and build an average-field game model based on raw data. The nodes include suppliers, warehouses, and logistics providers.

[0083] The solution module 330 can use the Hamilton-Jacobi-Bellman equation to solve the mean-field game model to obtain the initial optimal strategy of each participant, and obtain the initial strategy probability distribution based on the initial optimal strategy of each participant. The initial strategy probability distribution is used to represent the probability distribution of each participant choosing a strategy in the group state.

[0084] The optimization module 340 can use a deep reinforcement learning algorithm to iteratively optimize the initial policy probability distribution to obtain an optimized policy probability distribution. The optimized policy probability distribution is used to represent the probability distribution of each participant choosing a policy under the input state. The input state includes the group state, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment score.

[0085] The execution module 350 can determine and execute the optimal strategy for each participant under the input state based on the probability distribution of the optimization strategy. The optimal strategy for each participant includes the optimal order allocation quantity for each supplier, the optimal replenishment quantity for each warehouse, and the optimal transportation quantity for each logistics provider.

[0086] It should be noted that the acquisition module 310, construction module 320, solution module 330, optimization module 340, and execution module 350 are respectively used to execute the aforementioned steps 210 to 250. Here, the specific execution logic of each unit can be found in the description of steps 210 to 250 in method 200 above, and will not be repeated here.

[0087] According to the supply chain dynamic resource allocation method 200 in this embodiment of the invention, firstly, an average-field game model is constructed by treating each node in the supply chain (including suppliers, warehouses, and logistics providers) as participants, which can simulate the real dynamics of close relationships between the participants. Secondly, the Hamilton-Jacobi-Bellman equation is used to solve the average-field game model to obtain the initial optimal strategy for each participant, which can significantly reduce computational complexity. Furthermore, an initial strategy probability distribution is obtained based on the initial optimal strategy of each participant, and then a deep reinforcement learning algorithm is used to iteratively optimize the initial strategy probability distribution to obtain an optimized strategy probability distribution. Based on the optimized strategy probability distribution, the optimal strategy selected by each participant under the input state can be determined. Based on this, the strategy can be adjusted according to the dynamic changes in the group state of the supply chain, the inventory level and daily demand of each warehouse, and the popularity of social media sentiment, thereby effectively responding to the dynamic environment and realizing the dynamic optimization of supply chain resource allocation.

[0088] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0089] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the supply chain dynamic resource allocation method of the present invention according to instructions in the program code stored in the memory.

[0090] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0091] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0092] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0093] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof.

[0094] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0095] Unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

Claims

1. A method for dynamic resource allocation in a supply chain, executed in a computing device, comprising: Real-time acquisition of raw data, including core supply chain data and external environment data. The core supply chain data includes daily order volume, inventory level of each warehouse, transportation delay data for each product, and production capacity of each supplier. The external environment data includes social media sentiment data. By treating each node in the supply chain as a participant, an average-field game model is constructed based on the raw data. The nodes include suppliers, warehouses, and logistics providers. The Hamilton-Jacobi-Bellman equation is used to solve the mean-field game model to obtain the initial optimal strategy of each participant, and the initial strategy probability distribution is obtained based on the initial optimal strategy of each participant. The initial strategy probability distribution is used to represent the probability distribution of each participant choosing a strategy in the group state. The initial policy probability distribution is iteratively optimized using a deep reinforcement learning algorithm to obtain an optimized policy probability distribution. The optimized policy probability distribution is used to represent the probability distribution of each participant choosing a policy under the input state. The input state includes the group state, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment score. Based on the probability distribution of the optimization strategy, the optimal strategy for each participant under the input state is determined and executed, wherein the optimal strategy for each participant includes the optimal order allocation quantity for each supplier, the optimal replenishment quantity for each warehouse, and the optimal transportation quantity for each logistics provider.

2. The method as described in claim 1, wherein, Treating each node in the supply chain as a participant, a mean-field game model is constructed based on the raw data, including: The average inventory status is obtained based on the inventory level of each warehouse, and the average shipping delay status is obtained based on the shipping delay data of each product. Based on the average inventory status and the average transportation delay status, the group status of the supply chain is obtained; Based on the group state of the supply chain, an objective function is constructed with the goal of minimizing the expected utility of each participant. Based on the objective function, an average-field game model is obtained, wherein the expected utility of each participant is related to the participant's strategy and the group state.

3. The method as described in claim 1 or 2, wherein, The initial policy probability distribution is iteratively optimized using a deep reinforcement learning algorithm to obtain an optimized policy probability distribution, including: The group status of the supply chain, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment score are used as input status, and the initial optimal strategy of each participant is used as action. Construct a reward function between each participant's initial optimal strategy and the group state; Based on the input state, the action, and the reward function, the initial policy probability distribution is iteratively optimized by maximizing the cumulative reward to obtain an optimized policy probability distribution.

4. The method of claim 3, wherein, Based on the input state, the action, and the reward function, the initial policy probability distribution is iteratively optimized by maximizing the cumulative reward, including: Using a proximal policy optimization algorithm, based on the input state, the action, and the reward function, the parameters of the policy network are adjusted by maximizing the cumulative reward to iteratively optimize the initial policy probability distribution.

5. The method according to any one of claims 1-4, wherein, Also includes: Extract feature data from the original data; The feature data includes demand volatility, inventory health, and social media sentiment score. The inventory health is related to the maximum inventory capacity of each warehouse and the average daily demand of each warehouse.

6. The method of claim 5, wherein, The raw data also includes the total daily warehouse demand, the maximum inventory capacity of each warehouse, and the average daily demand of each warehouse; Extracting feature data from the raw data includes: Demand volatility is calculated based on the total daily warehouse demand. The inventory health is calculated based on the maximum inventory capacity of each warehouse and the average daily demand of each warehouse. The social media sentiment data is analyzed using an NLP model to obtain keywords related to the demand, and the popularity score of each keyword is calculated to obtain the social media sentiment popularity score.

7. The method according to any one of claims 1-6, wherein, Based on the probability distribution of the optimization strategy, determine and execute the optimal strategy for each participant under the input state, including: The probability distribution of the optimization strategy is pushed to the ERP system; The ERP system determines and executes the optimal strategy for each participant under the input state based on the probability distribution of the optimization strategy, and collects execution data.

8. The method according to any one of claims 1-7, wherein, The initial policy probability distribution is obtained based on the initial optimal policy of each participant, including: The initial policy probability distribution is obtained based on the initial optimal policy of each participant using the Fermi-Dirac distribution.

9. A supply chain dynamic resource allocation device, deployed in a computing device, the device comprising: The acquisition module is suitable for acquiring raw data in real time. The raw data includes core supply chain data and external environment data. The core supply chain data includes the daily order volume of the supply chain, the inventory level of each warehouse, the transportation delay data of each product, and the production capacity of each supplier. The external environment data includes social media sentiment data. The module is designed to include each node in the supply chain as a participant and to build an average-field game model based on the raw data. The nodes include suppliers, warehouses, and logistics providers. The solution module is adapted to solve the mean-field game model using the Hamilton-Jacobi-Bellman equation to obtain the initial optimal strategy of each participant, and to obtain the initial strategy probability distribution based on the initial optimal strategy of each participant. The initial strategy probability distribution is used to represent the probability distribution of each participant choosing a strategy in the group state. The optimization module is adapted to use a deep reinforcement learning algorithm to iteratively optimize the initial policy probability distribution to obtain an optimized policy probability distribution. The optimized policy probability distribution is used to represent the probability distribution of each participant choosing a policy under the input state, wherein the input state includes the group state, the inventory level of each warehouse, the average daily demand of each warehouse, and the social media sentiment heat score. The execution module is adapted to determine and execute the optimal strategy for each participant under the input state based on the probability distribution of the optimization strategy, wherein the optimal strategy for each participant includes the optimal order allocation quantity for each supplier, the optimal replenishment quantity for each warehouse, and the optimal transportation quantity for each logistics provider.

10. A computing device, comprising: At least one processor; and A memory storing program instructions, wherein the program instructions are configured to be processed by the at least one processor, the program instructions including instructions for processing the method as claimed in any one of claims 1-8.

11. A computer program product comprising computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method as described in any one of claims 1-8.

Images