Order scheduling method and equipment

Abstract

The present application relates to the field of intelligent decision-making technology, and discloses an order scheduling method and device, including: traversing pending orders to obtain the current order; deciding the order delivery strategy corresponding to the current order as a production delivery strategy or an inventory delivery strategy through a decision network; setting a corresponding order delivery schedule for the current order based on the order delivery strategy; constructing experience samples according to the current order and the order delivery strategy, and optimizing the decision network according to the experience samples; at the end of the traversal, generating order scheduling results according to the order delivery strategy and order delivery schedule corresponding to each pending order. Due to the introduction of reinforcement learning, the delivery strategy is intelligently decided through a deep neural network, and scheduling is automatically performed for different delivery strategies. Compared with the simplified delivery strategy, the degree of intelligence is improved, and the rationality of the delivery strategy and the order fulfillment rate are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent decision-making technology, and in particular to an order scheduling method and device. Background Art

[0002] In modern enterprise operations, planning and scheduling is a vital management activity. It involves the rational arrangement and optimal allocation of enterprise resources to ensure the orderly progress of various links such as production, sales, and procurement. Simply put, planning and scheduling is to pre-plan production work in the future and clarify the time nodes, resource requirements, and execution order of various production tasks. However, due to the large number of product types and the huge number of orders commonly found in large manufacturing companies, how to formulate a reasonable production plan to maximize the utilization efficiency of machine capacity and inventory while maximizing the order fulfillment rate is a very difficult problem.

[0003] For modern manufacturing enterprises, the delivery strategy for MTO and MTS mixed orders is relatively simple. Usually, the production plan is scheduled according to inventory, and the delivery strategy is not further optimized. The actual use effect is not good, which restricts the further improvement of the order fulfillment rate. Summary of the invention

[0004] The main purpose of this application is to provide an order scheduling method and device, aiming to solve the technical problem that the delivery strategy for MTO and MTS mixed orders in related technologies is relatively simple and the actual use effect is poor.

[0005] To achieve the above objectives, the present application proposes an order scheduling method, which includes:

[0006] Traverse the pending orders to obtain the current orders, where the pending orders are orders that have not been delivered, and the pending orders include MTO orders and MTS orders;

[0007] The order delivery strategy corresponding to the current order is determined to be a production delivery strategy or an inventory delivery strategy through a decision network, the decision network is a value network or a strategy network, and the order delivery strategy includes a finished product delivery strategy and / or a semi-finished product delivery strategy when delivering the current order;

[0008] Setting a corresponding order delivery schedule for the current order based on the order delivery strategy, wherein the order delivery schedule includes an order processing time and a processing quantity;

[0009] constructing an experience sample according to the current order and the order delivery strategy, and optimizing the decision network according to the experience sample;

[0010] At the end of the traversal, the order scheduling results are generated according to the order delivery strategies and order delivery schedules corresponding to each pending order.

[0011] In a possible implementation of the present application, setting a corresponding order delivery schedule for the current order based on the order delivery strategy includes:

[0012] Based on the order delivery strategy, use one-stage reverse scheduling to schedule the current order setting to determine whether it can be scheduled;

[0013] If it can be scheduled, then set the corresponding order delivery schedule for the current order according to the one-stage reverse scheduling result;

[0014] If it cannot be scheduled, then based on the order delivery strategy, the current order is scheduled according to the two-stage normal scheduling, and the corresponding order delivery schedule is set for the current order.

[0015] In a possible implementation of the present application, the order delivery strategy is to give priority to the use of inventory;

[0016] The setting a corresponding order delivery schedule for the current order based on the order delivery strategy includes:

[0017] Obtain the delivery deadline corresponding to the current order;

[0018] Determining whether the current order can be supplied within the delivery period according to the inventory information;

[0019] If the supply cannot be made within the delivery deadline, determining whether the production and delivery of the current order can be completed before the delivery deadline based on the production capacity information;

[0020] If the production and delivery can be completed before the delivery deadline, setting an order delivery schedule for the current order according to the inventory information and the production capacity information, and updating the inventory information and the production capacity information;

[0021] If production delivery cannot be completed before the delivery deadline, after determining the delivery deadline based on the inventory information and the production capacity information, the nearest available delivery time to the delivery deadline is used, and an order delivery schedule is set for the current order based on the available delivery time, and the inventory information and production capacity information are updated.

[0022] In a possible implementation of the present application, the order delivery strategy is to give priority to production;

[0023] The setting a corresponding order delivery schedule for the current order based on the order delivery strategy includes:

[0024] Obtain the delivery deadline corresponding to the current order;

[0025] Determine whether the current order can be completed before the delivery deadline based on inventory information and production capacity information;

[0026] If production cannot be completed, determine whether inventory can be supplied before the delivery deadline based on inventory information;

[0027] If the inventory can be supplied before the delivery deadline, setting an order delivery schedule for the current order according to the inventory information, and updating the inventory information and production capacity information;

[0028] If the inventory cannot be supplied before the delivery deadline, after determining the delivery deadline based on the inventory information and the production capacity information, the nearest available delivery time to the delivery deadline is used, and an order delivery schedule is set for the current order based on the available delivery time, and the inventory information and production capacity information are updated.

[0029] In a possible implementation of the present application, at the end of the traversal, an order scheduling result is generated according to the order delivery strategy and order delivery schedule corresponding to each pending order, including:

[0030] At the end of the traversal, check whether the preset termination condition is met;

[0031] If the preset termination conditions are met, the order scheduling results are generated according to the order delivery strategies and order delivery schedules corresponding to each pending order.

[0032] In a possible implementation of the present application, at the end of the traversal, detecting whether a preset termination condition is met includes:

[0033] At the end of the traversal, the current traversal round is obtained, where the current traversal round is the number of times the current pending orders are completely traversed;

[0034] If the current traversal round is greater than or equal to a preset round threshold, it is determined that the preset termination condition is met.

[0035] In a possible implementation of the present application, at the end of the traversal, detecting whether a preset termination condition is met includes:

[0036] At the end of the traversal, the order fulfillment rate of the current traversal round and the previous N traversal rounds is obtained, wherein the current traversal round is the number of times the pending orders are completely traversed, the order fulfillment rate is the proportion of the scheduled fulfilled orders in the pending orders, and the fulfilled orders are the orders that can be delivered before the delivery deadline after scheduling;

[0037] If it is determined based on the order fulfillment rates of the current traversal round and the previous N traversal rounds that the order fulfillment rate has not decreased in multiple consecutive traversal rounds, it is determined that the preset termination condition is met.

[0038] In a possible implementation of the present application, at the end of the traversal, detecting whether a preset termination condition is met includes:

[0039] At the end of the traversal, the order fulfillment rate of the current traversal round and the previous N traversal rounds is obtained, wherein the current traversal round is the number of times the pending orders are completely traversed, the order fulfillment rate is the proportion of the scheduled fulfilled orders in the pending orders, and the fulfilled orders are the orders that can be delivered before the delivery deadline after scheduling;

[0040] If the current traversal round is greater than or equal to the preset round threshold, or, based on the order fulfillment rates of the current traversal round and the previous N traversal rounds, it is determined that the order fulfillment rate has not decreased in multiple consecutive rounds of traversal, then it is determined that the preset termination condition is met.

[0041] In a possible implementation of the present application, after detecting whether a preset termination condition is met at the end of the traversal, the method further includes:

[0042] If the preset termination condition is not met, a dynamic ε-greedy adjustment strategy is used to adjust the exploration and utilization parameters of the decision network, where the exploration and utilization parameters are parameters that control the exploration and solution experience in the decision network;

[0043] When the adjustment is completed, return to the step of traversing the pending orders to obtain the current order.

[0044] In a possible implementation of the present application, the use of a dynamic ε-greedy adjustment strategy to adjust the exploration and utilization parameters of the decision network includes:

[0045] Determine the order fulfillment rate according to the order delivery schedule, and obtain the previous iteration detection parameters;

[0046] Generate a current exploration and utilization parameter according to the previous iteration detection parameter, the order fulfillment rate and the preset attenuation coefficient, wherein the previous iteration detection parameter is the exploration and utilization parameter generated in the previous iteration round, and each iteration round represents a complete traversal of the orders to be processed;

[0047] The exploration and utilization parameters in the decision network are updated to the current exploration and utilization parameters.

[0048] In a possible implementation of the present application, generating the current exploration and utilization parameter according to the previous iteration detection parameter, the order fulfillment rate and the preset attenuation coefficient includes:

[0049] Subtract the previous iteration detection parameter from the final value of the iteration parameter to obtain a first intermediate parameter;

[0050] The order fulfillment rate is multiplied by a preset attenuation coefficient, and the difference between the product and a preset constant is calculated to obtain a second intermediate parameter;

[0051] The first intermediate parameter is multiplied by the second intermediate parameter, and the product is added to the final value of the iteration parameter to generate a current exploration and utilization parameter.

[0052] In a possible implementation of the present application, the constructing an experience sample according to the current order and the order delivery strategy includes:

[0053] Obtain the delivery deadline corresponding to the current order;

[0054] Determining whether the current order is achievable according to the delivery deadline and the order delivery schedule;

[0055] If the current order can be fulfilled, constructing an experience sample according to the first preset score, the current order and the order delivery strategy;

[0056] If the current order cannot be fulfilled, constructing an experience sample according to the second preset score, the current order and the order delivery strategy;

[0057] Among them, the first preset score is greater than the second preset score.

[0058] In a possible implementation of the present application, optimizing the decision network according to the experience sample includes:

[0059] Adding the experience sample to the sample pool to obtain an updated sample pool;

[0060] Extracting training samples from the updated sample pool according to a preset sampling quantity;

[0061] The decision network is optimized and trained according to the extracted training samples.

[0062] In a possible implementation of the present application, traversing the pending orders to obtain the current order includes:

[0063] Sorting the pending orders according to a preset sorting rule to obtain a sorting order, wherein the preset sorting rule is set according to at least one of an order number, an order type, and an order requirement;

[0064] The pending orders are traversed in the sorting order to obtain the current order.

[0065] In addition, to achieve the above-mentioned purpose, the present application also proposes an order scheduling device, which includes:

[0066] An acquisition module is used to traverse pending orders and obtain current orders, wherein the pending orders are orders that have not been delivered, and the pending orders include MTO orders and MTS orders;

[0067] A decision module, used to decide through a decision network whether the order delivery strategy corresponding to the current order is a production delivery strategy or an inventory delivery strategy, wherein the decision network is a value network or a strategy network, and the decision network is a pre-trained value network or a decision network, and the order delivery strategy includes a finished product delivery strategy and / or a semi-finished product delivery strategy when delivering the current order;

[0068] A setting module, used to set a corresponding order delivery schedule for the current order based on the order delivery strategy, wherein the order delivery schedule includes an order processing time and a processing quantity;

[0069] A construction module, used to construct experience samples according to the current order and the order delivery strategy, and optimize the decision network according to the experience samples;

[0070] The generation module is used to generate order scheduling results according to the order delivery strategy and order delivery schedule corresponding to each pending order at the end of the traversal.

[0071] In addition, to achieve the above-mentioned purpose, the present application also proposes an order scheduling device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the order scheduling method as described above.

[0072] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the order scheduling method described above are implemented.

[0073] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the order scheduling method described above are implemented.

[0074] One or more technical solutions proposed in this application have at least the following technical effects: due to the introduction of reinforcement learning, the delivery strategy is intelligently decided through a deep neural network, and scheduling is automatically performed for different delivery strategies. Compared with the simple delivery strategy, the degree of intelligence is improved, the rationality of the delivery strategy and the order fulfillment rate are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0077] Figure 1 A flowchart of the first embodiment of the method for scheduling application orders is provided;

[0078] Figure 2 A schematic diagram of a production BOM according to an embodiment of the present application;

[0079] Figure 3 A schematic diagram of a priority inventory setting process according to an embodiment of the present application;

[0080] Figure 4 A schematic diagram of a priority production setting process according to an embodiment of the present application;

[0081] Figure 5 A brief flowchart of the order scheduling method provided in Example 2 of the present application;

[0082] Figure 6 This is a schematic diagram of the module structure of the order scheduling device according to an embodiment of the present application;

[0083] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the order scheduling method in the embodiment of the present application.

[0084] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0085] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0086] The present application embodiment provides an order scheduling method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the application order scheduling method.

[0087] In this embodiment, the order scheduling method includes steps S10 to S50:

[0088] Step S10: Traverse the orders to be processed and obtain the current order.

[0089] It should be noted that the execution subject of this embodiment may be an order scheduling device with data processing, network communication and program running functions, and the order scheduling device may be an electronic device such as a tablet computer, a personal computer, a mobile phone, or other electronic devices capable of realizing the above functions. In this embodiment and the following embodiments, the order scheduling method of this application is described by taking the order scheduling device as an example.

[0090] It should be noted that pending orders can be orders that have not been delivered, that is, orders placed by customers but have not yet been completed or delivered to customers. Pending orders can include MTO orders and MTS orders.

[0091] Among them, planning and scheduling is the process of optimizing the scheduling system based on limited production capacity, comprehensively considering the impact of market demand, materials, production capacity, process flow and other factors on production, and deriving a reasonable and effective production plan.

[0092] MTO (Make To Order), MTO orders are mostly customized materials. This type of order usually has a small demand for materials. Companies usually do not have inventory or have very little inventory, and they only produce when there is demand.

[0093] MTS (Make To Stock), MTS orders are mostly for standard materials. This type of order usually has a large demand for materials. Companies usually stockpile a certain amount of inventory and deliver directly from inventory when demand arises. If the inventory is insufficient, production will be carried out for delivery.

[0094] At present, for some products with smaller usage and higher degree of customization, they are produced in MTO mode, that is, production upon order. For some standard materials with larger usage, they are produced in MTS combined with MTO model, that is, products that can be delivered from stock will be delivered from stock first. If the stock cannot be delivered, it will be converted into MTO order for production.

[0095] In actual use, the pending orders are traversed to obtain the current order. The pending orders can be traversed in order of order numbers, order placement times and / or expected delivery dates of the orders, and the traversed pending orders are used as the current order.

[0096] For ease of understanding, now combined Figure 2 This is for illustration only, but not for limitation. Figure 2 This is a schematic diagram of the production BOM of this embodiment.

[0097] Take the production of materials A and B as an example, the production BOM diagram is as follows Figure 2As shown in the figure, the production of A requires one B and one C, and the production of C requires one D. A and C are both materials that can be sold directly. If there are orders for A and C at the same time, the delivery date of A is T+10, and the delivery date of C is T+4. The production capacity of the machine for producing A is sufficient, and there is only one C in stock at present. The machine for producing C has no production capacity before T+4, and has sufficient production capacity after that. The production priority of A is higher than that of C. If the semi-finished product is used in stock first, the only C will be used when scheduling A, and the order of C cannot be fulfilled due to the lack of production capacity before T+4 when scheduling C. At this time, if A gives priority to the production of semi-finished product C, both A and C can be fulfilled (the production capacity of C is sufficient after T+4, and the delivery date of A is T+10, so a C can be successfully produced for the delivery of A before the delivery date of A). It can be seen that optimizing the delivery strategy during material planning and scheduling can improve the order fulfillment rate and capacity utilization efficiency.

[0098] Among them, the delivery time is the expected delivery time, that is, the expected delivery date. It is the time from when a product is unlocked and the standard delivery time it takes. The expected delivery time is usually used to assess the delivery capability of the supply chain. An order is considered to be fulfilled if production is completed before the expected delivery time, and is considered to be unfulfilled if production is completed after the expected delivery time. The supply chain usually hopes to fulfill as many orders as possible.

[0099] Step S20: The order delivery strategy corresponding to the current order is determined by a decision network as a production delivery strategy or an inventory delivery strategy.

[0100] It should be noted that the decision network can be a value network or a strategy network, and the order delivery strategy includes the delivery strategy of finished products and / or the delivery strategy of semi-finished products when delivering the current order. Specifically, the decision network analyzes the current order and outputs the order delivery strategy that should be executed most for the current order. Executing this strategy can obtain the maximum expected benefit, including current and future returns.

[0101] The delivery strategy can be in the production schedule, to provide a finished product or a semi-finished product when delivering a certain order, and to select the source of supply. The delivery strategy includes production delivery strategy and inventory delivery strategy, such as direct delivery using inventory or delivery through production.

[0102] In actual use, through the decision network, it can be determined whether the order delivery strategy to be used when executing the current order is the production delivery strategy or the inventory delivery strategy. Among them, when an order includes the delivery of finished products and semi-finished products at the same time, the delivery strategies used for finished products and semi-finished products can be the same or different, for example: both finished products and semi-finished products use the inventory delivery strategy or the production delivery strategy, or the finished products use the production delivery strategy and the semi-finished products use the inventory delivery strategy.

[0103] An agent is an entity that performs actions and interacts with the environment in reinforcement learning. It is the subject of the learning process and has the ability to perceive, make decisions, and perform actions. The goal of the agent is to learn how to choose appropriate actions to maximize cumulative rewards by interacting with the environment.

[0104] State is a way to describe the environment. It includes all factors that affect decision-making. The state comes from the environment. Every time a decision is made, the state is input into the policy network, resulting in different decision results.

[0105] Reward: A reward is a scalar feedback signal given by the environment, which shows how well the agent performs at a certain step by adopting a certain strategy.

[0106] An action is an executable operation output by the policy network that maximizes the expected cumulative reward in the long term. In reinforcement learning, the agent affects the transition of the environment by executing actions. The choice of action is determined by the current state and the agent's strategy.

[0107] The value network is a neural network model that is used to approximate the action value function or the state value function. The structure of the value network is preset manually, and its parameters are randomly initialized at the beginning and learned through the interaction between the agent and the environment.

[0108] A policy network is a neural network model that approximates the behavior policy of an agent. In the context of reinforcement learning, a policy refers to a set of decision rules that an agent takes in a given environment, i.e., what action to choose in a particular state.

[0109] In order to facilitate understanding, the decision network is explained by taking the value network as an example. The order information of the current order is input into the value network. The value network selects the action to be executed in the preset action set according to the probability distribution to maximize the expected reward. Specifically, the value network outputs the value of executing each action, and then selects the action to be executed through the action selection strategy. When the value network is used as the decision network, the DQN network can be used, or the more robust A2C network can be used. Of course, other networks can also be used, and this embodiment does not limit this.

[0110] In actual use, the decision network may have been pre-trained, and the corresponding order delivery strategy may be directly set for the current order through the decision network, and the effect of the setting may still be guaranteed to be good.

[0111] In the specific implementation, since the delivery strategy of finished products and semi-finished products needs to be decided for each material of the order, the action set A=[a1, a2, a3, a4] in the decision network, where a1, a2, a3, a4 are all 0-1 variables. If ax=1, it means that the strategy is selected, and ax=0 means that the strategy is not selected. Only one action among a1, a2, a3, a4 will be selected, that is, only one value among a1-a4 will be equal to 1. The detailed strategies represented by each action are shown in the following table:

[0112] a1: Finished products are delivered by inventory first, and semi-finished products are delivered by inventory first;

[0113] a2: Finished products are delivered from inventory first, and semi-finished products are delivered from production first;

[0114] a3: Finished products are delivered first, and semi-finished products are delivered first from inventory;

[0115] a4: Finished products are given priority in production and delivery, while semi-finished products are given priority in production and delivery.

[0116] Among them, the decision network follows the ε-greedy strategy when selecting actions to avoid the emergence of local optimal solutions. The strategy is expressed as the following formula:

[0117]

[0118]

[0119] In the formula, A* represents the action with the maximum value obtained by the output of the policy network. represents the probability distribution of the next action, where the probability of selecting A* is approximately 1-ε, and the sum of the probabilities of selecting other actions is approximately ε, |A(S)| represents the number of actions in state s (4 in this example), and the probability of each other action being selected is ε / |A(S)|. ε-greedy enables the agent to have both the ability to exploit and explore during the learning process. Exploitation is to select the action with the "maximum value in the table" based on the experience of the current action-value function, while exploration gives the agent the ability to try new unknown actions.

[0120] In a specific implementation, in order to reduce the complexity of the decision network, step S20 in this embodiment may include:

[0121] Extracting features of the current order to obtain order feature information;

[0122] The order feature information is input into the decision network for decision making to obtain the order delivery strategy corresponding to the current order.

[0123] It should be noted that order information is generally in natural language or data assembled in a specific format. If the current order is directly transmitted to the decision network, the decision network needs to extract the order features first and then perform corresponding decision analysis. This will increase the complexity of the decision network and is not conducive to its training. Therefore, the features of the current order can be extracted first to obtain the order feature information, and then the order feature information can be input into the decision network for decision-making, thereby reducing the complexity of the decision network.

[0124] In actual use, for the convenience of processing, the order feature information may be stored in a one-dimensional array or a one-dimensional set, and the order feature information may include at least one of the following information:

[0125] 1) Order sort number: the sorting order of each order after sorting;

[0126] 2) Relative sort number: After sorting, the sorting position of the order among materials of the same type. For example, if there are three orders for material A, and the sorting positions are 1, 11, and 15 respectively, then the relative sort number of the order ranked 15th is 3;

[0127] 3) Order type: MTO order or MTS order;

[0128] 4) Demand: the quantity of a product required in the order;

[0129] 5) Material demand: the sum of the demand quantities of the material required in a certain order in all orders;

[0130] 6) Total order volume: the quantity of all orders;

[0131] 7) Total material order quantity: the sum of the order quantities of a material required in a certain order in all orders requiring the material;

[0132] 8) Material ID: the ID of the material required in an order;

[0133] 9)Future maximum inventory: the maximum inventory quantity of materials required for a certain order within a certain period of time in the future;

[0134] 10) Future minimum inventory: the minimum inventory quantity of materials required for an order within a certain period of time in the future;

[0135] 11) Remaining delivery time: the remaining time from the scheduled date to the expected delivery date of the order, in days;

[0136] 12) BOM material quantity: the quantity of materials involved in the production BOM of the required materials in a certain order;

[0137] 13) BOM depth: The maximum depth of the production BOM of the required material in a certain order, such as Figure 1The production BOM shown has a BOM depth of 3.

[0138] 14) Maximum inventory of BOM materials: the maximum inventory quantity of all materials involved in the production BOM of the material required in a certain order within a certain period of time in the future.

[0139] 15) BOM material minimum inventory: the minimum inventory quantity of all materials involved in the production BOM of the material required in a certain order within a certain period of time in the future.

[0140] After obtaining the order characteristics of the current order, all order characteristics can be organized into a structured one-dimensional array or one-dimensional set to obtain order characteristic information, and the order characteristic information is input into the decision network. Taking the above information as an example, the order characteristic information can be expressed as: [order sort number, relative sort number, order type, demand quantity, material demand quantity, total order quantity, total material order quantity, material id, future maximum inventory, future minimum inventory, remaining delivery time, BOM material quantity, BOM depth, BOM material maximum inventory, BOM material minimum inventory].

[0141] Among them, the above BOM is the production BOM diagram.

[0142] Step S30: Setting a corresponding order delivery schedule for the current order based on the order delivery strategy.

[0143] It should be noted that the order delivery schedule includes order processing time and processing quantity, wherein the order processing time can be the time when the order is delivered using inventory, or the time when the finished products or semi-finished products corresponding to the order are produced; the processing quantity can be the quantity delivered using inventory, or the quantity of finished products or semi-finished products corresponding to the order produced.

[0144] In actual use, after determining the order delivery strategy for the current order, the order scheduling device can determine the order delivery schedule for the current order by analyzing the inventory information and production information.

[0145] In a specific implementation, an intelligent scheduling system may be provided in the order scheduling device, and step S30 is executed by the intelligent scheduling system.

[0146] Step S40: constructing an experience sample according to the current order and the order delivery strategy, and optimizing the decision network according to the experience sample.

[0147] It should be noted that after determining the current order and the order delivery strategy for the current order, it is also possible to construct experience samples based on the current order and the order delivery strategy, and iteratively optimize the decision network through the experience samples to enable it to perform further learning and optimize the decision-making effect of the decision network.

[0148] Step S50: At the end of the traversal, an order scheduling result is generated according to the order delivery strategy and order delivery schedule corresponding to each pending order.

[0149] It is understandable that if the traversal is completed, the order delivery strategy and order delivery schedule corresponding to each pending order can be obtained. At this time, in order to facilitate the relevant management personnel to clarify the specific conclusion of the schedule, the order delivery strategy and order delivery schedule corresponding to each pending order can be assembled in a specific format to generate the order scheduling result. If the traversal is not completed, the traversal can be continued.

[0150] Specifically, in order to facilitate relevant management personnel to view the order scheduling results, after the order scheduling results are generated, the order scheduling results can also be constructed into a corresponding result display interface and displayed on a display screen connected to the order scheduling device.

[0151] In a possible implementation of this embodiment, in order to ensure the scheduling effect of the order scheduling result as much as possible, S50 in this embodiment may include:

[0152] At the end of the traversal, check whether the preset termination condition is met;

[0153] If the preset termination conditions are met, the order scheduling results are generated according to the order delivery strategies and order delivery schedules corresponding to each pending order.

[0154] It should be noted that the decision network may not have been trained in advance. At this time, the corresponding order delivery strategy is directly set for the current order through the decision network, and only one scheduling is performed based on the set order delivery strategy. The order scheduling result generated at this time may not be the global optimal solution or the local optimal solution, and the order fulfillment rate cannot be guaranteed to be high. At this time, it may be necessary to combine multiple rounds of iterative processing to ensure that the order scheduling result with a high order fulfillment rate is finally generated. Therefore, at the end of the traversal, further judgment can be made to detect whether the preset termination condition is met, so as to determine whether to directly generate the order scheduling result. Among them, the preset termination condition can be set in advance by the administrator of the order scheduling device.

[0155] It can be understood that if the preset termination conditions are met, it means that the order scheduling result at this time is the global optimal solution or the local optimal solution, and the order fulfillment rate is high. Therefore, the order scheduling result can be generated according to the order delivery strategy and order delivery schedule corresponding to each pending order.

[0156] If the preset termination condition is not met, it means that further optimization scheduling is required at this time. At this time, the process can return to the above step S10 to start a new round of traversal of the pending orders to iterate the optimization scheduling effect.

[0157] In a possible implementation of this embodiment, the preset termination condition may be set according to the iterative traversal round. In this case, the step of detecting whether the preset termination condition is met at the end of the traversal in this embodiment may include:

[0158] At the end of the traversal, get the current traversal round;

[0159] If the current traversal round is greater than or equal to a preset round threshold, it is determined that the preset termination condition is met.

[0160] It should be noted that the current traversal round is the number of complete traversals of the current pending order. For example, if the pending order has been traversed for 5 rounds, the current traversal round is 5. The preset round threshold may be the maximum number of traversals of the pending order, which may be pre-set by the administrator of the order scheduling device.

[0161] It can be understood that if the current traversal round is greater than or equal to the preset round threshold, it means that multiple rounds of complete traversal have been performed on the pending orders, and during the traversal process, the decision network has been optimized multiple times. At this time, it can be considered that the current scheduling result is good and meets the actual use requirements. Therefore, it can be determined that the preset termination condition is met. If the current traversal round is less than the preset round threshold, it means that the iterative traversal round has not yet met the requirements, and a new round of traversal can be continued. Therefore, it can return to step S10.

[0162] In a possible implementation of this embodiment, the preset termination condition may be set according to the change of the order fulfillment rate. In this case, the step of detecting whether the preset termination condition is met at the end of the traversal described in this embodiment may include:

[0163] At the end of the traversal, obtain the order fulfillment rate of the current traversal round and the previous N traversal rounds;

[0164] If it is determined based on the order fulfillment rates of the current traversal round and the previous N traversal rounds that the order fulfillment rate has not decreased in multiple consecutive traversal rounds, it is determined that the preset termination condition is met.

[0165] It should be noted that the current traversal round is the number of complete traversals of the current pending orders; the order fulfillment rate is the proportion of fulfilled orders in the pending orders after scheduling, that is, the ratio of the total number of fulfilled orders to the total number of pending orders; fulfilled orders are orders that can be delivered before the delivery deadline after scheduling. N is a positive integer and can be set in advance by the administrator of the order scheduling device.

[0166] It can be understood that if it is determined based on the order fulfillment rate of the current traversal round and the previous N traversal rounds that the order fulfillment rate has not decreased in multiple consecutive rounds of traversal, it can be determined that the scheduling result at this time is at least a local optimal solution. Therefore, it can be determined that the preset termination condition is met.

[0167] However, if it is determined based on the order fulfillment rates of the current traversal round and the previous N traversal rounds that the order fulfillment rate continues to decline in multiple consecutive rounds of traversal, it means that the order fulfillment rate is not the global optimal solution or the local optimal solution at this time. Therefore, a new round of traversal can be continued, and therefore, the execution of step S10 can be returned.

[0168] In a possible implementation of this embodiment, before a new round of traversal is performed, the decision network may be updated first. In this case, after the step of detecting whether a preset termination condition is met at the end of the traversal described in this embodiment, the following may also be included:

[0169] If the preset termination condition is not met, a dynamic ε-greedy adjustment strategy is used to adjust the exploration and utilization parameters of the decision network, where the exploration and utilization parameters are parameters that control the exploration and solution experience in the decision network;

[0170] When the adjustment is completed, return to the step of traversing the pending orders to obtain the current order.

[0171] The specific adjustment process of using the dynamic ε-greedy adjustment strategy to adjust the exploration and utilization parameters of the decision network can be:

[0172] Determine the order fulfillment rate according to the order delivery schedule, and obtain the previous iteration detection parameters;

[0173] Generate current exploration and utilization parameters according to the previous iteration detection parameters, the order fulfillment rate and the preset attenuation coefficient;

[0174] The exploration and utilization parameters in the decision network are updated to the current exploration and utilization parameters.

[0175] It should be noted that the order fulfillment rate can be the value of the number of fulfilled orders divided by the total number of pending orders. The previous iteration exploration parameter is the exploration and utilization parameter generated in the previous iteration round, and each iteration round represents a complete traversal of the pending orders. The exploration and utilization parameter is a parameter that controls the exploration and solution experience in the decision network. The current exploration and utilization parameter is the exploration and utilization parameter of the current round.

[0176] In actual use, the step of generating the current exploration and utilization parameters according to the previous iteration detection parameters, the order fulfillment rate and the preset attenuation coefficient may include:

[0177] Subtract the previous iteration detection parameter from the final value of the iteration parameter to obtain a first intermediate parameter;

[0178] The order fulfillment rate is multiplied by a preset attenuation coefficient, and the difference between the product and a preset constant is calculated to obtain a second intermediate parameter;

[0179] The first intermediate parameter is multiplied by the second intermediate parameter, and the product is added to the final value of the iteration parameter to generate a current exploration and utilization parameter.

[0180] It should be noted that the final value of the iteration parameter may be the final value of the iteration parameter after attenuation, which may be pre-set by the administrator of the order scheduling device. The preset constant may be pre-set by the administrator of the order scheduling device, such as setting the preset output to 1.

[0181] In the specific implementation, for parameter adjustment of the decision network, a dynamic ε-greedy adjustment strategy based on the order fulfillment rate can be used to adjust the probability of the algorithm exploring and utilizing experience during the iteration process. Assuming that the current iteration round is n+1 and the previous iteration round is n, the adjustment process of the exploration and utilization parameter ε is as follows:

[0182]

[0183]

[0184] Where 1 is a preset constant, ε is a parameter for controlling exploration and solving experience in the decision network; n Represents the ε parameter in the nth round (one round means that all orders complete a traversal); ε n+1 represents the ε parameter at the n+1th round, that is, the current exploration and utilization parameter; ε 1 represents the initial value of ε when training begins. This initial value is often large, which makes the probability of choosing an action to explore the environment greater; ε end Represents the final value of ε after attenuation in the late stage of training. This final value is the preset value, which represents the minimum value that ε is allowed to decrease to. This value is often small, so the probability of making decisions based on known experience will be greater; ε n They are all constants greater than 0 and less than 1, and they decrease as n increases.

[0185] τ n+1 represents the order fulfillment rate of the n+1th round. When all orders of the n+1th round are traversed, τ n+1 Equal to the number of orders k reached in round n+1 n+1 Total number of orders to be processed allThe value of division. β is the preset attenuation coefficient, which represents the attenuation strength of ε. The larger β is, the faster the attenuation speed of ε is. On the contrary, if β is smaller, the slower the attenuation speed of ε is. β can be set in advance by the administrator of the order scheduling device.

[0186] It can be understood that after obtaining the current exploration and utilization parameters, the exploration and utilization parameters in the decision network can be updated to the current exploration and utilization parameters, thereby updating the decision network, and after the update, return to step S10 to execute step S10.

[0187] In a possible implementation of this embodiment, the preset termination condition may be set in combination with the change in the iterative traversal round and the order fulfillment rate. In this case, the step of detecting whether the preset termination condition is met at the end of the traversal described in this embodiment may include:

[0188] At the end of the traversal, obtain the order fulfillment rate of the current traversal round and the previous N traversal rounds;

[0189] If the current traversal round is greater than or equal to the preset round threshold, or, based on the order fulfillment rates of the current traversal round and the previous N traversal rounds, it is determined that the order fulfillment rate has not decreased in multiple consecutive rounds of traversal, then it is determined that the preset termination condition is met.

[0190] It should be noted that the current traversal round is the number of complete traversals of the current pending orders; the order fulfillment rate is the proportion of completed orders in the pending orders after scheduling, that is, the ratio of the total number of completed orders to the total number of pending orders; completed orders are orders that can be delivered before the delivery deadline after scheduling; N is a positive integer and can be set in advance by the administrator of the order scheduling device.

[0191] In actual use, in some cases, the global optimal solution is actually difficult to obtain, or requires a very large number of iterations to obtain, which does not meet actual usage requirements. By setting a preset termination condition in combination with the changes in the iterative traversal rounds and the order fulfillment rate, it can be ensured that while obtaining the global optimal solution as much as possible, too many iterative traversal rounds are avoided, consuming too many computing resources or too many calculations to produce results. Therefore, if the current traversal round is greater than or equal to the preset round threshold, or, based on the order fulfillment rates of the current traversal round and the previous N traversal rounds, it is determined that the order fulfillment rate has not decreased in multiple consecutive traversal rounds, it can be determined that the preset termination condition is met; otherwise, it can be determined that the preset termination condition is not met, and a new round of traversal can be continued at this time, so the process can return to execute step S10.

[0192] In this embodiment, when setting the corresponding order delivery schedule for the current order, the order delivery schedule of the order may be set by using the first-order reverse and second-order normal schedules. At this time, step S30 in this embodiment may include:

[0193] Based on the order delivery strategy, use one-stage reverse scheduling to schedule the current order setting to determine whether it can be scheduled;

[0194] If it can be scheduled, then set the corresponding order delivery schedule for the current order according to the one-stage reverse scheduling result;

[0195] If it cannot be scheduled, then based on the order delivery strategy, the current order is scheduled according to the two-stage normal scheduling, and the corresponding order delivery schedule is set for the current order.

[0196] It should be noted that one-stage reverse scheduling refers to the process of reverse scheduling between the order unlocking date and the expected delivery date when the order can be completed before the expected delivery date. Its purpose is to make the actual order generation or delivery time as close to the expected delivery date as possible while ensuring that the order can be delivered normally.

[0197] The two-stage forward scheduling means that when an order cannot be completed before the expected delivery date, the nearest production or delivery date is determined after the expected delivery date. The goal of the two-stage forward scheduling is to complete the production of orders that cannot be produced or delivered in time within the expected delivery date as soon as possible outside the delivery date, that is, to make the order delivery or production date after the expected delivery date, as close to the expected delivery date as possible.

[0198] In this embodiment, when scheduling, different methods may be used to set order delivery schedules for different order delivery strategies. If the order delivery strategy is to give priority to using inventory, then step S30 in this embodiment may include:

[0199] Obtain the delivery deadline corresponding to the current order;

[0200] Determining whether the current order can be supplied within the delivery period according to the inventory information;

[0201] If the supply cannot be made within the delivery deadline, determining whether the production and delivery of the current order can be completed before the delivery deadline based on the production capacity information;

[0202] If the production and delivery can be completed before the delivery deadline, setting an order delivery schedule for the current order according to the inventory information and the production capacity information, and updating the inventory information and the production capacity information;

[0203] If production delivery cannot be completed before the delivery deadline, after determining the delivery deadline based on the inventory information and the production capacity information, the nearest available delivery time to the delivery deadline is used, and an order delivery schedule is set for the current order based on the available delivery time, and the inventory information and production capacity information are updated.

[0204] For ease of understanding, now combined Figure 3 This is for illustration only, but not for limitation. Figure 3 This is a schematic diagram of the priority inventory setting process of this embodiment. Figure 3 As shown, for the delivery strategy of finished products, the inventory is used first. In scheduling, the inventory information is queried first to determine whether the required material is in stock or can be supplied within the delivery period. If the required material is in stock or can be supplied within the delivery period (i.e., the expected delivery period), the inventory or future supply is used for delivery, and the corresponding inventory or supply is deducted. If the material does not exist or cannot be supplied within the delivery period, it is determined whether the production of the material can be completed before the delivery period based on the generated information. If the material cannot be produced before the delivery period due to production capacity or raw material constraints, the material production is completed as soon as possible after the delivery period.

[0205] In this embodiment, if the order delivery strategy is to give priority to production, then step S30 in this embodiment may include:

[0206] Obtain the delivery deadline corresponding to the current order;

[0207] Determine whether the current order can be completed before the delivery deadline based on inventory information and production capacity information;

[0208] If production cannot be completed, determine whether inventory can be supplied before the delivery deadline based on inventory information;

[0209] If the inventory can be supplied before the delivery deadline, setting an order delivery schedule for the current order according to the inventory information, and updating the inventory information and production capacity information;

[0210] If the inventory cannot be supplied before the delivery deadline, after determining the delivery deadline based on the inventory information and the production capacity information, the nearest available delivery time to the delivery deadline is used, and an order delivery schedule is set for the current order based on the available delivery time, and the inventory information and production capacity information are updated.

[0211] For ease of understanding, now combined Figure 4 This is for illustration only, but not for limitation. Figure 4 This is a schematic diagram of the priority production setting process of this embodiment. Figure 4As shown, for the delivery strategy with priority production of finished products, in scheduling, it is first determined whether the production of the material can be completed before the delivery date based on inventory information and production capacity information. If it cannot be produced due to production capacity or raw material constraints before the delivery date, it is checked whether there is inventory of the required material or supply within the delivery date. If there is inventory or supply within the delivery date, the inventory or supply within the delivery date is used for delivery, and the corresponding inventory or supply is deducted. If there is no inventory or supply is not available within the delivery date, an attempt is made to complete the production of the material as close as possible after the delivery date.

[0212] Among them, this embodiment provides a more detailed explanation of the completion of production of finished products before the delivery date. Completing production before the delivery date can be called the first-stage scheduling. The first-stage scheduling first determines the expected delivery date of the order and performs reverse scheduling between the order unlocking date and the expected delivery date. For example, the unlocking time of a finished product order is T1 day, and the expected delivery date is T2 day, T1 < T2. If the sum of the production and warehousing time of the finished product is S days, an attempt is first made to schedule at T2 - S day (i.e., the latest start time for order fulfillment). At this date, it is checked whether the production capacity, semi-finished products, and raw material constraints meet the order requirements. If they are met, it is successfully scheduled. If not, an attempt is made to schedule at T2 - S - 1 day, and this process is repeated continuously. When an attempt to schedule at the order locking period T1 day still fails to be scheduled, this order is considered unable to be scheduled before the delivery date. In addition, various order splitting strategies can be attempted during the first-stage scheduling. For example, when it is impossible to schedule within the time period [T1, T2 - S], the order can be split into two small orders and an attempt is made to reverse-schedule the two small orders into the production queue. The goal of the first-stage reverse scheduling is to complete more orders before the delivery date. By combining with the delivery strategy output by the algorithm, the order fulfillment rate of the algorithm can be further improved.

[0213] It should be noted that when checking whether the semi-finished products meet the constraints during the first-stage scheduling, the corresponding semi-finished product usage strategy output by the decision network should be used. If the semi-finished products give priority to using inventory, first check whether the inventory of the semi-finished products can meet the production of the finished products. If not, attempt the production process. If the production cannot be completed, the semi-finished products do not meet the constraints. If the semi-finished products give priority to production, first attempt the semi-finished product production process. If the production cannot be completed, attempt to use inventory. If the inventory is insufficient, the semi-finished products do not meet the constraints. When attempting to produce semi-finished products, the first-stage finished product scheduling mode is also adopted. Assuming that the sum of the production and warehousing time of the semi-finished products is S days and the upper-layer finished product attempts to schedule at T3, the scheduling time range of the semi-finished products is [T1, T3 - S - 1], that is, from the unlocking of the upper-layer finished product to the latest time when the semi-finished products meet the delivery requirements. The reverse scheduling mode is still adopted, that is, continuously attempt to schedule forward from the last day of the time range. In addition, the constraint detection method for the lower-layer semi-finished products of the semi-finished products is the same as that for the upper-layer semi-finished products.

[0214] Here, this solution explains the production and delivery at the nearest time outside the delivery date in more detail, and calls the completion of production at the nearest time outside the delivery date as two-stage scheduling. If the expected delivery date of the order is T2, and the sum of the production and warehousing time of the product is S days, the time range of the two-stage scheduling is [T2-S, +∞]. Orders produced within this time range are considered unattainable, so it is hoped that scheduling will be performed as early as possible within this range to minimize the delivery delay. The two-stage scheduling starts from T2-S day and performs forward scheduling to detect whether the capacity, semi-finished products, and raw material constraints of the day meet the order requirements. If they meet, they are successfully scheduled. If not, they will try to schedule on the next day until the order is scheduled. The goal of the two-stage forward scheduling is to complete the production of orders that have not been produced in time within the delivery date as soon as possible outside the delivery date, so that the demand side can get the goods as soon as possible after exceeding the agreed time. By combining with the delivery strategy output by the algorithm, it can improve the satisfaction of the demand side, maintain customer relationships, reduce semi-finished product inventory, and improve overall inventory turnover.

[0215] It should be noted that when checking whether semi-finished products meet the constraints in the second-stage scheduling, the semi-finished product usage strategy corresponding to the output of the decision network should be used. If the semi-finished products are given priority to use inventory, the semi-finished product inventory should be checked first to see if it can meet the production of finished products. If not, the production process should be tried. If the production cannot be completed, the semi-finished product does not meet the constraints. If the semi-finished products are given priority to production, the semi-finished product production process should be tried first. If the production cannot be completed, the inventory should be tried. If the inventory does not meet the constraints, the semi-finished product does not meet the constraints. When the semi-finished products are tried to be produced, assuming that the sum of the production and warehousing time of the semi-finished products is S days, the upper-level finished products are tried to be scheduled at T3, then the scheduling time range of the semi-finished products is [T1, T3-S-1], that is, from the unlocking of the upper-level finished products to the latest time when the semi-finished products meet the delivery, the reverse scheduling mode is still adopted, that is, the scheduling is continuously tried from the last day of the time range. This is consistent with the semi-finished products in the first stage and will not be repeated.

[0216] This embodiment provides an order scheduling method. Due to the introduction of reinforcement learning, the delivery strategy is intelligently decided through a deep neural network, and scheduling is automatically performed for different delivery strategies. Compared with the simple delivery strategy, the degree of intelligence is improved, the rationality of the delivery strategy and the order fulfillment rate are improved.

[0217] In a feasible implementation, step S10 may include steps S101-S102:

[0218] Step S101: sorting the orders to be processed according to a preset sorting rule to obtain a sorting order;

[0219] Step S102: traverse the pending orders in the sorting order to obtain the current order.

[0220] It should be noted that the preset sorting rules can be set according to at least one of the order number, order type, and order requirement. Of course, according to actual needs, the preset sorting rules can also be set according to more indicators, and this embodiment does not limit this.

[0221] It is understandable that in actual applications, different orders have different priorities, and the higher the order is in the sorting, the earlier it will be scheduled, and the earlier the order is scheduled, the higher the possibility of the order being fulfilled. Therefore, the pending orders can be sorted according to the preset sorting rules, and the pending orders can be traversed in the order of traversal based on the sorting rules, so as to ensure that the pending orders with high priority are traversed first.

[0222] In this embodiment, when traversing the pending orders, the pending orders are first sorted according to the preset sorting rules, and then the pending orders are traversed according to the sorting rules. This can ensure that the pending orders with higher priorities are traversed first, and ensure that the pending orders with higher priorities are scheduled first, which meets the actual scheduling requirements as much as possible.

[0223] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 5 , step S40, including steps S401 to S405:

[0224] Step S401: Obtain the delivery deadline corresponding to the current order.

[0225] It should be noted that the delivery deadline corresponding to the current order may be the expected delivery date corresponding to the current order, that is, the expected delivery date; obtaining the delivery deadline corresponding to the current order may be reading the expected delivery date of the current order, and using the read expected delivery date as the delivery deadline corresponding to the current order.

[0226] Step S402: Determine whether the current order is fulfillable according to the delivery deadline and the order delivery schedule.

[0227] It should be noted that, to determine whether the current order can be fulfilled based on the delivery deadline and the order delivery schedule, the order processing time can be read from the order delivery schedule of the current order, and the order processing time is compared with the delivery deadline corresponding to the current order. If the order processing time is before the delivery deadline of the current order, or the order processing time is the same as the delivery deadline of the current order, then the current order is determined to be fulfilled; conversely, if the order processing time is after the delivery deadline of the current order, then the current order is determined to be unfulfillable.

[0228] Step S403: If the current order can be fulfilled, construct an experience sample according to the first preset score, the current order and the order delivery strategy.

[0229] Step S404: If the current order cannot be fulfilled, construct an experience sample according to the second preset score, the current order and the order delivery strategy.

[0230] It should be noted that the first preset score and the second preset score can be set in advance by the administrator of the order scheduling device.

[0231] In actual use, constructing the experience sample according to the first preset score, the current order, and the order delivery strategy may be to use the first preset score as the reward value, and construct the experience sample according to the reward value, the current order, and the order delivery strategy. Constructing the experience sample according to the second preset score, the current order, and the order delivery strategy may be to use the second preset score as the reward value, and construct the experience sample according to the reward value, the current order, and the order delivery strategy.

[0232] It should be noted that if the order can be fulfilled, it means that the scheduling effect is good, and if the order cannot be fulfilled, it means that the scheduling effect is poor. Different reward values ​​can be given for these two types of orders to optimize the decision network.

[0233] Among them, in order to ensure the optimization effect, it is necessary to distinguish between positive incentives and negative incentives. Achievable orders serve as positive incentives for the optimization of the decision network, while unachievable orders serve as negative incentives. Therefore, the first preset score can be greater than the second preset score. In order to distinguish the incentives to a greater extent, the first preset score can be a positive value and the second preset score can be a negative value.

[0234] Step S405: Optimizing the decision network according to the experience samples.

[0235] It is understandable that after constructing the experience samples, the experience samples can be used to iteratively optimize the decision network, adjust its internal parameters, and optimize its decision-making effect.

[0236] In a possible implementation of this embodiment, in order to ensure that the decision network can be reasonably optimized, step S405 of this embodiment may include:

[0237] Adding the experience sample to the sample pool to obtain an updated sample pool;

[0238] Extracting training samples from the updated sample pool according to a preset sampling quantity;

[0239] The decision network is optimized and trained according to the extracted training samples.

[0240] It should be noted that the preset sampling number is the number of samples used in the iterative update of the decision network. When optimizing the decision network, the number of samples used in each round of iterative training is set. In order to ensure that the decision network can be reasonably optimized, it is necessary to reasonably select the samples for iterative optimization of the decision network. Therefore, after constructing the experience sample, the constructed experience sample can be added to the sample pool to obtain the updated sample pool, and then the training sample is extracted from the updated sample pool according to the preset sampling number, and the decision network is iteratively optimized and trained according to the extracted training samples.

[0241] For ease of understanding, an example is given below, but this solution is not limited to the following:

[0242] After the scheduling is completed, the reward for the current order can be output according to the order delivery schedule. If the current step is the tth step in a complete iteration round (i.e., the tth traversal in a complete traversal of the order to be processed), the reward function is defined as follows:

[0243]

[0244] If the order scheduled in the tth step is fulfilled by the intelligent scheduling system, the reward for the tth step is p1. Usually, p1 is set to a positive value, indicating that the successful fulfillment of the order has a positive incentive for the model. If it is not fulfilled by the intelligent scheduling system, the reward for the tth step is p2. Usually, p2 is set to a negative value, indicating that the failure to successfully fulfill the order has a negative penalty for the model.

[0245] Through the definition of positive and negative rewards, the reinforcement learning model (i.e., the decision network) will iteratively optimize parameters with the goal of maximizing the cumulative rewards. As the cumulative rewards continue to increase, the number of orders completed will gradually increase, reaching the goal of the highest number of orders completed.

[0246] During multiple iterations of the algorithm, the cumulative reward is defined as:

[0247]

[0248] In the formula, γ is the attenuation factor, and its value range is [0,1]. The smaller t is, the larger the attenuation factor is set.

[0249] Get the reward r for the current step t t After that, generate experience samples t , A t , S t+1 , r t , down>, where is the order feature information of the current order traversed in the t-th step, is the order delivery strategy of the current order traversed in the t-th step, rt is the reward of the current order traversed in the t-th step, S t+1 ​It is the status information of the corresponding order when the t+1 step is scheduled, obtained through the status acquisition module. Down is the end flag (0-1 variable). Down = 0 means that the current t step is not the last order, and down = 1 means that the current t step is the last order. It should be noted that when down = 1, there are no subsequent orders to be scheduled. The convention S t =S t+1 .

[0250] The generated experience samples are stored in the experience pool. When the number of samples in the experience pool is greater than the batch_size of the training model, batch_size samples are randomly sampled from the experience pool to train the model parameters.

[0251] If the order corresponding to the current step t is not the last order in the order sorting, then update t, set t=t+1, and continue traversing, switching to the next pending order. If the order corresponding to the current step t is the last order, determine whether the algorithm should terminate the iteration based on the algorithm termination condition. If the termination condition is met, the delivery strategy of the finished product, the use strategy of the semi-finished product and the scheduling results are output to the planner through the front-end page. If not, set t=0 and restart a new round of iterative traversal. The termination condition can be customized according to the user's needs, such as: complete iteration of n rounds, the achievement rate does not continue to decline within m rounds, the order achievement rate is greater than a certain expected target, etc.

[0252] In this embodiment, by distinguishing between achievable orders and unachievable orders and setting different reward values ​​for them, experience samples are constructed to ensure that when the decision network is optimized through the experience samples, the decision network is optimized in the direction of achieving the orders as much as possible, thereby improving the decision-making effect of the decision network and ensuring that the order fulfillment rate can be increased.

[0253] This application also provides an order scheduling device, please refer to Figure 6 , the order scheduling device comprises:

[0254] An acquisition module 10 is used to traverse pending orders and obtain current orders, wherein the pending orders are orders that have not been delivered, and the pending orders include MTO orders and MTS orders;

[0255] A decision module 20 is used to decide whether the order delivery strategy corresponding to the current order is a production delivery strategy or an inventory delivery strategy through a decision network, wherein the decision network is a value network or a strategy network, and the order delivery strategy includes a finished product delivery strategy and / or a semi-finished product delivery strategy when delivering the current order;

[0256] A setting module 30, configured to set a corresponding order delivery schedule for the current order based on the order delivery strategy, wherein the order delivery schedule includes an order processing time and a processing quantity;

[0257] A construction module 40, configured to construct an experience sample according to the current order and the order delivery strategy, and optimize the decision network according to the experience sample;

[0258] The generating module 50 is used to generate order scheduling results according to the order delivery strategies and order delivery schedules corresponding to each pending order at the end of the traversal.

[0259] The order scheduling device provided by the present application adopts the order scheduling method in the above embodiment, which can solve the technical problem that the delivery strategy for MTO and MTS mixed orders in the related art is relatively simple and the actual use effect is not good. Compared with the related art, the beneficial effects of the order scheduling device provided by the present application are the same as the beneficial effects of the order scheduling method provided by the above embodiment, and the other technical features in the order scheduling device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0260] The present application provides an order scheduling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the order scheduling method in the above-mentioned embodiment one.

[0261] Reference below Figure 7 , which shows a schematic diagram of the structure of an order scheduling device suitable for implementing the embodiment of the present application. The order scheduling device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The order scheduling device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0262] like Figure 7As shown, the order scheduling device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the order scheduling device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the order scheduling device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an order scheduling device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0263] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0264] The order scheduling device provided by the present application adopts the order scheduling method in the above embodiment, which can solve the technical problem that the delivery strategy for MTO and MTS mixed orders in the related technology is relatively simple and the actual use effect is not good. Compared with the related technology, the beneficial effects of the order scheduling device provided by the present application are the same as the beneficial effects of the order scheduling method provided by the above embodiment, and the other technical features in the order scheduling device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0265] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0266] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0267] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the order scheduling method in the above-mentioned embodiment.

[0268] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic or infrared systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0269] The computer-readable storage medium may be included in the order scheduling device; or it may exist independently without being assembled into the order scheduling device.

[0270] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the order scheduling device, the order scheduling device: traverses the pending orders to obtain the current order, the pending orders are orders that have not been delivered, and the pending orders include MTO orders and MTS orders; decides through a decision network that the order delivery strategy corresponding to the current order is a production delivery strategy or an inventory delivery strategy, the decision network is a value network or a strategy network, and the order delivery strategy includes a finished product delivery strategy and / or a semi-finished product delivery strategy when delivering the current order; sets a corresponding order delivery schedule for the current order based on the order delivery strategy, and the order delivery schedule includes order processing time and processing quantity; constructs an experience sample based on the current order and the order delivery strategy, and optimizes the decision network based on the experience sample; at the end of the traversal, generates an order scheduling result based on the order delivery strategy and order delivery schedule corresponding to each pending order.

[0271] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0272] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0273] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0274] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned order scheduling method, and can solve the technical problem that the delivery strategy for MTO and MTS mixed orders in the related art is relatively simple and the actual use effect is not good. Compared with the related art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the order scheduling method provided in the above-mentioned embodiment, and will not be repeated here.

[0275] The present application also provides a computer program product, including a computer program, which implements the steps of the order scheduling method as described above when executed by a processor.

[0276] The computer program product provided by this application can solve the technical problem that the delivery strategy for MTO and MTS mixed orders in the related art is relatively simple and the actual use effect is not good. Compared with the related art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the order scheduling method provided by the above embodiment, which will not be repeated here.

[0277] The above descriptions are only some embodiments of the present application, and do not limit the protection scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An order scheduling method, characterized in that: The order scheduling method comprises: Traverse the pending orders to obtain the current orders, where the pending orders are orders that have not been delivered, and the pending orders include MTO orders and MTS orders; The order delivery strategy corresponding to the current order is determined to be a production delivery strategy or an inventory delivery strategy through a decision network, the decision network is a value network or a strategy network, and the order delivery strategy includes a finished product delivery strategy and / or a semi-finished product delivery strategy when delivering the current order; Setting a corresponding order delivery schedule for the current order based on the order delivery strategy, wherein the order delivery schedule includes an order processing time and a processing quantity; constructing an experience sample according to the current order and the order delivery strategy, and optimizing the decision network according to the experience sample; At the end of the traversal, the order scheduling results are generated according to the order delivery strategies and order delivery schedules corresponding to each pending order; Wherein, at the end of the traversal, the order scheduling result is generated according to the order delivery strategy and order delivery schedule corresponding to each pending order, including: At the end of the traversal, check whether the preset termination condition is met; If the preset termination conditions are met, the order scheduling results are generated according to the order delivery strategies and order delivery schedules corresponding to each pending order; At the end of the traversal, detecting whether a preset termination condition is met includes: At the end of the traversal, the order fulfillment rate of the current traversal round and the previous N traversal rounds is obtained, wherein the current traversal round is the number of times the pending orders are completely traversed, the order fulfillment rate is the proportion of the scheduled fulfilled orders in the pending orders, and the fulfilled orders are the orders that can be delivered before the delivery deadline after scheduling; If it is determined based on the order fulfillment rates of the current traversal round and the previous N traversal rounds that the order fulfillment rate has not decreased in multiple consecutive traversal rounds, it is determined that the preset termination condition is met.

2. The order scheduling method according to claim 1, characterized in that: The setting a corresponding order delivery schedule for the current order based on the order delivery strategy includes: Based on the order delivery strategy, use one-stage reverse scheduling to schedule the current order setting to determine whether it can be scheduled; If it can be scheduled, then set the corresponding order delivery schedule for the current order according to the one-stage reverse scheduling result; If it cannot be scheduled, then based on the order delivery strategy, the current order is scheduled according to the two-stage normal scheduling, and the corresponding order delivery schedule is set for the current order.

3. The order scheduling method according to claim 1, characterized in that: The order delivery strategy is to use inventory first; The setting a corresponding order delivery schedule for the current order based on the order delivery strategy includes: Obtain the delivery deadline corresponding to the current order; Determining whether the current order can be supplied within the delivery period according to the inventory information; If the supply cannot be made within the delivery deadline, determining whether the production and delivery of the current order can be completed before the delivery deadline based on the capacity information; If the production and delivery can be completed before the delivery deadline, setting an order delivery schedule for the current order according to the inventory information and the production capacity information, and updating the inventory information and the production capacity information; If production delivery cannot be completed before the delivery deadline, after determining the delivery deadline based on the inventory information and the production capacity information, the nearest available delivery time to the delivery deadline is used, and an order delivery schedule is set for the current order based on the available delivery time, and the inventory information and production capacity information are updated.

4. The order scheduling method according to claim 1, wherein: The order delivery strategy is to give priority to production; The setting a corresponding order delivery schedule for the current order based on the order delivery strategy includes: Obtain the delivery deadline corresponding to the current order; Determine whether the current order can be completed before the delivery deadline based on inventory information and production capacity information; If production cannot be completed, determine whether inventory can be supplied before the delivery deadline based on inventory information; If the inventory can be supplied before the delivery deadline, setting an order delivery schedule for the current order according to the inventory information, and updating the inventory information and production capacity information; If the inventory cannot be supplied before the delivery deadline, after determining the delivery deadline based on the inventory information and the production capacity information, the nearest available delivery time to the delivery deadline is used, and an order delivery schedule is set for the current order based on the available delivery time, and the inventory information and production capacity information are updated.

5. The order scheduling method according to claim 1, characterized in that: At the end of the traversal, detecting whether a preset termination condition is met includes: At the end of the traversal, the current traversal round is obtained, where the current traversal round is the number of times the current pending orders are completely traversed; If the current traversal round is greater than or equal to a preset round threshold, it is determined that the preset termination condition is met.

6. The order scheduling method according to claim 1, wherein: At the end of the traversal, after detecting whether the preset termination condition is met, the method further includes: If the preset termination condition is not met, a dynamic ε-greedy adjustment strategy is used to adjust the exploration and utilization parameters of the decision network, where the exploration and utilization parameters are parameters that control the exploration and solution experience in the decision network; When the adjustment is completed, return to the step of traversing the pending orders to obtain the current order.

7. The order scheduling method according to claim 6, characterized in that: The step of using a dynamic ε-greedy adjustment strategy to adjust the exploration and utilization parameters of the decision network includes: Determine the order fulfillment rate according to the order delivery schedule, and obtain the previous iteration detection parameters; Generate a current exploration and utilization parameter according to the previous iteration detection parameter, the order fulfillment rate and the preset attenuation coefficient, wherein the previous iteration detection parameter is the exploration and utilization parameter generated in the previous iteration round, and each iteration round represents a complete traversal of the orders to be processed; The exploration and utilization parameters in the decision network are updated to the current exploration and utilization parameters.

8. The order scheduling method according to claim 7, characterized in that: The generating of the current exploration and utilization parameter according to the previous iteration detection parameter, the order fulfillment rate and the preset attenuation coefficient includes: Subtract the previous iteration detection parameter from the final value of the iteration parameter to obtain a first intermediate parameter; The order fulfillment rate is multiplied by a preset attenuation coefficient, and the difference between the product and a preset constant is calculated to obtain a second intermediate parameter; The first intermediate parameter is multiplied by the second intermediate parameter, and the product is added to the final value of the iteration parameter to generate a current exploration and utilization parameter.

9. The order scheduling method according to claim 1, wherein: The constructing of the experience sample according to the current order and the order delivery strategy includes: Obtain the delivery deadline corresponding to the current order; Determining whether the current order is achievable according to the delivery deadline and the order delivery schedule; If the current order can be fulfilled, constructing an experience sample according to the first preset score, the current order and the order delivery strategy; If the current order cannot be fulfilled, constructing an experience sample according to the second preset score, the current order and the order delivery strategy; Among them, the first preset score is greater than the second preset score.

10. The order scheduling method according to claim 1, wherein: The optimizing the decision network according to the experience sample comprises: Adding the experience sample to the sample pool to obtain an updated sample pool; Extracting training samples from the updated sample pool according to a preset sampling quantity; The decision network is optimized and trained according to the extracted training samples.

11. The order scheduling method according to claim 1, wherein: The traversal of the pending orders to obtain the current order includes: Sorting the pending orders according to a preset sorting rule to obtain a sorting order, wherein the preset sorting rule is set according to at least one of an order number, an order type, and an order requirement; The pending orders are traversed in the sorting order to obtain the current order.

12. The order scheduling method according to any one of claims 1 to 11, characterized in that: The step of setting a corresponding order delivery strategy for the current order through the decision network includes: Extracting features of the current order to obtain order feature information; The order feature information is input into the decision network for decision making to obtain the order delivery strategy corresponding to the current order.

13. An order scheduling device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the order scheduling method according to any one of claims 1 to 12.

Images