Order allocation method, model construction method and device in shared manufacturing mode

Abstract

The invention discloses an order allocation method and device and a model construction method and device in a shared manufacturing mode, and the model construction method comprises the steps: obtaining historical data which at least comprises order data and production enterprise feature data; constructing a shared manufacturing simulation model according to the historical data and the operation process of the production enterprise; obtaining an order allocation strategy initial model, wherein the order allocation strategy initial model is constructed according to the shared manufacturing simulation model and a DQN algorithm model taking the shared manufacturing simulation model as an interaction object; training the order allocation strategy initial model based on a DQN algorithm to obtain a trained order allocation strategy model; according to the method, various factors and interaction in the shared manufacturing system can be reflected more comprehensively and accurately, the order distribution problem of the shared manufacturing system is described in a mode closer to the actual situation, the operation efficiency is high, the performance is good, and the method can be suitable for the complex and dynamic shared manufacturing environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of shared manufacturing, and in particular relates to an order allocation method, a model building method and a device under a shared manufacturing mode. Background Art

[0002] Shared manufacturing, as an emerging manufacturing model, provides hierarchical shared manufacturing services in a point-to-point manner, supporting shared consumption between individuals or enterprises. Specifically, the tangible or intangible assets of suppliers are encapsulated into shared manufacturing services and integrated into platform services, supporting consumers to access them in a point-to-point manner. In this way, the utility of goods and services can be maximized, and the breadth and depth of sharing can be expanded. According to factors such as platform subject characteristics, business model, and shared content, shared manufacturing models can be divided into four types: intermediary, crowd innovation, service, and collaborative. Intermediary platforms are built by third-party Internet companies, connecting supply and demand by integrating multi-party production capacity; crowd innovation platforms are led by large manufacturing companies and provide comprehensive services to small and micro start-ups and internal entrepreneurial teams; service platforms are built by industrial technology companies, renting intelligent equipment to manufacturing companies and providing technical services; collaborative platforms are jointly built by third parties or small and micro enterprises, and the participating entities are mainly small and medium-sized enterprises.

[0003] Intermediary shared manufacturing platforms are built and operated by third-party companies. Such platforms do not have manufacturing capacity and are mainly responsible for matching transactions between supply and demand. Therefore, in this model, the matching and scheduling research between supply and demand, that is, the research on order allocation strategy, is more difficult and has outstanding practical significance.

[0004] At present, the modeling method of mixed integer programming is usually adopted, combined with heuristic algorithms to solve the problems of supply and demand matching and production scheduling. However, the modeling method of mixed integer programming is difficult to fully restore the complex dynamic shared manufacturing system, so it is difficult to formulate an allocation strategy that can effectively cope with the actual order allocation scenario. The shared manufacturing system involves multiple parties such as demanders and production enterprises. The modeling of its order allocation problem is inseparable from the characterization of the demand generation process and the enterprise production process. However, various types of demand are heterogeneous in terms of geographical location, demand volume and other characteristics, and between production enterprises in terms of equipment quantity, production rate and other factors. In addition, the internal production process of a single enterprise and the external demand of the shared manufacturing system are both random. Mathematical programming models are difficult to fully characterize the order allocation problem of the shared manufacturing system. In addition, although the heuristic algorithm can solve the problem modeled based on the simulation method, it can only solve its order allocation strategy for one shared manufacturing scenario, that is, one parameter combination at a time. Once the scenario changes, it is necessary to find the optimal solution again, that is, it lacks the ability to quickly adapt to the dynamic manufacturing environment.

[0005] Therefore, a new order allocation method is urgently needed to solve the order allocation problem under the shared manufacturing model. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes an order allocation method, a model building method and a device under a shared manufacturing mode, which can more comprehensively and accurately reflect the various factors and interactions in the shared manufacturing system, and characterize the order allocation problem of the shared manufacturing system in a way that is closer to the actual situation. It has high operating efficiency and good performance, and can be applied to complex and dynamic shared manufacturing environments.

[0007] In a first aspect, the present invention provides a method for constructing an order allocation strategy model, comprising:

[0008] Acquiring historical data, wherein the historical data at least includes historical order data and production enterprise characteristic data;

[0009] Building a shared manufacturing simulation model based on the historical data and the production enterprise operation process;

[0010] Acquire an initial model of an order allocation strategy, wherein the initial model of the order allocation strategy is constructed according to the shared manufacturing simulation model and a DQN algorithm model with the shared manufacturing simulation model as an interaction object;

[0011] The order allocation strategy initial model is trained based on the DQN (Deep Q-Network) algorithm to obtain a trained order allocation strategy model.

[0012] In an optional embodiment, the shared manufacturing simulation model at least includes a state acquisition module, a revenue acquisition module, an order arrival module, a queue module, an order allocation module and a plurality of production enterprise modules, wherein the order arrival module, the queue module and the order allocation module are connected in sequence, and each of the production enterprise modules includes a production submodule, a production equipment submodule and a truck submodule respectively connected to the production submodule; the order allocation module is respectively connected to the production submodule in each production enterprise module;

[0013] The state acquisition module is used to determine state variable information and send the state variable information to the DQN algorithm model;

[0014] The revenue acquisition module is used to determine the revenue information and send the revenue information to the DQN algorithm model; wherein the revenue information is expressed as:

[0015] FT j =WT j +PT j +DT j ;

[0016] In the above formula, FT j is the completion time of the jth order; WT j is the waiting time for the jth order; PT j is the production time of the jth order; DT j is the shipping time of the jth order;

[0017] The order arrival module is used to randomly generate multiple orders according to the demand points and determine the order information corresponding to each order;

[0018] The queue module is used to queue each order and send the orders to the order distribution module in sequence according to the queue order; and determine the waiting time for each order;

[0019] The order allocation module is used to determine the production enterprise module corresponding to each order according to the order allocation strategy information output by the DQN algorithm model, and send multiple order information to the corresponding production enterprise module in sequence;

[0020] The production submodule is used to call the production equipment submodule to produce products according to the order information;

[0021] The production equipment submodule is used to determine the production time used by the target production equipment to produce a single product;

[0022] The truck submodule is used to transport products to demand points according to a pre-established GIS map model and determine the transportation time.

[0023] In an optional implementation, the historical order data includes at least location data of the demand point to which each historical order belongs and product quantity data in each historical order.

[0024] In an optional embodiment, the order arrival module includes a demand point submodule, a product quantity submodule and an order arrival interval submodule;

[0025] The demand point submodule is used to determine the location information of the demand point to which each order belongs according to the location data and location distribution function of the demand point to which each historical order belongs;

[0026] The product quantity module is used to determine the product quantity information contained in each order according to the product quantity data and data distribution function in each historical order;

[0027] The order arrival interval module is used to determine the time interval for the arrival of each order, so as to generate multiple orders according to the time interval for the arrival of each order, wherein each generated order includes order information, and the order information includes the location information of the demand point to which each order belongs and the product quantity information contained in each order.

[0028] In an optional implementation, the manufacturing enterprise characteristic data includes at least the location data of each manufacturing enterprise, the number of production equipment of each manufacturing enterprise, and the production time data, failure time data and maintenance time data of each equipment in each manufacturing enterprise for a single product.

[0029] In an optional embodiment, the production equipment submodule includes a processing capacity module, a fault interval time module and a maintenance time module;

[0030] The failure time module is used to determine the failure occurrence time of the target production equipment according to the failure time data;

[0031] The maintenance time module is used to determine the maintenance time of the target production equipment according to the maintenance time data;

[0032] The processing capacity module is used to determine the production time used by the target production equipment to produce a single product according to the production time data of the single product;

[0033] The production time of the single product produced by the target production equipment is determined according to the production time used by the target production equipment to produce the single product, the maintenance time of the target production equipment and the failure time of the target production equipment.

[0034] In an optional implementation, the state variable information includes:

[0035] s t =(d t ,l t ,c it *k it ,w it );

[0036] In the above formula, s t is the state variable in round t; d t is the number of products required to be produced for the orders to be allocated in round t; l t is the location of the demand point to which the order to be allocated in round t belongs; c i *k it is the idle production capacity of the ith production enterprise; where c i k is the reciprocal of the average time required to produce a single product; it is the number of idle equipment in the ith production enterprise in round t; w it is the number of products to be produced by the i-th production enterprise.

[0037] In an optional implementation, the order allocation strategy initial model is trained based on the DQN algorithm to obtain a trained order allocation strategy model, including:

[0038] Initializing the experience pool and the network parameters of the DQN algorithm model, wherein the DQN algorithm model includes an action neural network and a target neural network, and the network parameters of the DQN algorithm model include network parameters of the action neural network and network parameters of the target neural network;

[0039] Repeat the following steps until the preset number of training times is reached:

[0040] Reset the shared manufacturing simulation model;

[0041] Repeat the following steps until all orders are completed:

[0042] Acquiring current state variable information sent by the shared manufacturing simulation model;

[0043] Determine order allocation strategy information according to the current state variable information and the ε-greedy strategy algorithm, and send the order allocation strategy information to the shared manufacturing simulation model;

[0044] Obtaining revenue information and next state variable information sent by the shared manufacturing simulation model;

[0045] Storing the experience tuple in the experience pool, wherein the experience tuple includes the current state variable information, the order allocation strategy information, the revenue information and the next state variable information;

[0046] Randomly selecting a preset number of experience tuples from the experience pool for training;

[0047] Determine the target value corresponding to each experience tuple according to the action neural network and the target neural network, and determine the loss function value;

[0048] Using a back propagation algorithm and a gradient descent algorithm to update the network parameters of the action neural network, after updating a preset number of times, copying the values ​​of the network parameters of the action neural network to the target neural network to stabilize the training process;

[0049] Output the trained order allocation strategy model.

[0050] In a second aspect, the present invention provides an order allocation method under a shared manufacturing mode, comprising:

[0051] Get the orders to be allocated;

[0052] The order to be allocated is input into a trained order allocation strategy model, and an allocation strategy for the order to be allocated is output, wherein the order allocation strategy model is constructed according to the order allocation strategy model construction method described in the first aspect.

[0053] In a third aspect, the present invention provides an order allocation strategy model construction device, comprising:

[0054] A data acquisition module, used to acquire historical data, wherein the historical data at least includes historical order data and production enterprise characteristic data;

[0055] A simulation model building module, used to build a shared manufacturing simulation model based on the historical data and the production enterprise operation process;

[0056] An initial model acquisition module, used to acquire an initial model of an order allocation strategy, wherein the initial model of the order allocation strategy is constructed according to the shared manufacturing simulation model and a DQN algorithm model with the shared manufacturing simulation model as an interaction object;

[0057] The training module is used to train the initial model of the order allocation strategy based on the DQN algorithm to obtain a trained order allocation strategy model.

[0058] In a fourth aspect, the present invention provides an order allocation device in a shared manufacturing mode, characterized in that it includes:

[0059] An order acquisition module is used to acquire orders to be allocated;

[0060] An order allocation module is used to input the order to be allocated into a trained order allocation strategy model and output an allocation strategy for the order to be allocated, wherein the order allocation strategy model is constructed by the order allocation strategy model construction method described in any one of the first aspects.

[0061] In a fifth aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the aforementioned implementations when executing the computer program.

[0062] In a sixth aspect, the present invention provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the method described in any one of the aforementioned embodiments.

[0063] The technical solution provided by the embodiment of the present invention has the following beneficial effects: in the order allocation method, model building method and device under the shared manufacturing mode of the present invention, a shared manufacturing simulation model is constructed according to historical data and the operation process of the production enterprise in the model building method, so that various factors and interactions in the shared manufacturing system can be more comprehensively and accurately reflected, and the order allocation problem of the shared manufacturing system can be characterized in a way that is closer to the actual situation, which solves the limitation that the mathematical programming model is difficult to fully restore the complex dynamic shared manufacturing system, and provides support for solving the order allocation problem; in the model building method, the shared manufacturing simulation model is also used as the interaction object, and the initial model of the order allocation strategy is trained based on the DQN algorithm to obtain a trained order allocation strategy model, that is, the shared manufacturing simulation model is used as the training environment, and the initial model of the order allocation strategy based on the DQN algorithm is trained, so that the trained order allocation strategy model can directly make order allocation decisions for orders under the shared manufacturing system without repeatedly running the simulation model to search for order allocation decisions, thereby improving the operation efficiency; during the training process, the shared manufacturing system under different scenarios can be simulated, thereby improving the adaptability; thus, it can be applicable to complex and dynamic shared manufacturing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A flowchart of a method for constructing an order allocation strategy model provided by an embodiment of the present invention;

[0065] Figure 2 A schematic diagram of the application scenario principle of the shared manufacturing system provided by an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of the interaction principle between the shared manufacturing simulation model and the DQN algorithm model provided in an embodiment of the present invention;

[0067] Figure 4 A schematic diagram of a shared manufacturing simulation model provided by an embodiment of the present invention;

[0068] Figure 5 A schematic diagram of a flow chart of an order allocation method in a shared manufacturing mode provided by an embodiment of the present invention;

[0069] Figure 6 A schematic diagram comparing the effects of an order allocation strategy generated by an order allocation method under a shared manufacturing mode and a nearby allocation strategy provided by an embodiment of the present invention;

[0070] Figure 7 A sensitivity analysis diagram of different order arrival rates under an order allocation strategy generated by an order allocation method under a shared manufacturing mode provided by an embodiment of the present invention;

[0071] Figure 8A sensitivity analysis diagram of order arrival following uniform distribution under an order allocation strategy generated by an order allocation method under a shared manufacturing mode provided by an embodiment of the present invention;

[0072] Fig. 9 A system schematic diagram of an order allocation strategy model building device provided by an embodiment of the present invention;

[0073] Fig.10 A system schematic diagram of an order distribution device in a shared manufacturing mode provided by an embodiment of the present invention;

[0074] Fig.11 A system schematic diagram of an electronic device provided by an embodiment of the present invention.

[0075] In the figure: 11-data acquisition module; 12-simulation model building module; 13-initial model acquisition module; 14-training module; 21-order acquisition module; 22-order allocation module; 400-electronic device; 401-communication interface; 402-processor; 403-memory; 404-bus. DETAILED DESCRIPTION

[0076] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0077] Reference Figure 1 , an order allocation strategy model construction method provided by an embodiment of the present invention includes steps S110 to S140.

[0078] Step S110, obtaining historical data, the historical data includes historical order data and production enterprise characteristic data; the historical order data includes at least location data and product quantity data. Here, the location data is used to determine the location of the demand point to which each order belongs, and the product quantity data is used to determine the number of products contained in each order. The production enterprise characteristic data includes at least the location data of each production enterprise, the number of production equipment of each production enterprise, and the production time data, failure time data and maintenance time data of each equipment in each production enterprise for a single product.

[0079] Step S120: construct a shared manufacturing simulation model based on historical data and the operation process of the manufacturing enterprise. Specifically, this embodiment uses simulation software to construct a shared manufacturing simulation model.

[0080] When building a shared manufacturing simulation model, it is necessary to combine the application scenario of the shared manufacturing system - the operation process of the production enterprise under the shared manufacturing system. Figure 2As shown in the figure, it includes a shared manufacturing platform, an enterprise cluster consisting of multiple enterprises producing the same products, and several demand points. Demand orders are randomly generated at the demand points and sent to the shared manufacturing platform. After receiving the order information, the platform needs to decide which enterprise to assign the order to. If the order is assigned to a production enterprise with idle production capacity, production will start immediately, otherwise the order will enter the waiting queue. An order contains multiple products, and a production enterprise contains several production equipment. A production equipment can only produce one product at a time, and there is a certain probability of failure. When a failure occurs during the production process, the task being processed will be suspended until the equipment is successfully repaired and restored to normal working condition. After the production enterprise produces all the products required for the order, it will deliver them to the demand point of the order. Among them, the demand point is used to represent the geographical location of the product demander, and the demander will generate one or more orders. The order information includes the quantity of products required for the order and the address of delivery after the order is completed (that is, the location of the demand point to which the order belongs).

[0081] Step S130, obtaining an initial model of an order allocation strategy, wherein the initial model of the order allocation strategy is constructed according to a shared manufacturing simulation model and a DQN algorithm model with the shared manufacturing simulation model as an interaction object.

[0082] The DQN algorithm model is a model built based on the DQN algorithm. The main process of the DQN algorithm includes the following (1)-(5).

[0083] (1) The DQN algorithm uses a neural network to predict the reward value of each action in a given state. The input of this neural network is the state s, and the output is the long-term reward value estimate for all possible actions a.

[0084] (2) The DQN algorithm uses an experience replay mechanism to store and replay past experiences (i.e. states, actions, rewards, and new states). This disrupts the correlation between experiences during training, making the update of the neural network more effective.

[0085] (3) The DQN algorithm also uses a fixed Q target mechanism, that is, two neural networks (an action network and a target network) with the same structure but different parameters are used to predict the Q value. The action network is used to select actions, while the target network is used to calculate the target value of the Q value. This mechanism can prevent the target value from being affected by the latest parameters and improve the stability of the neural network.

[0086] (4) During the training process, the DQN algorithm updates the parameters of the neural network by minimizing the gap between the predicted Q value and the target Q value. This gap is used as a loss function and optimized through backpropagation and gradient descent.

[0087] (5) In order to balance exploration and exploitation, the DQN algorithm adopts the ε-greedy strategy algorithm, selecting random actions (exploration) with a certain probability, and selecting the action currently estimated to be optimal (exploitation) with a higher probability.

[0088] Step S140: training the initial order allocation strategy model based on the DQN algorithm to obtain a trained order allocation strategy model.

[0089] Reference Figure 3 , which is the interaction principle between the shared manufacturing simulation model and the DQN algorithm model. The shared manufacturing simulation model includes m demand points, n production enterprises and a shared manufacturing platform, which is used to simulate the operation process of the shared manufacturing system. The DQN algorithm model interacts with the shared manufacturing simulation model and continuously optimizes the initial model of the order allocation strategy in a large number of interaction rounds. Among them, each demand point in the shared manufacturing simulation model randomly generates orders. After generating an order, the DQN algorithm model returns the state s according to the shared manufacturing simulation model in round t. t , make order allocation decision a t , that is, to decide which enterprise will produce the order. The shared manufacturing simulation model returns the reward value r t , used to evaluate the quality of the decision-making and strategy itself based on the order allocation strategy. t ,a t ,r t ,s t+1 ) trains and updates the DQN algorithm model. After the training is completed, a trained order allocation strategy model is obtained, and the order allocation strategy model can output an order allocation strategy, which is how the shared manufacturing system should allocate orders under different states.

[0090] The shared manufacturing simulation model of this embodiment includes a state acquisition module, a revenue acquisition module, an order arrival module, a queue module, an output order module, an order allocation module and multiple production enterprise modules, wherein the order arrival module, the queue module, the order allocation module and the output order module are connected in sequence, and each production enterprise module includes a receiving order sub-module, a production sub-module, an output product information module connected in sequence, and a production equipment sub-module and a truck sub-module respectively connected to the production sub-module; the order allocation module is respectively connected to the production sub-module in each production enterprise module.

[0091] Specifically, the shared manufacturing simulation model constructed is as follows: Figure 4 shown. Figure 4In (a), demand_points[...] is an agent, representing a cluster of demand point objects; orders[..] is an agent, representing a cluster of order objects, i.e., the order objects generated by Order_arrival are stored in orders[..] for order data statistics; factories[..] is an agent, representing a cluster of factory objects, for factory data statistics; order_id is a variable, representing an order identifier; amoutDistribution represents the quantity distribution function; l ocationDistribution represents the location distribution function.

[0092] max_order_size is a parameter of the Order_arrival module, indicating the maximum order size. For example, the value of max_order_size is 1000, which means that when the order volume reaches 1000, the shared manufacturing simulation model stops running; arrival_interval is a parameter of the Order_arrival module, indicating the order arrival interval. For example, the value of arrival_interval is 10 minutes, which means that an order object is generated every 10 minutes; dqn is a DQN algorithm parameter, which is used when dqn is the first preset threshold, instead of using the DQN algorithm, the proximity allocation strategy is used (i.e., calling the Proximity_allocation proximity allocation function); when dqn is the second preset threshold, the DQN algorithm is used; episode_stop is a parameter used to determine whether the training is finished. For example, the value of episode_stop is 1000, which means 1000 rounds of training.

[0093] Proximity_allocation represents the proximity allocation function, which is used to find the production enterprise closest to the order demand point and allocate the order to the enterprise; getstate represents the state acquisition function, that is, the state acquisition module mentioned above; getreward represents the reward acquisition function, that is, the reward acquisition module mentioned above; takeaction is the function of the shared manufacturing simulation model to execute the order allocation decision generated by the DQN algorithm.

[0094] Order_arrival represents the order arrival module; queue represents the queue module, decision represents the order allocation module; exit represents the output order module; in the figure, Order_arrival order arrival module, queue module, decision order allocation module and exit output order module are connected in sequence.

[0095] Figure 4In (b), trucks[..] represents the truck submodule; Take_order represents the order receiving submodule; produce represents the production submodule; equipment represents the production equipment submodule; and exit represents the output product module. In the figure, Take_order represents the order receiving submodule and produce represents the production submodule, and the exit represents the output product module. The exit represents the output product module, which is connected to the trucks[..] truck submodule and the equipment production equipment submodule.

[0096] Figure 4 In (b), name is a parameter of the produce submodule, indicating the name of the manufacturing company; process_capacity is a parameter of the equipment submodule, indicating the production capacity of a single device; num_eguipment is a parameter of the equipment submodule, indicating the number of devices; num_wait_process is a variable indicating the number of products to be produced.

[0097] exist Figure 4 In , the state acquisition module getstate is used to determine the state variable information of the current order and send the state variable information to the DQN algorithm model. The state variable information is shown in formula (1).

[0098] s t =(d t ,l t ,c i *k it ,w it ), (1).

[0099] In the above formula, s t is the state variable in round t; d t is the number of products required to be produced for the orders to be allocated in round t; l t is the location of the demand point to which the order to be allocated in round t belongs; c i *k it is the idle production capacity of the ith production enterprise; where c i k is the reciprocal of the average time required to produce a single product; it is the number of idle equipment in the ith production enterprise in round t; w it is the number of products to be produced by the i-th production enterprise in round t, where a “round” is defined as the time interval between two orders that arrive one after the other.

[0100] The revenue acquisition module getreward is used to determine the revenue information and send the revenue information to the DQN algorithm model; wherein the revenue information is shown in formula (2). The optimization goal of this embodiment is to minimize the order completion time in the shared manufacturing system.

[0101] FT j =WT j +PT j +DT j , (2).

[0102] In the above formula (2), FT j is the completion time of the jth order; WT j is the waiting time for the jth order; PT j is the production time of the jth order; DT j is the shipping time of the jth order.

[0103] The order arrival module Order_arrival is used to randomly generate multiple orders based on demand points and determine the order information corresponding to each order. Among them, the order arrival module includes a demand point submodule (i.e., the location distribution function of locationDistribution mentioned above), a product quantity submodule (amoutDistribution quantity distribution function) and an order arrival interval submodule (i.e., when setting the parameters of the order arrival module, demand_points[...], amoutDistribution and locationDistribution are associated with Order_arrival). The demand point submodule represents the location information of the demand point to which the order belongs. It calls the location data of the demand point in the historical order data and the location distribution function contained in it to determine the location information of the demand point. The product quantity module represents the number of products contained in the order. It calls the product quantity data in the historical order data and the data distribution function contained in it to determine the number of products contained in the order. The order arrival interval module is triggered by an event based on a time interval, and the order arrival interval submodule arrival_interval is used to determine the time interval for the arrival of each order, so as to generate multiple orders according to the time interval for the arrival of each order. These orders are accompanied by order information, where the order information includes not only the time interval for the arrival of each order, but also the location information of the demand point to which each order belongs and the product quantity information contained in each order. In this way, the location information of the demand point to which each order belongs and the product quantity information contained in each order will be transmitted to the production equipment submodule of each production enterprise along with the order. The production equipment submodule will produce the corresponding number of products according to the product quantity information. The products produced by the production equipment submodule will also carry the location information of the demand point. When the products are transported to the truck submodule, the truck submodule transports the products according to the location information of the demand point.

[0104] The queue module queue is used to queue each order and send the orders to the order allocation module in sequence according to the queue order; and determine the waiting time WT for each order. The order allocation module is used to determine the production enterprise module corresponding to each order according to the order allocation strategy information output by the DQN algorithm model. The output order module exit is used to output the order to the corresponding production enterprise module and send multiple order information to the corresponding production enterprise module in sequence.

[0105] The production capacity of each production enterprise is different, and the workflow of each enterprise is implemented using the process modeling library module. Figure 4 (b) lists the model of one of the established production enterprise modules. The order receiving submodule Take_order in the production enterprise module is used to receive orders and order information, and output the orders and order information to the production submodule. The production submodule produce is used to call the production equipment submodule to produce products according to the order information. The production equipment submodule equipments is used to determine the production time used by the target production equipment to produce a single product; the output order module is used to output the produced products to the truck submodule after the order is completed. The truck submodule is used to transport products to the demand point according to the pre-established GIS map model and determine the transportation time. In addition, the truck submodule can also customize its transportation speed according to actual conditions.

[0106] Preferably, the production equipment submodule includes a processing capacity module, a fault interval time module and a maintenance time module. The fault time module is used to determine the fault occurrence time of the target production equipment according to the fault time data of each equipment in each production enterprise in the production enterprise characteristic data; the maintenance time module is used to determine the maintenance time of the target production equipment according to the maintenance time data of each equipment in each production enterprise in the production enterprise characteristic data; the processing capacity module is used to determine the production time used by the target production equipment to produce a single product according to the production time data of each equipment in each production enterprise in the production enterprise characteristic data; wherein, the production time used by the target production equipment to produce a single product is determined according to the production time used by the target production equipment to produce a single product, the number of products in a single order, the maintenance time of the target production equipment and the fault occurrence time of the target production equipment.

[0107] In some embodiments, the order allocation strategy initial model is trained based on the DQN algorithm to obtain a trained order allocation strategy model, including the following steps (1)-(3).

[0108] (1) Initializing the experience pool and the network parameters of the DQN algorithm model, wherein the DQN algorithm model includes an action neural network and a target neural network, and the network parameters of the DQN algorithm model include network parameters of the action neural network and network parameters of the target neural network.

[0109] (2) Repeat the following steps (21)-(22) until the maximum number of training times is reached.

[0110] (21) Reset the shared manufacturing simulation model.

[0111] (22) Repeat the following steps (221)-(227) until all orders are produced.

[0112] (221) Obtain the current state variable information sent by the shared manufacturing simulation model.

[0113] (222) Determine order allocation strategy information according to current state variable information and the ε-greedy strategy algorithm, and send the order allocation strategy information to the shared manufacturing simulation model.

[0114] (223) Obtain the revenue information and next state variable information sent by the shared manufacturing simulation model.

[0115] (224) The experience tuple is stored in the experience pool, wherein the experience tuple includes current state variable information, order allocation strategy information, revenue information and next state variable information.

[0116] (225) A preset number of experience tuples are randomly selected from the experience pool for training.

[0117] (226) The target value corresponding to each experience tuple is determined based on the action neural network and the target neural network, and the loss function value is determined.

[0118] (227) The network parameters of the action neural network are updated using the back propagation algorithm and the gradient descent algorithm. After updating for a preset number of times, the values ​​of the updated network parameters of the action neural network are copied to the target neural network to stabilize the training process.

[0119] (3) Output the trained order allocation strategy model.

[0120] In the above steps, the corresponding algorithm pseudo code is shown in Table 1.

[0121] Table 1

[0122]

[0123]

[0124] In each round, the DQN algorithm model needs to decide which manufacturer to assign the order to. The decision variable is a = (a1, a2, .., a i ), if the order is assigned to the i-th factory, a i =1, otherwise 0. In this embodiment, the optimization goal of the order allocation strategy model is to minimize the order completion time, so the profit value is the negative value of the order completion time of this round. Here, the smaller the order completion time, the better, so in the algorithm, the order completion time is taken as a negative value, and the larger the profit value, the better.

[0125] In Table 1, at the beginning of each round, the aforementioned algorithm model needs to call the “getstate” function in the shared manufacturing simulation model to obtain the state variable s t , and then according to s t Make decision a with the ε-greedy strategy t , which determines which manufacturer the order is assigned to. Then, the algorithm model calls the “takeaction” function to assign the order to the manufacturer. t The shared manufacturing simulation model is passed in and executed. At the end of the round, the algorithm model calls the "getstate" function again to obtain the current production system state variable s t+1 Since the order may not be completed at the end of the round, the corresponding order completion time cannot be obtained. Therefore, at the end of each round, that is, the order completion time of each round is known, the starting state s of each round is t 、Actions takena t , the corresponding benefit value r t And the end state s t+1 In the algorithm training phase, J samples are drawn from the experience pool each time, and the target value of sample j, i.e., the action benefit value r, is calculated. j The sum of the discounted values ​​of the long-term benefits of the optimal action predicted by the target neural network in the next state; and the predicted value is the long-term total benefit value of the action neural network predicting the current state. Based on the difference between the sample target value and the predicted value, the action neural network parameters are continuously adjusted, and the action neural network parameters are copied to the target neural network parameters after C adjustments. Each round of training (episode) contains several rounds, and at the end of each round, the shared manufacturing simulation model will be restored to the initial stage. Through repeated training and continuous optimization of neural network parameters, the algorithm can more accurately predict the long-term total benefits of taking various actions under different states and generate an order allocation strategy that optimizes the target.

[0126] It should be noted that the focus of this embodiment is on the order allocation strategy of the platform to optimize the supply and demand matching and scheduling process, and does not consider the platform's costs, benefits, etc. That is, this embodiment makes decisions by comprehensively considering the demand and the production capacity of each production enterprise.

[0127] The method of this embodiment can effectively solve the order allocation problem in the shared manufacturing mode. Figure 5 As shown, it is the training curve of the order allocation strategy model of this embodiment, that is, the change of the average completion time of all orders in this round as the number of training rounds increases. After every 5 rounds of training, the shared manufacturing simulation model will run 20 times according to the DQN strategy at this time, and a different random seed will be set for each run. Subsequently, the mean of the average order completion time of these 20 simulation runs and the 95% confidence interval are plotted. The horizontal dotted line represents the average order completion time when the nearest allocation strategy is executed. The results show that the order allocation strategy model of this embodiment converges quickly, and after convergence, its order completion time is shortened by 25.5% compared to the nearest allocation strategy used by real enterprises.

[0128] Reference Figure 6 , this embodiment provides an order allocation method under a shared manufacturing mode, including steps S210-S220.

[0129] Step S210, obtaining orders to be distributed.

[0130] Step S220, inputting the order to be allocated into the trained order allocation strategy model, and outputting the allocation strategy of the order to be allocated, wherein the order allocation strategy model is constructed by the aforementioned order allocation strategy model construction method.

[0131] The order allocation method in the shared manufacturing mode of this embodiment can be used to adapt to different order arrival rates and order arrival distributions without additional training. ① The allocation strategy trained with the order arrival following the Poisson distribution with λ=22 is used for the order arrival following the Poisson distribution with λ=1 to 50. Figure 7 As shown in the figure, the results show that in different order arrival rate scenarios, the DQN optimization strategy is better than the nearest allocation strategy, and the order completion time is shortened by an average of 55.47%, at least 51.81%, and is less affected by changes in the order arrival rate. ② The allocation strategy trained with the Poisson distribution of order arrivals with λ=22 is used in the scenario where the order arrivals follow a uniform distribution. Figure 8 As shown, compared with the nearest allocation strategy, the DQN strategy shortens the average order completion time by 22.44% under U[17, 27], shortens the average order completion time by 18.66% under U[12, 32], shortens the average order completion time by 24.32% under U[7, 37], and shortens the average order completion time by 22.05% under U[2, 42].

[0132] Reference Fig. 9, an order allocation strategy model construction device provided in this embodiment includes: a data acquisition module 11, a simulation model construction module 12, an initial model acquisition module 13 and a training module 14. The data acquisition module 11 is used to acquire historical data, and the historical data at least includes historical order data and production enterprise characteristic data. The simulation model construction module 12 is used to build a shared manufacturing simulation model based on historical data and the production enterprise operation process. The initial model acquisition module 13 is used to obtain an order allocation strategy initial model, wherein the order allocation strategy initial model is constructed based on the shared manufacturing simulation model and a DQN algorithm model with the shared manufacturing simulation model as an interaction object. The training module 14 is used to train the order allocation strategy initial model based on the DQN algorithm to obtain a trained order allocation strategy model.

[0133] In an optional embodiment, the shared manufacturing simulation model at least includes a state acquisition module, a revenue acquisition module, an order arrival module, a queue module, an order allocation module and a plurality of production enterprise modules, wherein the order arrival module, the queue module and the order allocation module are connected in sequence, each production enterprise module includes a production submodule, a production equipment submodule and a truck submodule respectively connected to the production submodule; the order allocation module is respectively connected to the production submodule in each production enterprise module;

[0134] The state acquisition module is used to determine the state variable information and send the state variable information to the DQN algorithm model. The benefit acquisition module is used to determine the benefit information and send the benefit information to the DQN algorithm model; wherein the benefit information is expressed as:

[0135] FT j =WT j +PT j +DT j ;

[0136] In the above formula, FT j is the completion time of the jth order; WT j is the waiting time for the jth order; PT j is the production time of the jth order; DT j is the shipping time of the jth order.

[0137] The order arrival module is used to randomly generate multiple orders according to the demand point and determine the order information corresponding to each order. The queue module is used to queue each order and send the orders to the order allocation module in sequence according to the queue order; and determine the waiting time for each order. The order allocation module is used to determine the production enterprise module corresponding to each order according to the order allocation strategy information output by the DQN algorithm model, and send multiple order information to the corresponding production enterprise module in sequence. The production submodule is used to call the production equipment submodule to produce products according to the order information. The production equipment submodule is used to determine the production time used by the target production equipment to produce a single product. The truck submodule is used to transport products to the demand point according to the pre-established GIS map model and determine the transportation time.

[0138] In an optional embodiment, the historical order data includes at least location data of the demand point to which each historical order belongs and product quantity data in each historical order.

[0139] In an optional embodiment, the order arrival module includes a demand point submodule, a product quantity submodule and an order arrival interval submodule. The demand point submodule is used to determine the location information of the demand point to which each order belongs based on the position data and the location distribution function of the demand point to which each historical order belongs. The product quantity module is used to determine the product quantity information contained in each order based on the product quantity data and the data distribution function in each historical order. The order arrival interval module is used to determine the time interval for the arrival of each order, so as to generate multiple orders according to the time interval for the arrival of each order, wherein each generated order includes order information, and the order information includes the location information of the demand point to which each order belongs and the product quantity information contained in each order.

[0140] In an optional embodiment, the production enterprise characteristic data includes at least the location data of each production enterprise, the number of production equipment of each production enterprise, and the production time data, failure time data and maintenance time data of each equipment in each production enterprise for a single product.

[0141] In an optional embodiment, the production equipment submodule includes a processing capacity module, a fault interval time module and a maintenance time module. The fault time module is used to determine the fault occurrence time of the target production equipment according to the fault time data. The maintenance time module is used to determine the maintenance time of the target production equipment according to the maintenance time data; the processing capacity module is used to determine the production time used by the target production equipment to produce a single product according to the production time data of a single product; wherein, the production time of the target production equipment to produce a single product is determined according to the production time used by the target production equipment to produce a single product, the maintenance time of the target production equipment and the fault occurrence time of the target production equipment.

[0142] In an optional implementation, the state variable information includes:

[0143] s t =(d t ,l t ,c it *k it ,w it );

[0144] In the above formula, s t is the state variable in round t; d t is the number of products required to be produced for the orders to be allocated in round t; l t is the location of the demand point to which the order to be allocated in round t belongs; c i *k it is the idle production capacity of the ith production enterprise; where c i k is the reciprocal of the average time required to produce a single product; it is the number of idle equipment in the ith production enterprise in round t; w it is the number of products to be produced by the i-th production enterprise in round t.

[0145] In an optional embodiment, the training module 14 includes: an initialization module, a repetition module, a selection module, a loss function module, an update parameter module and an output module.

[0146] The initialization module is used to initialize the experience pool and the network parameters of the DQN algorithm model, wherein the DQN algorithm model includes an action neural network and a target neural network, and the network parameters of the DQN algorithm model include network parameters of the action neural network and network parameters of the target neural network.

[0147] The repetition module is used to repeatedly execute the following modules: a first acquisition submodule, a determination submodule, a second acquisition submodule and a storage submodule until the production of all orders is completed. The first acquisition submodule is used to obtain the current state variable information sent by the shared manufacturing simulation model. The determination submodule is used to determine the order allocation strategy information based on the current state variable information and the ε-greedy strategy algorithm, and send the order allocation strategy information to the shared manufacturing simulation model. The second acquisition submodule is used to obtain the revenue information and the next state variable information sent by the shared manufacturing simulation model. The storage submodule is used to store the experience tuple in the experience pool, wherein the experience tuple includes the current state variable information, the order allocation strategy information, the revenue information and the next state variable information.

[0148] The selection module is used to randomly select a preset number of experience tuples from the experience pool for training. The loss function module is used to determine the target value corresponding to each experience tuple according to the action neural network and the target neural network, and to determine the loss function value. The parameter update module is used to update the network parameters of the action neural network using the back propagation algorithm and the gradient descent algorithm. After updating the preset number of times, the values ​​of the updated network parameters of the action neural network are copied to the target neural network to stabilize the training process. The output module is used to output the order allocation strategy model that has reached the maximum number of training times.

[0149] Reference Fig.10 The present embodiment provides an order allocation device in a shared manufacturing mode, comprising an order acquisition module 21 and an order allocation module 22. The order acquisition module 21 is used to acquire orders to be allocated. The order allocation module 22 is used to input the orders to be allocated into a trained order allocation strategy model, and output an allocation strategy for the orders to be allocated, wherein the order allocation strategy model is constructed according to the above-mentioned order allocation strategy model construction method.

[0150] The device provided in the embodiment of the present application is adopted. Since the device adopts the same inventive concept as the above-mentioned method provided in the embodiment of the present application, on the premise that the method can solve the technical problem, the device can also solve the technical problem, which will not be repeated here.

[0151] Reference Fig.11 The embodiment of the present invention further provides an electronic device 400, including a communication interface 401, a processor 402, a memory 403 and a bus 404, wherein the processor 402, the communication interface 401 and the memory 403 are connected via the bus 404; the memory 403 is used to store a computer program that supports the processor 402 to execute the order allocation strategy model construction method and the order allocation method under the shared manufacturing mode, and the processor 402 is configured to execute the program stored in the memory 403.

[0152] Optionally, an embodiment of the present invention further provides a computer-readable medium having a non-volatile program code executable by the processor 402, the program code enabling the processor 402 to execute the order allocation strategy model building method and the order allocation method under the shared manufacturing mode as described in the above embodiments.

[0153] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. A method for constructing an order allocation strategy model, characterized in that: include: Acquiring historical data, wherein the historical data at least includes historical order data and production enterprise characteristic data; Building a shared manufacturing simulation model based on the historical data and the production enterprise operation process; Acquire an initial model of an order allocation strategy, wherein the initial model of the order allocation strategy is constructed according to the shared manufacturing simulation model and a DQN algorithm model with the shared manufacturing simulation model as an interaction object; The order allocation strategy initial model is trained based on the DQN algorithm to obtain a trained order allocation strategy model.

2. The order allocation strategy model construction method according to claim 1, characterized in that: The shared manufacturing simulation model at least includes a state acquisition module, a revenue acquisition module, an order arrival module, a queue module, an order allocation module and a plurality of production enterprise modules, wherein the order arrival module, the queue module and the order allocation module are connected in sequence, and each of the production enterprise modules includes a production submodule, a production equipment submodule and a truck submodule respectively connected to the production submodule; the order allocation module is respectively connected to the production submodule in each production enterprise module; The state acquisition module is used to determine state variable information and send the state variable information to the DQN algorithm model; The revenue acquisition module is used to determine the revenue information and send the revenue information to the DQN algorithm model; wherein the revenue information is expressed as: FT j =WT j +PT j +DT j ; In the above formula, FT j is the completion time of the jth order; WT j is the waiting time for the jth order; PT j is the production time of the jth order; DT j is the shipping time of the jth order; The order arrival module is used to randomly generate multiple orders and determine the order information corresponding to each order; The queue module is used to queue each order and send the orders to the order distribution module in sequence according to the queue order; and determine the waiting time for each order; The order allocation module is used to determine the production enterprise module corresponding to each order according to the order allocation strategy information output by the DQN algorithm model, and send multiple order information to the corresponding production enterprise module in sequence; The production submodule is used to call the production equipment submodule to produce products according to the order information; The production equipment submodule is used to determine the production time used by the target production equipment to produce a single product; The truck submodule is used to transport products to demand points according to a pre-established GIS map model and determine the transportation time.

3. The order allocation strategy model construction method according to claim 2, characterized in that: The historical order data at least includes location data of the demand point to which each historical order belongs and product quantity data in each historical order.

4. The order allocation strategy model construction method according to claim 3 is characterized in that: The order arrival module includes a demand point submodule, a product quantity submodule and an order arrival interval submodule; The demand point submodule is used to determine the location information of the demand point to which each order belongs according to the location data and location distribution function of the demand point to which each historical order belongs; The product quantity module is used to determine the product quantity information contained in each order according to the product quantity data and data distribution function in each historical order; The order arrival interval module is used to determine the time interval for the arrival of each order, so as to generate multiple orders according to the time interval for the arrival of each order, wherein each generated order includes order information, and the order information includes the location information of the demand point to which each order belongs and the product quantity information contained in each order.

5. The order allocation strategy model construction method according to claim 2, characterized in that: The production enterprise characteristic data at least includes the location data of each production enterprise, the number of production equipment of each production enterprise, and the production time data, failure time data and maintenance time data of each equipment in each production enterprise for a single product.

6. The order allocation strategy model construction method according to claim 5, characterized in that: The production equipment submodule includes a processing capacity module, a fault interval time module and a maintenance time module; The failure time module is used to determine the failure occurrence time of the target production equipment according to the failure time data; The maintenance time module is used to determine the maintenance time of the target production equipment according to the maintenance time data; The processing capacity module is used to determine the production time used by the target production equipment to produce a single product according to the production time data of the single product; The production time of the single product produced by the target production equipment is determined according to the production time used by the target production equipment to produce the single product, the maintenance time of the target production equipment and the failure time of the target production equipment.

7. The order allocation strategy model construction method according to claim 2, characterized in that: The state variable information includes: s t =(d t ,l t ,c it *k it ,w it ); In the above formula, s t is the state variable in round t; d t is the number of products required to be produced for the orders to be allocated in round t; l t is the location of the demand point to which the order to be allocated in round t belongs; c i *k it is the idle production capacity of the ith production enterprise; where c i k is the reciprocal of the average time required to produce a single product; it is the number of idle equipment in the ith production enterprise in round t; w it is the number of products to be produced by the i-th production enterprise in round t.

8. The order allocation strategy model construction method according to claim 2, characterized in that: The order allocation strategy initial model is trained based on the DQN algorithm to obtain a trained order allocation strategy model, including: Initializing the experience pool and the network parameters of the DQN algorithm model, wherein the DQN algorithm model includes an action neural network and a target neural network, and the network parameters of the DQN algorithm model include network parameters of the action neural network and network parameters of the target neural network; Repeat the following steps until the preset number of training times is reached: Reset the shared manufacturing simulation model; Repeat the following steps until all orders are completed: Acquiring current state variable information sent by the shared manufacturing simulation model; Determine order allocation strategy information according to the current state variable information and the ε-greedy strategy algorithm, and send the order allocation strategy information to the shared manufacturing simulation model; Obtaining revenue information and next state variable information sent by the shared manufacturing simulation model; Storing the experience tuple in the experience pool, wherein the experience tuple includes the current state variable information, the order allocation strategy information, the revenue information and the next state variable information; Randomly selecting a preset number of experience tuples from the experience pool for training; Determine the target value corresponding to each experience tuple according to the action neural network and the target neural network, and determine the loss function value; Using a back propagation algorithm and a gradient descent algorithm to update the network parameters of the action neural network, after updating a preset number of times, copying the values ​​of the network parameters of the action neural network to the target neural network to stabilize the training process; Output the trained order allocation strategy model.

9. A method for allocating orders in a shared manufacturing mode, characterized in that: include: Get the orders to be allocated; The order to be allocated is input into a trained order allocation strategy model, and an allocation strategy for the order to be allocated is output, wherein the order allocation strategy model is constructed according to the order allocation strategy model construction method according to any one of claims 1-8.

10. An order allocation strategy model construction device, characterized in that: include: A data acquisition module, used to acquire historical data, wherein the historical data at least includes historical order data and production enterprise characteristic data; A simulation model building module, used to build a shared manufacturing simulation model based on the historical data and the production enterprise operation process; An initial model acquisition module, used to acquire an initial model of an order allocation strategy, wherein the initial model of the order allocation strategy is constructed according to the shared manufacturing simulation model and a DQN algorithm model with the shared manufacturing simulation model as an interaction object; The training module is used to train the initial model of the order allocation strategy based on the DQN algorithm to obtain a trained order allocation strategy model.

Images