Production scheduling method and device of magnetic drive conveying system, electronic equipment, readable storage medium and program product

Abstract

The invention relates to a production scheduling method and device for a magnetic drive conveying system, electronic equipment, a readable storage medium and a program product, and the method comprises the steps: obtaining the state monitoring data of the magnetic drive conveying system in a production environment, inputting the state monitoring data into a pre-trained production scheduling model for data calculation, obtaining a scheduling strategy, and carrying out the data calculation. And carrying out scheduling processing on the magnetic drive conveying system according to the scheduling strategy. Therefore, the waiting time of the conveying equipment is shortened, the idle time of the production station is shortened, the production efficiency is improved, and the production cost is reduced. The production scheduling model in the embodiment is obtained by performing deep reinforcement learning model training based on the sample data of the magnetic drive conveying system through interaction with the simulation environment, so that the model can understand the dynamic characteristics of the magnetic drive conveying system, gradually learn and optimize the scheduling strategy, and improve the production scheduling efficiency. Therefore, the method can adapt to dynamic changes of an actual production environment to meet the actual production environment.

Description

Technical Field

[0001] The present application relates to the technical field of production scheduling, and in particular to a production scheduling method, device, electronic equipment, readable storage medium and program product for a magnetic drive conveying system. Background Art

[0002] The magnetic drive conveying system is a kind of equipment widely used in the field of industrial automation. Its main function is to drive the trolley on the track through magnetic force to realize the transportation and transfer of materials.

[0003] In modern industrial production, magnetic drive conveyor lines are widely used in various production lines due to their high efficiency and flexibility. However, how to optimize the production rhythm of magnetic drive conveyor lines and reduce idle time in production to improve production efficiency, reduce production costs, and enhance production flexibility is an urgent problem to be solved in the current industrial automation field. Summary of the invention

[0004] Based on this, it is necessary to provide a production scheduling method, device, electronic device, readable storage medium and program product for a magnetic drive conveying system that can adapt to the actual production environment to improve production efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a production scheduling method for a magnetic drive conveying system, wherein the magnetic drive conveying system comprises a magnetic drive conveying track and at least one conveying device, wherein a plurality of production stations are arranged on the magnetic drive conveying track, and the method comprises:

[0006] Acquiring status monitoring data of the magnetic drive conveying system, wherein the status monitoring data includes the monitored load of the production station in the magnetic drive conveying system and the position of the conveying equipment;

[0007] Inputting the state monitoring data into a production scheduling model for data calculation to obtain a scheduling strategy, wherein the scheduling strategy includes at least one of a movement instruction for at least one of the conveying devices and a state instruction for at least one of the production stations; the production scheduling model is obtained by deep reinforcement learning training based on sample data of the magnetic drive conveying system;

[0008] The magnetic drive conveying system is scheduled according to the scheduling strategy.

[0009] In one of the embodiments, after the magnetic drive conveying system is scheduled according to the scheduling strategy, the method further includes: obtaining first state change data and a first reward parameter after the magnetic drive conveying system is scheduled; when the first reward parameter does not meet the set target value, determining first interaction data based on the production environment according to the state monitoring data, the scheduling strategy, the first state change data and the first reward parameter; and optimizing the production scheduling model according to the first interaction data.

[0010] In one of the embodiments, the training method of the production scheduling model includes: determining a corresponding simulation environment based on the magnetic-driven conveying system and initializing network parameters; obtaining initial state data of the magnetic-driven conveying system in the simulation environment, and determining a sample scheduling strategy based on the initial state data; obtaining state change data and sample reward parameters of the magnetic-driven conveying system after scheduling based on the sample scheduling strategy; determining sample interaction data based on the simulation environment based on the initial state data, the sample scheduling strategy, the state change data and the sample reward parameters; and updating the network parameters based on a deep reinforcement learning algorithm based on the sample interaction data to obtain the trained production scheduling model.

[0011] In one embodiment, the network parameters are updated based on the deep reinforcement learning algorithm according to the sample interaction data to obtain the trained production scheduling model, including: calculating an advantage function based on the sample interaction data, the advantage function being used to measure the difference between the expected return of scheduling based on the sample scheduling strategy and the value estimate corresponding to the initial state data; updating the strategy network parameters by maximizing the expected return of the sample scheduling strategy according to the advantage function; and updating the value network parameters by minimizing the square error between the expected return and the value estimate according to the advantage function to obtain the trained production scheduling model.

[0012] In one embodiment, the method for obtaining the sample reward parameters includes: determining the load imbalance degree of each production station based on the state change data; obtaining the idle time and product production time of the magnetic drive conveying system; and calculating the sample reward parameters based on a preset reward function according to the load imbalance degree, the idle time and the product production time.

[0013] In one embodiment, the reward function is: t = -α·T idle -β·T total -γ·L balance , where R trepresents the calculated sample reward parameter determined based on the sample scheduling strategy at time t, T idle Indicates the idle time, T total Indicates the production time of the product, L balance represents the load imbalance degree, and α, β, and γ represent the corresponding weights.

[0014] In one of the embodiments, the deep reinforcement learning algorithm is implemented based on any one of a deep Q-network, a deep deterministic policy gradient, and a proximal policy optimization algorithm.

[0015] In a second aspect, the present application further provides a production scheduling device for a magnetic drive conveying system, wherein the magnetic drive conveying system comprises a magnetic drive conveying track and at least one conveying device, wherein a plurality of production stations are arranged on the magnetic drive conveying track, and the device comprises:

[0016] A status monitoring module, used for acquiring status monitoring data of the magnetic drive conveying system, wherein the status monitoring data includes the load of the production station in the magnetic drive conveying system and the position of the conveying equipment;

[0017] a strategy acquisition module, configured to input the state monitoring data into a production scheduling model for data calculation to obtain a scheduling strategy, wherein the scheduling strategy includes at least one of a movement instruction for at least one of the conveying devices and a state instruction for at least one of the production stations; the production scheduling model is obtained by deep reinforcement learning training based on sample data of the magnetic drive conveying system;

[0018] The scheduling processing module is used to schedule the magnetic drive conveying system according to the scheduling strategy.

[0019] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0021] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0022] The production scheduling method, device, electronic device, readable storage medium and program product of the magnetic drive conveying system mentioned above obtain the status monitoring data of the magnetic drive conveying system in the production environment, and input the status monitoring data into a pre-trained production scheduling model for data calculation to obtain a scheduling strategy, and schedule the magnetic drive conveying system according to the scheduling strategy. Thereby reducing the waiting time of the conveying equipment and reducing the idle time of the production station to improve production efficiency and reduce production costs. In addition, since the production scheduling model in this embodiment is obtained through interaction with the simulation environment and deep reinforcement learning model training based on sample data of the magnetic drive conveying system, the model can understand the dynamic characteristics of the magnetic drive conveying system, gradually learn and optimize the scheduling strategy, so that it can adapt to the dynamic changes of the actual production environment to meet the actual production environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic flow chart of a production scheduling method for a magnetic drive conveying system in one embodiment;

[0025] Figure 2 A schematic diagram of a flow chart of a model optimization step in an embodiment;

[0026] Figure 3 A schematic diagram of a flow chart of a model training step in an embodiment;

[0027] Figure 4 A schematic flow chart of a production scheduling method for a magnetic drive conveying system in another embodiment;

[0028] Figure 5 It is a structural block diagram of a production scheduling device of a magnetic drive conveying system in one embodiment;

[0029] Figure 6 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] Magnetic drive conveyor lines are widely used in various production lines due to their high efficiency and flexibility. How to optimize the production rhythm of magnetic drive conveyor lines and reduce idle time in production to improve production efficiency, reduce production costs, and enhance production flexibility is an urgent problem to be solved in the current industrial automation field. In addition, in traditional technology, the production rhythm of magnetic drive conveyor lines is mainly optimized through manual experience or simple mathematical models. For example, by statistically analyzing historical data, the average processing time of each process is calculated, and then the production plan is arranged based on these data. In addition, there are some solutions that try to use traditional optimization algorithms, such as genetic algorithms, simulated annealing algorithms, etc., to optimize the production rhythm. However, these methods often require a lot of prior knowledge and assumptions, lack the ability to adapt to the dynamic changes of the actual production environment, and thus are difficult to meet the actual production environment.

[0032] Based on this, an embodiment of the present application provides a production scheduling method, device, electronic device, readable storage medium and program product of a magnetic drive conveying system, wherein the magnetic drive conveying system includes a magnetic drive conveying track and at least one conveying device running in the magnetic drive conveying track, and a plurality of production stations are also arranged on the magnetic drive conveying track. During the operation of the magnetic drive conveying system, the conveying device is used to transfer materials from one production station to another production station through the magnetic drive conveying track for production and processing. However, since the processing tasks of each production station are different, the time required for their processing is also different, which makes the conveying device need to wait in front of some production stations, resulting in low production efficiency. The present application aims to realize dynamic production scheduling of the magnetic drive conveying system through deep reinforcement learning, so as to reduce idle time in production, improve production efficiency, reduce production costs, and enhance production flexibility.

[0033] In one embodiment, Figure 1 As shown, a production scheduling method for a magnetic drive conveying system is provided, which may specifically include the following steps:

[0034] Step 101, obtaining status monitoring data of the magnetic drive conveying system.

[0035] Among them, the status monitoring data is the relevant data obtained by monitoring the status of the magnetic drive conveying system during its operation, including the load of each production station in the monitored magnetic drive conveying system and the position of the conveying equipment. Specifically, the load of the production station can be the current queue length of the production station, which can be calculated based on the queuing theory. The position of the conveying equipment indicates the production station where the conveying equipment is currently located, that is, at which production station the conveying equipment is currently located, which can be determined based on the specific settings of the magnetic drive conveying system in the production environment and the positioning of the conveying equipment. The production environment refers to the real environment in which production and manufacturing are carried out in the production site.

[0036] In this embodiment, during the operation of the magnetic drive conveying system, its status can be periodically monitored to obtain corresponding status monitoring data, that is, the load of the production station in the magnetic drive conveying system and the position of the conveying equipment, etc., and production scheduling can be performed based on subsequent steps.

[0037] Step 102: Input the status monitoring data into the production scheduling model for data calculation to obtain a scheduling strategy.

[0038] Among them, the production scheduling model is obtained by deep reinforcement learning training based on sample data of the magnetic drive conveying system through interaction with the simulation environment. The trained production scheduling model can output the corresponding scheduling strategy based on the input state monitoring data. The scheduling strategy is a specific scheduling task or scheduling rule for production scheduling. Specifically, the scheduling strategy may include at least one of a movement instruction for the conveying equipment and a state instruction for the production station. For example, a movement instruction may be a command to instruct a specific conveying equipment to move from one production station to another. A state instruction may be a command to instruct a specific production station to be closed or opened.

[0039] Step 103, scheduling the magnetic drive conveying system according to the scheduling strategy.

[0040] Among them, the scheduling process is the process of performing specific production scheduling based on the scheduling strategy. Specifically, the scheduling process can be to move a specific conveying device from one production station to another production station based on the scheduling strategy, or to close or open a specific production station based on the scheduling strategy, thereby reducing the waiting time of the conveying device and reducing the idle time of the production station, so as to improve production efficiency and reduce production costs.

[0041] Specifically, suppose that in a magnetic drive conveying system of an automobile manufacturing production line, there are three production stations on the magnetic drive conveying track, and each station completes different assembly tasks, such as welding, painting, and installing parts. The magnetic drive conveying track is responsible for transporting the conveying equipment from one production station to the next production station, so as to realize the transmission of the materials transported by the conveying equipment from one production station to the next production station. In the process of the magnetic drive conveying system working, the load of each production station on the magnetic drive conveying track can be monitored in real time or periodically, and the position of the conveying equipment on the magnetic drive conveying track can be located to obtain the state monitoring data of the magnetic drive conveying system. Based on the state monitoring data, using the pre-trained production scheduling model, it can be understood which production station has more conveying equipment waiting for processing and which production station may be a bottleneck, so as to give the corresponding scheduling strategy, such as allocating the conveying equipment with a longer waiting time at a certain production station to the production station with a lighter load, thereby reducing the waiting time of the conveying equipment and improving production efficiency.

[0042] In the production scheduling method of the magnetic drive conveying system, the state monitoring data of the magnetic drive conveying system in the production environment is obtained, and the state monitoring data is input into the pre-trained production scheduling model for data calculation to obtain a scheduling strategy, and the magnetic drive conveying system is scheduled according to the scheduling strategy. Thereby reducing the waiting time of the conveying equipment and reducing the idle time of the production station to improve production efficiency and reduce production costs. In addition, since the production scheduling model in this embodiment is obtained through interaction with the simulation environment and deep reinforcement learning model training based on sample data of the magnetic drive conveying system, the model can understand the dynamic characteristics of the magnetic drive conveying system, gradually learn and optimize the scheduling strategy, so that it can adapt to the dynamic changes of the actual production environment to meet the actual production environment.

[0043] In an exemplary embodiment, Figure 2 As shown, in step 103, after the magnetic drive conveying system is scheduled according to the scheduling strategy, the above method may further include:

[0044] Step 201, obtaining the first state change data and the first reward parameter of the magnetic drive conveying system after the scheduling process.

[0045] The first state change data and the first reward parameter are obtained after the magnetic drive conveying system is scheduled according to the scheduling strategy in step 103. Specifically, the first state change data is the state data monitored after the state of the magnetic drive conveying system is updated after the magnetic drive conveying system is scheduled according to the scheduling strategy, and includes the current load of the production station in the magnetic drive conveying system and the current position of the conveying equipment.

[0046] The first reward parameter is the feedback value obtained immediately after performing an action in Deep Reinforcement Learning (DRL), that is, the immediate reward. The immediate reward can be used to evaluate the quality of the corresponding action. In this embodiment, the first reward parameter is the feedback value obtained immediately after the magnetic drive conveying system executes the scheduling strategy. Specifically, the first reward parameter can be calculated based on a preset reward function.

[0047] Step 202: when the first reward parameter does not meet the set target value, determine first interaction data based on the production environment.

[0048] The set target value may be a preset minimum instant reward for evaluating whether the corresponding action meets expectations. That is, if the first reward parameter meets the set target value, it can be determined that the corresponding action meets expectations, and if the first reward parameter does not meet the set target value, it can be determined that the corresponding action does not meet expectations.

[0049] The first interaction data may be empirical data used for optimizing the model. In this embodiment, when it is determined that the first reward parameter does not meet the set target value, the first interaction data based on the production environment may be determined according to the state monitoring data, the scheduling strategy, the first state change data and the first reward parameter.

[0050] Step 203: Optimize the production scheduling model according to the first interaction data.

[0051] The optimization process can be a process of online training or fine-tuning the model based on the experience of the actual production environment collected. In this embodiment, the production scheduling model is optimized by using the first interaction data so that the model can adapt to the new production environment and optimize the next scheduling strategy, thereby improving the accuracy of the model output.

[0052] It is understandable that the process of optimizing the production scheduling model according to the first interactive data can be performed periodically, such as optimizing or iterating the production scheduling model based on the data of the magnetic drive conveying system for one week or one month, so as to update and upgrade the production scheduling model, so that the model can learn the dynamic changes of the actual production environment. Specifically, when optimizing the production scheduling model, the corresponding magnetic drive conveying system can be shut down and wait, so as to avoid the optimization process affecting the magnetic drive conveying system.

[0053] In the above embodiment, by obtaining the first state change data and the first reward parameter of the magnetic drive conveying system after scheduling, and when the first reward parameter does not meet the set target value, the first interaction data based on the production environment is determined, and the production scheduling model is optimized according to the first interaction data. It continuously adjusts and optimizes the model through feedback data in the actual production environment, thereby further improving the adaptability and flexibility of the model, so that the model can adapt to the new production environment, and optimizes the subsequent scheduling strategy to improve the accuracy of the model output.

[0054] In an exemplary embodiment, Figure 3 As shown, the training method of the production scheduling model includes the following steps:

[0055] Step 301, determine the corresponding simulation environment based on the magnetic drive conveying system and initialize network parameters.

[0056] The simulation environment is an environment that simulates the behavior and characteristics of the real world or virtual system on a computer. Specifically, the simulation environment of this embodiment is an environment that simulates the behavior and characteristics of the magnetic drive conveying system in the production environment on a computer, and the behavior and characteristics of the magnetic drive conveying system may include the length of the magnetic drive conveying track in the magnetic drive conveying system, the number of conveying equipment, the number and distribution of production stations, the process flow of each production station, and the processing time of each production station.

[0057] In this embodiment, the corresponding simulation environment is determined based on the magnetic drive conveying system, and the network parameters are initialized. Since the production scheduling model in this embodiment is trained based on deep reinforcement learning (DRL), and deep reinforcement learning is a machine learning method that combines deep learning (DL) and reinforcement learning (RL), it uses a deep neural network to approximate the strategy or value function in reinforcement learning to solve complex decision-making problems. In DRL, it usually includes a policy network (PolicyNetwork) and a value network (Value Network), wherein the policy network is used to predict the action (Action) taken under a given state (State), which is usually a deep neural network that can output a probability distribution, indicating the probability of taking each possible action under a given state. The value network is used to predict the expected return of taking a certain strategy under a given state, which helps to evaluate the quality of different strategies. The initialization network parameters can specifically be the weights and biases of the initialization policy network and the value network, and can specifically be randomly initialized.

[0058] Step 302, obtaining initial state data of the magnetic drive conveying system in the simulation environment, and determining a sample scheduling strategy according to the initial state data.

[0059] The initial state data refers to the state data of the magnetic drive conveying system initialized to a certain state in the simulation environment, including the load of the production station and the position of the conveying equipment in the corresponding state. The sample scheduling strategy is the specific scheduling task or scheduling rule for production scheduling selected according to the initial state data.

[0060] In this embodiment, the initial state data of the magnetic drive conveying system in the simulation environment is obtained, and a sample scheduling strategy is determined according to the initial state data, and then model training is performed based on subsequent steps to obtain a trained production scheduling model.

[0061] Step 303, obtaining the state change data and sample reward parameters of the magnetic drive conveying system after scheduling based on the sample scheduling strategy.

[0062] It can be understood that the state change data and sample reward parameters in this embodiment are obtained after scheduling according to the sample scheduling strategy. Specifically, the state change data is the state data obtained after the state of the magnetic drive conveying system in the simulation environment is updated after scheduling according to the sample scheduling strategy. The sample reward parameter is the feedback value obtained immediately after the sample scheduling strategy is executed.

[0063] In this embodiment, after scheduling based on the sample scheduling strategy, the state change data and sample reward parameters of the magnetic drive conveying system can be obtained. Specifically, the load imbalance degree of each production station can be determined according to the state change data, and the idle time and product production time of the magnetic drive conveying system can be obtained, and then the sample reward parameters can be calculated based on the preset reward function according to the load imbalance degree, idle time and product production time.

[0064] Step 304, determining sample interaction data based on the simulation environment according to the initial state data, the sample scheduling strategy, the state change data and the sample reward parameter.

[0065] The sample interaction data may be empirical data for model training. In this embodiment, the corresponding sample interaction data is generated and stored according to the initial state data, the sample scheduling strategy, the state change data and the sample reward parameter for model learning and training.

[0066] Step 305: Update network parameters based on the deep reinforcement learning algorithm according to the sample interaction data to obtain a trained production scheduling model.

[0067] Among them, the deep reinforcement learning algorithm can be implemented by any one of the deep Q-network (Deep Q-Network, DQN for short), the deep deterministic policy gradient (Deep Deterministic Policy Gradient, DDPG for short) and the proximal policy optimization (Proximal Policy Optimization, PPO for short) algorithms. In this embodiment, through multiple iterations of the simulation environment, a large amount of sample interaction data can be generated, based on which deep reinforcement learning training is performed, so that it can understand the dynamic characteristics of the magnetic drive conveying system, and gradually learn and optimize the scheduling strategy to find the optimal strategy, and maximize the long-term cumulative reward to update the network parameters, thereby obtaining a trained production scheduling model.

[0068] In the above embodiment, the corresponding simulation environment is determined based on the magnetic drive conveying system, and the network parameters are initialized. The initial state data of the magnetic drive conveying system in the simulation environment is obtained, and the sample scheduling strategy is determined according to the initial state data, and the state change data and sample reward parameters of the magnetic drive conveying system after scheduling based on the sample scheduling strategy are obtained. According to the initial state data, the sample scheduling strategy, the state change data and the sample reward parameters, the sample interaction data based on the simulation environment is determined, and according to the sample interaction data, the network parameters are updated based on the deep reinforcement learning algorithm to obtain the trained production scheduling model. Since the training process is based on the interaction with the simulation environment, and the simulation environment is an environment that simulates the behavior and characteristics of the magnetic drive conveying system in the production environment, therefore, through training in the simulation environment, the model can learn the dynamic characteristics of the magnetic drive conveying system, so that it can be applied in the actual production environment to improve production efficiency.

[0069] In an exemplary embodiment, Figure 4 As shown, the production scheduling method of the magnetic drive conveying system is further described below, which specifically includes:

[0070] Step 401, data collection, establishing a simulation environment.

[0071] Specifically, the corresponding simulation environment is established by collecting relevant data of the magnetic drive conveying system in the production environment, wherein the relevant data of the magnetic drive conveying system includes but is not limited to the actual length of the magnetic drive conveying track, the number of conveying equipment on the magnetic drive conveying track, the number and location of production stations, the detailed process of processing products at each production station, and the required processing time, etc.

[0072] Step 402: Use queuing theory to describe the system state.

[0073] First, determine the system type. The magnetic drive conveying system can be regarded as a multi-stage queuing system, where each production station corresponds to a service node, and each conveying device carries one or more products to be processed and moves between various production stations.

[0074] Then define key parameters, such as arrival rate (λ), which indicates the frequency of the conveyor equipment arriving at a certain production station per unit time; service rate (μ), which indicates the frequency of the production station completing processing per unit time; queue length (L), which indicates the number of conveyor equipment waiting for processing in front of a certain production station; waiting time (W), which indicates the waiting time of the conveyor equipment in front of a certain production station, and the time the conveyor equipment waits for processing to be completed at the production station, i.e., the service time.

[0075] Then a mathematical model is established. Specifically, the waiting time and queue length of each production station can be calculated using queue theory formulas (such as the M / M / 1 model, the M / M / c model, etc.). Among them, M / M / 1 represents a single server, Poisson arrival, and exponential service time. M / M / c represents multiple servers, Poisson arrival, and exponential service time.

[0076] Specifically, taking the M / M / 1 model as an example, assume that on a magnetic drive conveying system, there is a production station responsible for the welding process (single server), the conveying equipment arrives in a Poisson process (that is, the arrival interval time obeys an exponential distribution), and the welding time of the production station also obeys an exponential distribution. Then:

[0077] λ (arrival rate): the number of conveying equipment arriving at the production station per hour (e.g. 10 vehicles per hour).

[0078] μ (service rate): the number of conveying equipment that completes welding at a workstation per hour (e.g. 15 vehicles per hour).

[0079] System utilization (ρ):

[0080] Average queue length (Lq):

[0081] Average waiting time (Wq) (i.e. the average waiting time of the transport equipment in the queue):

[0082]

[0083] Average number of transport devices in the system (L) (including transport devices in queue and in service):

[0084] L=Lq+ρ

[0085] Average system time (W) (including the waiting time of the conveyor equipment in the system and the service time at the production station):

[0086]

[0087] Taking the M / M / c model as an example, suppose the same production station now has 3 parallel jobs (such as 3 welding robots), and the arrival and service time distribution remains unchanged. Then:

[0088] λ (arrival rate): the number of transport equipment arriving per hour (e.g. 30 vehicles per hour).

[0089] μ (service rate): the number of conveyor equipment that completes welding on a single server per hour (e.g. 15 vehicles per hour).

[0090] c(Number of servers): 3.

[0091] System utilization (ρ):

[0092] Average queue length (Lq):

[0093]

[0094]

[0095] Average waiting time (Wq):

[0096] Average number of transport devices in the system (L) (including transport devices in queue and in service):

[0097] L=Lq+ρc

[0098] Average system time (W) (including the waiting time of the conveyor equipment in the system and the service time at the production station):

[0099]

[0100] Step 403, design a deep reinforcement learning framework.

[0101] Specifically, define the state space, action space and reward function, and select a suitable deep reinforcement learning algorithm. The state space defines the current state of the system. In this embodiment, the state space can be defined as the position of the workstation load and conveyor equipment of the current system. For example, assuming there are i production stations and j conveyor equipment, the state space can be expressed as:

[0102] S t ={L1,……,L i ,C1,……,C j}

[0103] Among them, L i represents the current load of the i-th station, C j Represents the current position and status of the jth conveyor device.

[0104] The action space defines the actions that can be taken. In this embodiment, the action space can include the movement decision of the conveyor (such as moving the conveyor from one station to another) and the adjustment of the station setting (such as opening or closing a station). For example, the action can be expressed as: A t ={A move ,A setup}, where A move represents the movement decision of the conveying equipment, A setup Indicates adjustments to the workstation settings.

[0105] The reward function comprehensively considers the waiting time of each workstation, the total production time and the load balance of the workstation. The reward function can be specifically expressed as: R t = -α·T idle -β·T total -γ·L balance . R t represents the immediate reward at time step t, T idle It represents the total idle time in the magnetic drive conveying system (to reduce the idle time of the trolley and workstation in the magnetic drive line), T total Indicates the production time of the product (to reduce the total production time of the product), L balance It indicates the degree of load imbalance and can be defined as the standard deviation of the load of each workstation (to balance the load of each workstation and avoid some workstations being too busy while others are idle). α, β, and γ are the weight parameters of the corresponding items, corresponding to the weights of idle time, total production time, and load balance, respectively.

[0106] The deep reinforcement learning algorithm can adopt any of the deep Q network DQN (applicable to discrete action space), deep deterministic policy gradient DDPG (applicable to continuous action space) and proximal policy optimization PPO (combines policy gradient and value function methods and is widely applicable).

[0107] Step 404: model training.

[0108] Through interaction with the simulation environment, training is performed based on the deep reinforcement learning algorithm to obtain a trained production scheduling model. The training process can specifically include: initializing the environment and network parameters; at each time step, selecting actions according to the current strategy, executing actions, updating states, calculating immediate rewards, and storing experience (S t ,A t ,R t ,S t+1 ); Periodically update the strategy and value network and calculate the advantage function F t , update the policy network to maximize the expected return of the policy, update the value network to minimize the time difference error; iterate the above process until the model learns the optimal scheduling strategy. Through this training process, the model can learn and optimize the scheduling strategy in a complex production environment, improve production efficiency, reduce production costs, and enhance production flexibility.

[0109] The following takes the deep reinforcement learning algorithm PPO as an example to further illustrate the model training process.

[0110] Assume that the magnetic drive conveying system includes 3 workstations (G1, G2, G3) and 5 conveying equipment (C1, C2, C3, C4, C5). Each workstation can process materials transported by one conveying equipment at the same time. The service time of the workstation follows an exponential distribution, and the arrival time of the vehicle also follows an exponential distribution.

[0111] Then generate the corresponding simulation environment and initialize the environment, and initialize the system to the initial state. The initial state includes the current queue length of the workstation and the position and state of the conveying equipment. If the initial state S0 = {L1 = 2, L2 = 3, L3 = 0, C1 = G1, C2 = G2, C3 = G3, C4 = G2, C5 = G1}, where L i represents the current load (queue length) of station i, C j Indicates the position of the conveying device j (eg C1 = G1 indicates that the conveying device C1 is waiting at the workstation G1).

[0112] In the first time step, an action A0 can be selected according to the current state and interact with the simulation environment, such as moving the conveyor C1 from station 1 to station 3. Then the new state S1 may be: S1 = {L1 = 1, L2 = 3, L3 = 2, C1 = G3, C2 = G2, C3 = G3, C4 = G2, C5 = G1}. Calculate the immediate reward R0, which can be calculated by the reward function: R0 = -α·T idle (S0,A0)-β·T total (S0,A0)-γ·L balance (S0, A0). The PPO algorithm updates the strategy and value function based on the (S0, A0, R0, S1) experience to optimize future scheduling decisions.

[0113] It can be understood that during the training process, for each time step t, the current state S can be obtained. t , and select an action A t , perform action A t After that, the system transfers to the new state S t+1 , and receive an instant reward R t , get experience (S t ,A t ,R t ,S t+1 ) and stored in the experience pool, the PPO algorithm iteratively updates the strategy and value function based on the stored experience. The update process is as follows:

[0114] Compute the advantage function:

[0115]

[0116] The advantage function F is used to measure the difference between the actual return and the estimated value. Among them, γ is the discount factor, V(S t ) is the state S at time t t The value estimate of V(S t+K ) is the state S at time t+k t+K The value of R t+k is the instant reward at time t+k. Then, by maximizing the expected return of the strategy, the strategy network parameters are updated, specifically:

[0117]

[0118] Among them, L CLIP (θ) represents the loss function of the policy network, where θ represents the network parameters. Represents the expected value, usually calculated by averaging over a batch of samples (state-action pairs). θ (A t |S t ) is the current strategy network in state S t Next take action A t probability. The old strategy network is in state S t Next take action A t ∈ is the clipping range, which is used to limit the strategy changes. t Indicates action A t The corresponding reward (advantage), which measures the benefit of taking action A t The return relative to the average action.

[0119] The above formula represents that for each time step t, the new strategy π can be calculated θ (A t |S t ) and the old strategy The probability ratio between the two is calculated and multiplied by the reward (advantage) of the action. Then, this product is clipped to a predetermined interval to ensure that the policy update is not too large. Finally, the expected value of this clipped value is taken as the loss of the policy network. This loss function encourages the policy network to learn a new policy that is at least as good as the old policy in terms of expected reward, while limiting the magnitude of the policy update so that the difference between the new policy and the old policy is not too large. This is also the key to the PPO algorithm's ability to effectively balance exploration and utilization.

[0120] And update the value network parameters by minimizing the temporal difference error (that is, minimizing the squared error between the expected return and the value estimate):

[0121]

[0122] Among them, L VF (φ) represents the loss function of the value network, and φ represents the network parameters. is the expected value, usually computed by averaging over a batch of examples (state-reward pairs). V φ (S t ) gives the state S at time t for the value network t The state value estimate, R t is the instantaneous reward at time t. The purpose of this loss function is to enable the value network to accurately predict the expected return under a given state. By minimizing this loss function, the value network learns a value function V φ , which can estimate the t The cumulative reward that can be obtained by starting and following the current policy. In reinforcement learning, value functions are often used together with policy networks to guide the model in choosing actions. For example, in policy iteration methods, the value function is first used to evaluate the current policy, and then the policy network is used to improve the policy in the hope of obtaining higher rewards. Accurate prediction of the value function is crucial to the success of reinforcement learning algorithms because it provides the model with information about the potential rewards of different states, thereby helping the model make better decisions.

[0123] Specifically, the Policy Network and the Value Network are responsible for generating the model's action strategy and evaluating the value of the state or state-action pair, respectively.

[0124] The policy network parameters are used to generate the probability distribution of the model's choice of action in a given state. The policy network is usually composed of a deep neural network whose parameters include weights and biases. Its network structure includes:

[0125] Input layer: receives the feature vector of the current state S.

[0126] Hidden layer: Several hidden layers, using activation functions (such as ReLU, Tanh) for nonlinear transformation.

[0127] Output layer: Outputs the probability distribution of each action. For discrete action spaces, the Softmax function is usually used to convert the output into a probability distribution.

[0128] The network parameters include:

[0129] Weight W: The weight matrix of each layer of neuron connections.

[0130] Bias b: The bias vector of each layer of neurons.

[0131] Assuming that the parameter of the policy network is θ, the action probability π output by the policy network θ(a|s) represents the probability of selecting action a in state s. It can be expressed as: π θ (a|s)=Softmax(W n ReLU(W n-1 ...ReLU(W1·s+b1)...+b n-1 )+b n ), where ReLU is the activation function, W n is the weight of the nth layer of neuron connections, b n is the bias vector of the nth layer of neurons.

[0132] The value network parameters are used to evaluate the expected cumulative reward in a given state. The value network is also composed of a deep neural network, whose parameters include weights and biases. The network structure includes:

[0133] Input layer: receives the feature vector of the current state S.

[0134] Hidden layer: Several hidden layers, using activation functions for nonlinear transformation.

[0135] Output layer: Outputs a scalar value representing the value of the state (in the state-value function) or the value of the state-action pair (in the action-value function).

[0136] The network parameters include:

[0137] Weight W: The weight matrix of each layer of neuron connections.

[0138] Bias b: The bias vector of each layer of neurons.

[0139] Assuming that the parameter of the value network is φ, the state value V output by the value network is φ (s) represents the expected cumulative reward in state s. It can be expressed as:

[0140] V φ (s) = W m ReLU(W m-1 ...ReLU(W1·s+b1)...+b m-1 )+b m , where ReLU is the activation function, W n is the weight of the nth layer of neuron connections, b n is the bias vector of the nth layer of neurons.

[0141] In deep reinforcement learning, the policy network and the value network work together to optimize the model's behavior strategy. Specifically:

[0142] Policy network: Generates the probability of selecting different actions in the current state and guides the model on how to act.

[0143] Value network: Evaluates the value of the current state or state-action pair, providing feedback about the quality of the state.

[0144] Assume that the state space S includes the current position and workload information of the conveyor equipment, and the action space A includes the movement strategy of the conveyor equipment and the setting allocation of the workstation. The training process specifically includes:

[0145] Initialize network parameters: Randomly initialize the weights and biases of the policy network θ and the value network φ.

[0146] State input: Get the current state S in the simulation environment t , input into the policy network and value network.

[0147] Action selection: The policy network selects actions based on the state S. t Generate action probability distribution π θ (a|s), select action A from it t .

[0148] Environment interaction: Execute action A in the simulation environment t , get the new state S t+1 and reward R t .

[0149] Value Assessment: Value Network Assessment Current Status S t The value of V φ (S t ).

[0150] Calculate Advantage: Calculate the advantage function F t =R t +γV φ (S t+1 )-V φ (S t ), where γ is the discount factor.

[0151] Policy update: Use the gradient ascent method to update the policy network parameters θ to maximize the expected reward:

[0152] Value update: Use gradient descent to update the value network parameter φ to minimize the squared error of the state value:

[0153] Repeat the above steps until the policy network and value network converge. Through the above process, the policy network and value network are continuously optimized, so that the model can make the best decision in the magnetic drive conveying system to obtain the trained production scheduling model.

[0154] Step 405: Model deployment application.

[0155] The trained model is deployed in the actual production environment for production scheduling in the actual production process, and can be adjusted in real time based on actual feedback.

[0156] Taking the application of the model to actual production scheduling as an example, assume that at a certain moment, the state of the magnetic drive conveying system is as follows:

[0157] The queue length of station 1 is 2, and the load is 2;

[0158] The queue length of workstation 2 is 2, and the load is 2;

[0159] The queue length of station 3 is 1 and the load is 1;

[0160] Conveying equipment 1 is at station 1, and the status is waiting;

[0161] The conveying device 2 is at station 2 and is in working state;

[0162] Conveying equipment 3 is at station 3, and the status is waiting;

[0163] The conveying device 4 is at station 2 and the status is waiting;

[0164] The conveying device 5 is at the workstation 1 and is in working state.

[0165] Then the input of the model can be expressed as:

[0166] S t ={L1=2,L2=2,L3=1,C1=G1,C2=G2,C3=G3,C4=G2,C5=G1}

[0167] The model outputs possible scheduling decisions, such as:

[0168] Move the conveying device 1 from station 1 to station 3;

[0169] Move the conveying device 4 from station 2 to station 1;

[0170] Close station 2.

[0171] Therefore, production scheduling can be carried out according to the actual status of the system, reducing the waiting time of conveying equipment and the idle time of workstations to improve production efficiency.

[0172] Furthermore, the model can be dynamically optimized based on actual feedback to adapt to the new production environment. Specifically, the model can calculate the immediate reward R based on the execution results and the state changes of the magnetic drive conveying system. t , and update the experience. If the model performs poorly, the collected experience can be used to train or fine-tune the model online so that the model can adapt to the new production environment.

[0173] Suppose that the following situations are encountered in the actual production process:

[0174] (1) Workstation 2 suddenly breaks down and needs to be temporarily closed;

[0175] (2) The load at station 1 suddenly increases, and the conveying equipment waits too long at station 1.

[0176] At this time, since the fault of station 2 is detected, the system status is updated as follows:

[0177] S t ={L1=4, L2=0, L3=1, C1=G1, C2=G1, C3=G3, C4=G1, C5=G1}

[0178] Enter the updated state S t to the model, which outputs a new scheduling decision:

[0179] Move the conveying device 1 from station 1 to station 3;

[0180] Move the conveying device 2 from station 1 to station 3;

[0181] The conveying device 4 is moved from the workstation 1 to the workstation 3 .

[0182] According to the model output, the above scheduling operation is performed to reduce the load of station 1, thereby avoiding excessive waiting time of the conveying equipment at station 1 and improving production efficiency.

[0183] However, if the new scheduling strategy is not effective, for example, the waiting time of the conveying equipment at station 3 is too long, the system will collect this experience and adjust the model parameters to achieve dynamic optimization of the model to adapt to changes in the production environment, optimize the next decision, and improve the accuracy of the model decision, thereby further improving production efficiency and reducing production costs.

[0184] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0185] Based on the same inventive concept, the embodiment of the present application also provides a scheduling device for implementing the production scheduling method of the magnetic drive conveying system involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more scheduling device embodiments provided below can refer to the limitations on the production scheduling method above, and will not be repeated here.

[0186] In an exemplary embodiment, Figure 5 As shown, a production scheduling device for a magnetic drive conveying system is provided, wherein the magnetic drive conveying system comprises a magnetic drive conveying track and at least one conveying device, wherein a plurality of production stations are arranged on the magnetic drive conveying track, and the device comprises: a state monitoring module 501, a strategy acquisition module 502 and a scheduling processing module 503, wherein:

[0187] A state monitoring module 501 is used to obtain state monitoring data of the magnetic drive conveying system, wherein the state monitoring data includes the load of the production station in the magnetic drive conveying system and the position of the conveying equipment;

[0188] A strategy acquisition module 502 is used to input the state monitoring data into a production scheduling model for data calculation to obtain a scheduling strategy, wherein the scheduling strategy includes at least one of a movement instruction for at least one of the conveying equipment and a state instruction for at least one of the production stations; the production scheduling model is obtained by deep reinforcement learning training based on sample data of the magnetic drive conveying system;

[0189] The scheduling processing module 503 is used to schedule the magnetic drive conveying system according to the scheduling strategy.

[0190] In an exemplary embodiment, the device also includes an optimization module, which is used to: after scheduling the magnetic drive conveying system according to the first scheduling strategy, the method also includes: obtaining the first state change data and the first reward parameter of the magnetic drive conveying system after the scheduling process; when the first reward parameter does not meet the set target value, determine the first interaction data based on the production environment according to the state monitoring data, the first scheduling strategy, the first state change data and the first reward parameter; and optimize the production scheduling model according to the first interaction data.

[0191] In an exemplary embodiment, the device also includes a model training module, which is used to: determine the corresponding simulation environment based on the magnetic-driven conveying system and initialize network parameters; obtain the initial state data of the magnetic-driven conveying system in the simulation environment, and determine the sample scheduling strategy based on the initial state data; obtain the state change data and sample reward parameters of the magnetic-driven conveying system after scheduling processing based on the sample scheduling strategy; determine the sample interaction data based on the simulation environment according to the initial state data, the sample scheduling strategy, the state change data and the sample reward parameters; and update the network parameters based on the deep reinforcement learning algorithm according to the sample interaction data to obtain the trained production scheduling model.

[0192] In an exemplary embodiment, the model training module is also used to: calculate an advantage function based on the sample interaction data, wherein the advantage function is used to measure the difference between the expected return of scheduling processing based on the sample scheduling strategy and the value estimate corresponding to the initial state data; according to the advantage function, update the strategy network parameters by maximizing the expected return of the sample scheduling strategy; according to the advantage function, update the value network parameters by minimizing the square error between the expected return and the value estimate to obtain the trained production scheduling model.

[0193] In an exemplary embodiment, the model training module is also used to: determine the degree of load imbalance of each production station based on the state change data; obtain the idle time and product production time of the magnetic drive conveying system; and calculate the sample reward parameters based on a preset reward function according to the degree of load imbalance, the idle time and the product production time.

[0194] In an exemplary embodiment, the reward function is: t = -α·T idle -β·T total -γ·L balance , where R t represents the calculated sample reward parameter determined based on the sample scheduling strategy at time t, T idle Indicates the idle time, T total Indicates the production time of the product, L balance represents the load imbalance degree, and α, β, and γ represent the corresponding weights.

[0195] In an exemplary embodiment, the deep reinforcement learning algorithm is implemented based on any one of a deep Q network, a deep deterministic policy gradient, and a proximal policy optimization algorithm.

[0196] Each module in the production scheduling device of the magnetic drive conveying system can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in the electronic device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0197] In an exemplary embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a production scheduling method for a magnetic drive conveying system is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0198] Those skilled in the art will understand that Figure 6 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0199] In an exemplary embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0200] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0201] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0202] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0203] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0204] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A production scheduling method for a magnetic drive conveying system, characterized in that: The magnetic drive conveying system comprises a magnetic drive conveying track and at least one conveying device, a plurality of production stations are arranged on the magnetic drive conveying track, and the method comprises: Acquiring status monitoring data of the magnetic drive conveying system, wherein the status monitoring data includes the monitored load of the production station in the magnetic drive conveying system and the position of the conveying equipment; Inputting the state monitoring data into a production scheduling model for data calculation to obtain a scheduling strategy, wherein the scheduling strategy includes at least one of a movement instruction for at least one of the conveying devices and a state instruction for at least one of the production stations; the production scheduling model is obtained by deep reinforcement learning training based on sample data of the magnetic drive conveying system; The magnetic drive conveying system is scheduled according to the scheduling strategy.

2. The method according to claim 1, characterized in that The training method of the production scheduling model includes: Determine a corresponding simulation environment based on the magnetic drive conveying system and initialize network parameters; Acquire initial state data of the magnetic drive conveying system in the simulation environment, and determine a sample scheduling strategy according to the initial state data; Acquire state change data and sample reward parameters of the magnetic drive conveying system after scheduling based on the sample scheduling strategy; Determining sample interaction data based on the simulation environment according to the initial state data, the sample scheduling strategy, the state change data and the sample reward parameter; According to the sample interaction data, the network parameters are updated based on a deep reinforcement learning algorithm to obtain the trained production scheduling model.

3. The method according to claim 2, characterized in that The updating of the network parameters based on the deep reinforcement learning algorithm according to the sample interaction data to obtain the trained production scheduling model includes: Calculating an advantage function based on the sample interaction data, the advantage function being used to measure the difference between an expected return of scheduling processing based on the sample scheduling strategy and a value estimate corresponding to the initial state data; updating policy network parameters by maximizing the expected return of the sample scheduling policy according to the advantage function; According to the advantage function, the value network parameters are updated by minimizing the square error between the expected return and the value estimate to obtain the trained production scheduling model.

4. The method according to claim 2, characterized in that: The method for obtaining the sample reward parameter includes: Determining the load imbalance degree of each production station according to the state change data; Obtaining the idle time and product production time of the magnetic drive conveying system; The sample reward parameter is calculated based on a preset reward function according to the load imbalance degree, the idle time and the product production time.

5. The method according to claim 4, characterized in that The reward function adopts: R t = -α·T idle -β·T total -γ·L balance , where R t represents the calculated sample reward parameter determined based on the sample scheduling strategy at time t, T idle Indicates the idle time, T total Indicates the production time of the product, L balance represents the load imbalance degree, and α, β, and γ represent the corresponding weights.

6. The method according to any one of claims 2 to 5, characterized in that: The deep reinforcement learning algorithm is implemented based on any one of a deep Q network, a deep deterministic policy gradient, and a proximal policy optimization algorithm.

7. A production scheduling device for a magnetic drive conveying system, characterized in that: The magnetic drive conveying system comprises a magnetic drive conveying track and at least one conveying device, a plurality of production stations are arranged on the magnetic drive conveying track, and the device comprises: A status monitoring module, used for acquiring status monitoring data of the magnetic drive conveying system, wherein the status monitoring data includes the load of the production station in the magnetic drive conveying system and the position of the conveying equipment; a strategy acquisition module, configured to input the state monitoring data into a production scheduling model for data calculation to obtain a scheduling strategy, wherein the scheduling strategy includes at least one of a movement instruction for at least one of the conveying devices and a state instruction for at least one of the production stations; the production scheduling model is obtained by deep reinforcement learning training based on sample data of the magnetic drive conveying system; The scheduling processing module is used to schedule the magnetic drive conveying system according to the scheduling strategy.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images