Pulsating production line scheduling method, device, equipment, storage medium and product

Abstract

This invention provides a method, apparatus, equipment, storage medium, and product for scheduling a pulsed production line. The method involves inputting static information about the current scheduling problem into a resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; inputting the static information about the current scheduling problem, the predicted peak value of each type of production resource, and the current shift schedule into a process scheduling agent to obtain the probability value of each process as the next shift schedule output by the process scheduling agent; determining the next scheduling plan based on the probability value of each process as the next shift schedule; and executing the scheduling process on the pulsed production line according to the next scheduling plan. If there are still processes on the pulsed production line that have not been started, the above steps are repeated; otherwise, the scheduling of the pulsed production line ends. This invention achieves load balancing of the pulsed production line and improves the scheduling accuracy of the pulsed production line.

Description

Technical Field

[0001] This invention relates to the field of production line scheduling technology, and in particular to a method, apparatus, equipment, storage medium, and product for scheduling a pulsed production line. Background Technology

[0002] Pulsed production lines are advanced assembly lines primarily used in the aerospace industry. As an example, aircraft assembly processes are implemented using pulsed production lines. Aircraft assembly is a critical link in aircraft manufacturing, and how to rationally schedule aircraft assembly pulsed production lines to achieve cost reduction and efficiency improvement is an important scientific problem in the field of intelligent manufacturing. However, the aircraft assembly pulsed production line scenario is complex; assembling a single aircraft involves tens of thousands of processes, which brings new challenges to efficiently solving the aircraft assembly scheduling problem.

[0003] In existing technologies, pointer network with actor-critic (PN-AC) models based on reinforcement learning are used for scheduling pulsating pipelines. Currently, Markov decision processes (MDPs) for pulsating pipeline scheduling problems are often based on greedy frameworks. In practical applications, reinforcement learning agents dynamically select the task with the highest priority, while the initial execution time of the task is determined by the greedy framework.

[0004] However, greedy algorithms aim to advance the start time of tasks as much as possible within a locally optimal timeframe. Therefore, current pulsed production line scheduling methods suffer from load imbalance and poor scheduling accuracy for aircraft assembly lines. Summary of the Invention

[0005] This invention provides a method, apparatus, equipment, storage medium, and product for scheduling pulse production lines, which solves the defects of existing pulse production line scheduling methods, such as load imbalance and poor scheduling accuracy, and achieves load balancing of pulse production lines, thereby improving the scheduling accuracy of pulse production lines.

[0006] This invention provides a method for scheduling a pulsed production line, comprising: acquiring static information of the current scheduling problem of the pulsed production line; inputting the static information of the current scheduling problem into a resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; wherein the resource prediction agent is constructed based on a reinforcement learning algorithm and pre-trained based on a resource prediction training set; the resource prediction training set includes static information of historical scheduling problems and the historical peak values ​​of each type of production resource; inputting the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift schedule into a process scheduling agent to obtain the current peak value of each process output by the process scheduling agent. The probability value is used as the next scheduling value; wherein, the process scheduling agent is constructed based on a reinforcement learning algorithm and is pre-trained based on a process scheduling training set; the process scheduling training set includes the static information of the historical scheduling problem, the peak value of each type of production resource in history, the historical scheduling situation, and the probability value of each process in history as the next scheduling; based on the probability value of each current process as the next scheduling, the next scheduling plan is determined; according to the next scheduling plan, the scheduling process of the pulsed production line is executed; if there are still processes that have not started on the pulsed production line, the step of obtaining the static information of the current scheduling problem of the pulsed production line is returned; otherwise, the scheduling of the pulsed production line ends.

[0007] According to a pulsed production line scheduling method provided by the present invention, before obtaining static information of the current scheduling problem of the pulsed production line, the method further includes: constructing the resource prediction training set and the process scheduling training set based on historical pulsed production line assembly data; constructing an initial resource prediction agent and an initial process scheduling agent based on a reinforcement learning algorithm; and alternately training the initial resource prediction agent and the initial process scheduling agent based on the resource prediction training set and the process scheduling training set until the resource prediction agent and the process scheduling agent are obtained.

[0008] According to a pulsed production line scheduling method provided by the present invention, the alternating training of the initial resource prediction agent and the initial process scheduling agent specifically includes: gradually increasing the training scale through a course-based training method, and alternatingly training the initial resource prediction agent and the initial process scheduling agent.

[0009] According to a pulsed production line scheduling method provided by the present invention, the method further includes: normalizing the reward function under different training scales; during the training process, after the currently trained agent performs an action, obtaining the current reward function fed back by the external environment, and inputting the current reward function into the currently trained agent to optimize the currently trained agent.

[0010] According to a pulsed production line scheduling method provided by the present invention, the step of determining the next scheduling scheme based on the probability value of each current process as the next scheduling step includes: if the variance of the probability value of each current process as the next scheduling step is greater than a first threshold, then determining the next scheduling scheme based on a search-based multiple inference method; if the variance of the probability value of each current process as the next scheduling step is not greater than the first threshold, then constructing multiple sub-problems corresponding to the current scheduling problem based on a data augmentation method; calculating the solution of each sub-problem to obtain multiple equivalent solutions corresponding to the current scheduling problem; evaluating the multiple equivalent solutions corresponding to the current scheduling problem through an evaluation function, and using the optimal solution as the next scheduling scheme.

[0011] According to a pulsed production line scheduling method provided by the present invention, after obtaining the static information of the current scheduling problem of the pulsed production line, the method further includes: constructing an input graph of the current scheduling problem based on the static information of the current scheduling problem through a graph attention network; the step of inputting the static information of the current scheduling problem into a resource prediction agent specifically includes: inputting the input graph of the current scheduling problem into the resource prediction agent.

[0012] This invention also provides a pulsed production line scheduling device, comprising the following modules: an acquisition module, used to acquire static information of the current scheduling problem of the pulsed production line; a first input module, used to input the static information of the current scheduling problem into a resource prediction agent, and obtain the predicted peak value of each type of production resource output by the resource prediction agent; wherein, the resource prediction agent is constructed based on a reinforcement learning algorithm and pre-trained based on a resource prediction training set; the resource prediction training set includes historical scheduling problem static information and historical peak values ​​of each type of production resource; a second input module, used to input the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift schedule into a process scheduling agent, and obtain the predicted peak value of each type of production resource output by the process scheduling agent. Each current process is used as a probability value for the next scheduling step; wherein, the process scheduling agent is constructed based on a reinforcement learning algorithm and pre-trained based on a process scheduling training set; the process scheduling training set includes static information of the historical scheduling problem, the peak value of each type of production resource in history, the historical scheduling situation, and the probability value of each current process as the next scheduling step; a determination module is used to determine the next scheduling scheme based on the probability value of each current process as the next scheduling step; an execution module is used to perform scheduling processing on the pulsed production line according to the next scheduling scheme; a processing module is used to return to the step of obtaining the static information of the current scheduling problem of the pulsed production line if there are still unstarted processes on the pulsed production line; otherwise, the pulsed production line scheduling is terminated.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the pulsed production line scheduling method described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pulsed production line scheduling method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the pulsed production line scheduling method as described above.

[0016] The pulsed production line scheduling method, apparatus, equipment, storage medium, and product provided by this invention involve: inputting static information of the current scheduling problem into a resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; inputting the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift status into a process scheduling agent to obtain the probability value of each process as the next shift output by the process scheduling agent; determining the next scheduling plan based on the probability value of each process as the next shift; performing scheduling processing on the pulsed production line according to the next scheduling plan; if there are still unstarted processes on the pulsed production line, returning to the above steps; otherwise, ending the pulsed production line scheduling. The present invention employs a two-layer reinforcement learning model to decompose the scheduling problem of a pulsed production line into two sub-problems: resource prediction and process scheduling. Resource prediction intelligence predicts the peak value of each type of production resource. Then, based on the predicted peak value of each type of production resource, the process scheduling intelligence outputs the probability value of each current process as the next scheduling step. Furthermore, based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. The next scheduling plan is then executed, and after execution, it is determined whether there are any unstarted processes on the pulsed production line. If there are, the process returns to the previous steps until there are no unstarted processes on the pulsed production line, thus achieving load balancing of the pulsed production line and improving the scheduling accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts of the pulsed production line scheduling method provided by the present invention.

[0019] Figure 2 This is the second flowchart of the pulsed production line scheduling method provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the pulsed production line scheduling device provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] The technical solution of this application and how it solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The following is a combination of... Figure 1 and Figure 2 The present invention describes the pulsed production line scheduling method.

[0024] In practical applications, the executing entity of this pulsed production line scheduling method can be a pulsed production line scheduling device. There are various ways to implement a pulsed production line scheduling device. For example, it can be implemented through a computer program, such as application software; or, for example, a chip. It can also be implemented as a medium storing the relevant computer program, such as a USB flash drive or cloud storage; or, it can be implemented through a physical device that integrates or installs the relevant computer program, such as a server.

[0025] Figure 1 This is one of the flowcharts illustrating the pulsed production line scheduling method provided by the present invention, such as... Figure 1 As shown, the method includes steps 101 to 107.

[0026] Step 101: Obtain static information on the current scheduling problem of the pulsed production line.

[0027] The static information of the current scheduling problem includes the topological relationships between all processes in the pulsed production line, the number of workers required for each process, and the processing time for each process. In this embodiment, the static information of the current scheduling problem can be preset according to current production needs. As an example, the pulsed production line can be applied to the aircraft assembly production process, and the aircraft assembly pulsed production line sets the static information of the current scheduling problem according to the aircraft assembly requirements.

[0028] In practical applications, pulsed production lines involve multiple processes, requiring the rational allocation of various types of production resources within the production system. The scheduling problem, in particular, refers to the allocation of these diverse production resources. The objectives of the scheduling problem are generally to minimize completion time, achieve the most balanced production line load, or reduce the consumption of production resources.

[0029] Step 102: Input the static information of the current scheduling problem into the resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent.

[0030] Among them, the resource prediction agent is constructed based on the reinforcement learning algorithm and is pre-trained based on the resource prediction training set; the resource prediction training set includes static information of historical scheduling problems and the peak values ​​of each type of production resource in history.

[0031] In practical applications, production systems include various types of production resources, such as human resources, production equipment resources, and material resources. In this embodiment, the resource prediction agent receives static information about the current scheduling problem and uses it to predict the peak value of each type of production resource.

[0032] Specifically, the resource prediction agent can be represented as Let K be the number of types of production resources, and let the resource prediction agent be... Receive static information about the current scheduling problem, and the resource prediction agent... The output is the predicted peak value for each type of production resource, denoted as . .

[0033] Furthermore, to obtain more accurate predicted peak values ​​for each type of production resource, features can be extracted from the static information of the current scheduling problem, and then the extracted information can be input into the resource prediction agent. In this embodiment, the feature extraction method is not specifically limited. In one example, an attention mechanism (AM) is used to extract features from the static information of the current scheduling problem. In another example, a graph attention network (GAT) can be used to extract features from the static information of the current scheduling problem. In one possible implementation, after step 101, the method further includes: constructing an input graph of the current scheduling problem based on the static information of the current scheduling problem using a graph attention network; step 102 specifically includes: inputting the input graph of the current scheduling problem into the resource prediction agent to obtain the predicted peak values ​​for each type of production resource output by the resource prediction agent.

[0034] In this embodiment, the input graph G of the current scheduling problem is a topological graph constructed by the Graph Attention Network (GAT) based on the static information of the current scheduling problem. In the input graph G, each task operation is a node in the graph, and the constraints between nodes are directed edges. The structure of the topological graph and the node information change as the solution progresses. For the i-th layer network, the graph attention network representation calculation formula for the j-th node is as follows:

[0035]

[0036] in express The successor node of a node. Indicates the first The value matrix in a layered network It is an attention mechanism, and its calculation formula is:

[0037]

[0038] in, and They represent the first The query and key matrix in the layered network.

[0039] Specifically, resource prediction intelligent agents The input is the current scheduling problem input graph G, and the resource prediction agent is... The output of a two-layer perceptron (MLP) network is a K-dimensional Gaussian distribution with a variance of 1 and a mean of 1. ,in, Resource prediction agent The probability output, This represents the predicted peak value for each type of production resource.

[0040] Step 103: Input the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift status into the process scheduling agent to obtain the probability value of each current process output by the process scheduling agent as the next shift.

[0041] Among them, the process scheduling agent is constructed based on the reinforcement learning algorithm and is pre-trained based on the process scheduling training set; the process scheduling training set includes static information of historical scheduling problems, peak values ​​of each type of production resource in history, historical shift schedules, and the probability value of each process in history as the next shift.

[0042] Specifically, the process scheduling agent can be represented as Process scheduling intelligent agent It receives static information about the current scheduling problem, the predicted peak value of each type of production resource, and the current shift status. It outputs the probability value of each process as the next shift through the Pointer Network (PN).

[0043] In practical applications, process scheduling intelligent agents Output distribution ,distributed The dimension is consistent with the number of processes in the pulsed production line, through distribution. It is possible to determine the probability value of each current process as the next scheduling step.

[0044] Step 104: Determine the next scheduling plan based on the probability value of each current process as the next shift.

[0045] Step 105: Perform scheduling processing on the pulsed production line according to the next scheduling plan.

[0046] In practical applications, based on the probability value of each current process being used as the next scheduling step, an unstarted process is selected from all processes and assigned to begin execution at the earliest time allowed by resources. Specifically, for the process scheduling agent... Output distribution In the case of distribution Select one process from the sample and assign it to start execution at the earliest time that resources allow.

[0047] Step 106: Determine if there are any unstarted processes on the pulsed production line; if there are no unstarted processes on the pulsed production line, the process ends.

[0048] Step 107: If there are still unstarted processes on the pulsed production line, return to step 101.

[0049] Furthermore, after the scheduling plan is executed, it is determined whether there are any unstarted processes in the pulsed production line. If there are unstarted processes, the above steps are repeated until there are no unstarted processes in the pulsed production line.

[0050] In existing technologies, a typical strategy for constructing Markov Decision Processes (MDPs) in reinforcement learning scenarios that solve combinatorial optimization problems is based on a traditional greedy framework. In this method, the reinforcement learning agent dynamically selects the task with the highest priority, and the initial execution time of the task is determined by the greedy framework. The greedy algorithm aims to achieve a locally optimal execution time, thus trying to advance the task start time as much as possible. Therefore, current pulsed production line scheduling methods suffer from load imbalance and poor scheduling accuracy in aircraft assembly lines.

[0051] To address the aforementioned technical problems, this invention constructs a two-layer reinforcement learning framework to decompose the scheduling problem of a pulsed production line into two mutually cooperating sub-problems: resource prediction and process scheduling. Within this framework, the resource prediction agent predicts the peak value of each type of production resource. Further, constrained by the predicted peak value of each type of production resource, the process scheduling agent outputs the probability value of each current process as the next scheduling step. Then, based on the probability value of each current process as the next scheduling step, the next scheduling scheme is determined. The next scheduling scheme is then executed, and after execution, it is determined whether there are any unstarted processes on the pulsed production line. If so, the process returns to the previous steps until there are no unstarted processes on the pulsed production line, thus achieving load balancing and improving the scheduling accuracy of the pulsed production line.

[0052] Based on the above description, both the resource prediction agent and the process scheduling agent are constructed based on reinforcement learning algorithms and pre-trained. Specifically, regarding the training process of the resource prediction agent and the process scheduling agent, as an example, in one possible implementation, Figure 2 This is the second flowchart illustrating the pulsed production line scheduling method provided by the present invention, as shown below. Figure 2 As shown, before step 101 above, the method further includes steps 201 to 203.

[0053] Step 201: Based on historical pulsed production line assembly data, construct resource prediction training set and process scheduling training set.

[0054] Based on the above explanation, the resource prediction training set includes static information about historical scheduling problems and the peak values ​​of each type of production resource in history. The process scheduling training set includes static information about historical scheduling problems, the peak values ​​of each type of production resource in history, historical shift schedules, and the probability value of each process in history as the next shift in the schedule.

[0055] In practical applications, historical pulsed production line assembly data refers to the actual pulsed production line assembly data during a historical period. This historical data includes at least: static information about historical scheduling problems, peak values ​​for each type of production resource, and the probability value of each process as the next scheduling step. As an example, during the operation of the pulsed production line, assembly data is recorded in a log in real time. Correspondingly, historical pulsed production line assembly data can be obtained from the log, and based on this data, resource prediction training sets and process scheduling training sets can be constructed.

[0056] Step 202: Based on the reinforcement learning algorithm, construct the initial resource prediction agent and the initial process scheduling agent.

[0057] As an example, the initial resource prediction agent could be based on a multilayer perceptron (MLP) network model. The initial process scheduling agent could be based on a pointer network with actor-critic (PN-AC) model using reinforcement learning.

[0058] Step 203: Based on the resource prediction training set and the process scheduling training set, the initial resource prediction agent and the initial process scheduling agent are trained alternately until the resource prediction agent and the process scheduling agent are obtained.

[0059] In practical applications, an alternating training mode, mimicking coordinate descent, is used when training the resource prediction agent and the process scheduling agent. Specifically, during the training of the resource prediction agent, the process scheduling agent is placed in evaluative mode, treating it as the external environment of the resource prediction agent. Static information from historical scheduling problems is input into the current resource prediction agent to obtain the predicted peak value for each type of production resource. The current resource prediction agent receives the current reward function from the external environment and optimizes itself based on this function. By comparing the predicted peak value for each type of production resource output by the current resource prediction agent with the historical peak values, the performance of the current resource prediction agent can be evaluated.

[0060] Specifically, resource prediction intelligent agents The reward function is expressed as minimizing the total number of produced resources, that is... In practical applications, to prevent resource prediction agents... If the output of total production resources is too low, the process scheduling agent will... Unable to predict resource intelligence agents Given production resources, a resource prediction agent can construct a suitable scheduling scheme. A significant penalty term will be added to the reward function.

[0061] Correspondingly, when training the process scheduling agent, the resource prediction agent is placed in evaluative mode, treating it as the external environment for the process scheduling agent. Static information from historical scheduling problems, the peak values ​​of each type of production resource in history, and historical shift schedules are input into the current process scheduling agent to obtain the probability value of each current process as the next shift in the schedule. The current process scheduling agent receives the corresponding reward function and optimizes itself based on this function. By comparing the probability value of each current process as the next shift with the historical probability values, the performance of the current process scheduling agent can be evaluated.

[0062] Specifically, process scheduling intelligent agent After a complete scheduling scheme is generated, the external environment will evaluate and return the reward signal for the entire decision trajectory. The reward function is the negative of the total time to complete all processes. If the final generated scheduling scheme cannot satisfy all constraints, a significant penalty term will be added to the reward function.

[0063] As an example, when training resource prediction agents and process scheduling agents, algorithms such as reinforcement learning (REINFORCE), proximal policy optimization (PPO), and advantage actor-critic (A2C) can be used for optimization. For example, the REINFORCE algorithm formula is as follows:

[0064]

[0065] Among them, This represents a set of exploratory solutions generated. This represents the reward assessment for this set of solutions. In the question Using the baseline as a training base can reduce variance when training with the REINFORCE algorithm.

[0066] Based on the above explanation, the resource prediction agent The input can be the graph G of the current scheduling problem, and the resource prediction agent. The output is a Gaussian distribution of dimension K. In practical applications, it is used to train resource prediction agents. At that time, resource prediction agent The actions can be sampled from a Gaussian distribution, and the mean of the Gaussian distribution is used for evaluation. It should be noted that, due to the resource prediction agent... The actions are highly interpretable. To achieve more stable and efficient exploration, resource prediction agents can be optimized. Additional restrictions are imposed on the sampling process. For example, it is stipulated that the sampling result must be an integer and can only be within the mean of a Gaussian distribution. Within the specified range, the value of a certain production resource can be increased or decreased accordingly. Based on this, the resource prediction agent... Based on the final returned reward function, production resources that can be reduced can be further optimized, or the region of illegal solutions can be escaped.

[0067] In this embodiment, a resource prediction agent and a process scheduling agent are pre-trained based on a resource prediction training set and a process scheduling training set, resulting in a two-layer reinforcement learning model. Based on this, the scheduling problem of the pulsed production line is decomposed into two sub-problems: resource prediction and process scheduling. The resource prediction agent predicts the peak value of each type of production resource. Then, based on the predicted peak value of each type of production resource, the process scheduling agent outputs the probability value of each current process as the next scheduling step. Furthermore, based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. Further, the next scheduling plan is executed, and after execution, it is checked whether there are still unstarted processes on the pulsed production line. If there are still unstarted processes, the above steps are returned until there are no unstarted processes on the pulsed production line, thus achieving load balancing of the pulsed production line and improving the scheduling accuracy of the pulsed production line.

[0068] In practical applications, pulsed production lines involve multiple processes. For example, a complex aircraft assembly pulsed production line involves dozens of workstations and tens of thousands of processes. The training scale of reinforcement learning agents is large, and it is difficult for the agent to obtain meaningful strategies in this large-scale exploration space in the early stages of training. The training is time-consuming and difficult. To improve training efficiency, as an example, in one possible implementation, step 203 above alternates the training of the initial resource prediction agent and the initial process scheduling agent. Specifically, this includes gradually increasing the training scale through a course-based training method, and alternating the training of the initial resource prediction agent and the initial process scheduling agent.

[0069] In this embodiment, the agent is trained by gradually increasing the training scale, meaning the agent learns from easy to difficult problems. For example, the agent is first trained on an assembly problem with LB=50 steps, and its learning performance is evaluated. Then, based on the reward function value obtained by the agent, the upper bound UB of the problem size is gradually increased until an assembly problem of a given size can be solved. The training scale is sampled from a uniform distribution U(LB, UB) for each iteration. It is understood that the method in this embodiment effectively reduces the difficulty for the agent in exploring difficult problems, significantly reduces training time, and improves training efficiency.

[0070] Specifically, for the aircraft assembly problem, the solution process involves breaking down the original problem into smaller subproblems. For example, an aircraft assembly problem with 100 steps can be reduced to an aircraft assembly problem with 97 steps after completing 3 steps. Therefore, training on smaller-scale problems has a positive benefit for learning larger-scale problems, which is why the scale of course learning training is not directly increased, but rather sampled. However, during agent training, larger-scale problems can potentially yield higher rewards, while smaller-scale problems offer lower rewards, leading to instability in optimization. To improve the stability and reliability of agent training, as an example, in one possible implementation, the above-mentioned pulsed production line scheduling method further includes: normalizing the reward function under different training scales; during training, after the currently trained agent performs an action, obtaining the current reward function from the external environment feedback, and inputting the current reward function into the currently trained agent to optimize the agent.

[0071] In this embodiment, the reward function under different training scales is normalized to ensure that the reward function under different training scales is on the same benchmark, thereby improving the stability and reliability of agent training.

[0072] Based on the above explanation, after obtaining the probability value of each current process as the next scheduling step from the output of the process scheduling agent, the next scheduling plan is determined according to the probability value of each current process as the next scheduling step. In one example, the next scheduling plan is determined through a single inference. Specifically, based on a greedy algorithm, the process with the highest probability value is directly selected as the next process and assigned to start execution at the earliest time allowed by resources.

[0073] In another example, the next scheduling scheme is determined through multiple inferences. As an example, in one possible implementation, step 104 includes: if the variance of the probability value of each current process as the next scheduling step is greater than a first threshold, then the next scheduling scheme is determined based on the search-based multiple inference method; if the variance of the probability value of each current process as the next scheduling step is not greater than the first threshold, then multiple sub-problems corresponding to the current scheduling problem are constructed based on the data augmentation method; the solution to each sub-problem is calculated to obtain multiple equivalent solutions corresponding to the current scheduling problem; the multiple equivalent solutions corresponding to the current scheduling problem are evaluated through an evaluation function, and the optimal solution is taken as the next scheduling scheme.

[0074] During the inference process, when the reinforcement learning agent's training approaches convergence, the variance of the probability value of each current process as the next scheduling step is very small. This makes it difficult to sample solutions with significant differences using search techniques, and makes it easier to get trapped in local optima. In this implementation, based on a data augmentation method, multiple sub-problems corresponding to the current scheduling problem are constructed; the solution to each sub-problem is calculated, resulting in multiple equivalent solutions to the current scheduling problem; these equivalent solutions are evaluated using an evaluation function, and the optimal solution is used as the next scheduling scheme, improving the accuracy of the pulsed production line scheduling. Correspondingly, when the variance of the probability value of each current process as the next scheduling step exceeds a first threshold, a multiple inference method using search is employed to determine the next scheduling scheme.

[0075] Specifically, data augmentation methods can be based on Directed Acyclic Graph (DAG) modeling. As an example, considering the multi-process parallel nature of aircraft assembly and the dependency relationships modeled using DAGs, this invention proposes solving several subproblems of the original problem in addition to solving the original problem. The solution spaces of these subproblems are subsets of the original problem. Specifically, on one hand, several nodes with a degree of 0 and their corresponding edges are randomly removed from the DAG and added to the scheduling scheme, equivalent to directly starting some unconstrained processes. On the other hand, several edges are randomly added to the DAG, equivalent to adding constraints to some processing steps. The DAGs of these subproblems differ significantly from the original problem, leading to different agent strategies. This approach allows obtaining some equivalent solutions to the original problem without compromising solution quality. The optimal solution is then found through an evaluation function as the final output scheduling scheme.

[0076] In the pulsed production line scheduling method provided in this embodiment, the static information of the current scheduling problem is input into the resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift status are input into the process scheduling agent to obtain the probability value of each process as the next shift output by the process scheduling agent; based on the probability value of each process as the next shift, the next scheduling plan is determined; the pulsed production line is scheduled according to the next scheduling plan; if there are still processes on the pulsed production line that have not started, the above steps are returned; otherwise, the pulsed production line scheduling ends. This embodiment uses a two-layer reinforcement learning model to decompose the pulsed production line scheduling problem into two sub-problems: resource prediction and process scheduling. The resource prediction agent predicts the peak value of each type of production resource. Then, based on the predicted peak value of each type of production resource, the process scheduling agent outputs the probability value of each process as the next shift. Finally, based on the probability value of each process as the next shift, the next scheduling plan is determined. Furthermore, the next scheduling plan is executed, and after execution, it is determined whether there are still unstarted processes on the pulsed production line. If there are still unstarted processes on the pulsed production line, the above steps are returned; otherwise, the pulsed production line scheduling is terminated, thus achieving load balancing of the pulsed production line and improving the scheduling accuracy of the pulsed production line.

[0077] The pulsed production line scheduling device provided by the present invention is described below. The pulsed production line scheduling device described below can be referred to in correspondence with the pulsed production line scheduling method described above.

[0078] Figure 3 This is a schematic diagram of the structure of the pulsed production line scheduling device provided by the present invention, as shown below. Figure 3 As shown, the pulsed production line scheduling device includes: an acquisition module 31, a first input module 32, a second input module 33, a determination module 34, an execution module 35, and a processing module 36.

[0079] The aforementioned acquisition module 31 is used to acquire static information about the current scheduling problem of the pulsed production line; wherein, the static information about the current scheduling problem includes the topological relationship between all processes in the pulsed production line, the number of workers required for each process, and the processing time of each process.

[0080] In this embodiment, the static information of the current scheduling problem can be preset according to the current production needs. As an example, a pulsed production line can be applied to the aircraft assembly production process. The aircraft assembly pulsed production line sets the static information of the current scheduling problem according to the aircraft assembly requirements.

[0081] In practical applications, pulsed production lines involve multiple processes, requiring the rational allocation of various types of production resources within the production system. The scheduling problem, in particular, refers to the allocation of these diverse production resources. The objectives of the scheduling problem are generally to minimize completion time, achieve the most balanced production line load, or reduce the consumption of production resources.

[0082] The first input module 32 is used to input the static information of the current scheduling problem into the resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent. The resource prediction agent is constructed based on the reinforcement learning algorithm and is pre-trained based on the resource prediction training set. The resource prediction training set includes the static information of historical scheduling problems and the peak value of each type of production resource in history.

[0083] In practical applications, production systems include various types of production resources, such as human resources, production equipment resources, and material resources. In this embodiment, the resource prediction agent receives static information about the current scheduling problem and uses it to predict the peak value of each type of production resource.

[0084] Specifically, the resource prediction agent can be represented as Let K be the number of types of production resources, and let the resource prediction agent be... Receive static information about the current scheduling problem, and the resource prediction agent... The output is the predicted peak value for each type of production resource, denoted as . .

[0085] Furthermore, to obtain more accurate predicted peak values ​​for each type of production resource, features can be extracted from the static information of the current scheduling problem, and then the extracted information can be input into the resource prediction agent. In this embodiment, the feature extraction method is not specifically limited. In one example, an attention mechanism (AM) is used to extract features from the static information of the current scheduling problem. In another example, a graph attention network (GAT) can be used to extract features from the static information of the current scheduling problem. In one possible implementation, the above-mentioned pulsed production line scheduling device further includes: a feature extraction module, used to construct an input graph of the current scheduling problem based on the static information of the current scheduling problem using a graph attention network; the first input module 32 is specifically used to: input the input graph of the current scheduling problem into the resource prediction agent to obtain the predicted peak values ​​for each type of production resource output by the resource prediction agent.

[0086] In this embodiment, the input graph G of the current scheduling problem is a topological graph constructed by the Graph Attention Network (GAT) based on the static information of the current scheduling problem. In the input graph G, each task operation is a node in the graph, and the constraints between nodes are directed edges. The structure of the topological graph and the node information change as the solution progresses. For the i-th layer network, the graph attention network representation calculation formula for the j-th node is as follows:

[0087]

[0088] in express The successor node of a node. Indicates the first The value matrix in a layered network It is an attention mechanism, and its calculation formula is:

[0089]

[0090] in, and They represent the first The query and key matrix in the layered network.

[0091] Specifically, resource prediction intelligent agents The input is the current scheduling problem input graph G, and the resource prediction agent is... The output of a two-layer perceptron (MLP) network is a Gaussian distribution of dimension K, with a variance of 1 and a mean of 1. ,in, Resource prediction agent The probability output, This represents the predicted peak value for each type of production resource.

[0092] The second input module 33 is used to input the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift status into the process scheduling agent to obtain the probability value of each current process as the next shift output by the process scheduling agent. The process scheduling agent is constructed based on a reinforcement learning algorithm and is pre-trained based on the process scheduling training set. The process scheduling training set includes the static information of historical scheduling problems, the peak value of each type of production resource in history, the shift status in history, and the probability value of each historical process as the next shift.

[0093] Specifically, the process scheduling agent can be represented as Process scheduling intelligent agent It receives static information about the current scheduling problem, the predicted peak value of each type of production resource, and the current shift status. It outputs the probability value of each process as the next shift through the Pointer Network (PN).

[0094] In practical applications, process scheduling intelligent agents Output distribution ,distributed The dimension is consistent with the number of processes in the pulsed production line, through distribution. It is possible to determine the probability value of each current process as the next scheduling step.

[0095] The aforementioned determining module 34 is used to determine the next scheduling plan based on the probability value of each current process as the next scheduling step.

[0096] The aforementioned execution module 35 is used to perform scheduling processing on the pulsed production line according to the next scheduling plan.

[0097] In practical applications, the execution module 35 selects an unstarted process from all processes based on the probability value of each current process being used as the next scheduling step, and assigns it to start execution at the earliest time allowed by resources. Specifically, for the process scheduling agent... Output distribution In this case, execution module 35 is distributed Select one process from the sample and assign it to start execution at the earliest time that resources allow.

[0098] The aforementioned processing module 36 is used to return to the step of obtaining static information about the current scheduling problem of the pulse production line if there are still unstarted processes on the pulse production line; otherwise, the pulse production line scheduling is terminated.

[0099] Furthermore, after the scheduling plan is executed, the processing module 36 determines whether there are any unstarted processes in the pulsed production line. If there are unstarted processes, it returns to execute the above steps until there are no unstarted processes in the pulsed production line.

[0100] In this embodiment, a two-layer reinforcement learning framework is constructed to decompose the scheduling problem of a pulsed production line into two mutually cooperating sub-problems: resource prediction and process scheduling. Under this framework, the resource prediction agent predicts the peak value of each type of production resource. Further, constrained by the predicted peak value of each type of production resource, the process scheduling agent outputs the probability value of each current process as the next scheduling step. Then, based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. The next scheduling plan is then executed, and after execution, it is determined whether there are any unstarted processes on the pulsed production line. If so, the process returns to the previous steps until there are no unstarted processes on the pulsed production line, thus achieving load balancing and improving the scheduling accuracy of the pulsed production line.

[0101] In one possible implementation, the above-mentioned pulsed production line scheduling device further includes: a first construction module, a second construction module, and a training module.

[0102] The first construction module mentioned above is used to build a resource prediction training set and a process scheduling training set based on historical pulsed production line assembly data.

[0103] Based on the above explanation, the resource prediction training set includes static information about historical scheduling problems and the peak values ​​of each type of production resource in history. The process scheduling training set includes static information about historical scheduling problems, the peak values ​​of each type of production resource in history, historical shift schedules, and the probability value of each process in history as the next shift in the schedule.

[0104] In practical applications, historical pulsed production line assembly data refers to the actual pulsed production line assembly data during a historical period. This historical data includes at least: static information about historical scheduling problems, peak values ​​for each type of production resource, and the probability value of each process as the next scheduling step. As an example, during the operation of the pulsed production line, the assembly data is recorded in a log in real time. Correspondingly, the first construction module can obtain historical pulsed production line assembly data from the log and, based on this data, construct a resource prediction training set and a process scheduling training set.

[0105] The second building module mentioned above is used to build an initial resource prediction agent and an initial process scheduling agent based on reinforcement learning algorithms.

[0106] As an example, the initial resource prediction agent could be based on a multilayer perceptron network (MLP) model. The initial process scheduling agent could be based on a pointer network model (PN-AC) using reinforcement learning.

[0107] The aforementioned training module is used to alternately train the initial resource prediction agent and the initial process scheduling agent based on the resource prediction training set and the process scheduling training set until the resource prediction agent and the process scheduling agent are obtained.

[0108] In practical applications, the training module employs an alternating training mode mimicking coordinate descent when training the resource prediction agent and the process scheduling agent. Specifically, when training the resource prediction agent, the training module places the process scheduling agent in evaluative mode, treating it as the external environment for the resource prediction agent. Static information from historical scheduling problems is input into the current resource prediction agent to obtain the predicted peak value for each type of production resource. The current resource prediction agent receives the current reward function from the external environment and optimizes itself based on this function. By comparing the predicted peak value for each type of production resource output by the current resource prediction agent with the historical peak values, the performance of the current resource prediction agent can be evaluated.

[0109] Specifically, resource prediction intelligent agents The reward function is expressed as minimizing the total number of produced resources, that is... In practical applications, to prevent resource prediction agents... If the output of total production resources is too low, the process scheduling agent will... Unable to predict resource intelligence agents Given production resources, a resource prediction agent can construct a suitable scheduling scheme. A significant penalty term will be added to the reward function.

[0110] Correspondingly, during the training of the process scheduling agent, the training module places the resource prediction agent in evaluative mode, treating the resource prediction agent as the external environment for the process scheduling agent. Static information from historical scheduling problems, the peak values ​​of each type of production resource in history, and historical shift schedules are input into the current process scheduling agent to obtain the probability value of each current process as the next shift in the schedule. The current process scheduling agent receives the corresponding reward function and optimizes itself based on this function. By comparing the probability value of each current process as the next shift with the historical probability values, the performance of the current process scheduling agent can be evaluated.

[0111] Specifically, process scheduling intelligent agent After a complete scheduling scheme is generated, the external environment will evaluate and return the reward signal for the entire decision trajectory. The reward function is the negative of the total time to complete all processes. If the final generated scheduling scheme cannot satisfy all constraints, a significant penalty term will be added to the reward function.

[0112] As an example, when training resource prediction agents and process scheduling agents, algorithms such as reinforcement learning (REINFORCE), proximal policy optimization (PPO), and advantage actor-commentator (A2C) can be used for optimization.

[0113] Based on the above explanation, the resource prediction agent The input can be the graph G of the current scheduling problem, and the resource prediction agent. The output is a Gaussian distribution of dimension K. In practical applications, it is used to train resource prediction agents. At that time, resource prediction agent The actions can be sampled from a Gaussian distribution, and the mean of the Gaussian distribution is used for evaluation. It should be noted that, due to the resource prediction agent... The actions are highly interpretable. To achieve more stable and efficient exploration, resource prediction agents can be optimized. Additional restrictions are imposed on the sampling process. Based on this, the resource prediction agent... Based on the final returned reward function, production resources that can be reduced can be further optimized, or the region of illegal solutions can be escaped.

[0114] In this embodiment, a resource prediction agent and a process scheduling agent are pre-trained based on a resource prediction training set and a process scheduling training set, resulting in a two-layer reinforcement learning model. Based on this, the scheduling problem of the pulsed production line is decomposed into two sub-problems: resource prediction and process scheduling. The resource prediction agent predicts the peak value of each type of production resource. Then, based on the predicted peak value of each type of production resource, the process scheduling agent outputs the probability value of each current process as the next scheduling step. Furthermore, based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. The next scheduling plan is then executed, and after execution, it is determined whether there are any unstarted processes on the pulsed production line. If there are still unstarted processes, the above steps are repeated until there are no unstarted processes on the pulsed production line, thus achieving load balancing of the pulsed production line and improving the scheduling accuracy of the pulsed production line.

[0115] To improve training efficiency, as an example, in one possible implementation, the training module described above is used to alternately train the initial resource prediction agent and the initial process scheduling agent, specifically by: gradually increasing the training scale through a course-based training method, and alternately training the initial resource prediction agent and the initial process scheduling agent.

[0116] In this embodiment, the agent is trained by gradually increasing the training scale, that is, the agent learns in a way that progresses from easy to difficult problems. This can effectively reduce the difficulty for the agent to explore difficult problems, reduce the training time, and improve the training efficiency.

[0117] To improve the stability and reliability of agent training, as an example, in one possible implementation, the above-mentioned pulsed production line scheduling device further includes: a normalization module for normalizing the reward function under different training scales; and a third input module for obtaining the current reward function fed back from the external environment after the currently trained agent performs an action during the training process, and inputting the current reward function into the currently trained agent to optimize the currently trained agent.

[0118] In this embodiment, the reward function under different training scales is normalized to ensure that the reward function under different training scales is on the same benchmark, thereby improving the stability and reliability of agent training.

[0119] Based on the above explanation, after obtaining the probability value of each current process as the next scheduling step from the output of the process scheduling agent, the next scheduling plan is determined according to the probability value of each current process as the next scheduling step. In one example, the next scheduling plan is determined through a single inference. Specifically, based on a greedy algorithm, the process with the highest probability value is directly selected as the next process and assigned to start execution at the earliest time allowed by resources.

[0120] In another example, the next scheduling scheme is determined through multiple inferences. As an example, in one possible implementation, the determination module 34 is specifically used to: if the variance of the probability value of each current process as the next scheduling step is greater than a first threshold, then determine the next scheduling scheme based on the multiple inference method of search; if the variance of the probability value of each current process as the next scheduling step is not greater than the first threshold, then construct multiple sub-problems corresponding to the current scheduling problem based on the data augmentation method; calculate the solution of each sub-problem to obtain multiple equivalent solutions corresponding to the current scheduling problem; evaluate the multiple equivalent solutions corresponding to the current scheduling problem through an evaluation function, and take the optimal solution as the next scheduling scheme.

[0121] During the inference process, when the reinforcement learning agent's training approaches convergence, the variance of the probability value of each current process as the next scheduling step is very small. This makes it difficult to sample solutions with significant differences using search techniques, and makes it easier to get trapped in local optima. In this implementation, based on a data augmentation method, multiple sub-problems corresponding to the current scheduling problem are constructed; the solution to each sub-problem is calculated, resulting in multiple equivalent solutions to the current scheduling problem; these equivalent solutions are evaluated using an evaluation function, and the optimal solution is used as the next scheduling scheme, improving the accuracy of the pulsed production line scheduling. Correspondingly, when the variance of the probability value of each current process as the next scheduling step exceeds a first threshold, a multiple inference method using search is employed to determine the next scheduling scheme.

[0122] In the pulsed production line scheduling device provided by this invention, the acquisition module acquires static information of the current scheduling problem of the pulsed production line; the first input module inputs the static information of the current scheduling problem into the resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; the second input module inputs the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift status into the process scheduling agent to obtain the probability value of each process as the next shift output by the process scheduling agent; the determination module determines the next scheduling plan based on the probability value of each process as the next shift; the execution module performs scheduling processing on the pulsed production line according to the next scheduling plan; if there are still processes that have not started on the pulsed production line, the processing module returns to execute the above steps; otherwise, the processing module ends the pulsed production line scheduling. This embodiment employs a two-layer reinforcement learning model to decompose the scheduling problem of a pulsed production line into two sub-problems: resource prediction and process scheduling. Resource prediction intelligence predicts the peak value of each type of production resource. Then, based on the predicted peak value of each type of production resource, the process scheduling intelligence outputs the probability value of each current process as the next scheduling step. Furthermore, based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. The next scheduling plan is then executed, and after execution, it is determined whether there are any unstarted processes. If there are still unstarted processes on the pulsed production line, the above steps are returned; otherwise, the pulsed production line scheduling ends. This achieves load balancing of the pulsed production line and improves the scheduling accuracy.

[0123] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a pulsed production line scheduling method. This method includes: acquiring static information of the current scheduling problem of the pulsed production line; inputting the static information of the current scheduling problem into a resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; wherein the resource prediction agent is constructed based on a reinforcement learning algorithm and pre-trained based on a resource prediction training set; the resource prediction training set includes static information of historical scheduling problems and the historical peak values ​​of each type of production resource; inputting the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift schedule into a process scheduling agent to obtain the process scheduling... The process scheduling agent outputs the probability value of each current process as the next scheduling step. This process scheduling agent is constructed based on a reinforcement learning algorithm and pre-trained on a process scheduling training set. The training set includes static information about historical scheduling problems, the peak values ​​of each type of production resource in history, historical scheduling data, and the probability value of each historical process as the next scheduling step. Based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. The scheduling process is then executed on the pulsed production line according to the next scheduling plan. If there are still processes on the pulsed production line that have not yet started, the process returns to the step of obtaining the static information of the current scheduling problem of the pulsed production line; otherwise, the pulsed production line scheduling ends.

[0124] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pulse production line scheduling method provided by the above methods. The method includes: acquiring static information of the current scheduling problem of the pulse production line; inputting the static information of the current scheduling problem into a resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; wherein the resource prediction agent is constructed based on a reinforcement learning algorithm and is pre-trained based on a resource prediction training set; the resource prediction training set includes static information of historical scheduling problems and the peak value of each type of production resource in history; inputting the static information of the current scheduling problem and the peak value of each type of production resource in history ... training set to obtain the predicted peak value of each type of production resource in history; inputting the static information of the current scheduling problem and the peak value of each type of production resource in history into a resource prediction training set to obtain the predicted peak value of each type of production resource in history; inputting the static information of the current scheduling problem and the peak value of each type of production resource in history into a resource prediction training set to obtain the predicted peak value of each type of production resource in history; inputting the static information of the current scheduling problem and the peak value of each type of production resource in history into a resource prediction training set to obtain the predicted peak value of each type of production resource in history; inputting the static information of the current scheduling problem and the peak value of each type of production resource in history into a resource prediction training set to obtain the predicted peak value of each type of production resource in history; inputting the static information of the current scheduling problem and the peak value of each type of production resource in history into a resource prediction training set to obtain the The predicted peak value of resources and the current scheduling status are input into the process scheduling agent to obtain the probability value of each current process as the next scheduling step. The process scheduling agent is constructed based on a reinforcement learning algorithm and pre-trained on a process scheduling training set. The training set includes static information of historical scheduling problems, historical peak values ​​of each type of production resource, historical scheduling status, and historical probability values ​​of each process as the next scheduling step. Based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. The scheduling process is then executed on the pulsed production line according to the next scheduling plan. If there are still processes on the pulsed production line that have not yet started, the process returns to the step of obtaining the static information of the current scheduling problem of the pulsed production line; otherwise, the pulsed production line scheduling ends.

[0126] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the pulsed production line scheduling method provided by the above methods. The method includes: acquiring static information of the current scheduling problem of the pulsed production line; inputting the static information of the current scheduling problem into a resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; wherein the resource prediction agent is constructed based on a reinforcement learning algorithm and pre-trained based on a resource prediction training set; the resource prediction training set includes static information of historical scheduling problems and the historical peak values ​​of each type of production resource; and inputting the static information of the current scheduling problem, the predicted peak values ​​of each type of production resource, and the current shift schedule information... The input is fed into the process scheduling agent to obtain the probability value of each current process as the next scheduling step. The process scheduling agent is constructed based on a reinforcement learning algorithm and pre-trained on a process scheduling training set. The training set includes static information about historical scheduling problems, peak values ​​for each type of production resource, historical scheduling data, and the probability value of each historical process as the next scheduling step. Based on the probability value of each current process as the next scheduling step, the next scheduling plan is determined. The scheduling process is then executed on the pulsed production line according to the next scheduling plan. If there are still processes that have not started on the pulsed production line, the process returns to the step of obtaining the static information of the current scheduling problem of the pulsed production line; otherwise, the pulsed production line scheduling ends.

[0127] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for scheduling a pulsed production line, characterized in that, include: Obtain static information about the current scheduling problem of the pulsed production line; The static information of the current scheduling problem is input into the resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; wherein, the resource prediction agent is constructed based on the reinforcement learning algorithm and is pre-trained based on the resource prediction training set; the resource prediction training set includes the static information of historical scheduling problems and the peak value of each type of production resource in history. The static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift schedule are input into the process scheduling agent to obtain the probability value of each current process as the next shift schedule output by the process scheduling agent. The process scheduling agent is constructed based on a reinforcement learning algorithm and pre-trained on a process scheduling training set. The process scheduling training set includes the static information of historical scheduling problems, the historical peak values ​​of each type of production resource, historical shift schedules, and the historical probability value of each process as the next shift schedule. Based on the probability value of each current process as the next scheduling step, determine the next scheduling plan; and execute the scheduling process on the pulsed production line according to the next scheduling plan. If there are still unstarted processes on the pulsed production line, then return to the step of obtaining static information about the current scheduling problem of the pulsed production line; otherwise, end the scheduling of the pulsed production line. Before obtaining static information about the current scheduling problem of the pulsed production line, the method further includes: Based on historical pulsed production line assembly data, construct the resource prediction training set and the process scheduling training set; Based on reinforcement learning algorithms, an initial resource prediction agent and an initial process scheduling agent are constructed. Based on the resource prediction training set and the process scheduling training set, the initial resource prediction agent and the initial process scheduling agent are trained alternately until the resource prediction agent and the process scheduling agent are obtained. The step of determining the next scheduling plan based on the probability value of each current process as the next scheduling step includes: If the variance of the probability value of each current process as the next scheduling step is greater than the first threshold, then the next scheduling scheme is determined based on the multiple reasoning method of search. If the variance of the probability value of each current process as the next scheduling step is not greater than the first threshold, then based on the data augmentation method, multiple sub-problems corresponding to the current scheduling problem are constructed; the solution of each sub-problem is calculated to obtain multiple equivalent solutions corresponding to the current scheduling problem; the multiple equivalent solutions corresponding to the current scheduling problem are evaluated through an evaluation function, and the optimal solution is used as the next scheduling scheme.

2. The pulsed production line scheduling method according to claim 1, characterized in that, The alternating training of the initial resource prediction agent and the initial process scheduling agent specifically includes: Through course-based training, the training scale is gradually increased, and the initial resource prediction agent and the initial process scheduling agent are trained alternately.

3. The pulsed production line scheduling method according to claim 2, characterized in that, The method further includes: Normalize the reward function for different training scales; During training, after the currently trained agent performs an action, the current reward function fed back from the external environment is obtained, and the current reward function is input into the currently trained agent to optimize the agent.

4. The pulsed production line scheduling method according to any one of claims 1-3, characterized in that, After obtaining the static information of the current scheduling problem of the pulsed production line, the method further includes: Based on the static information of the current scheduling problem, an input graph of the current scheduling problem is constructed using a graph attention network; The step of inputting the static information of the current scheduling problem into the resource prediction agent specifically includes: The current scheduling problem input graph is input into the resource prediction agent.

5. A pulsed production line scheduling device, characterized in that, include: The acquisition module is used to acquire static information about the current scheduling problem of the pulsed production line; The first input module is used to input the static information of the current scheduling problem into the resource prediction agent to obtain the predicted peak value of each type of production resource output by the resource prediction agent; wherein, the resource prediction agent is constructed based on the reinforcement learning algorithm and is pre-trained based on the resource prediction training set; the resource prediction training set includes the static information of historical scheduling problems and the peak value of each type of production resource in history; The second input module is used to input the static information of the current scheduling problem, the predicted peak value of each type of production resource, and the current shift schedule into the process scheduling agent to obtain the probability value of each current process as the next shift schedule output by the process scheduling agent; wherein, the process scheduling agent is constructed based on a reinforcement learning algorithm and is pre-trained based on a process scheduling training set; the process scheduling training set includes the static information of the historical scheduling problems, the peak value of each type of production resource in history, the historical shift schedule, and the probability value of each historical process as the next shift schedule; The determination module is used to determine the next scheduling plan based on the probability value of each current process as the next shift schedule; The execution module is used to perform scheduling processing on the pulsed production line according to the next scheduling plan; The processing module is configured to, if there are still unstarted processes on the pulsed production line, return to the step of obtaining static information on the current scheduling problem of the pulsed production line; otherwise, terminate the scheduling of the pulsed production line. Before obtaining the static information of the current scheduling problem of the pulsed production line, the method further includes: Based on historical pulsed production line assembly data, construct the resource prediction training set and the process scheduling training set; Based on reinforcement learning algorithms, an initial resource prediction agent and an initial process scheduling agent are constructed. Based on the resource prediction training set and the process scheduling training set, the initial resource prediction agent and the initial process scheduling agent are trained alternately until the resource prediction agent and the process scheduling agent are obtained. The step of determining the next scheduling plan based on the probability value of each current process as the next scheduling step includes: If the variance of the probability value of each current process as the next scheduling step is greater than the first threshold, then the next scheduling scheme is determined based on the multiple reasoning method of search. If the variance of the probability value of each current process as the next scheduling step is not greater than the first threshold, then based on the data augmentation method, multiple sub-problems corresponding to the current scheduling problem are constructed; the solution of each sub-problem is calculated to obtain multiple equivalent solutions corresponding to the current scheduling problem; the multiple equivalent solutions corresponding to the current scheduling problem are evaluated through an evaluation function, and the optimal solution is used as the next scheduling scheme.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the pulsed production line scheduling method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pulsed production line scheduling method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the pulsed production line scheduling method as described in any one of claims 1 to 4.

Images