Inconsistent batched lot flow flexible job shop scheduling method and system integrating mathematical model and reinforcement learning, and storage medium

Abstract

The application belongs to the field of workshop scheduling, and particularly discloses a method and system for inconsistent batched batch flow flexible job shop scheduling integrating mathematical model and reinforcement learning, and a storage medium, which comprises the following steps: firstly, a strict workshop state representation is introduced, which contains information such as decision-related processes, machines and batches, and a double attention network is used to extract state feature information; then, a scheduling decision framework based on a near segment strategy optimization algorithm is designed to solve the process sequencing sub-problem and obtain a batch division adaptive coefficient; finally, considering the processing preparation time, a process-level batch machine collaborative optimization mixed integer linear programming model is constructed to realize comprehensive optimization of the batch division sub-problem and the machine selection sub-problem. The scheduling method has significant advantages in solving the batch flow flexible job shop scheduling problem, and can quickly solve the inconsistent batched batch flow flexible job shop scheduling problem to obtain a high-quality workshop scheduling method.

Description

Technical Field

[0001] This invention belongs to the field of shop floor scheduling, and more specifically, relates to a flexible job shop scheduling method, system, and storage medium that integrates mathematical models and reinforcement learning for inconsistent batch flow. Background Technology

[0002] As a core component of intelligent manufacturing, intelligent optimization decision-making drives the execution of production activities and has a significant impact on enterprise production efficiency. Production scheduling, which allocates limited resources to different tasks within a reasonable timeframe under certain conditions to meet the goals of optimization decision-making, is a key core technology for production planning and control in manufacturing enterprises. With the widespread application of multi-functional equipment such as CNC machine tools and machining centers, and the rise of personalized and customized production models, the Flexible Job Shop Scheduling Problem (FJSP) has emerged.

[0003] In traditional flexible workshop scheduling, a workpiece cannot be transferred to the next machine for further processing until its current step is completed. However, manufacturing enterprises typically operate on an order-based production model, where orders often contain several identical workpieces. Processing the entire order as a single workpiece would fail to fully utilize the processing capacity of the flexible workshop equipment, resulting in prolonged equipment idle time. Conversely, splitting all workpieces in an order into individual workpieces for scheduling and production presents a significant challenge to scheduling algorithms due to the increased problem scale. This could prevent the scheduling algorithm from finding an optimal solution. More seriously, frequent workpiece switching during actual production leads to increased switchover time, higher transfer costs, reduced processing efficiency, difficulty in guaranteeing processing quality, and difficulties in various aspects of production organization and management (including material preparation, shop floor logistics, and delivery management). Therefore, the common approach is to split orders into sub-batch processing, i.e., batch flow production. Batch flow is classified according to two criteria: (1) based on whether the batch size of each sub-batch of the same workpiece is the same, it can be divided into equal batching and unequal batching; (2) based on whether the batching strategies of different processes of the same workpiece are the same, it can be divided into consistent batching and inconsistent batching.

[0004] Batch flow flexible workshops are very common in real-world production. Current research on this problem mainly focuses on consistent batching, which suffers from insufficient batch flexibility and low machine utilization. Research on inconsistent batching is limited. Zhang Chunjiang, Fan Jiaxin, and others fixed the machine allocation and processing order variables of each sub-batch in the elite solution, constructing a mixed-integer linear programming model with sub-batch batch size as one of the decision variables. They then used an intelligent optimization algorithm to iteratively solve the mixed-integer linear programming model. While fixing some variables in the elite solution can accelerate local optimization, it may limit the search space and affect global optimality. Furthermore, the encoding / decoding-based intelligent optimization algorithm requires a long iteration time to find a satisfactory solution, especially as the problem size increases, its time complexity rises significantly.

[0005] Reinforcement learning (RL) teaches strategies through interaction between an agent and its environment. When the state changes, RL can quickly generate decisions using pre-trained strategies, offering greater real-time performance and adaptability. With the development of computer science, deep neural networks have been introduced into RL to handle continuous state and action spaces, known as deep reinforcement learning (DRL). Introducing neural networks not only effectively addresses the challenges of continuous state and action spaces but also significantly improves the model's generalization ability and learning performance, resulting in more flexible and intelligent decision-making capabilities. Li Junqing et al.'s deep reinforcement learning framework based on graph neural networks optimizes the solution of the flexible job shop scheduling problem for batch flow. While maintaining a constant material quantity in each sub-batch across all batches, their invention limits the adjustment capability of sub-batches in state representation and action decision-making, leading to insufficient resource utilization of some machines and deficiencies in flexibility and resource utilization. Therefore, designing a fast and high-quality solution for flexible job shops with inconsistent batch flow is of great significance. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a flexible job shop scheduling method, system, and storage medium that integrates mathematical models and reinforcement learning for inconsistent batch flow. The aim is to quickly solve for a high-quality job shop scheduling method under inconsistent batch conditions, thereby improving the efficiency of job shop processing.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a flexible job shop scheduling method integrating mathematical models and reinforcement learning for inconsistent batch flow is proposed, comprising the following steps:

[0008] The flexible job shop scheduling problem is represented as a heterogeneous graph, where processes and machines are different types of nodes, and the processing relationship between processes and machines is represented as edges. The flexible job shop scheduling problem is transformed into a Markov decision process, and an action space, reward function, and decision network are designed. The action space is a combination of processes to be processed and batch partitioning coefficient γ, which is used to allocate processes to machines with lower processing time costs. The decision network is constructed based on an actor-commentator network.

[0009] A mixed-integer linear programming model for process scheduling is established, and the objective function of the mixed-integer linear programming model is constructed based on the batch partitioning coefficient.

[0010] Reinforcement learning training in a simulated environment is performed using a proximal policy optimization algorithm, including:

[0011] S1. Extract environmental state features based on the current heterogeneous graph;

[0012] S2. The actor network makes decisions based on environmental state characteristics to determine the current processing step and batch division coefficient;

[0013] S3. For the current process to be processed, solve the mixed integer linear programming model to obtain the scheduling scheme, that is, the batch division and machine selection of the process to be processed;

[0014] S4. Update the heterogeneous graph based on the scheduling scheme and optimize the decision network parameters according to the reward function;

[0015] S5. Repeat S1 to S4 until all processing steps are decided, thus obtaining the flexible workshop scheduling method.

[0016] As a further preferred embodiment, the action space A(t) is represented as:

[0017] A(t)={(O i,j ,γ)|O i,j ∈J c (t),γ∈Γ}

[0018] Among them, O i,j J represents the j-th process of the i-th type of operation, i.e., the process to be processed. c (t) represents the set of candidate processes at time t; Γ represents the discretized set of batch partitioning coefficient γ, Γ={γ1,γ2,…,γ l}, v = 1, 2, ..., l, where l represents the number of discretizations, γ v ∈[0,1].

[0019] As a further preferred embodiment, the mixed-integer linear programming model employs batch partitioning variables, machine selection variables, and completion time variables in a coordinated manner, with the optimization objective being to minimize the maximum completion time and processing preparation time.

[0020] As a further preferred option, for the j-th process O of the i-th type of operation i,j The objective function of the mixed-integer linear programming model is:

[0021]

[0022] Among them, C i,j O i,j Maximum completion time of sub-batch, B i,j O i,j Total number of sub-batches, S i,j,k O i,j In M i,j,k Preparation time for processing, x i,j,k O i,j Is it in M? i,j,k Upper processing, M i,j,k O i,j The kth processable machine.

[0023] As a further preferred option, when using the near-end strategy optimization algorithm for reinforcement learning training, the objective function is to minimize the total completion time of the workpiece.

[0024] As a further preferred embodiment, the reward function r t Represented as:

[0025]

[0026] Among them, C((O) i,j ,γ),s t ), C((O) i,j ,γ),s t+1 ) represent process O respectively i,j The estimated completion times at time step t and time step t+1 are s t s t+1 These represent the states at time step t and time step t+1, respectively.

[0027] As a further preferred embodiment, the environmental state characteristics include process characteristics. Machine characteristics Sub-batch features Process machine for features and global features h G .

[0028] As a further preferred embodiment, the method for determining the environmental state characteristics is as follows:

[0029] Based on the current heterogeneous graph, obtain the initial process features. Initial machine characteristics Sub-batch features Features of the machine in the process

[0030] Initial process characteristics Initial machine characteristics Pay attention to the network in the input graph to obtain process features. Machine characteristics

[0031] Process characteristics Machine characteristics Features of Sub-batch After performing average pooling separately, the data are concatenated to obtain the global feature h. G .

[0032] According to a second aspect of the present invention, an inconsistent batch flow flexible job shop scheduling system integrating mathematical models and reinforcement learning is provided, comprising a processor for executing the aforementioned inconsistent batch flow flexible job shop scheduling method integrating mathematical models and reinforcement learning.

[0033] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described integrated mathematical model and reinforcement learning-based inconsistent batch flow flexible job shop scheduling method.

[0034] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0035] This invention introduces a batch partitioning coefficient and, based on a scheduling decision framework using a near-segment strategy optimization algorithm, solves the process sequencing subproblem and obtains the adaptive batch partitioning coefficient. Then, combined with a mixed-integer linear programming model, it achieves comprehensive optimization of both the batch partitioning and machine selection subproblems. This invention's method has significant advantages in solving the batch flow flexible job shop scheduling problem, enabling rapid solution of a high-quality job shop scheduling method under inconsistent batching conditions, thereby improving job shop processing efficiency. Attached Figure Description

[0036] Figure 1 This is a flowchart of the inconsistent batch flow flexible job shop scheduling method integrating mathematical models and reinforcement learning, as described in an embodiment of the present invention.

[0037] Figure 2 This diagram illustrates network feature extraction in an embodiment of the present invention.

[0038] Figure 3This is a schematic diagram of the action space of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0040] This invention provides a flexible job shop scheduling method for inconsistent batch flow that integrates mathematical models and reinforcement learning.

[0041] For the flexible job shop scheduling problem involving inconsistent batch flow: In the shop, there are n types of jobs J = {J1, J2, ..., J...} n} and m machines M = {M1, M2, ..., M} m}, the i-th type of assignment J i The CCP has D i The same assignment, J i A single job contains N i Each process To be processed in a given order, a sub-batch is the smallest unit of processing.

[0042] Each process O i,j Divided into B i,j Individual batch, B i,j,k Indicates process O i,j The kth sub-batch. i,j The set of machinable machines is represented as x i,j,k O i,j Is it in M? i,j, Upper processing, O i,j In M i,j,k The size of the sub-batch processed above is represented by σ. i,j,k O i,j In M i,j,k The sub-batch processed by τ i,j,k O indicates i,j In M i,j,k The processing time is represented by P. i,j,k O i,j In M i,j,k The earliest available time for processing is denoted as ST. i,j,k M i,j,k The preparation time is denoted as S. i,j,k c i,j,k O i,j In M i,j,k The processing time, C ijO i,j The complete processing time.

[0043] The scheduling problem in this embodiment has the following characteristics:

[0044] 1) All devices are available at time 0;

[0045] 2) All tasks can be started at time 0;

[0046] 3) The processing sequence of the processing task cannot be changed, and there is no sequential constraint on the processing of different sub-batches;

[0047] 4) Workpieces within each sub-batch of the task process must be processed continuously without interruption;

[0048] 5) A sub-batch can only be transferred to the next process after all workpieces have completed one process.

[0049] 6) A sub-batch of a certain process in a task can come from multiple sub-batches in the previous process. In this case, the processing of this sub-batch can only begin after the processing of these multiple sub-batches in the previous process is completed.

[0050] 7) The equipment buffer is infinitely large, and the transfer time is ignored;

[0051] 8) The same equipment can only process one operation of a workpiece in a certain sub-batch within the same time period;

[0052] 9) A workpiece in the same sub-batch can only be processed by a certain machine in one operation within the same time period.

[0053] To realize an inconsistent batch flow flexible job shop scheduling method integrating mathematical models and reinforcement learning, this invention makes the following design:

[0054] I. Introducing State Representation and Extracting State Feature Information Based on Graph Attention Network

[0055] The state space includes information such as processes, machines, batch sizes, and equipment availability, used to describe the current scheduling state of the workshop. Based on the process-machine structure of the scheduling problem, a heterogeneous graph is constructed, treating processes and machines as different types of nodes, and the processing relationships between processes and machines as edges. Edge weights represent process-machine information (such as processing time). A Graph Attention Network (GAT) is employed to capture the interaction relationships between nodes and extract global state information; GAT aggregates the features of neighboring nodes through an attention mechanism to generate high-dimensional feature representations of each node, used for subsequent process ranking decisions.

[0056] 1. State representation

[0057] The state space design in this invention comprehensively covers the dynamic information of workshop scheduling, specifically including elements such as processes, workpieces, machines, and batches.

[0058] Process characteristics describe O i,j The status includes information such as its progress and remaining workload, specifically including: whether the process has been scheduled, the earliest completion time lower bound of the process, the processing time span, the number of remaining processes in the job, the waiting time of the process, the remaining workload of the process, the number of remaining processes in the job, the remaining workload of the job, and the number of available machines.

[0059] Machine characteristics describe the machine's status and resource availability, mainly including: the number of jobs currently available on the machine, the number of operations currently available on the machine, the minimum processing time of the machine, the remaining workload of the machine, the idle time of the machine, and whether the machine is currently working.

[0060] Sub-batch features describe the attributes of each sub-batch, mainly including: the earliest completion time lower bound of the sub-batch, the average processing time of the sub-batch, the processing time span of the sub-batch, the maximum processing time of the sub-batch, the minimum processing time of the sub-batch, the average preparation time of the sub-batch, the maximum preparation time of the sub-batch, the minimum preparation time of the sub-batch, and the preparation time span of the sub-batch.

[0061] The process machine pair features represent the relationship between each process and the machine, mainly including: the processing time of the current candidate sub-batch, the maximum sub-batch processing time, the maximum processing time of the current candidate sub-batch, the maximum preparation time of the current candidate sub-batch, the minimum preparation time of the current candidate sub-batch, and the preparation time span of the current candidate sub-batch.

[0062] The four types of features (process features, machine features, sub-batch features, and process-machine pair features) comprehensively describe the status of processes, machine availability, process-machine matching, and sub-batch processing characteristics in job scheduling. By effectively combining and normalizing these features, the model can better understand and predict the system's scheduling behavior, optimize scheduling decisions, and improve resource utilization and job completion efficiency. Furthermore, feature normalization helps avoid the impact of scale differences on model training, thereby improving the model's stability and generalization ability.

[0063] 2. Construction of Heterogeneous Diagrams for Flexible Workshops

[0064] Heterogeneous graph structures are used to model operations and machines in the Flexible Job Shop Scheduling Problem (FJSP), treating operations and machines as different types of nodes. Operation nodes represent the specific operations of each job, including information such as operation number, demand, and processing time; machine nodes contain information such as machine number, remaining workload of the machine, and whether the machine is currently working.

[0065] In heterogeneous graphs, edge features represent the relationship between processes and machines. Edge weights typically reflect the processing time of a process on a given machine or the matching correlation between the two. This provides a clear basis for optimizing batch partitioning coefficients. Specifically, smaller edge weights indicate a higher matching degree between the machine and the process, and a shorter processing time, thus helping to determine the optimal batch partitioning coefficient and ensuring that sub-batch processing is more effectively allocated to resources. Conversely, larger edge weights indicate a lower matching degree between the process and the machine, or a longer processing time, suggesting that the machine should be avoided during batch partitioning.

[0066] Edges can be categorized into two types: connecting edges and scheduling edges. Connecting edges represent the potential processing relationships between operations and machines, while scheduling edges represent the actual scheduling and execution order of a particular operation on a specific machine. Updating scheduling edges reflects the job scheduling process. Scheduled operations update the direction of their edges to reflect their execution order, thus transforming the graph structure into a directed acyclic graph (DAG), ensuring no circular dependencies occur during scheduling. This graph structure allows for flexible management and optimization of job shop scheduling, taking into account factors such as machine availability, operation dependencies, and job requirements.

[0067] 3. Feature Extraction

[0068] Graph Attention Networks (GAT) dynamically extract the interaction features between process nodes and machine nodes through an attention mechanism. They focus on capturing the complex dynamic interactions between processes and machines, and employ a dual-attention network design: a process attention block and a machine fit attention block, which respectively handle specific features of the process and the machine. For example... Figure 2 As shown, the specific implementation method is as follows:

[0069] (1) Model Input

[0070] Initial process characteristics Each process node The input features are represented as This includes information such as processing time and priority.

[0071] Initial machine characteristics Each machine node The input features are represented as This includes information such as current load and idle time;

[0072] Sub-batch features Each B i,j,k Sub-batch characteristics are denoted as in This includes information such as sub-batch processing time and preparation time.

[0073] Process machine for features This includes information such as the processing time of the current candidate sub-batch and the preparation time span of the candidate sub-batch.

[0074] (2) Process attention block

[0075] This module captures the characteristics of processes within the same job by analyzing the sequential relationships between processes, and models their priorities.

[0076] Attention coefficient calculation: for process O i,j Input features and its neighboring nodes O i,j-1 and O i,j+1 Calculate the attention coefficient:

[0077]

[0078]

[0079] in For linear transformation, This represents the attention weight.

[0080] Dynamic masking and normalization: If a node O i,j-1 Or O i,j+1 If it is missing, then its corresponding e i,j-1 or e i,j+1 Set a mask. Softmax normalize the attention scores to obtain the attention coefficient α. i,j-1 α i,j+1 .

[0081] Feature update: using normalized attention coefficients α i,j-1 α i,j+1 The neighbor features are weighted and summed, and the output features are updated using the linear activation function LeakyReLU:

[0082]

[0083] If a neighboring node is missing, its corresponding attention score and feature weighting are set to 0. This allows for direct calculation of the attention scores of predecessors and successors and their impact on the features of the current node. This method is more compact while maintaining effective modeling of the relationships between nodes.

[0084] (3) Machine adaptability attention block

[0085] The Machine Fit Attention Module aims to model the fit relationships and dynamic competition between machines, thereby supporting the generation of high-quality machine features. In production scheduling, different machines may differ in their manufacturability, current state, and processing efficiency for unscheduled processes. Therefore, the Machine Fit Attention Module captures the dynamic fit between machines by explicitly modeling this competitive relationship.

[0086] Fit definition: Assume C k,q Indicates machine M k and M q The fit set between them contains unscheduled processes that can be jointly processed by the two machines; c k,q The compatibility weights between them are defined as follows: Among them, J C Represents the set of currently unscheduled operations, when O i,j ∈C k,q When c is empty k,q It is 0.

[0087] Attention coefficient calculation: M per machine k Computation and other machines M q Attention weights u k,q The calculation formula is as follows: in, It is machine M k and M q Features, c k,q These are their adaptation weights. and It is a linear transformation matrix. It is a weight vector.

[0088] Normalization and Feature Update: The attention weights are normalized to β using softmax. k,q , indicating machine M k For other machines M q Adaptability, Machine M k The updated features are obtained by weighted summation: Where, N k Is with M k A set of machines that have a compatibility relationship.

[0089] (4) Global features

[0090] Process O i,j Machine M k The initial features are respectively The features of the process and the machine are updated respectively after passing through an L-layer attention network. right Average pooling was applied separately, and the results of the pooling were expressed as follows: The global features are then obtained; these global features are then concatenated to form the global features of a complete FJSPLS example. The specific expression is as follows:

[0091]

[0092] II. Process Sequencing Decision Based on Proximity Strategy Optimization

[0093] This invention designs a process sequencing decision network based on proximal policy optimization to optimize the batch flow flexible job shop scheduling problem (FJSPLS). A deep reinforcement learning framework based on proximal policy optimization (PPO) is employed. This framework leverages the size independence of the attention model and combines it with the proximal policy optimization algorithm to achieve efficient decision-making. Specifically, the reinforcement learning policy selects the current processing process based on feature information and generates adaptive batch partitioning coefficients. The objective function is to minimize the total job completion time. Reinforcement learning is trained in a simulated environment, and the policy network continuously optimizes through a feedback loop of state-action rewards. After training, the policy model can be deployed for real-time decision-making and scheduling.

[0094] Specifically, such as Figure 3 As shown, the flexible job shop scheduling problem is transformed into a Markov decision process:

[0095] 1. Motion space

[0096] In a flexible batch production workshop, the processing time for different sub-batches on different machines may vary. How can we dynamically adjust the batching strategy to ensure more efficient task processing on different machines and avoid resource waste? Introducing a batch partitioning coefficient (γ) helps optimize overall production scheduling. The batch partitioning coefficient γ represents the ratio of the weight of the process completion time to the weight of the process preparation time, and is used to allocate processes to machines with lower processing time costs.

[0097] Specifically, the discretized set Γ = {γ1, γ2, ..., γ} of the batch partitioning coefficient γ l}, v = 1, 2, ..., l, where l represents the number of discretizations, γ v ∈[0,1]. The larger the number of discretizations l, the higher the degree of batch refinement, but the greater the difficulty of convergence of the reinforcement learning network. Therefore, the value of l is determined according to the actual situation and specific needs.

[0098] The action space A(t) is designed as a combination of the process to be processed and the batch division coefficient, expressed as:

[0099] A(t)={(Oi,j ,γ)|O i,j ∈J c (t),γ∈Γ}

[0100] Among them, J c (t) represents the set of candidate processes at time t.

[0101] 2. State transition

[0102] At each time step t, perform an action a. t This action corresponds to selecting a process O to be processed. i,j Process it with the corresponding batch partitioning coefficient γ. After executing a t The system will update the status s t The set of processes in Machine collection Sub-batch set and action set A t And obtain the new state s based on the new production state. t+1 .

[0103] 3. Reward Design

[0104] Reward r t It should be designed to guide the agent in choosing a path that helps reduce the maximum completion time (i.e., makespan, denoted as C) for all task operations. max The action of ) is predicted using heuristic rules. i,j The completion time C((O) at time t i,j ,γ),s t Reward r t Defined as the difference between the estimated completion time at step t and t+1, and standardized, it can be expressed as:

[0105]

[0106] 4. Strategy

[0107] This invention employs a random strategy π(a) t |s t The distribution of these distributions is generated by a deep reinforcement learning algorithm (in this invention, an actor critic network) and has trainable parameters θ. Given a state s... t In this case, the distribution will return the selection of each action a. t The probability of ∈A(t).

[0108] 5. Decision-making network

[0109] This invention designs a decision network based on an actor-critic model, preserving the size-independent property of the attention model. It uses two multilayer perceptrons (MLPs) as the actor and critic, with parameters θ and θ, respectively.

[0110] The actor network aims to generate a random policy π in two steps. θ (a t |s t First, for each with a t All features related to A(t) are concatenated. To reduce the action space size, feature selection (FS) is performed in the candidate region based on the "remaining maximum workload". The selected feature vector is then input into the MLP. θ In this process, we obtain the vector μ(a) t |s t ), represented as: Then use the softmax function to output the desired distribution, represented as: Commentator networks with global features h G Generate a vector from the input. As an estimate of the state value.

[0111] III. MILP Model for Process-Level Batch Machine Collaborative Optimization

[0112] A process-level batch machine collaborative optimization mixed-integer linear programming (MILP) model is constructed to simultaneously solve the batch partitioning and machine selection subproblems. The model aims to minimize the maximum completion time and processing setup time (including changeover setup time for different workpieces processed on different machines). It employs collaborative modeling of batch partitioning variables, machine selection variables, and completion time variables, ensuring logical consistency between processing requirements, batch partitioning, and machine selection through constraints. Through dynamic batching and flexible allocation strategies, the model achieves comprehensive optimization of both the batch partitioning and machine selection subproblems, improving workshop processing efficiency and equipment utilization.

[0113] Specifically, regarding O i,j Establish a process-level batch machine collaborative optimization model that can optimize batch partitioning (number of sub-batches, sub-batch size), machine allocation, and processing time under given job and machine constraints, while taking into account O(n) time constraints. i,j Makespan and machine utilization during processing. The batch adjustment coefficient in the model allows for flexible adjustment of the optimization objective according to actual needs.

[0114] 1. Variable Definition

[0115] Table 1 Variable Definitions

[0116]

[0117] 2. Objective function

[0118] The introduction of a batch partitioning factor, by influencing the number of sub-batches, can effectively reduce idle machine resources and uneven load distribution, while also reducing additional setup time costs, such as machine adjustments and task switching. The introduction of the batch partitioning factor makes the scheduling framework more flexible, enabling it to dynamically adjust the batching strategy based on different production needs and resource constraints, ensuring optimal production scheduling decisions. Therefore, the objective function is expressed as:

[0119]

[0120] 3. Constraints

[0121] The total quantity of each workpiece in a sub-batch is equal to J. i Total number of assignments:

[0122]

[0123] Sub-batch completion time constraints:

[0124]

[0125] Batch partitioning is only effective when a machine is selected:

[0126]

[0127] Process O i,j Maximum completion time C i,j Greater than the completion time of any of its sub-batches:

[0128]

[0129] 4. Solution process

[0130] To solve this batch machine collaborative optimization problem at the process level, a mixed-integer linear programming (MILP) model was constructed and solved using the Gurobi optimizer. Through optimization, the model calculated the number of workpieces σ allocated to each machine. i,j,k And the processing time σ for each machine i,j,k ·P i,j,k Two key variables. These two optimization results are then passed to the environment, which updates its state based on this information.

[0131] IV. Scheduling Methods

[0132] Based on the designs described in sections one through three above, when performing flexible job shop scheduling for inconsistent batch flow in embodiments of the present invention, such as... Figure 1 As shown, it includes the following steps:

[0133] Reinforcement learning training is performed in a simulated environment using a proximal policy optimization algorithm:

[0134] S1. Extract environmental state features based on the current heterogeneous graph;

[0135] S2. The actor network makes decisions based on environmental state characteristics to determine the current processing step and batch division coefficient;

[0136] S3. For the current process to be processed, solve the mixed integer linear programming model to obtain the scheduling scheme, that is, the batch division and machine selection of the process to be processed;

[0137] S4. Update the heterogeneous graph based on the scheduling scheme and optimize the decision network parameters according to the reward function;

[0138] S5. Repeat S1 to S4 until all processing steps are decided, thus obtaining the flexible workshop scheduling method.

[0139] V. Numerical Experiments and Analysis

[0140] The effectiveness and advancement of the proposed method are verified through ablation experiments and numerical comparisons with several baseline algorithms. Several popular baseline algorithms were selected, including the traditional Priority Dispatching Rules (PDR), the exact Gurobi algorithm, and a genetic algorithm employing an advanced batching strategy. The main objective of the experiments was to evaluate the advantages of the proposed algorithm in terms of scheduling performance, generalization ability, and computational efficiency.

[0141] 1. Test Case

[0142] The examples are derived from an extension of the classic Flexible Job Shop Scheduling Problem (FJSP) examples, and the baseline examples are configured as follows:

[0143] Number of assignments D i ∈[200,400], machine initial setup time S i,j ∈[50,100], machine setting time S i,j,i′,j′ ∈[100,200], processing time P (i,j,k) ∈[3,7].

[0144] The specific sizes of the training examples are 10×5, 15×5, 15×10, 20×5, 20×10, 30×10, 40×20, and 60×20. In this invention, training examples are randomly generated from benchmark examples of corresponding sizes, while test and validation examples are pre-generated from each benchmark example.

[0145] 2. Training Configuration

[0146] The training process is configured as follows: training period (Nep) = 3000, training set size |Etr| = 5, reset frequency (Nr) = 3, validation set size (Nval) = 20, weight factor dimension n = 21.

[0147] When the number of candidate jobs is no more than 20, the job dimension is selected as 10; otherwise, it is 20. The output dimension of the first layer of the attention network is set to 64. (MLP) θ and All layers are set to 3, with the hidden layer having a dimension of 64.

[0148] For the parameter configuration based on the genetic algorithm: Termination condition: number of job types * number of machines * 10s, population size: 600, crossover probability: 0.9, mutation probability: 0.2, number of elite individuals: 6, tournament selection size: 2.

[0149] For the parameter configuration of the PPO algorithm: policy coefficient α policy =1, value coefficient α value =0.5, entropy coefficient α entropy =0.01, clipping parameter ∈ =0.2, GAE parameter λ =0.95, discount factor γ =0.99, learning rate lr =5×10 4 .

[0150] All experiments were conducted on machines equipped with an Intel(R) Core(TM) i5 12600KF CPU and an NVIDIA GeForce RTX4060Ti 16G GPU.

[0151] 3. Baseline Algorithm and Performance Metrics

[0152] To compare the performance of the proposed algorithm, four types of priority dispatch rules (PDRs) for process ordering and three types of batch partitioning PDRs were selected, totaling 12 composite scheduling rules:

[0153] Operation Sequencing PDRs: FIFO (First In, First Out): Selects the earliest ready candidate operation and the earliest ready compatible machine; MOPNR (Most Operations Remaining): Selects the candidate operation with the most subsequent operations and selects the machine that can be processed immediately; SPT (Shortest Processing Time): Selects the compatible operation-machine pair with the shortest processing time; MWKR (Most Work Remaining): Selects the operation with the longest average processing time of subsequent operations and selects the machine that can be processed immediately.

[0154] Batch-Machine Collaborative Division of PDRs: Binary Batching: Dividing tasks into two batches with the quantity as even as possible; Equal-Quantity Inconsistent Batching: Minimizing the completion time of each process while ensuring that the processing quantity on each machine is distributed as evenly as possible; Unequal-Quantity Inconsistent Batching: Minimizing the completion time of each process and performing unequal batching based on the machine's load.

[0155] Each composite scheduling rule was run 20 times independently in the experiment, and the average result of each experiment was recorded to evaluate the performance of the algorithm.

[0156] 4. Experimental Results and Analysis

[0157] Through the above experiments, a detailed comparison was made between the proposed algorithm and the baseline algorithm. Table 2 shows the scheduling performance of the algorithm and the composite rule, demonstrating the algorithm's strong solution capability. Table 3 shows the performance of the algorithm on examples of different sizes and characteristics, indicating its strong generalization ability. Table 4 compares the computation time of different intelligent optimization algorithms, showing that the proposed algorithm has higher computational efficiency when solving large-scale examples. These experiments verify that the proposed algorithm can provide a more efficient and accurate solution than the baseline algorithm when dealing with complex scheduling problems.

[0158] Table 2 Comparison of Experiments with Composite Scheduling Rules

[0159]

[0160] Table 3. Generalization Experiment Comparison with Composite Scheduling Rules

[0161]

[0162] Table 4 Comparison of Intelligent Optimization Algorithm with Intelligent Optimization Algorithm

[0163]

[0164] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A flexible job shop scheduling method integrating mathematical models and reinforcement learning for inconsistent batch flow, characterized in that, Includes the following steps: The flexible workshop scheduling problem is represented as a heterogeneous graph, where processes and machines are different types of nodes, and the processing relationship between processes and machines is represented as edges. The flexible job shop scheduling problem is transformed into a Markov decision process, and the action space, reward function, and decision network are designed; where the action space consists of the processes to be processed and the batch partitioning coefficients. The combination of batch partitioning coefficients Used to assign processing steps to machines with lower processing time costs; the decision network is built based on an actor-commentator network; The action space Represented as: in, Indicates the first i The first type of assignment j The next process step, also known as the process to be processed. express The set of candidate processes at any given time; Indicates batch partitioning coefficient Discretized sets, ,in Indicates the number of discretizations. ; A mixed-integer linear programming model for process scheduling is established, and the objective function of the mixed-integer linear programming model is constructed based on the batch partitioning coefficient. For the i The first type of assignment j Steps The objective function of the mixed-integer linear programming model is: in, express Maximum completion time for sub-batch express Total number of sub-batch express exist Preparation time for processing express Is it in Upward processing, express The k One machine that can be processed; Reinforcement learning training in a simulated environment is performed using a proximal policy optimization algorithm, including: S1. Extract environmental state features based on the current heterogeneous graph; S2. The actor network makes decisions based on environmental state characteristics to determine the current processing step and batch division coefficient; S3. For the current process to be processed, solve the mixed integer linear programming model to obtain the scheduling scheme, that is, the batch division and machine selection of the process to be processed; S4. Update the heterogeneous graph based on the scheduling scheme and optimize the decision network parameters according to the reward function; S5. Repeat S1 to S4 until all processing steps are decided, thus obtaining the flexible workshop scheduling method.

2. The method for flexible job shop scheduling based on inconsistent batch flow using an integrated mathematical model and reinforcement learning as described in claim 1, characterized in that, The mixed-integer linear programming model employs batch partitioning variables, machine selection variables, and completion time variables in a coordinated manner, with the optimization objective being to minimize the maximum completion time and processing preparation time.

3. The method for flexible job shop scheduling based on inconsistent batch flow using an integrated mathematical model and reinforcement learning as described in claim 1, characterized in that, When using the near-end policy optimization algorithm for reinforcement learning training, the objective function is to minimize the total completion time of the workpiece.

4. The inconsistent batch flow flexible job shop scheduling method based on integrated mathematical models and reinforcement learning as described in claim 1, characterized in that, The reward function Represented as: in, , Each represents a process. At time step Time step The estimated completion time, , These represent the time steps. Time step The state.

5. The integrated mathematical model and reinforcement learning-based inconsistent batch flow flexible job shop scheduling method as described in any one of claims 1-4, characterized in that, The environmental state characteristics include process characteristics. Machine characteristics Sub-batch features Process machine for features and global features .

6. The inconsistent batch flow flexible job shop scheduling method based on integrated mathematical models and reinforcement learning as described in claim 5, characterized in that, The environmental state characteristics are determined as follows: Based on the current heterogeneous graph, obtain the initial process features. Initial machine characteristics Sub-batch features Features of the machine in the process ; Initial process characteristics Initial machine characteristics Pay attention to the network in the input graph to obtain process features. Machine characteristics ; Process characteristics Machine characteristics Features of Sub-batch After performing average pooling separately, the data are concatenated to obtain the global features. .

7. A flexible job shop scheduling system integrating mathematical models and reinforcement learning for inconsistent batch flow, characterized in that, The system includes a processor for executing the inconsistent batch flow flexible job shop scheduling method based on an integrated mathematical model and reinforcement learning as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the inconsistent batch flow flexible job shop scheduling method of the integrated mathematical model and reinforcement learning as described in any one of claims 1-6.

Images