A structure-aware and heuristic-guided flexible job-shop scheduling method

Abstract

The application discloses a flexible job shop scheduling method based on structure perception and heuristic guidance, comprising the following steps: constructing a scheduling model to minimize the maximum completion time; constructing a job-process-machine heterogeneous graph; generating a candidate action set; extracting and fusing multi-hop information of the heterogeneous graph through hierarchical ring diffusion coding; extracting heuristic features to construct action representation; jointly modeling structure and heuristic representation to output scores; selecting scheduling actions according to the scores, updating the environment and network parameters, and recycling to output a scheme. The application can accurately depict multi-hop structure dependence, effectively improve state representation accuracy, action discrimination and decision stability, overcome the defects of insufficient long-range dependence representation, insufficient utilization of heuristic information and unstable action sorting in the prior art, adapt to different scale and complex scheduling scenarios, and significantly improve scheduling performance and robustness.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing scheduling technology, and in particular to a flexible job shop scheduling method based on structure perception and heuristic guidance. Background Technology

[0002] Currently, in the field of flexible job shop scheduling, the Flexible Job Shop Scheduling (FJSP) problem, where multiple candidate machines can be selected for each process, requires not only determining the processing order of processes but also selecting suitable processing equipment for each process. Furthermore, this problem is subject to various constraints such as process sequence relationships, machine capacity, resource conflicts, and machine heterogeneity, making it a typical NP-hard problem characterized by a large solution space, high combinatorial complexity, and difficulty in solving. With the continuous increase in the number of jobs, machine scale, and process flexibility, the problem scale expands rapidly, and traditional methods struggle to meet actual production needs in terms of real-time performance and scalability. Existing exact algorithms, such as mixed-integer programming, dynamic programming, and branch and bound, can obtain relatively good or even optimal solutions in small-scale scenarios, but their computational overhead is high, making it difficult to meet the real-time scheduling requirements of complex manufacturing scenarios. While heuristic and metaheuristic methods can obtain feasible solutions in a short time, they typically rely on human experience, rule design, or parameter settings, and are prone to insufficient generalization ability and getting trapped in local optima in complex dynamic scenarios. Furthermore, in recent years, some methods have incorporated deep reinforcement learning into FJSP solving and combined it with graph neural networks to model processes, machines, and their constraints, thereby enhancing the representation of scheduling states. However, existing methods typically rely on stacked graph message passing layers to obtain long-distance structural information, which can easily lead to the mixing of structural features at different levels, affecting the state representation effect. Existing methods usually focus more on extracting graph structural state features, and do not make sufficient use of heuristic information directly related to scheduling decisions, such as the number of remaining processes, remaining processing time, candidate machine processing time, and earliest available machine time. They also lack mechanisms for effectively comparing and coordinating candidate action sets, thus affecting the stability of action ordering and the final scheduling effect. Therefore, existing technologies still have shortcomings in multi-hop structural dependency modeling, utilization of action-level heuristic information, and candidate action coordination and discrimination, and there is still room for further improvement. Summary of the Invention

[0003] The purpose of this invention is to provide a flexible job shop scheduling method based on structure awareness and heuristic guidance. Addressing the flexible job shop scheduling problem, the method represents the relationship between jobs, processes, and machines as a heterogeneous graph structure. During the reinforcement learning decision-making process, it simultaneously considers long-range structural information in the scheduling state and heuristic information directly related to candidate actions, thereby improving the decision quality and training stability in complex scheduling environments.

[0004] To achieve the above objectives, this invention provides a structure-aware and heuristic-guided flexible job shop scheduling method, comprising the following steps: S1. Construct a flexible job shop scheduling model, define jobs, processes, machines and their constraints, and take minimizing the maximum completion time as the scheduling optimization objective; S2. Construct a job-process-machine heterogeneous diagram based on the current scheduling status to describe the current workshop status; S3. Determine the schedulable processes based on the current scheduling status, and generate a set of candidate actions by combining the candidate machines corresponding to each schedulable process. S4. Using a hierarchical ring diffusion coding mechanism, neighborhood information with different hop counts in heterogeneous graphs is extracted and fused hierarchically to obtain the state representation result; S5. Extract heuristic features from candidate actions and construct action-level heuristic representations; S6. Use the dual context coordination mechanism to jointly model the state representation results obtained in step S4 and the heuristic representation obtained in step S5, and output the score results of each candidate action. S7. Select the scheduling action at the current moment based on the score output of the reinforcement learning strategy network, and apply the scheduling action to the flexible job shop scheduling environment. S8. The scheduling environment updates the process status, machine status, and scheduling time information according to the selected scheduling action, and outputs the corresponding reward value and the next time state. Based on the reward value and the next time state, the advantage function is calculated, the reinforcement learning policy network parameters are updated, and it is determined whether the current scheduling process has ended. If it has not ended, it returns to step S2 to continue execution. If it has ended, the final scheduling scheme is output.

[0005] Preferably, in step S2, when constructing the job-process-machine heterogeneous graph, the current scheduling state is represented as a bipartite heterogeneous graph, with the following formula: ; in, It is a two-part heterogeneous diagram as a whole. For a set of process nodes, For a set of machine nodes, For process priority constraint edge set, For each process - a set of machine-processable edges; The characteristics of a process node include whether the process has been completed, whether the process is in a schedulable state, the estimated processing time of the process, and the cumulative processing information of the unfinished processes in the corresponding job. Machine node characteristics include whether the machine is idle, the machine's current remaining processing time, the last time the machine became idle, and the machine's current load level. Process priority constraint edges represent the successor relationship between adjacent processes within the same operation; The process-machine machinable edge represents the processing adaptation relationship between a certain process and a certain machine, and carries the processing time of the corresponding process on the corresponding machine as an equal edge attribute.

[0006] Preferably, in step S4, when performing hierarchical extraction and fusion using the hierarchical ring diffusion coding mechanism, the following steps are performed: S41. Perform initial feature mapping on process nodes and machine nodes, project the two types of nodes onto a unified latent space to form a unified initial node representation. S42. Construct the enhanced heterogeneous adjacency matrix, the formula is: ; in, To enhance the overall heterogeneous adjacency matrix, This is a process-to-process adjacency submatrix. This is a process-machine adjacency submatrix. For machine-process adjacency submatrix, The machine-to-machine adjacency submatrix; It is obtained by adding the predecessor adjacency matrix and the successor adjacency matrix of the process, and the formula is: ; in, This is the predecessor adjacency matrix of the process. This is the adjacency matrix of the process successors; for The transpose of the matrix, It is a zero matrix.

[0007] Preferably, in step S4, when performing hierarchical extraction and fusion using the hierarchical ring diffusion coding mechanism, the following steps are performed after constructing the enhanced heterogeneous adjacency matrix: S43. Perform cyclic hop neighborhood decomposition to obtain cyclic hop adjacency matrices with different hop numbers; S44. Normalize each cyclic adjacency matrix to obtain the aggregation matrix, using the following formula: ; in, node With participating nodes Between the first Skip adjacency relationship For summation index, used to represent the summation index of nodes. In the same position Any node within the calculation range of the skip ring skip neighborhood, For the first Circular hop adjacency matrix This is a numerically stable term used to prevent the denominator from being zero; S45. Perform multi-hop diffusion on the initial node representation to obtain the aggregation results of each hop. Adaptively fuse the aggregation results of each hop using a hop-level attention mechanism to obtain the multi-hop fused representation.

[0008] Preferably, in step S4, when performing hierarchical extraction and fusion using the hierarchical ring diffusion coding mechanism, the following steps are performed after obtaining the multi-hop fused representation: S46. A two-stage hierarchical update mechanism is adopted to update the node representation. The first stage updates the machine node representation, and the second stage updates the process node representation based on the machine representation obtained in the first stage. S47. Perform average pooling on the machine node representation and the process node representation respectively, and then concatenate them to obtain the graph-level state representation.

[0009] Preferably, the heuristic features extracted in step S5 include the number of remaining operations, the remaining workload, the candidate machine processing time, and the earliest available time of the machine. When constructing an action-level heuristic representation, for any candidate action... Construct a four-dimensional heuristic feature vector using the following formula: ; in, For a single candidate action, For the first The first assignment One process, For the first 1 machine; Candidate actions The corresponding four-dimensional heuristic feature vector, This represents the number of processes that have not yet been completed between the start of the current process and the completion of its associated task. This represents the remaining workload of the current process. The processing time of the current process on the candidate machine. For machines The earliest available time at the current moment; The heuristic feature vectors are normalized within instances using the following formula: ; in, The normalized heuristic feature vector, This is the mean vector of heuristic features within the candidate action set for the current instance. The standard deviation vector of the heuristic features within the current instance's candidate action set. It is a numerically stable term; By concatenating the process embedding, machine embedding, process graph-level convergence vector, machine graph-level convergence vector, and normalized heuristic feature vector, we obtain the action enhancement representation.

[0010] Preferably, when performing joint modeling using the dual context coordination mechanism in step S6, the following steps are executed: S61. Construct a structural action representation for any candidate action. The structural action representation is composed of process embedding, machine embedding and graph-level state representation. Concatenate the structural action representation with the normalized heuristic features to obtain a joint action representation. S62. Use a shared parameter scoring network to score each joint action representation and calculate the attention weights in the candidate action set, using the following formula: ; in, Candidate actions The corresponding attention weights Candidate actions The corresponding feasibility mask has a value of 1 indicating that the action is legal and a value of 0 indicating that the action is illegal. Candidate actions The corresponding score for the combined action. For any candidate action in the candidate action set at the current time, This is the set of candidate actions for the current moment.

[0011] Preferably, in step S6, when performing joint modeling using the dual context coordination mechanism, the following steps are performed after calculating the attention weights: S63. Project the joint action representation onto the context space, and converge the candidate action set context vector according to the attention weights; S64. Calculate the weighted cost reference value of the candidate action set based on the same set of attention weights, using the following formula: ; in, This is the weighted cost reference vector for the candidate action set at the current moment. Candidate actions The corresponding attention weights Candidate actions The corresponding normalized heuristic feature vector; S65. Define a reference residual for each candidate action, using the following formula: ; in, Candidate actions The deviation vector relative to the set's average cost level. Candidate actions The corresponding normalized heuristic feature vector, This is the weighted cost reference vector for the candidate action set at the current moment; S66. The structural action representation, the normalized heuristic features, and the reference residual are concatenated to obtain the final action input representation.

[0012] Preferably, in step S8, when updating the reinforcement learning policy network parameters based on the reward value output by the scheduling environment and the state at the next time step, an Actor-Critic architecture is adopted. The policy network takes the final action input representation as input and outputs the action selection probability distribution. The value network takes the graph-level state representation and the candidate action set context vector as input and outputs the state value estimate.

[0013] Preferably, in step S8, the parameters of the reinforcement learning policy network and value network are updated using a proximal policy optimization method, with the objective function being: ; in, Let the objective function be the near-end policy optimization algorithm. These are the learnable parameters of the policy network; To Calculation of the expected value of the sampled trajectory at each time step. For the new and old strategies The probability ratio at time 1. for The estimated value of the dominance function at time 1; This is the clipping function, used to limit the range of fluctuation in the probability ratio. This is the cutting factor; The formula for the probability ratio of the new and old strategies is: ; in, and The current policy and the old policy are respectively in the state. Select action The probability of; During the training phase, actions are sampled and scheduled from the candidate action set based on the action probability distribution. During the inference phase, the candidate action with the highest probability is directly selected as the current scheduled action.

[0014] Therefore, the flexible job shop scheduling method based on structure awareness and heuristic guidance described above, as used in this invention, has the following beneficial effects: (1) Hierarchical ring diffusion coding can explicitly split the structural information of different hop counts, avoid the feature mixing problem caused by stacking layers in traditional graph neural networks, effectively improve the joint representation ability of long-term process dependence and local resource conflict, and enhance the representation accuracy of scheduling state.

[0015] (2) Action-level heuristic feature enhancement mechanism: The prior information of scheduling domain such as the number of remaining processes and the remaining workload is explicitly introduced into the action representation, which makes up for the shortcomings of existing methods that only rely on graph structure features, effectively improves the distinguishability of candidate actions and enhances the accuracy of decision-making.

[0016] (3) The dual context coordination mechanism realizes the joint coordination of structural and heuristic contexts. It completes the relative comparison of candidate actions by referring to residuals, effectively improving the stability of action ranking and enhancing the stability of reinforcement learning training. It can adapt to complex scheduling scenarios of different scales.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the overall framework of a flexible job shop scheduling method based on structure awareness and heuristic guidance according to the present invention. Figure 2 This is a schematic diagram illustrating the dynamic update of the heterogeneous graph state in this invention. Detailed Implementation

[0019] like Figure 1 As shown, this invention discloses a structure-aware and heuristic-guided flexible job shop scheduling method, the steps of which include: S1. Construct a flexible job shop scheduling model, defining jobs, processes, machines, and their constraints, with the goal of minimizing the maximum completion time. Specifically: S11. Construct a flexible job shop scheduling model: Assume that the flexible work workshop includes a set of jobs. and machine collection A job consists of several steps performed in a fixed order, represented as follows: ,in, The total number of assignments. The total number of machines. For the job, each process Corresponding to a set of candidate machines The processing time for a process can vary on different candidate machines, denoted as... In this scheduling problem, two core decisions need to be made simultaneously: first, selecting a specific processing machine for each operation; and second, determining the processing sequence of operations on each machine. The entire scheduling process satisfies the following constraints: each machine processes at most one operation at any given time; once an operation begins, it cannot be interrupted; any operation can only begin after its predecessor operation has been completed; each operation can only select one machine from its pre-set set of candidate machines for processing. Based on these definitions, flexible job shop scheduling can be formalized as a combinatorial optimization problem with coupled process and resource constraints.

[0020] S12. Establish optimization objectives and construct the scheduling environment: The optimization objective is to minimize the maximum completion time, which is defined as: ; in, Indicate homework The completion time is therefore the optimization objective is... The maximum completion time reflects the total cycle required for the entire workshop to complete all production tasks. Therefore, minimizing this indicator can directly improve the overall scheduling efficiency.

[0021] S13. Model the scheduling process as a Markov decision process: The scheduling process is modeled as a Markov decision process: ; in, Representing the state space, Represents the action space, Indicates state transition, Represents the reward function, This represents the discount factor.

[0022] At every decision moment The agent observes the current scheduling state. From the set of actionable actions Choose one action After the environment performs this action, it transitions to the next state. and output the reward value. The decision-making moments here include the initial moment and the moment when the system state is updated after any process is completed. In other words, the system enters the next round of decision-making when one or more of the following occurs: machine release, subsequent processes become schedulable, or the set of candidate actions changes.

[0023] S14. Construct the reward function: The reward function is based on the improvement of the maximum completion time of a partial scheduling scheme: ; in, This represents the estimated maximum completion time corresponding to the current partial scheduling scheme. This represents the estimated maximum completion time after an action is performed. An action that reduces the maximum completion time of subsequent scheduling plans will receive a positive reward; conversely, an action that is detrimental to shortening the overall cycle time will receive a reduced reward. When the discount factor is set to... At that time, the cumulative reward for a single round is: ; in, Index for scheduling decision moments; The number of scheduling decision steps in a single scheduling round. , The maximum completion time corresponding to the initial scheduling state. The maximum completion time corresponding to the final scheduling scheme at the end of the scheduling process is determined by the initial state. Since it is a constant, maximizing the cumulative reward and minimizing the maximum completion time of the final scheduling scheme are consistent in the optimization direction.

[0024] S2. Construct a job-process-machine heterogeneous diagram based on the current scheduling status to describe the current workshop status, such as... Figure 2 As shown. Specifically, the current scheduling state is represented as a bipartite heterogeneous graph, with the following formula: ; in, It is a two-part heterogeneous diagram as a whole. For a set of process nodes, For a set of machine nodes, For process priority constraint edge set, For each process - a set of machine-processable edges; The characteristics of a process node include whether the process has been completed, whether the process is in a schedulable state, the estimated processing time of the process, and the cumulative processing information of the unfinished processes in the corresponding job. Machine node characteristics include whether the machine is idle, the machine's current remaining processing time, the last time the machine became idle, and the machine's current load level. Process priority constraint edges represent the successor relationship between adjacent processes within the same operation; The process-machine machinable edge represents the processing adaptation relationship between a certain process and a certain machine, and carries the processing time of the corresponding process on the corresponding machine as an equal edge attribute.

[0025] In this way, process flow information and resource status information are uniformly represented within the same graph structure, facilitating subsequent information propagation and structural modeling within a unified graph domain. The current scheduling state is represented by a bipartite heterogeneous graph containing process nodes, machine nodes, process priority constraint edges, and process-machine processable edges, thus providing a state basis for subsequent candidate action generation and dynamic graph structure updates.

[0026] S3. Determine schedulable processes based on the current scheduling status, and generate a set of candidate actions by combining the candidate machines corresponding to each schedulable process; specifically: At the current decision point, firstly, based on the process priority constraints, determine which processes have already met the condition that all preceding processes have been completed, thus obtaining the current set of schedulable processes; then, based on the machine status, determine which machines are currently idle, and combine this with the candidate machine sets for each process to filter out all legal process-machine combinations, thereby generating a candidate action set. , where each candidate action This represents an executable process—a machine assignment action. When a candidate action is selected, the environment immediately executes the corresponding scheduling action, retaining the process-machine edge between the selected action and the chosen machine, and removing the edges between the action and the remaining candidate machines. Simultaneously, the processing status of the action, the busy status of the machine, and the corresponding remaining processing time are updated. Afterward, the environment advances the time to the next event point, i.e., the moment when the next process completes processing, and re-evaluates whether subsequent processes enter a schedulable state, thus forming a new heterogeneous graph state. When a candidate action is selected, the process-machine edge between the selected action and the target machine is retained, and the processable edges between the action and the remaining candidate machines are deleted. At the same time, the process state and machine state are updated, thereby achieving a dynamic transition from the current heterogeneous graph state to the updated heterogeneous graph state.

[0027] S4. A hierarchical ring diffusion coding mechanism is used to extract and fuse neighborhood information with different hop counts in the heterogeneous graph, resulting in a state representation. Specifically: S41. Perform initial feature mapping on process nodes and machine nodes: Let the input feature matrix of the process node be... The machine node input feature matrix is First, project the two types of nodes onto a unified latent space using linear mapping, and obtain: ; ; in, This is the initial hidden representation obtained after the initial linear mapping of the process nodes. This is the initial hidden representation obtained after the initial linear mapping of the machine node. and To obtain the learnable parameter matrix, the two types of node representations are concatenated along the node dimension to form a unified initial node representation: ; This step allows both types of nodes to enter the same implicit representation space while maintaining the semantic differences between process nodes and machine nodes, providing input for subsequent unified diffusion propagation on the heterogeneous graph.

[0028] Project the two types of nodes onto a unified latent space to form a unified initial node representation; S42. Construct the enhanced heterogeneous adjacency matrix, the formula is: ; in, To enhance the overall heterogeneous adjacency matrix, This is a process-to-process adjacency submatrix. This is a process-machine adjacency submatrix. For machine-process adjacency submatrix, This is a machine-machine adjacency submatrix; this matrix places process nodes and machine nodes in the same graph domain. The upper left block represents the successor-successor connections between processes, the upper right block and the lower left block represent the machinable relationships between processes and machines, and machines and processes, respectively. The lower right block is an empty matrix, indicating that machine nodes do not directly establish similar connections. This block matrix approach allows for the expression of process constraints and resource matching relationships on a unified graph topology.

[0029] It is obtained by adding the predecessor adjacency matrix and the successor adjacency matrix of the process, and the formula is: ; in, This is the predecessor adjacency matrix of the process. This is the adjacency matrix of the process successors; for The transpose of the matrix, It is a zero matrix.

[0030] S43. Perform cyclic neighborhood decomposition to obtain cyclic adjacency matrices with different hop counts; specifically: first, remove self-loops and construct binary adjacency matrices: ; in, For indicator functions, For element-wise multiplication, 1 represents a matrix of all 1s. It is the identity matrix. (Definition) , ,in, The zero-hop circular adjacency matrix is ​​taken as the identity matrix. Used to represent the information of each node itself. Given a 1-hop circular adjacency matrix, take the binary adjacency matrix after removing self-loops. , used to represent a one-hop direct adjacency relationship between nodes in an enhanced heterogeneous graph, for any ≥2, calculate first Step reachability matrix Then put all those smaller than Remove nodes that are already reachable within the jump range to obtain exactly... Circular hop adjacency matrix of hops: ; After the above processing, we can obtain Each of them Both indicate that the shortest path length is exactly 1 / 2. The set of nodes. The purpose of this step is to explicitly split the structural information under different hop counts, so as to avoid the direct mixing of semantics at different scales during deep propagation.

[0031] S44. Normalize each cyclic adjacency matrix to obtain the aggregation matrix, using the following formula: ; in, node With participating nodes Between the first Skip adjacency relationship For summation index, used to represent the summation index of nodes. In the same position Any node within the calculation range of the skip ring skip neighborhood, For the first Circular hop adjacency matrix This is a numerically stable term used to prevent the denominator from being zero; S45. Initial node representation Perform multi-hop diffusion to obtain the aggregation results of each hop. And retain 0-hop itself. To adaptively fuse information with different hop counts, a hop-level attention mechanism is used to adaptively fuse the aggregated results of each hop. The multi-hop fusion representation is obtained. ,in, This is the order of the maximum cyclic diffusion.

[0032] Further enhance training stability by employing gated residual connections: ; in, For learnable parameters, The activation function is used to restrict the gating coefficients to between 0 and 1. Finally, a new node representation is output through the feedforward network. This step allows for the adaptive fusion of structural dependency information under different hop counts into the node representation while preserving the original node information, thereby improving the joint characterization capability of local resource conflicts and long-term process dependencies.

[0033] S46. To avoid semantic confusion between process nodes and machine nodes during the same propagation process, a two-stage hierarchical update method of "machine first, process subsequent" is adopted. The node representation is updated using a two-stage hierarchical update mechanism. The first stage updates the machine node representation, resulting in: ; in, For the first stage of the loop-jump diffusion and fusion operator, This represents the order of the loop propagation during the machine node update phase. In this stage, machine nodes mainly aggregate information such as shared resource contention, machine load, and local processing context from adjacent process nodes; in the second stage, the process node representation is updated based on the machine representation obtained in the first stage, resulting in: ; in, For the second stage of the loop-jump diffusion and fusion operator, The order of loop propagation during the process node update phase; In this stage, the process nodes further aggregate process priority relationships, critical path semantics, and current resource context information. After these two stages of updates, the output includes machine node representations that better reflect resource contention relationships. And process node representations that better reflect process dependencies under resource context conditions. ; S47. Perform average pooling on the machine node representation and the process node representation respectively to obtain: ; ; in, Reflects the overall status of the process. Reflecting the overall state of the machine, the data is pieced together to obtain a diagram-level state representation. It is used to characterize global information about the current scheduling state and serves as an important input for subsequent value estimation and scheduling decisions.

[0034] S5. Extract heuristic features from candidate actions and construct action-level heuristic representations; the extracted heuristic features include the number of remaining processes, the remaining workload, the processing time of candidate machines, and the earliest available time of the machine. When constructing an action-level heuristic representation, for any candidate action... Construct a four-dimensional heuristic feature vector using the following formula: ; in, For a single candidate action, For the first The first assignment One process, For the first 1 machine; Candidate actions The corresponding four-dimensional heuristic feature vector, This represents the number of processes that have not yet been completed between the start of the current process and the completion of its associated task. This represents the remaining workload of the current process. The processing time of the current process on the candidate machine. For machines The earliest available time at the current moment; The heuristic feature vectors are normalized within instances using the following formula: ; in, The normalized heuristic feature vector, This is the mean vector of heuristic features within the candidate action set for the current instance. The standard deviation vector of the heuristic features within the current instance's candidate action set. It is a numerically stable term; By concatenating the process embedding, machine embedding, process graph-level convergence vector, machine graph-level convergence vector, and the normalized heuristic feature vector, we obtain the action augmentation representation: ; in, This represents the embedded representation of the process node in the candidate action. This represents the embedded representation of the machine node in the candidate action.

[0035] This action enhancement means that, on the one hand, the graph structure encoding results are preserved, and on the other hand, heuristic cost information directly related to scheduling decisions is explicitly incorporated, thereby improving the comparability between different candidate actions.

[0036] S6. Utilize a dual-context coordination mechanism to jointly model the state representation obtained in step S4 and the heuristic representation obtained in step S5, and output the scoring results for each candidate action; specifically: S61. Construct a structural action representation for any candidate action: ; ; in, Candidate actions The corresponding structural action representation, Representation of structural actions Dimensions This is based on the dimensions of process embedding and machine embedding. The dimension for the graph-level state representation; The structural action representation is composed of process embedding, machine embedding, and graph-level state representation; the joint action representation is obtained by concatenating the structural action representation with normalized heuristic features. ; ; in, Candidate actions The corresponding joint action is represented as follows: To indicate joint actions The dimension; S62. Use a shared parameter scoring network to score each joint action representation and calculate the attention weights in the candidate action set, using the following formula: ; in, Candidate actions The corresponding attention weights Candidate actions The corresponding feasibility mask has a value of 1 indicating that the action is legal and a value of 0 indicating that the action is illegal. Candidate actions The corresponding score for the combined action. For any candidate action in the candidate action set at the current time, This is the set of candidate actions for the current moment.

[0037] S63. Project the joint action representation onto the context space to obtain the projection vector. ,in, For learnable projection matrices, The activation function is used; the candidate action set and context vector are obtained by weighted aggregation based on attention weights. ; S64. Calculate the weighted cost reference value of the candidate action set based on the same set of attention weights, using the following formula: ; in, Used to describe the overall contextual features of the entire candidate action set. This is the weighted cost reference vector for the candidate action set at the current moment. Candidate actions The corresponding attention weights Candidate actions The corresponding normalized heuristic feature vector, The dimension of the normalized heuristic feature vector; S65. Define a reference residual for each candidate action, using the following formula: ; in, Candidate actions The deviation vector relative to the set's average cost level. Candidate actions The corresponding normalized heuristic feature vector, This is the weighted cost reference vector for the candidate action set at the current time; by simultaneously using and It can stabilize the action ranking results when candidate actions are highly similar.

[0038] S66. The structural action representation, the normalized heuristic features, and the reference residual are concatenated to obtain the final action input representation.

[0039] S7. Select the scheduling action at the current moment based on the score output of the reinforcement learning strategy network, and apply the scheduling action to the flexible job shop scheduling environment. S8. The scheduling environment updates the process status, machine status, and scheduling time information based on the selected scheduling action, outputting the corresponding reward value and the next time-instance state; based on the reward value and the next time-instance state, it calculates the advantage function, updates the reinforcement learning policy network parameters, and determines whether the current scheduling process has ended. If not, it returns to step S2 to continue execution, reconstructing the heterogeneous graph state, generating a candidate action set, and executing subsequent decisions; when all processes of all jobs have been completed, the final scheduling scheme is output. This forms a complete closed loop of "state construction—candidate action generation—structure encoding—action enhancement—dual context coordination—scheduling decision—environment update—parameter optimization". Specifically: When updating the reinforcement learning policy network parameters based on the reward value output by the scheduling environment and the state at the next time step, an Actor-Critic architecture is adopted. The policy network takes the final action input representation as input and outputs the action selection probability distribution. The value network takes the graph-level state representation and the context vector of the candidate action set as input and outputs the state value estimate. Specifically: In the policy network, the structural action representation, normalized heuristic features, and reference residuals are concatenated to obtain the final action input representation. In the value network, the graph-level state representation is concatenated with the context vector of the candidate action set to obtain the state value estimation input. Subsequently, the policy network considers any candidate action. Output unnormalized score The action selection probability distribution is obtained after softmax normalization. ,in, It is the final candidate action representation sent to the Actor. This is the context representation of the candidate action set fed into the Critic. During the training phase, actions are sampled and scheduled from the candidate action set based on the action probability distribution to enhance the exploration capability; during the inference phase, the candidate action with the highest probability is directly selected as the current scheduled action.

[0040] The Proximal Policy Optimization (PPO) method is used to update the parameters of the reinforcement learning policy network and value network. The objective function is: ; in, Let the objective function be the near-end policy optimization algorithm. These are the learnable parameters of the policy network; To Calculation of the expected value of the sampled trajectory at each time step. For the new and old strategies The probability ratio at time 1. for The estimated value of the dominance function at time 1; This is the clipping function, used to limit the range of fluctuation in the probability ratio. This is the cutting factor; The formula for the probability ratio of the new and old strategies is: ; in, and The current policy and the old policy are respectively in the state. Select action The probability of; During the training phase, actions are sampled and scheduled from the candidate action set based on the action probability distribution. During the inference phase, the candidate action with the highest probability is directly selected as the current scheduled action.

[0041] Example 1 In a preferred embodiment, the training environment adopts a 10×5 flexible job shop scale, combined with 15×10 and 20×5 scales for training and verification; the process node feature dimension is 6, the machine node feature dimension is 3, and the node embedding dimension is 8; the policy network and value network adopt a multilayer perceptron structure; the heuristic action feature dimension is 4; the hop count parameter of the hierarchical loop-diffusion encoder is 2; the reinforcement learning training adopts a proximal policy optimization algorithm with a learning rate of 2×10⁻⁴, a discount factor of 1, and a pruning factor of 0.2. The above parameters are only preferred embodiments and do not constitute a limitation on the scope of protection of this invention. To verify the effectiveness and generalization ability of the method of this invention, comparative experiments were conducted on synthetic examples and public benchmarks, and the results are shown in Tables 1 and 2.

[0042] Table 1. Comparison of scheduling performance of different methods on 10×5, 15×10, and 20×5 composite instances.

[0043] Note: 1. Instance size "10×5" means 10 jobs and 5 machines.

[0044] 2. Relative deviation indicates the percentage deviation from the reference result.

[0045] 3. Bold data indicates the best results at this instance size.

[0046] Table 1 shows the results on synthetic instances of scales 10×5, 15×10, and 20×5, with consistent training and testing scales used to evaluate performance in scenarios with the same distribution. It can be seen that the method of this invention outperforms HGNN overall in terms of maximum completion time and relative bias.

[0047] Table 2. Comparison of scheduling performance of different methods on public benchmarks

[0048] Note: 1. Relative deviation represents the percentage deviation from the publicly available benchmark reference value.

[0049] 2. Time represents the average solution time for a single instance, in seconds.

[0050] 3. "—" indicates that the method does not provide comparable results on the corresponding dataset.

[0051] 4. Bold data indicates the superior results achieved by the method of the present invention on the corresponding dataset.

[0052] Table 2 shows the results on publicly available benchmark datasets such as Behnke, Brandimarte, Hurink (rdata), Hurink (edata), and Hurink (vdata). The model is still trained only on synthetic data and is not fine-tuned on publicly available datasets. The method of this invention outperforms comparative methods such as genetic algorithms, priority rules, and HGNNs in terms of relative bias on most publicly available benchmark datasets, and also demonstrates better solution efficiency.

[0053] Therefore, this invention employs a structure-aware and heuristic-guided flexible job shop scheduling method. By constructing a heterogeneous graph of job-process-machine, combined with hierarchical ring diffusion coding, action-level heuristic feature enhancement, and a dual-context coordination mechanism, and relying on the Actor-Critic architecture and proximal policy optimization algorithm to achieve scheduling decisions, it effectively solves the technical defects of existing methods, such as insufficient long-range structural dependency representation, inadequate utilization of heuristic information, unstable candidate action ranking, and weak policy training robustness. It significantly improves the accuracy of scheduling state representation, decision accuracy, and training stability, and can efficiently adapt to flexible job shop scheduling scenarios of different scales and complex constraints, possessing excellent scheduling performance and broad engineering application value. Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of this invention and not to limit it. Although the invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solution of this invention, and these modifications or equivalent substitutions should not cause the modified technical solution to deviate from the spirit and scope of the technical solution of this invention.

Claims

1. A flexible job shop scheduling method based on structure awareness and heuristic guidance, characterized by the following steps: include: S1. Construct a flexible job shop scheduling model, define jobs, processes, machines and their constraints, and take minimizing the maximum completion time as the scheduling optimization objective; S2. Construct a job-process-machine heterogeneous diagram based on the current scheduling status to describe the current workshop status; S3. Determine the schedulable processes based on the current scheduling status, and generate a set of candidate actions by combining the candidate machines corresponding to each schedulable process. S4. Using a hierarchical ring diffusion coding mechanism, neighborhood information with different hop counts in heterogeneous graphs is extracted and fused hierarchically to obtain the state representation result; S5. Extract heuristic features from candidate actions and construct action-level heuristic representations; S6. Use the dual context coordination mechanism to jointly model the state representation results obtained in step S4 and the heuristic representation obtained in step S5, and output the score results of each candidate action. S7. Select the scheduling action at the current moment based on the score output of the reinforcement learning strategy network, and apply the scheduling action to the flexible job shop scheduling environment. S8. The scheduling environment updates the process status, machine status, and scheduling time information according to the selected scheduling action, and outputs the corresponding reward value and the next time state. Based on the reward value and the next time state, the advantage function is calculated, the reinforcement learning policy network parameters are updated, and it is determined whether the current scheduling process has ended. If it has not ended, it returns to step S2 to continue execution. If it has ended, the final scheduling scheme is output.

2. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 1, characterized in that: In step S2, when constructing the job-process-machine heterogeneous graph, the current scheduling state is represented as a bipartite heterogeneous graph, with the following formula: ； in, It is a two-part heterogeneous diagram as a whole. For a set of process nodes, For a set of machine nodes, For process priority constraint edge set, For each process - a set of machine-processable edges; The characteristics of a process node include whether the process has been completed, whether the process is in a schedulable state, the estimated processing time of the process, and the cumulative processing information of the unfinished processes in the corresponding job. Machine node characteristics include whether the machine is idle, the machine's current remaining processing time, the last time the machine became idle, and the machine's current load level. Process priority constraint edges represent the successor relationship between adjacent processes within the same operation; The process-machine machinable edge represents the processing adaptation relationship between a certain process and a certain machine, and carries the processing time of the corresponding process on the corresponding machine as an equal edge attribute.

3. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 1, characterized in that, In step S4, when performing hierarchical extraction and fusion using the hierarchical ring diffusion coding mechanism, the following steps are executed: S41. Perform initial feature mapping on process nodes and machine nodes, project the two types of nodes onto a unified latent space to form a unified initial node representation. S42. Construct the enhanced heterogeneous adjacency matrix, the formula is: ； in, To enhance the overall heterogeneous adjacency matrix, This is a process-to-process adjacency submatrix. This is a process-machine adjacency submatrix. For machine-process adjacency submatrix, The machine-to-machine adjacency submatrix; It is obtained by adding the predecessor adjacency matrix and the successor adjacency matrix of the process, and the formula is: ； in, This is the predecessor adjacency matrix of the process. This is the adjacency matrix of the process successors; for The transpose of the matrix, It is a zero matrix.

4. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 3, characterized in that: In step S4, when performing hierarchical extraction and fusion using the hierarchical ring diffusion coding mechanism, the following steps are performed after constructing the enhanced heterogeneous adjacency matrix: S43. Perform cyclic hop neighborhood decomposition to obtain cyclic hop adjacency matrices with different hop numbers; S44. Normalize each cyclic adjacency matrix to obtain the aggregation matrix, using the following formula: ； in, node With participating nodes Between the first Skip adjacency relationship For summation index, used to represent the summation index of nodes. In the same position Any node within the calculation range of the skip ring skip neighborhood, For the first Circular hop adjacency matrix This is a numerically stable term used to prevent the denominator from being zero; S45. Perform multi-hop diffusion on the initial node representation to obtain the aggregation results of each hop. Adaptively fuse the aggregation results of each hop using a hop-level attention mechanism to obtain the multi-hop fused representation.

5. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 4, characterized in that: In step S4, when performing hierarchical extraction and fusion using the hierarchical ring diffusion coding mechanism, the following steps are performed after obtaining the multi-hop fused representation: S46. A two-stage hierarchical update mechanism is adopted to update the node representation. The first stage updates the machine node representation, and the second stage updates the process node representation based on the machine representation obtained in the first stage. S47. Perform average pooling on the machine node representation and the process node representation respectively, and then concatenate them to obtain the graph-level state representation.

6. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 1, characterized in that: The heuristic features extracted in step S5 include the number of remaining operations, the remaining workload, the candidate machine processing time, and the earliest available time of the machine. When constructing an action-level heuristic representation, for any candidate action... Construct a four-dimensional heuristic feature vector using the following formula: ； in, For a single candidate action, For the first The first assignment One process, For the first 1 machine; Candidate actions The corresponding four-dimensional heuristic feature vector, This represents the number of processes that have not yet been completed between the start of the current process and the completion of its associated task. This represents the remaining workload of the current process. The processing time of the current process on the candidate machine. For machines The earliest available time at the current moment; The heuristic feature vectors are normalized within instances using the following formula: ； in, The normalized heuristic feature vector, This is the mean vector of heuristic features within the candidate action set for the current instance. The standard deviation vector of the heuristic features within the current instance's candidate action set. It is a numerically stable term; By concatenating the process embedding, machine embedding, process graph-level convergence vector, machine graph-level convergence vector, and normalized heuristic feature vector, we obtain the action enhancement representation.

7. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 1, characterized in that: When performing joint modeling using the dual context coordination mechanism in step S6, the following steps are executed: S61. Construct a structural action representation for any candidate action. The structural action representation is composed of process embedding, machine embedding and graph-level state representation. Concatenate the structural action representation with the normalized heuristic features to obtain a joint action representation. S62. Use a shared parameter scoring network to score each joint action representation and calculate the attention weights in the candidate action set, using the following formula: ； in, Candidate actions The corresponding attention weights Candidate actions The corresponding feasibility mask has a value of 1 indicating that the action is legal and a value of 0 indicating that the action is illegal. Candidate actions The corresponding score for the combined action. For any candidate action in the candidate action set at the current time, This is the set of candidate actions for the current moment.

8. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 7, characterized in that, In step S6, when performing joint modeling using the dual context coordination mechanism, the following steps are executed after calculating the attention weights: S63. Project the joint action representation onto the context space, and converge the candidate action set context vector according to the attention weights; S64. Calculate the weighted cost reference value of the candidate action set based on the same set of attention weights, using the following formula: ； in, This is the weighted cost reference vector for the candidate action set at the current moment. Candidate actions The corresponding attention weights Candidate actions The corresponding normalized heuristic feature vector; S65. Define a reference residual for each candidate action, using the following formula: ； in, Candidate actions The deviation vector relative to the set's average cost level. Candidate actions The corresponding normalized heuristic feature vector, This is the weighted cost reference vector for the candidate action set at the current moment; S66. The structural action representation, the normalized heuristic features, and the reference residual are concatenated to obtain the final action input representation.

9. The flexible job shop scheduling method based on structure awareness and heuristic guidance according to claim 1, characterized in that: In step S8, when updating the reinforcement learning policy network parameters based on the reward value output by the scheduling environment and the state at the next time step, an Actor-Critic architecture is adopted. The policy network takes the final action input representation as input and outputs the action selection probability distribution. The value network takes the graph-level state representation and the candidate action set context vector as input and outputs the state value estimate.

10. A structure-aware and heuristic-guided flexible job shop scheduling method according to claim 9, characterized in that, In step S8, the parameters of the reinforcement learning policy network and value network are updated using the proximal policy optimization method. The objective function is: ； in, Let the objective function be the near-end policy optimization algorithm. These are the learnable parameters of the policy network; To Calculation of the expected value of the sampled trajectory at each time step. For the new and old strategies The probability ratio at time 1. for The estimated value of the dominance function at time 1; This is the clipping function, used to limit the range of fluctuation in the probability ratio. This is the cutting factor; The formula for the probability ratio of the new and old strategies is: ； in, and The current policy and the old policy are respectively in the state. Select action The probability of; During the training phase, actions are sampled and scheduled from the candidate action set based on the action probability distribution. During the inference phase, the candidate action with the highest probability is directly selected as the current scheduled action.

Images