Evolutionary strategy and meta-reinforcement learning based flexible job shop scheduling method and system

Abstract

The application provides a flexible job shop scheduling method and system based on an evolutionary strategy and meta-reinforcement learning, comprising the following steps: a certain number of flexible job shop scheduling problem instances are randomly generated to form a training data set, and the training data set is replaced every fixed update round; a meta-reinforcement learning framework based on an evolutionary strategy is constructed to train a meta-model, the optimal parameters of the meta-model are determined by minimizing the total average completion time of a verification set, and the meta-model is used as an initialization model to adapt to new tasks in an inference process; the completion time of test data is obtained by using the trained meta-model, and the optimal result for each instance in the test data is obtained by fine-tuning each instance a limited number of times.

Description

Technical Field

[0001] This invention belongs to the field of flexible job shop scheduling technology, and particularly relates to a flexible job shop scheduling method and system based on evolutionary strategies and meta-reinforcement learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The development of industrial technology is transforming how enterprises manufacture and distribute products, shifting towards rapid, intelligent, and flexible manufacturing, thus fundamentally changing enterprise production capabilities. The Flexible Job Scheduling Problem (FJSP) is a classic problem representing a typical scenario faced by flexible manufacturing. It allows each operation to be processed on multiple different machines. The machine allocation problem increases manufacturing flexibility, giving FJSP more complex topologies and a larger solution space. FJSP has been proven to be a strongly NP-hard (Nondeterministic Polynomial time problem). The combinatorial nature of FJSP makes finding (near-optimal) solutions using traditional operations research methods (such as constrained programming) challenging. The computational costs of these methods are cumbersome and increase dramatically with problem size, making them unsuitable for large-scale applications. To strike a balance between solution quality and computational cost, research in this field is increasingly shifting from traditional heuristics and metaheuristics to intelligent methods, such as data-driven deep learning and deep reinforcement learning (DRL).

[0004] Metaheuristic algorithms, including genetic algorithms, particle swarm optimization, differential evolution, and artificial bee colony optimization, have been widely applied to scheduling problems, often finding high-quality solutions through complex solution search procedures. Conversely, rule-based heuristics, such as Priority Scheduling Rules (PDR), are more practical due to their ease of implementation and efficiency. PDR repeatedly selects the highest-priority operation or machine according to some prescribed rules until a complete schedule is generated. However, designing effective PDRs often requires significant expertise and research, and they may only perform well on specific tasks. Currently, DRL methods have emerged as promising approaches for solving FJSP, modeling the scheduling process as a Markov Decision Process (MDP). In these methods, a parameterized neural network model is designed to receive information about the production environment as a state and output the priority of each feasible scheduling operation, such as assigning an operation to a machine, forming an end-to-end learning approach. By training on a set of production process data, the DRL model learns to adaptively select the optimal action in a given state to maximize the total reward associated with the production objective.

[0005] However, these methods generally have some limitations. Typically, the optimal model trained on a large training set achieves average performance optimal under a specific size or distribution. However, this training approach often ignores the specificity of individual instances in the scheduling problem. Therefore, although the model can achieve good global performance in a statistical sense, its actual performance may deviate significantly from the ideal reward when facing specific instances. This phenomenon indicates that existing DRL methods rely too heavily on the breadth of the training set and fail to fully consider the unique needs and scheduling characteristics of individual instances, thus limiting their generalization ability and practical application effectiveness in complex and dynamic environments. Therefore, algorithmic frameworks optimized for individual instances, especially by incorporating attention to instance specificity during model training, have become a key research direction for improving the application effect of DRL in FJSP.

[0006] Existing patents disclose heuristic methods for solving the flexible shop floor scheduling problem. However, heuristic methods require the development of numerous hand-crafted rules and domain expertise for each specific problem, and they may only perform well on specific tasks. Currently, deep reinforcement learning-based methods are more promising; however, these methods generally suffer from some limitations. Typically, the optimal model trained on a large training set achieves average performance optimal under a specific size or distribution. However, this training approach often ignores the particularity of a single instance of the scheduling problem. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, this invention provides a flexible job shop scheduling method based on evolutionary strategies and meta-reinforcement learning, which fully considers the unique needs and scheduling characteristics of different instances in order to further minimize the total completion time.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] Firstly, a flexible job shop scheduling method based on evolutionary strategies and meta-reinforcement learning is disclosed, including:

[0010] A set number of flexible shop floor scheduling problem instances are randomly generated to form a training dataset, and the training dataset is replaced every fixed update round.

[0011] We construct a meta-reinforcement learning framework based on an evolutionary strategy. The training dataset is used to train the meta-model. The optimal parameters of the meta-model are determined by minimizing the total average completion time of the validation set, which serves as the initial model and is used to adapt to new tasks during inference.

[0012] Using the trained meta-model, the completion time of the test data is obtained, and each instance in the test data is fine-tuned a finite number of times to obtain the optimal result for each instance.

[0013] The above-mentioned minimizing of the total average completion time of the validation set is to minimize the average total completion time of all instances in the validation set.

[0014] As a further technical solution, the meta-model training process includes inner loop optimization and outer loop optimization in each iteration. The inner loop optimization iteratively optimizes the task-specific model, and the outer loop optimization updates the meta-model, with the aim of maximizing the few-shot generalization performance of the task-specific model.

[0015] As a further technical solution, the training dataset includes:

[0016] The set consisting of n jobs and m machines is denoted by J and M, respectively.

[0017] Assume all jobs occur within system production time T. s =0 arrive simultaneously, each job J i ∈J contains n i These operations must be assembled in a specific order, by... Description; The set of all operations for all jobs is represented by O, and each operation has an O value. ij Processed by multiple machines, but only on a set M of available and compatible machines. ij Processed on one machine in M;

[0018] Machine M k ∈M ij The relevant processing time is determined by Provided.

[0019] As a further technical solution, the flexible shop floor scheduling problem is used to determine the appropriate processing machine and start time for each operation, while adhering to the following constraints:

[0020] 1.J i The operation must be performed according to O i Sequential processing in the process;

[0021] 2. Each operation must be assigned to exactly one compatible machine;

[0022] 3. Each machine can process a maximum of one operation at a time;

[0023] The goal of FJSP is to minimize the maximum completion time of all jobs, i.e., to minimize the total completion time.

[0024] As a further technical solution, the solution process of FJSP instance G is formulated as MDP, and the strategy is parameterized by a neural network constructed by encoder-decoder to learn the node selection for constructing the solution.

[0025] The encoder in the policy network outputs a global representation of the instance, which, together with the context representation, captures the current state. The decoder takes the global and context representations as input to calculate the probability of the node to be visited, i.e., the action, and selects the node in turn until the complete journey τ is constructed.

[0026] As a further technical solution, the meta-target is defined as follows:

[0027]

[0028] in, In instance T i Fine-tuning the model after K gradient updates on θ0; It is instance T i The same loss function is used for different instances.

[0029] Secondly, a flexible job shop scheduling system based on evolutionary strategies and meta-reinforcement learning is disclosed, including:

[0030] The training dataset building module is configured to randomly generate a set number of flexible shop floor scheduling problem instances to form a training dataset, and replace the training dataset every fixed update round.

[0031] The meta-model training module is configured to: construct a meta-reinforcement learning framework based on an evolutionary strategy for training the meta-model; determine the optimal parameters of the meta-model by minimizing the average completion time of the validation set; and use this as an initial model to adapt to new tasks during inference.

[0032] The scheduling module is configured to: use the trained meta-model to obtain the completion time of the test data, and perform a limited number of fine-tuning operations on each instance in the test data to obtain the optimal result for each instance.

[0033] The above one or more technical solutions have the following beneficial effects:

[0034] This invention improves the generalization ability and practical application effect of the scheduling system by fully considering the unique needs and scheduling characteristics of individual instances. This meta-learning-based strategy enables the scheduling system to automatically adjust its decision-making strategy when facing different production environments, machine configurations, and job characteristics, thereby optimizing scheduling performance. Furthermore, this invention replaces the policy gradient-based optimization algorithm with an evolutionary strategy algorithm. As a gradient-free optimization method, the evolutionary strategy has significant parallelization advantages, enabling efficient and simultaneous independent optimization of multiple scheduling instances in a single GPU environment. Moreover, during training, the evolutionary strategy uses global search rather than local gradient updates, making the optimization process less prone to getting trapped in local optima, thus enhancing the model's global generalization ability and further improving the training efficiency and fine-tuning performance of the meta-model.

[0035] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0036] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0037] Figure 1 This is a diagram of the meta-learning framework of an embodiment of the present invention;

[0038] Figure 2 This is a parallel computation diagram of the evolutionary strategy in an embodiment of the present invention;

[0039] Figure 3 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0040] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0041] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0042] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0043] Example 1

[0044] This embodiment discloses a flexible job shop scheduling method based on evolutionary strategies and meta-reinforcement learning, including:

[0045] Step 1: Randomly generate 20 FJSP instances to form a training dataset, and replace them every fixed update round. That is, every fixed update round, 20 new instances will be generated to replace the data in the previous training set.

[0046] In this implementation example, the training dataset includes 20 FJSP (Flexible Job Space Scheduling) instances. Each instance represents a specific job space situation, namely the number of jobs, the number of machines, the number of operations per job, and the processing time of each operation on different machines.

[0047] Step 2: Generate 100 FJSP instances respectively, forming a verification dataset and a test dataset;

[0048] Step 3: Construct a meta-reinforcement learning framework based on an evolutionary strategy. The goal is to train a meta-model that serves as a well-initiated model, capable of effectively adapting to new tasks during inference. The meta-training process includes inner and outer loop optimization in each iteration. The inner loop iteratively optimizes the task-specific model, similar to the fine-tuning phase in inference. The outer loop updates the meta-model, aiming to maximize the few-shot generalization performance of the task-specific model.

[0049] The meta-model is trained, and the optimal parameters of the meta-model are determined by minimizing the average completion time of the validation set.

[0050] The meta-model is a deep neural network with tens of thousands of parameters. These parameters are updated through backpropagation, which gradually improves the model's output. Finally, the model parameters corresponding to the best output are taken as the optimal parameters.

[0051] Step 4: Using the trained meta-model, obtain the completion time of the test data, and perform a limited number of fine-tunings on each instance in the test data to obtain the optimal result for each instance, i.e., the total time required to complete the workshop scheduling.

[0052] For each instance, the workshop information represented by that instance (number of workpieces, number of machines, number of operations per workpiece, and processing time for each operation on the machine) is embedded as a vector. This vector serves as the model's input data. After the vector is input into the model, a series of calculations determine the probability of the next action, i.e., matching workpieces with machines. After matching, the workshop information changes and is embedded into a new vector. This process is repeated to obtain the complete workshop scheduling flow. The output is the total time required to complete the workshop scheduling.

[0053] The fine-tuning process described above is the same as the training process, in which the model is backpropagated and updated for a single instance, but the magnitude of the update is reduced.

[0054] The flexible job shop scheduling method based on evolutionary strategies and meta-reinforcement learning in this embodiment improves its generalization ability and practical application effect by fully considering the unique needs and scheduling characteristics of individual instances. This meta-learning-based strategy enables the scheduling system to automatically adjust its decision-making strategy when facing different production environments, machine configurations, and job characteristics, thereby optimizing scheduling performance. Therefore, the method of this invention has good application prospects.

[0055] In the specific implementation of the technical solution of this application, a limited number of fine-tunings are made to the specific production environment, machine configuration and operation characteristics, so that after the model adapts to the specific workshop conditions, the output scheduling results are better than the original model.

[0056] FJSP can be formally expressed as follows. Consider a set consisting of n jobs and m machines, denoted by J and M respectively. Assume all jobs have a system production time T. s =0 arrive simultaneously. Each job J i ∈J contains n i There are 10 operations, where J represents the set of jobs. i and n i In this context, 'i' represents the i-th job in the set. These operations must be assembled in a specific order (i.e., priority constraints), determined by... Description. The set of all operations for all jobs is represented by O. Each operation has an O value. ij It can be processed by multiple machines, but only on a set M of available and compatible machines. ij Processed on one machine in M. Machine M k ∈M ij The relevant processing time is determined by Given i, the job number; j, the operation number within the job (e.g., Oij refers to the j-th operation of the i-th job); K, the machine number; p... ij k The meaning is the time required for the k-th machine to process the j-th operation of the i-th job.

[0057] In this implementation example, FJSP attempts to design a schedule that determines the appropriate processing machine and start time for each operation, while adhering to the following constraints:

[0058] 1.J i The operation must be performed according to O i The sequential processing (i.e., priority constraints) in the process;

[0059] 2. Each operation must be assigned to exactly one compatible machine;

[0060] 3. Each machine can process at most one operation at a time. The goal of FJSP is to minimize the maximum completion time of all jobs, i.e., to minimize the total completion time.

[0061] The scheduling process can be understood as dynamically allocating ready operations to compatible idle machines. In this way, the decision point t is the production system time T. S (t), at this point there exists at least one compatible operating machine pair (O) ij M k ), making O ij It can be done at time T S (t) in machine M k The above was processed.

[0062] A decision point is a node that needs to be paired with a machine during the scheduling process. It is equivalent to step t. Since the operation has processing time on the machine, each decision point has at least one node that is compatible with the machine when the operation is performed.

[0063] In step t, for example, if a scheduling process requires 50 steps to complete, then t is from 1 to 50. The DRL model receives state s from the environment. t And take action to designate a compatible pair at time T S (t) Start processing immediately, where Ts(t) refers to the time the workshop has been scheduled at step t. For this purpose, the FJSP environment returns a reward r related to the maximum completion time. t By repeating this process |O| times on the entire set of operations in the task, the path τ of the solution to the FJSP can be obtained.

[0064] State S t This refers to: the number of jobs in the workshop, the number of operations per job, the number of machines, the processing time of each operation on the machine, the jobs and machines that have been scheduled and completed in the workshop, and the jobs and machines that have not been scheduled in the workshop. The status of the workshop changes continuously as scheduling proceeds. Therefore, the current status of the workshop must be received at each scheduling time.

[0065] In this example, the cost function c(·) is the total completion time after scheduling. The goal of FJSP is to find the optimal path τ with the shortest total completion time. * .

[0066]

[0067] Where S is a discrete search space containing all feasible paths and subject to the constraints of a specific problem, and G is a single instance.

[0068] The neural construction method formulates the solution process for an FJSP instance G as an MDP, parameterizing the policy through an encoder-decoder neural network to learn node selections for constructing solutions. The encoder in the policy network outputs a global representation of the instance, which, along with a representation of the context (e.g., scheduled operations and machines), captures the current state. The decoder takes the global and context representations as input to compute the probability of visiting a node (i.e., an action). This node is selected sequentially until a complete journey τ is constructed. Therefore, the path probabilities are decomposed using a chain rule:

[0069]

[0070] Where, π θ (t) and π θ (<t) represents the selected node at time step t and the current partial solution, respectively; T represents the total number of steps. The reward is defined as the negative cost of a single trip, i.e.

[0071] The purpose of formula (2) is to calculate the probability of each possible scheduling under the condition of fixed model parameters. By knowing the probability of each path, we can obtain the expected return of the model and thus optimize the model according to the expectation.

[0072] To train a policy network, reinforcement learning algorithms are typically used to estimate the expected reward. gradient

[0073] Define a task T{(n*m), D} as a class of instances of size (n*m) with distribution d∈D, where n and m are the number of operands and the number of machines, respectively; It is a set of distributions; an instance is represented by Ti. A general meta-learning framework is applied to improve the generalization ability of the VRP neural method. This framework is model-independent and compatible with any model trained with gradient updates. The framework is as follows: Figure 1 As shown.

[0074] In the meta-learning framework, the goal is to train a meta-model θ0 as a well-initialized model that can effectively adapt to new tasks during inference. Formally, the meta-objective is defined as follows:

[0075]

[0076] in In instance T i Fine-tuning the model after K gradient updates on θ0; It is instance T i The loss function is given by E, which represents the expectation.

[0077] In this implementation example, the same loss function (e.g., enhancement loss) is used for different instances. To directly optimize this objective, the meta-training process includes inner and outer loop optimizations at each iteration, specifically:

[0078] Step 1: Initialize the metamodel;

[0079] Step 2: Randomly generate instances;

[0080] Step 3: Fine-tune the inner loop simulation;

[0081] Step 4: Update the metamodel in the outer loop;

[0082] Step 5: Repeat steps 3 and 4 until training is complete.

[0083] The pseudocode for meta-training is as follows.

[0084]

[0085] Inner loop optimization: It iteratively optimizes an instance-specific model, similar to the fine-tuning phase in the inference process. Specifically, given an instance T... i ∈T, initialize the instance-specific model through the metamodel, i.e. The training instances are adapted to Ti by performing K gradient update steps. The loss function at step k is... The gradient is calculated as follows:

[0086]

[0087] Formula (4) simulates fine-tuning and updating the model of the corresponding instance by calculating the inner loop optimization gradient.

[0088] Outer loop optimization: It uses the objective optimization meta-model in equation (3). Specifically, for each instance T i Evaluate task-specific models The generalization performance of fewer lenses is improved. The meta-gradient is obtained as follows:

[0089]

[0090] Formula (5) updates the meta-model by calculating the outer loop optimization gradient.

[0091] In a batch of tasks After inner-loop optimization, the metamodel θ0 is updated once. Intuitively, inner-loop optimization acts as the task adaptation phase, mimicking the fine-tuning process during inference, while outer-loop optimization updates the metamodel, aiming to maximize the instance-specific model. The generalization performance of a few lenses.

[0092] Therefore, after meta-training, a good initialization model can be obtained. This model can effectively adapt to new instances using only limited data. Note that equation (5) requires a second derivative because we expect to obtain the gradient direction with respect to the metamodel θ0. Its first-order approximation method will be introduced later.

[0093] The meta-gradient in the equation involves the gradient through the gradient (i.e., the second derivative), and obtaining it is computationally expensive due to the calculation of the Hessian vector product. To address this issue, a first-order approximation method is applied, which simply eliminates the second-order term. Empirical evidence of its effectiveness has been validated in a few supervised learning studies. Specifically, the first-order approximation of the meta-model update can be expressed as:

[0094]

[0095] In solving the flexible job shop scheduling problem, policy gradient-based optimization methods are commonly used, which update policy parameters through local gradient information. However, within the meta-reinforcement learning framework, policy gradient-based optimization algorithms face high computational overhead when training multiple scheduling instances, especially in a single-GPU environment. Specifically, policy gradient-based optimization algorithms rely on backpropagation to calculate gradients, typically requiring updates to the policy of each instance over multiple training epochs. This gradient-dependent iterative optimization approach is not only computationally inefficient but also struggles to fully utilize parallel computing resources.

[0096] To address the aforementioned issues, the policy gradient-based optimization algorithm is replaced with an evolutionary strategy (ES) algorithm. Evolutionary strategies, as a gradient-free optimization method, optimize the parameter population by simulating the process of natural selection. Its core idea is to simultaneously perturb multiple candidate solutions and evaluate their fitness in each generation, thereby guiding the optimization of model parameters through population-level updates. This method offers significant parallelization advantages, enabling efficient and independent optimization of multiple scheduling instances simultaneously in a single GPU environment. The parallel computing framework diagram is shown below. Figure 2 As shown.

[0097] During training, the evolutionary strategy uses global search rather than local gradient updates, which makes the optimization process less likely to get stuck in local optima, thereby enhancing the model's global generalization ability and further improving the fine-tuning performance of the meta-model.

[0098] The gradient estimation of the evolutionary policy is obtained as follows:

[0099]

[0100] Where, ∈ i It is a noise disturbance that follows a Gaussian distribution; σ is the noise standard deviation; n is the number of evolutionary populations.

[0101] The evolutionary strategy updates model parameters by repeatedly executing two phases:

[0102] 1. The parameters of the random perturbation strategy are generated into n populations, and the resulting parameters are evaluated by running the strategy in the environment for one round.

[0103] 2. Combine the results from all populations to calculate the stochastic gradient estimate and update the parameters.

[0104] The evolutionary strategy of this example is not to directly calculate the gradient, but to obtain a population by perturbing the model and then to simulate the gradient calculation through the population.

[0105] The pseudocode is as follows:

[0106]

[0107] This embodiment employs a meta-learning framework during model training. The aim is to further optimize scheduling performance by fine-tuning individual instances after training, thus reflecting the principle of "considering the unique needs and scheduling characteristics of individual instances." Specifically, the training process includes an inner loop and an outer loop. The inner loop simulates fine-tuning for individual instances, while the outer loop updates the meta-model. The inner loop's role is to consider the unique needs and scheduling characteristics of individual instances. The inner loop of the meta-learning framework introduces attention to the specificities of each instance.

[0108] See appendix Figure 3 As shown in the specific example, the flexible job shop scheduling method based on evolutionary strategy and meta-reinforcement learning includes:

[0109] Step 1: Generate 100 instances as the validation set, and randomly generate 20 instances as the training set every fixed number of rounds.

[0110] Step 2: Initialize the metamodel.

[0111] Step 3: Assign the initial parameters of the meta-model to the model corresponding to each instance in the training set, and perform inner loop simulation fine-tuning on the model.

[0112] Step 4: The inner loop ends, and the outer loop optimizes the meta-model.

[0113] Step 5: After training, select the best-performing model from the validation set as the final result.

[0114] Step 6: Input the actual engineering example into the meta-model.

[0115] Step 7: Make a limited number of fine-tunings to the instance so that the model can better adapt to the specific instance.

[0116] Step 8: Select the optimal scheduling scheme during the fine-tuning process.

[0117] Example 2

[0118] The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0119] Example 3

[0120] The purpose of this embodiment is to provide a computer-readable storage medium.

[0121] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0122] Example 4

[0123] The purpose of this embodiment is to provide a flexible job shop scheduling system based on evolutionary strategies and meta-reinforcement learning, including:

[0124] The training dataset building module is configured to randomly generate a set number of flexible shop floor scheduling problem instances to form a training dataset, and replace the training dataset every fixed update round.

[0125] The meta-model training module is configured to: construct a meta-reinforcement learning framework based on an evolutionary strategy for training the meta-model; determine the optimal parameters of the meta-model by minimizing the average completion time of the validation set; and use this as an initial model to adapt to new tasks during inference.

[0126] The scheduling module is configured to: use the trained meta-model to obtain the completion time of the test data, and perform a limited number of fine-tuning operations on each instance in the test data to obtain the optimal result for each instance.

[0127] Example 5

[0128] The purpose of this embodiment is to provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods and functions involved in any of the above embodiments.

[0129] The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0130] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0131] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A flexible job shop scheduling method based on evolutionary strategies and meta-reinforcement learning, characterized by including: A set number of flexible shop floor scheduling problem instances are randomly generated to form a training dataset, and the training dataset is replaced every fixed update round. The training dataset includes: The set consisting of n jobs and m machines is denoted by J and M, respectively. Assuming all jobs occur during system production time =0 arrive simultaneously, each job Include These operations must be assembled in a specific order, by... Description; The set of all operations for all jobs is represented by o, and each operation... Processed by multiple machines, but only on a set of available and compatible machines. Processed on one of the machines; machine The relevant processing time is determined by Give; p ij k The meaning is the first k The machine processes the first i The first assignment j Time required for each operation? The flexible shop floor scheduling problem is used to determine the appropriate processing machine and start time for each operation, while adhering to the following constraints: (1). The operation must be carried out in accordance with Sequential processing in the process; (2) Each operation must be assigned to exactly one compatible machine; (3) Each machine can process a maximum of one operation at a time; The goal of FJSP is to minimize the maximum completion time of all jobs, i.e., to minimize the total completion time. An evolutionary strategy-based meta-reinforcement learning framework is constructed to train the meta-model. The optimal parameters of the meta-model are determined by minimizing the average completion time of the validation set, which serves as the initial model for adapting to new tasks during inference. Using the trained meta-model, the completion time of the test data is obtained, and each instance in the test data is fine-tuned a finite number of times to obtain the optimal result for each instance; The solution process for FJSP instance G is formulated as an MDP, which uses a neural network constructed by an encoder-decoder to parameterize the strategy and learn the node selection for constructing the solution. In the policy network, the encoder outputs a global representation of the instance, which, along with the context representation, captures the current state. The decoder takes the global and context representations as input to calculate the probability of visiting nodes (actions), selecting nodes sequentially until a complete journey is constructed. .

2. The flexible job shop scheduling method based on evolutionary strategy and meta-reinforcement learning as described in claim 1, characterized in that, The meta-model training process includes inner loop optimization and outer loop optimization in each iteration. The inner loop optimization iteratively optimizes the task-specific model, while the outer loop optimization updates the meta-model. The goal is to maximize the few-shot generalization performance of the task-specific model.

3. The flexible job shop scheduling method based on evolutionary strategy and meta-reinforcement learning as described in claim 1, characterized in that, The meta-target is defined as follows: in, In the instance superior Fine-tuning the model after K gradient updates; It is an example The same loss function is used for different instances.

4. A flexible job shop scheduling system based on evolutionary strategies and meta-reinforcement learning, employing the flexible job shop scheduling method based on evolutionary strategies and meta-reinforcement learning as described in any one of claims 1-3, characterized in that, include: The training dataset building module is configured to randomly generate a set number of flexible shop floor scheduling problem instances to form a training dataset, and replace the training dataset every fixed update round. The meta-model training module is configured to: construct a meta-reinforcement learning framework based on an evolutionary strategy for training the meta-model; determine the optimal parameters of the meta-model by minimizing the average completion time of the validation set; and use this as an initial model to adapt to new tasks during inference. The scheduling module is configured to: use the trained meta-model to obtain the completion time of the test data, and perform a limited number of fine-tuning operations on each instance in the test data to obtain the optimal result for each instance.

5. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 3.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the method described in any one of claims 1-3 above.

Images