Supply chain process job scheduling method and system based on hybrid expert model

Abstract

The invention discloses a supply chain process job scheduling method and system based on a hybrid expert model, and the method comprises the steps: obtaining a target job shop scheduling problem, and selecting a target gated network from an expert selection gated network based on static similarity and a gated network based on weight learning according to a preset threshold value; when the target gated network is the gated network based on weight learning, constructing a scheduling graph according to the target job shop scheduling problem, and processing the scheduling graph based on a plurality of preset expert networks and the trained gated network based on weight learning to obtain a first target strategy; and when the target gated network is an expert selection gated network based on static similarity, selecting a target expert network from a plurality of preset expert networks according to the target job shop scheduling problem, and processing the target job shop scheduling problem through the target expert network to obtain a second target strategy. According to the invention, the job shop scheduling problem can be accurately processed.

Description

Technical Field

[0001] The present invention relates to the field of job shop scheduling, and in particular to a supply chain process job scheduling method, system, terminal and computer-readable storage medium based on a hybrid expert model. Background Art

[0002] The Job Shop Scheduling Problem (JSSP) is a key issue in optimizing the production process. Its goal is to rationally arrange the execution order of each process to minimize the total time (makespan) to complete all tasks. Currently, traditional optimization algorithms for JSSP, such as heuristic or metaheuristic algorithms, perform well for small-scale or specific types of problems. However, they are often inefficient when faced with the complexity and scale of problems in real applications and are difficult to adapt to the variability of production environments.

[0003] Currently, deep reinforcement learning (DRL) is employed to handle JSSP, and these methods have demonstrated the potential to solve complex scheduling problems by learning scheduling policies directly from data.

[0004] However, although deep reinforcement learning has superior learning capabilities in theory, models with different scales of training data have different adaptability to corresponding problems. Some DRL methods still perform poorly when faced with a large number of job shop scheduling problem instances with diverse scales. Therefore, when inputting problems of unknown scale, it is impossible to give scheduling results close to the optimal model based on the existing solution model, resulting in the inability to accurately handle different job shop scheduling problems.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0006] The main purpose of the present invention is to provide a supply chain process job scheduling method, system, terminal and computer-readable storage medium based on a hybrid expert model, aiming to solve the problem that the models of different scales of training data in the existing technology have different adaptability to problems, and some DRL methods still perform poorly when faced with a large number of job shop scheduling problem instances with diverse scales. Therefore, when inputting problems of unknown scale, it is impossible to give a scheduling result close to the optimal model based on the existing solution model, resulting in the inability to accurately handle different job shop scheduling problems.

[0007] To achieve the above object, the present invention provides a supply chain process operation scheduling method based on a hybrid expert model, the supply chain process operation scheduling method based on a hybrid expert model comprising the following steps:

[0008] Obtain a target job shop scheduling problem, and select a target gating network from a static similarity-based expert selection gating network and a weight learning-based gating network according to a preset threshold;

[0009] When the target gating network is a weight learning-based gating network, a scheduling graph is constructed according to the target job shop scheduling problem, and the scheduling graph is processed based on the pre-set multiple expert networks and the trained weight learning-based gating network to obtain a first target strategy;

[0010] When the target gating network is an expert selection gating network based on static similarity, a target expert network is selected from a plurality of pre-set expert networks according to the target job shop scheduling problem, and the target job shop scheduling problem is processed by the target expert network to obtain a second target strategy.

[0011] Optionally, the training process of the gating network based on weight learning specifically includes:

[0012] Generate multiple training instances within a preset range, process each training instance to obtain the corresponding node state feature, and input it into a preset expert network to obtain the corresponding action probability distribution and policy entropy, and record the logarithmic probability and policy entropy of each action probability distribution corresponding to the action;

[0013] According to the action probability distribution, a gating network based on weight learning is used to select the final action of the corresponding training instance from the action probability distribution generated by all expert networks, and the next stage state is obtained based on the final action and the corresponding training instance;

[0014] Each training step is processed in turn, and the parameters of the weight-learning-based gating network are updated after a preset number of times. When the training reaches the preset conditions, the training is terminated and the trained weight-learning-based gating network is output.

[0015] Optionally, updating the parameters of the gating network based on weight learning after every preset number of times specifically includes:

[0016] After every preset number of times, the next stage state corresponding to each training instance in the preset number of times is obtained;

[0017] Calculating the cumulative discount reward based on the next stage state, traversing the cumulative discount reward for a preset number of times, and calculating the future reward based on the discount factor;

[0018] Normalize the future returns and obtain the first tensor and the second tensor based on the log probability and the policy entropy;

[0019] According to the first tensor, the second tensor and future rewards, the final loss function is calculated, and the parameters of the gating network based on weight learning are updated according to the loss function.

[0020] Optionally, constructing a scheduling graph according to the target job shop scheduling problem, processing the scheduling graph based on a plurality of pre-set expert networks and a trained gating network based on weight learning to obtain a first target strategy specifically includes:

[0021] When the target gating network is a gating network based on weight learning, constructing a scheduling graph according to the target job shop scheduling problem;

[0022] Iteratively processing the scheduling graph based on a plurality of pre-set expert networks and a trained weight-learning-based gating network, and outputting an optimal action during each iteration;

[0023] When all actions corresponding to the target job shop scheduling problem are selected, the first target strategy is generated based on the best action obtained in each iteration.

[0024] Optionally, the iterative processing of the scheduling graph based on the pre-set multiple expert networks and the trained weight-learning-based gating network, and outputting the optimal action in each iteration process, specifically includes:

[0025] Processing the scheduling graph based on a target graph neural network to obtain node state features;

[0026] Inputting the node state features into a plurality of pre-set expert networks respectively, and obtaining the expert strategy generated by each expert network;

[0027] The expert strategy of each expert network is input into the trained weight-learning-based gating network to obtain the best action sampled by the gating network, and the scheduling graph is updated according to the best action.

[0028] Optionally, selecting a target expert network from a plurality of pre-set expert networks according to the target job shop scheduling problem specifically includes:

[0029] Obtaining size information of the target job shop scheduling problem and obtaining the sizes of a plurality of pre-set expert networks;

[0030] Based on the size of each expert network and the size of the target job shop scheduling problem, the similarity of each expert network is calculated;

[0031] According to the similarity of all expert networks, a target expert network is selected from multiple pre-set expert networks.

[0032] Optionally, the similarity of each expert network is calculated based on the size of each expert network and the size information of the target job shop scheduling problem, specifically including:

[0033] Based on the size of each expert network and the size of the target job shop scheduling problem, Manhattan distance, Euclidean distance, Chebyshev distance, and Minkowski distance are calculated;

[0034] After normalizing the Manhattan distance, Euclidean distance, Chebyshev distance, and Minkowski distance corresponding to each expert network, the similarity of each expert network is generated.

[0035] In addition, to achieve the above-mentioned purpose, the present invention further provides a supply chain process operation scheduling system based on a hybrid expert model, wherein the supply chain process operation scheduling system based on a hybrid expert model includes:

[0036] A selection module is used to obtain a target job shop scheduling problem and select a target gating network from a static similarity-based expert selection gating network and a weight learning-based gating network according to a preset threshold;

[0037] a first processing module configured to construct a scheduling graph according to a target job shop scheduling problem when the target gating network is a weight learning-based gating network, and to process the scheduling graph based on a plurality of pre-set expert networks and the trained weight learning-based gating network to obtain a first target strategy;

[0038] The second processing module is used to select a target expert network from a plurality of pre-set expert networks according to the target job shop scheduling problem when the target gating network is an expert selection gating network based on static similarity, and process the target job shop scheduling problem through the target expert network to obtain a second target strategy.

[0039] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a supply chain process job scheduling program based on a hybrid expert model stored on the memory and runnable on the processor, and when the supply chain process job scheduling program based on a hybrid expert model is executed by the processor, the steps of the supply chain process job scheduling method based on a hybrid expert model as described above are implemented.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a supply chain process job scheduling program based on a hybrid expert model, and when the supply chain process job scheduling program based on a hybrid expert model is executed by a processor, the steps of the supply chain process job scheduling method based on a hybrid expert model as described above are implemented.

[0041] In the present invention, a target job shop scheduling problem is obtained, and a target gating network is selected from a static similarity-based expert selection gating network and a weight learning-based gating network according to a preset threshold. When the target gating network is a weight learning-based gating network, a scheduling graph is constructed according to the target job shop scheduling problem, and the scheduling graph is processed based on multiple pre-set expert networks and the trained weight learning-based gating network to obtain a first target strategy. When the target gating network is a static similarity-based expert selection gating network, a target expert network is selected from multiple pre-set expert networks according to the target job shop scheduling problem, and the target job shop scheduling problem is processed using the target expert network to obtain a second target strategy. The present invention utilizes the respective advantages of different expert networks and effectively integrates them through a gating network to achieve a more flexible and robust scheduling strategy. The gating network dynamically calculates the weight of each expert model based on the current environmental state to determine which expert strategy or which expert set's candidate action should be most adopted in a specific situation. In this way, the hybrid expert model can not only significantly improve the adaptability and efficiency of the scheduling strategy, but also adapt to changes in the production environment through continuous learning, thereby achieving continuous optimization of complex scheduling tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of a preferred embodiment of the supply chain process operation scheduling method based on the hybrid expert model of the present invention;

[0043] Figure 2 It is a schematic diagram of the process of two gated networks participating in the processing of the supply chain process operation scheduling method based on the hybrid expert model of the present invention;

[0044] Figure 3 Schematic diagram of the training algorithm in the supply chain process operation scheduling method based on the hybrid expert model of the present invention;

[0045] Figure 4 It is a schematic diagram of a parameter updating algorithm in a supply chain process operation scheduling method based on a hybrid expert model of the present invention;

[0046] Figure 5 Schematic diagram of a gating network framework for expert selection based on static similarity in a supply chain process job scheduling method based on a hybrid expert model of the present invention;

[0047] Figure 6 This is a structural diagram of a preferred embodiment of the supply chain process operation scheduling system based on a hybrid expert model of the present invention;

[0048] Figure 7 FIG. 4 is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] The Job Shop Scheduling Problem (JSSP) is a key issue in optimizing the production process. The goal is to rationally arrange the execution sequence of each process to minimize the total time (makespan) to complete all tasks. Traditional optimization algorithms for JSSP, such as heuristic or metaheuristic algorithms, perform well for small-scale or specific types of problems. However, they are often inefficient when faced with the complexity and scale of real-world problems and struggle to adapt to the variability of production environments. Deep reinforcement learning (DRL) is currently being used to address JSSP. These methods demonstrate potential for solving complex scheduling problems by learning scheduling policies directly from data. However, despite the theoretical superiority of deep reinforcement learning, models with different training data sizes have different adaptability to the corresponding problems. Some DRL methods still perform poorly when faced with a large number of JSP instances with varying scales. Consequently, when fed with problems of unknown scale, existing models cannot produce scheduling results close to the optimal model, resulting in an inability to accurately address diverse JSP problems.

[0051] In response to one or more of the above problems, the present invention obtains a target job shop scheduling problem, and selects a target gating network from an expert selection gating network based on static similarity and a gating network based on weight learning according to a preset threshold; when the target gating network is a gating network based on weight learning, a scheduling graph is constructed according to the target job shop scheduling problem, and the scheduling graph is processed based on multiple pre-set expert networks and the trained gating network based on weight learning to obtain a first target strategy; when the target gating network is an expert selection gating network based on static similarity, a target expert network is selected from multiple pre-set expert networks according to the target job shop scheduling problem, and the target job shop scheduling problem is processed by the target expert network to obtain a second target strategy.

[0052] The supply chain process operation scheduling method based on hybrid expert model described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the supply chain process operation scheduling method based on the hybrid expert model includes the following steps:

[0053] Step S10: Obtain a target job shop scheduling problem, and select a target gating network from an expert selection gating network based on static similarity and a gating network based on weight learning according to a preset threshold.

[0054] Specifically, in this invention, when processing JSSP instances of varying sizes, the model adaptively selects the expert network most appropriate for the current task size. For example, when processing smaller JSSP instances, the model may favor expert network decisions optimized for small-scale problems; whereas, when faced with larger problem instances, it may rely more heavily on strategies from larger expert networks. This dynamic adjustment strategy enables the model to fully leverage the strengths of different experts in their respective fields, effectively expanding the breadth and richness of its decision-making, overcoming the limitations of relying on a single network, and significantly improving overall scheduling efficiency and performance.

[0055] Specifically, the present invention employs different gating networks to select the most appropriate processing method for different situations. Specifically, in application, the present invention allows the user to set a threshold for the number of experts and a threshold for scheduling time, depending on the application scenario. Three threshold selection methods are also provided: a first threshold selection method selects an expert number threshold, a second threshold selection method selects a scheduling time threshold, and a third threshold selection method selects both an expert number threshold and a scheduling time threshold. Furthermore, when applying, the corresponding threshold selection method is selected according to the application scenario, and the expert number threshold and scheduling time threshold are set. When the first threshold selection method is used, when the number of pre-set multiple expert networks is less than the expert number threshold, the expert selection gating network based on static similarity is adopted, otherwise the target gating network is selected from the gating network based on weight learning; when the second threshold selection method is selected, when the actual requirement for scheduling time is higher than the scheduling time threshold, that is, the actual scheduling time is less than the scheduling time threshold, the expert selection gating network based on static similarity is adopted, otherwise the target gating network is selected from the gating network based on weight learning; when the third threshold selection method is selected, when the number of pre-set multiple expert networks is less than the expert number threshold and the actual requirement for scheduling time is higher than the scheduling time threshold, the expert selection gating network based on static similarity is adopted, otherwise the target gating network is selected from the gating network based on weight learning.

[0056] Further, if Figure 2 As shown in the figure, the expert selection gating network based on static similarity and the gating network based on weight learning are drawn in the same figure. When the target gating network is the gating network based on weight learning, the scheduling graph is obtained, and after further processing through the graph structure network, it is input into the expert network, and the gating network is used to obtain the corresponding action; for the expert selection gating network based on static similarity, the expert network is selected according to the scale of the problem instance to process the corresponding problem.

[0057] Step S20: When the target gating network is a gating network based on weight learning, a scheduling graph is constructed according to the target job shop scheduling problem, and the scheduling graph is processed based on a plurality of pre-set expert networks and the trained gating network based on weight learning to obtain a first target strategy.

[0058] When the target gating network is a gating network based on weight learning, the solution process of the job shop scheduling problem (JSSP) is modeled as a Markov decision process (MDP). Among them, the state (State): The state space S fully reflects the current various characteristics of the job shop, including the remaining processing time of the job, the current state of the machine, and the number of completed jobs. This information provides a comprehensive basis for decision-making, enabling the model to understand and respond to the operating status of the shop in real time. For example, the number of jobs, the total number of machines, the number of remaining jobs, the number of idle machines, and the ratio of jobs to machines are all key factors affecting decision-making. The dynamic changes of these characteristics, such as the reduction in the number of remaining jobs due to the completion of the job, or the update of the state of the machine after completing the task, are reflected in real time in the state s t Action: The action space A includes adjustments to the weights of each expert model in the gating network. These adjustments affect the strategy of selecting different expert model combinations to deal with the current problem, that is, the action at taken at each time step t. The gating network dynamically allocates the weights of each expert model by analyzing the current state in real time to optimize the decision-making process. The weight allocation can be based on a specific algorithm (such as static weight allocation based on distance metric) or dynamically learned through deep reinforcement learning. Reward: The reward function R(s t , a t ) According to the action a taken t The model provides feedback on the impact of a decision on the job shop scheduling efficiency. For example, if a decision significantly reduces the maximum completion time Makespan, the action will receive a positive reward; if the decision result increases Makespan, it will receive a negative reward. This mechanism encourages the model to continuously optimize decisions to shorten the total completion time. State transition (Transition Probability): Transition probability P(s t+1 |s t , a t ) describes the transition from the current state s t Through action a t Transition to the next state s t+1This process is influenced by both the actual operational dynamics of the shop floor and the decisions made by the gating network. State transitions reflect the progress of the job and the changes in machine status, and are a key step in the system's progress toward its optimization goal. Through this MDP-based model, the hybrid expert system is able to learn and self-optimize through continuous interaction with the job shop environment. By dynamically adjusting the weights of each expert model, the system can more effectively address various scheduling challenges, gradually improving scheduling efficiency and productivity. This approach not only improves the adaptability and accuracy of decision-making but also continuously optimizes the operational efficiency of problem-solving across the entire job shop through real-time learning and feedback mechanisms.

[0059] Correspondingly, n JSSP solution models are selected, namely {E1, E2, ..., En}, corresponding to different expert networks. The gating network G receives the current JSSP state s of the environment t As input, and output the corresponding weight vector w t =[w t1 , w t2 ,...,w tn ], these weights satisfy the following conditions:

[0060]

[0061] This condition ensures that at any time step t, the sum of the weights is 1, effectively distributing the decision weights to each expert network. i Based on the current state s t Output an action probability distribution π i (a t |s t ). The final action probability distribution π(a t |s t ) is calculated by the following weighted sum:

[0062]

[0063] This formula shows that the gating network assigns weights to each expert network and integrates the decision-making capabilities of different experts in the current state, enabling the model to dynamically adjust the decision-making strategy according to the characteristics of the problem.

[0064] In addition, the present invention sets the definition of the reward function R and the discount factor γ, and the expected return V(st) can be expressed as:

[0065]

[0066] Among them, V(S t ) is the value function, which means that in state s tThe expected cumulative return (long-term value) that can be obtained in the future after taking action according to a certain strategy; Expected Value represents the weighted average return of all possible future paths under the guidance of the strategy; represents the accumulation from the current time step k = 0 to the infinite time steps in the future, which represents the cumulative future returns; γ k Represents the k-th power of the discount factor, which is used to weigh the impact of current and future rewards; γ∈[0,1), when γ is closer to 1, the more emphasis is placed on long-term returns; when it is closer to 0, the more emphasis is placed on immediate rewards; R(s t+k ,a t+k ) represents the reward function, which means taking action a at time t+k t+k After that, in state s t+k The instant reward obtained under V(S t ) represents the value function, which means that in state s t The expected cumulative return (long-term value) that can be obtained in the future after taking action according to a certain strategy; t+k represents the state of the current time step t, i.e., the observation of the environment or the description of the system; a t+k Represents the action taken at the current time step, which is used to interact with the environment and affect the next state.

[0067] This expression describes the t Next take action a t The potential long-term value provides a quantitative basis for evaluating the model's decision-making in complex environments. Training the gating network not only optimizes immediate decision-making but also seeks the optimal strategy in multi-step decision-making processes, providing an efficient solution to job shop scheduling problems.

[0068] Furthermore, the training process of the gating network based on weight learning specifically includes:

[0069] Generate multiple training instances within a preset range, process each training instance to obtain the corresponding node state feature, and input it into a preset expert network to obtain the corresponding action probability distribution and policy entropy, and record the logarithmic probability and policy entropy of each action probability distribution corresponding to the action;

[0070] According to the action probability distribution, a gating network based on weight learning is used to select the final action of the corresponding training instance from the action probability distribution generated by all expert networks, and the next stage state is obtained based on the final action and the corresponding training instance;

[0071] Each training step is processed in turn, and the parameters of the weight-learning-based gating network are updated after a preset number of times. When the training reaches the preset conditions, the training is terminated and the trained weight-learning-based gating network is output.

[0072] Correspondingly, the specific training process of the gating network based on weight learning is as follows Figure 3 As shown in the figure, the basic model configuration is first defined. The algorithm aims to train a gating network and a set of policy models for the job shop scheduling problem (JSSP). The algorithm begins by loading a preset list of policy models, which serve as the expert systems in the training process. Next, the algorithm initializes the gating network and sets its structure and initial weights based on the provided configuration parameters. During training, the algorithm iteratively generates a series of JSSP instances. For each instance, the algorithm initializes the simulation environment and obtains its initial state. Each expert model generates an action probability distribution based on the current state, while the gating network provides expert weights. These weights are combined to generate a new action probability distribution, from which actions are ultimately sampled. The selected actions are applied to the environment, updating the environment state and obtaining new reward information. Every ten steps, the algorithm updates the parameters of the gating network based on the collected data to optimize the network's decision-making capabilities. The goal of this training process is to enable the gating network to more effectively integrate the knowledge of different experts through iterative learning, improving its ability to handle complex scheduling tasks. Ultimately, the training algorithm outputs an optimized gating network and a set of fine-tuned policy models, which work together to provide a solution to the JSSP problem.

[0073] Furthermore, during training, the model is initialized. This involves loading multiple expert policy models from the training_config configuration file and storing them in policy_list. The gating network gating_net is initialized, which dynamically assigns weights to each expert policy. An optimizer is configured for the gating network to prepare for subsequent reinforcement learning training.

[0074] Afterwards, a training loop is performed, setting the corresponding training set, generating random training instances, and constructing the environment. Within a preset range, JSSP instances are randomly generated; batches of batch_size JSSP scheduling instances inst are generated; the model input tensor input_shape_tensor is constructed for subsequent model use; the scheduling environment env is initialized, and the expert network's initial policy type and parameters are configured; the initial state G, the executable action set action_set, the optimal mark optimal_mark, and the path information paths are obtained from the environment. Training is then performed on each JSSP instance, looping through the reinforcement learning decision operations. This creates an empty list sampled_a_list to store the action probability distribution generated by each expert. Each expert policy is iterated over, and during the iteration, the decision of each expert network, the corresponding action set action_set, and the probability distribution sampled_a of the sampled action are obtained. The logarithmic probability log_p of the action under the current expert and the policy entropy ent (used for reward calculation) are also recorded. The sampled action probability distribution of the current expert is added to sampled_a_list. Then, the gating mechanism select_action is called to select the final execution action final_a according to the gating network and the expert action. Final_a is applied to the environment to obtain the next state, including the immediate reward reward, the new action set action_set, the optimal path information, etc. In addition, the present invention is set to trigger the reinforcement learning parameter update of the MoE network every ten iterations; at the same time, the training progress information is displayed to facilitate monitoring of the training process.

[0075] Furthermore, the updating of the parameters of the gating network based on weight learning after every preset number of times specifically includes:

[0076] After every preset number of times, the next stage state corresponding to each training instance in the preset number of times is obtained;

[0077] Calculating the cumulative discount reward based on the next stage state, traversing the cumulative discount reward for a preset number of times, and calculating the future reward based on the discount factor;

[0078] Normalize the future returns and obtain the first tensor and the second tensor based on the log probability and the policy entropy;

[0079] According to the first tensor, the second tensor and future rewards, the final loss function is calculated, and the parameters of the gating network based on weight learning are updated according to the loss function.

[0080] Specifically, if Figure 4As shown, in the present invention, when updating parameters, the environment is initialized first: the scheduling environment env is initialized according to the scheduling instance inst; max_itr, that is, the maximum number of iterations for cyclic training is set, and training is stopped when the number of training times is met; at the same time, an empty sampled_a_list list is created to save the action probability distribution generated by each expert. When updating parameters, the expert strategy list is traversed, and the corresponding action probability distribution and entropy are obtained through the strategy network of the current expert list. The action probability distribution sampled_a of the currently traversed expert is added to the sampled_a_list, and the gating network gating_net is used to score or weightedly sample all expert actions. The final action final_a is sampled from the new action probability distribution for actual scheduling operations, and the action final_a is executed in the environment. The system returns new state information to update the environment, including: scheduling graph G, reward reward, new action set action_set, marking information optimal_mark and path path information paths; perform a policy learning operation every ten iterations. When optimizing parameters, calculate the cumulative discounted reward, traverse the historical rewards, and use the discount factor gamma to calculate the future returns from back to front; standardize the returns (zero mean, unit variance) to improve learning stability; then merge the log probability and entropy values ​​output by all experts into a unified tensor, which contains the first tensor log_probs and the second tensor ents, to prepare for loss calculation, construct the reinforcement learning loss function (REINFORCE), calculate the final loss value and perform backpropagation, and update the gating network parameters.

[0081] The process of normalizing the future returns is expressed as:

[0082]

[0083] normalized_returns are the normalized future returns.

[0084] The process of obtaining the first tensor and the second tensor from the logarithmic probability and policy entropy is expressed as:

[0085] log_probs←concat(log_p1,...,log_p N )

[0086] ents←torch.cat(ents,dim=-1);

[0087] The corresponding loss function is:

[0088] loss←-(logprobs*normalized_return+args.ent_coef*ents).sum(dim=-1).mean( );

[0089] Among them, loss is the loss value.

[0090] Furthermore, the method of constructing a scheduling graph based on the target job shop scheduling problem, processing the scheduling graph based on a plurality of pre-set expert networks and a trained weight learning-based gating network to obtain a first target strategy specifically includes:

[0091] When the target gating network is a gating network based on weight learning, constructing a scheduling graph according to the target job shop scheduling problem;

[0092] Iteratively processing the scheduling graph based on a plurality of pre-set expert networks and a trained weight-learning-based gating network, and outputting an optimal action during each iteration;

[0093] When all actions corresponding to the target job shop scheduling problem are selected, the first target strategy is generated based on the best action obtained in each iteration.

[0094] Specifically, in the present invention, after training the corresponding weight learning-based gating network, a disjunctive graph model for job shop scheduling is constructed according to the input JSSP instance, wherein the nodes represent the processes of the jobs and the edges represent the constraint relationships between the processes; the input includes the processing time matrix of the jobs on each machine and the job operation specified machine matrix, and the scheduling disjunctive graph structure is generated in combination with the job sequence and resource constraints; then the information of the nodes and edges in the scheduling graph is initialized and encoded, and features such as job identification, process location, processing time, resource occupancy, etc. are embedded to generate a high-dimensional vector representation for subsequent processing; the feature extraction layer in the graph neural network (here the graph attention network GAT is selected) is used to perform multiple rounds of aggregation and update of node features, and local and global structural information is extracted through the message passing mechanism to obtain a state representation reflecting the constraint relationship and resource status between jobs; the node state features after k-layer feature extraction processing of the graph neural network are output, and the state features are used as the input of the expert gating network, the extracted feature network is used as the state representation in the MDP process, and the gating network constitutes a reinforcement learning framework to provide basic data support for scheduling strategy learning and optimization.

[0095] Furthermore, the scheduling graph is iteratively processed based on the pre-set multiple expert networks and the trained weight-learning-based gating network, and the optimal action is output during each iteration, specifically including:

[0096] Processing the scheduling graph based on a target graph neural network to obtain node state features;

[0097] Inputting the node state features into a plurality of pre-set expert networks respectively, and obtaining the expert strategy generated by each expert network;

[0098] The expert strategy of each expert network is input into the trained weight-learning-based gating network to obtain the best action sampled by the gating network, and the scheduling graph is updated according to the best action.

[0099] Specifically, the gating network is the core coordination component of the entire framework. It receives the problem size information (Question Size) from the feature extraction module as input and generates weights (Expert 1 Weight, Expert 2 Weight, ..., Expert n Weight) corresponding to each expert network. These weights are combined with the action probability distribution output by the expert network through a weighted average operation to generate a new comprehensive probability distribution (New Distribution). Finally, a sampling operation selects an action (Sampled Action) from the new probability distribution. This action will interact with the JSSP environment (JSSP ENV). This process not only optimizes the contribution of each expert network but also enables the model to flexibly handle JSSP problems of various sizes. For example, when the gating network determines that the current problem is more suitable for a large-scale expert network, it will assign a higher weight to the large-scale expert network, thereby making the final decision more inclined to the recommendations of the large-scale expert network.

[0100] After completing an action selection, the loop operation is performed again until all action selections corresponding to the target job shop scheduling problem are completed. According to the best action obtained in each iteration, the first target strategy is generated, that is, the first target strategy includes the action selection of each step of the target job shop scheduling problem.

[0101] Step S30: When the target gating network is an expert selection gating network based on static similarity, a target expert network is selected from a plurality of pre-set expert networks according to the target job shop scheduling problem, and the target job shop scheduling problem is processed by the target expert network to obtain a second target strategy.

[0102] Specifically, when selecting an expert selection gating network based on static similarity, the network uses a multi-dimensional distance metric as the selection logic, without the need for a complex learning process, and directly selects the target expert network according to the corresponding situation, thereby performing corresponding processing.

[0103] The step of selecting a target expert network from a plurality of pre-set expert networks according to the target job shop scheduling problem specifically includes:

[0104] Obtaining size information of the target job shop scheduling problem and obtaining the sizes of a plurality of pre-set expert networks;

[0105] Based on the size of each expert network and the size of the target job shop scheduling problem, the similarity of each expert network is calculated;

[0106] According to the similarity of all expert networks, a target expert network is selected from multiple pre-set expert networks.

[0107] Specifically, if Figure 5 As shown, the gating network first receives the problem instance size information P(j), P(m) and compares it with the scale EXPi(j), EXPi(m), 1≤i≤n of each expert in the expert list; here j and m represent the number of jobs and the number of machines, respectively.

[0108] Then, the similarity of each expert network is calculated based on the size of each expert network and the size information of the target job shop scheduling problem, and the target expert network corresponding to the seat with the smallest similarity is selected.

[0109] Furthermore, the similarity of each expert network is calculated based on the size of each expert network and the size information of the target job shop scheduling problem, specifically including:

[0110] Based on the size of each expert network and the size of the target job shop scheduling problem, Manhattan distance, Euclidean distance, Chebyshev distance, and Minkowski distance are calculated;

[0111] After normalizing the Manhattan distance, Euclidean distance, Chebyshev distance, and Minkowski distance corresponding to each expert network, the similarity of each expert network is generated.

[0112] Specifically, the Manhattan distance, Euclidean distance, Chebyshev distance, and Minkowski distance are calculated using the scale of each expert network and the size information of the target job shop scheduling problem.

[0113] Among them, Manhattan Distance:

[0114]

[0115] Euclidean Distance:

[0116]

[0117] Chebyshev Distance:

[0118]

[0119] Minkowski Distance:

[0120]

[0121] Among them, p is the parameter of Minkowski distance, which is usually set to 3. The smaller the value calculated by the above distance formula, the higher the similarity between the expert scale and the problem instance.

[0122] The Manhattan distance, Euclidean distance, Chebyshev distance and Minkowski distance corresponding to each expert network are standardized, that is, one or more of the Manhattan distance, Euclidean distance, Chebyshev distance and Minkowski distance are selected as the value for calculating similarity according to actual needs and subsequent optimization.

[0123] In one embodiment of the present invention, four values ​​are selected and averaged to obtain the similarity. The calculation process is expressed as follows:

[0124]

[0125] The expert with the smallest distance is selected as the decision model. This selection logic is reflected in the following formula:

[0126]

[0127] The present invention obtains a target job shop scheduling problem and selects a target gating network from a static similarity-based expert selection gating network and a weight learning-based gating network based on a preset threshold. When the target gating network is a weight learning-based gating network, a scheduling graph is constructed based on the target job shop scheduling problem. The scheduling graph is processed based on multiple pre-set expert networks and a trained weight learning-based gating network to obtain a first target strategy. When the target gating network is a static similarity-based expert selection gating network, a target expert network is selected from multiple pre-set expert networks based on the target job shop scheduling problem. The target job shop scheduling problem is processed using the target expert network to obtain a second target strategy. The present invention utilizes the respective advantages of different expert networks and effectively integrates them through a gating network to achieve a more flexible and robust scheduling strategy. The gating network dynamically calculates the weight of each expert model based on the current environmental state to determine which expert strategy or which expert set's candidate action is most appropriate to adopt in a specific situation. In this way, the hybrid expert model can not only significantly improve the adaptability and efficiency of the scheduling strategy, but also adapt to changes in the production environment through continuous learning, thereby achieving continuous optimization of complex scheduling tasks.

[0128] Further, if Figure 6 As shown, based on the above-mentioned supply chain process operation scheduling method based on the hybrid expert model, the present invention also provides a supply chain process operation scheduling system based on the hybrid expert model, wherein the supply chain process operation scheduling system based on the hybrid expert model includes:

[0129] A selection module 91 is configured to obtain a target job shop scheduling problem and select a target gating network from among the expert selection gating network based on static similarity and the gating network based on weight learning according to a preset threshold;

[0130] A first processing module 92 is configured to construct a scheduling graph according to the target job shop scheduling problem when the target gating network is a weight learning-based gating network, and process the scheduling graph based on the pre-set multiple expert networks and the trained weight learning-based gating network to obtain a first target strategy;

[0131] The second processing module 93 is used to select a target expert network from a plurality of pre-set expert networks according to the target job shop scheduling problem when the target gating network is an expert selection gating network based on static similarity, and process the target job shop scheduling problem through the target expert network to obtain a second target strategy.

[0132] Further, if Figure 7 As shown, based on the above-mentioned supply chain process operation scheduling method and system based on the hybrid expert model, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 7 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0133] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a supply chain process job scheduling program 40 based on a hybrid expert model is stored on the memory 20, and the supply chain process job scheduling program 40 based on a hybrid expert model can be executed by the processor 10, thereby realizing the supply chain process job scheduling method based on a hybrid expert model in the present invention.

[0134] In some embodiments, the processor 10 can be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the supply chain process job scheduling method based on the hybrid expert model.

[0135] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch screen, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0136] In one embodiment, when the processor 10 executes the supply chain process job scheduling program 40 based on the hybrid expert model in the memory 20 , the steps of the above supply chain process job scheduling method based on the hybrid expert model are implemented.

[0137] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a supply chain process job scheduling program based on a hybrid expert model, and when the supply chain process job scheduling program based on a hybrid expert model is executed by a processor, the steps of the supply chain process job scheduling method based on a hybrid expert model as described above are implemented.

[0138] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0139] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0140] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A supply chain process operation scheduling method based on a hybrid expert model, characterized in that: The supply chain process operation scheduling method based on the hybrid expert model includes: Obtain a target job shop scheduling problem, and select a target gating network from a static similarity-based expert selection gating network and a weight learning-based gating network according to a preset threshold; When the target gating network is a weight learning-based gating network, a scheduling graph is constructed according to the target job shop scheduling problem, and the scheduling graph is processed based on the pre-set multiple expert networks and the trained weight learning-based gating network to obtain a first target strategy; When the target gating network is an expert selection gating network based on static similarity, a target expert network is selected from a plurality of pre-set expert networks according to the target job shop scheduling problem, and the target job shop scheduling problem is processed by the target expert network to obtain a second target strategy.

2. The supply chain process operation scheduling method based on hybrid expert model according to claim 1 is characterized in that: The training process of the gating network based on weight learning specifically includes: Generate multiple training instances within a preset range, process each training instance to obtain the corresponding node state feature, and input it into a preset expert network to obtain the corresponding action probability distribution and policy entropy, and record the logarithmic probability and policy entropy of each action probability distribution corresponding to the action; According to the action probability distribution, a gating network based on weight learning is used to select the final action of the corresponding training instance from the action probability distribution generated by all expert networks, and the next stage state is obtained based on the final action and the corresponding training instance; Each training step is processed in turn, and the parameters of the weight-learning-based gating network are updated after a preset number of times. When the training reaches the preset conditions, the training is terminated and the trained weight-learning-based gating network is output.

3. The supply chain process operation scheduling method based on hybrid expert model according to claim 2 is characterized in that: The updating of the parameters of the gating network based on weight learning after every preset number of times specifically includes: After every preset number of times, the next stage state corresponding to each training instance in the preset number of times is obtained; Calculating the cumulative discount reward based on the next stage state, traversing the cumulative discount reward for a preset number of times, and calculating the future reward based on the discount factor; Normalize the future returns and obtain the first tensor and the second tensor based on the log probability and the policy entropy; According to the first tensor, the second tensor and future rewards, the final loss function is calculated, and the parameters of the gating network based on weight learning are updated according to the loss function.

4. The supply chain process operation scheduling method based on hybrid expert model according to claim 1 is characterized in that: The method of constructing a scheduling graph based on the target job shop scheduling problem and processing the scheduling graph based on a plurality of pre-set expert networks and a trained gating network based on weight learning to obtain a first target strategy specifically includes: When the target gating network is a gating network based on weight learning, constructing a scheduling graph according to the target job shop scheduling problem; Iteratively processing the scheduling graph based on a plurality of pre-set expert networks and a trained weight-learning-based gating network, and outputting an optimal action during each iteration; When all actions corresponding to the target job shop scheduling problem are selected, the first target strategy is generated based on the best action obtained in each iteration.

5. The supply chain process operation scheduling method based on hybrid expert model according to claim 4 is characterized in that: The scheduling graph is iteratively processed based on the pre-set multiple expert networks and the trained weight-learning-based gating network, and the optimal action is output during each iteration, specifically including: Processing the scheduling graph based on a target graph neural network to obtain node state features; Inputting the node state features into a plurality of pre-set expert networks respectively, and obtaining the expert strategy generated by each expert network; The expert strategy of each expert network is input into the trained weight-learning-based gating network to obtain the best action sampled by the gating network, and the scheduling graph is updated according to the best action.

6. The supply chain process operation scheduling method based on hybrid expert model according to claim 1 is characterized in that: The step of selecting a target expert network from a plurality of pre-set expert networks according to the target job shop scheduling problem specifically includes: Obtaining size information of the target job shop scheduling problem and obtaining the sizes of a plurality of pre-set expert networks; Based on the size of each expert network and the size of the target job shop scheduling problem, the similarity of each expert network is calculated; According to the similarity of all expert networks, a target expert network is selected from multiple pre-set expert networks.

7. The supply chain process operation scheduling method based on hybrid expert model according to claim 1 is characterized in that: The similarity of each expert network is calculated based on the size of each expert network and the size of the target job shop scheduling problem, specifically including: Based on the size of each expert network and the size of the target job shop scheduling problem, Manhattan distance, Euclidean distance, Chebyshev distance, and Minkowski distance are calculated; After normalizing the Manhattan distance, Euclidean distance, Chebyshev distance, and Minkowski distance corresponding to each expert network, the similarity of each expert network is generated.

8. A supply chain process operation scheduling system based on a hybrid expert model, characterized by: The supply chain process operation scheduling system based on the hybrid expert model includes: A selection module is used to obtain a target job shop scheduling problem and select a target gating network from a static similarity-based expert selection gating network and a weight learning-based gating network according to a preset threshold; a first processing module configured to construct a scheduling graph according to a target job shop scheduling problem when the target gating network is a weight learning-based gating network, and to process the scheduling graph based on a plurality of pre-set expert networks and the trained weight learning-based gating network to obtain a first target strategy; The second processing module is used to select a target expert network from a plurality of pre-set expert networks according to the target job shop scheduling problem when the target gating network is an expert selection gating network based on static similarity, and process the target job shop scheduling problem through the target expert network to obtain a second target strategy.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a supply chain process job scheduling program based on a hybrid expert model stored in the memory and runnable on the processor. When the supply chain process job scheduling program based on a hybrid expert model is executed by the processor, the steps of the supply chain process job scheduling method based on a hybrid expert model as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a supply chain process job scheduling program based on a hybrid expert model. When the supply chain process job scheduling program based on a hybrid expert model is executed by a processor, the steps of the supply chain process job scheduling method based on a hybrid expert model as described in any one of claims 1 to 7 are implemented.

Images