Hybrid flow shop dynamic scheduling method and system considering machine predictive maintenance

Abstract

The invention belongs to the field of workshop scheduling, and particularly discloses a hybrid flow workshop dynamic scheduling method and system considering machine predictive maintenance, and the method comprises the steps: taking the minimization of total completion time, maintenance cost and processing cost as the target, building a hybrid flow workshop scheduling problem as a multi-target joint optimization model, and carrying out the optimization of the multi-target joint optimization model; setting intelligent agents with the same number as the processing stages, and constructing a Markov decision process; each agent has an independent scheduling network, including a workpiece scheduling network and a machine scheduling network; based on a Markov decision process, training an intelligent agent, maintaining a workshop at an operation and maintenance point, and respectively calling a workpiece scheduling network and a machine scheduling network at a scheduling point to select workpieces and machines; and after training is completed, workshop dynamic scheduling is realized based on the trained intelligent agent. According to the method, the problem of hybrid flow shop dynamic scheduling considering machine predictive maintenance is effectively solved through workshop scheduling integrating machine operation and maintenance, and the method has good dynamics and adaptivity.

Description

Technical Field

[0001] This invention belongs to the field of workshop scheduling, and more specifically, relates to a dynamic scheduling method and system for hybrid assembly line workshops that takes into account predictive maintenance of machines. Background Technology

[0002] Under intense market competition, the manufacturing industry faces enormous economic pressure and environmental challenges. To meet diverse customer needs and save costs, companies continuously strive to improve the scalability, flexibility, and reliability of their manufacturing systems. In actual production, machine reliability gradually decreases over time. Predictive maintenance can significantly increase the production reliability of manufacturing systems. However, integrating predictive maintenance with shop floor scheduling to achieve joint scheduling presents a challenge. Furthermore, actual manufacturing systems are prone to various disruptive events, increasing the complexity and uncertainty of the shop floor.

[0003] Therefore, considering predictive maintenance of machines, researching dynamic and real-time hybrid flow shop scheduling methods to rationally arrange machine maintenance and production in the shop, thereby improving enterprise operational efficiency, has significant theoretical and practical value. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a dynamic scheduling method and system for hybrid assembly line workshops that takes into account predictive maintenance of machines. Its purpose is to effectively handle interference events and improve the efficiency and reliability of dynamic scheduling of hybrid assembly line workshops.

[0005] To achieve the above objectives, according to one aspect of the present invention, a dynamic scheduling method for hybrid flow shop considering predictive maintenance of machines is proposed, comprising the following steps: With the goal of minimizing total completion time, maintenance cost, and processing cost, the hybrid flow shop scheduling problem is established as a multi-objective joint optimization model; The system sets up an agent with the same number of processing stages. Each agent makes decisions for the corresponding processing stage, and a Markov decision process is constructed by combining a multi-objective joint optimization model. Each agent has an independent scheduling network, including a workpiece scheduling network and a machine scheduling network. All agents share a global evaluation network, including a workpiece global evaluation network and a machine global evaluation network, which are used to evaluate the scheduling quality of the workpiece scheduling network and the machine scheduling network, respectively. Based on the Markov decision process, the agent is trained by reinforcement learning. During the training process, the workshop is maintained at the operation and maintenance point, and at the scheduling point, the workpiece scheduling network is called to select workpieces according to the current state, and then the machine scheduling network is called to select machines for the selected workpieces. After training, the global evaluation network is removed, and dynamic scheduling of the hybrid assembly line is achieved based on the scheduling network trained by each agent.

[0006] As a further preferred embodiment, the job scheduling network includes: an LSTM-based encoding layer for encoding job input sequences of different lengths; an attention-based decoding layer for decoding the encoded sequences and assigning different score values ​​to the jobs; and a Softmax-based pointer layer for generating corresponding selection probabilities based on the different score values ​​of the jobs, and then outputting the selected job sequence number.

[0007] As a further preferred embodiment, the machine scheduling network includes: a state expansion layer, which expands the dimensions of the input machine state through MLP; a Transformer-based encoding layer, which encodes the expanded machine state; and an MLP-based decoding layer, which assigns corresponding weights to the machine according to the encoded state through MLP, generates the machine selection probability, and outputs the selected machine number.

[0008] As a further preferred option, maintenance of the workshop is carried out at the operation and maintenance point, including: Obtain the machine's actual processing time, and then calculate the machine's health factors. RM c And compare it with the predictive maintenance threshold W1 and the opportunistic maintenance threshold W2; if RM c If W1 ≥ W2, then predictive maintenance is performed on the machine; if W2 ≤ W1, then predictive maintenance is performed on the machine. RM c If W1 < W1, then the machine will undergo opportunistic maintenance; otherwise, the machine will not be maintained. If a machine in a certain processing stage requires maintenance, then all machines in that stage must be shut down until the maintenance activity is completed.

[0009] As a further preferred option, after maintenance is completed, the scheduling point is determined: if there are workpieces that need to be scheduled and there are available machines, then the current time point is the scheduling point; in addition, when a workpiece is inserted at a certain time point, it is also considered a scheduling point.

[0010] As a further preferred embodiment, the input of the workpiece global evaluation network is the real-time status of the workpiece to be processed in all processing stages, and the output is the global workpiece scheduling evaluation value. The workpiece scheduling network is optimized based on the global workpiece scheduling evaluation value. The input of the machine global evaluation network is the real-time status of the machine in all processing stages, and the output is the global machine scheduling evaluation value. The machine scheduling network is optimized based on the global machine scheduling evaluation value.

[0011] As a further preferred embodiment, the real-time status of the workpiece includes eight workpiece status features: the current stage number of the workpiece, the average processing time of the current process of the workpiece, the remaining completion time of the workpiece, the average processing cost of the current process of the workpiece, the remaining processing cost, the current estimated completion time of the workpiece, the fastest completion time of the current process of the workpiece, and the minimum processing cost of the current process of the workpiece. The real-time machine status includes eight machine status features: current machine status, machine processing time, remaining machine processing time, time for processing the selected workpiece, cost of processing the selected workpiece, current machine health factor, current machine maintenance cost, and number of machine maintenance operations.

[0012] As a further optimization, based on the Markov decision process, the agent is trained by reinforcement learning using the training set; after training, multiple scheduling networks are selected during the reward function fluctuation phase, and the optimal scheduling network is selected from them using the test set as the final scheduling network for each agent.

[0013] As a further optimization, the reward function used during training is a weighted combination of total completion time, maintenance cost, and processing cost.

[0014] According to another aspect of the present invention, a hybrid flow shop dynamic scheduling system considering predictive machine maintenance is provided, including a processor for executing the above-described hybrid flow shop dynamic scheduling method considering predictive machine maintenance.

[0015] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages: 1. This invention divides intelligent agents into multiple stages and constructs an end-to-end scheduling network. It improves the previous centralized decision-making into decentralized decision-making at each stage, which can reduce the computational pressure on each intelligent agent and avoid the curse of dimensionality. At the same time, it sets up a workpiece scheduling network and a machine scheduling network, which can adapt to workpiece / machine input sequences of different lengths, perceive the production status of the corresponding processing stage in real time, and output scheduling actions. By training the heterogeneous network in each intelligent agent, it can learn excellent scheduling strategies to optimize the scheduling goal. Furthermore, it combines machine operation and maintenance with the scheduling process, which can effectively handle interference events and ensure the reliability of the manufacturing system while minimizing the total completion time, workpiece processing cost, and machine maintenance cost.

[0016] 2. This invention designs a predictive maintenance strategy for machines. Specifically, by comparing the machine's health factors with predictive maintenance thresholds and opportunistic maintenance thresholds, effective predictive maintenance of equipment is performed before resource scheduling. This effectively combines machine maintenance with production resource scheduling, overcoming problems such as interference events and unforeseen events in the workshop production process. It has strong versatility and superiority for production systems of different sizes and can provide excellent scheduling solutions based on the real-time status of the workshop. Attached Figure Description

[0017] Figure 1 The flowchart illustrates a hybrid flow shop dynamic scheduling method for considering predictive machine maintenance, as described in an embodiment of the present invention.

[0018] Figure 2 This is a diagram of a multi-agent collaborative scheduling framework based on MADRL, as described in an embodiment of the present invention.

[0019] Figure 3 This is a flowchart illustrating the joint decision point determination process in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] This invention provides a dynamic scheduling method for hybrid flow workshops that considers predictive machine maintenance, such as... Figures 1 to 3 As shown, it includes the following steps: Step 1: With the goal of minimizing total completion time, maintenance cost and processing cost, the unrelated parallel machine scheduling problem of hybrid flow workshop considering predictive maintenance is established as a multi-objective joint optimization model.

[0022] Specifically, the first optimization objective is to minimize the total completion time:

[0023] in, C T This represents the total completion time for workpiece machining. J Represents a set of workpieces. C Represents a set of machines. F j,c Representative workpiece j In the machine c The completion time.

[0024] The second optimization objective is to minimize the total processing cost:

[0025] in C m Represents the total cost of machining the workpiece. PC j,c Representative workpiece j In the machine c On processing costs, A j,c Indicates workpiece j Is it in the machine? c Upward processing, J Represents a set of workpieces. C It represents a set of machines.

[0026] The third optimization objective is to minimize the total maintenance cost:

[0027] in C n Represents the total maintenance cost of the machine. CPM c Representative machine c The cost of a single predictive maintenance, NPM c Representative machine c Number of predictive maintenance operations CCM c Representative machine c The cost of a single opportunistic maintenance, NCM c Representative machine c Number of opportunistic maintenance sessions C It represents a set of machines.

[0028] After achieving three goals C T , C m and C n Then, by assigning weights, the three optimization objectives are transformed into a single objective. f The details are as follows:

[0029] in ω 1, ω 2 and ω 3 represents C T , C m and C nThe weights are set to 0.6, 0.2, and 0.2 in this embodiment.

[0030] Step 2: Construct a multi-agent collaborative scheduling model based on multi-agent deep reinforcement learning, and combine it with a multi-objective joint optimization model to construct a Markov decision process.

[0031] (1) Multi-agent partitioning and end-to-end scheduling network construction: A hybrid assembly line workshop has multiple processing stages, through which workpieces sequentially complete their processes. Essentially, different processing stages collaborate to achieve the same production goal: completing all workpiece processes. The production environment can be viewed as a fully collaborative one, with each processing stage acting as a stage agent capable of allocating processing resources and responding to various dynamic events. Each stage agent's scheduling actions involve workpiece sorting and machine selection. Therefore, each stage agent employs two heterogeneous scheduling networks: a workpiece scheduling network and a machine scheduling network, used for workpiece sorting and machine selection, respectively. In total, there are n workpiece scheduling networks and n machine scheduling networks, the number depending on the number of processing stages, n.

[0032] Meanwhile, to evaluate the quality of the output actions of each scheduling network, two shared global evaluation networks are constructed: a workpiece global evaluation network and a machine global evaluation network. The input of the workpiece global evaluation network is the real-time status of the workpiece to be processed in all processing stages, and the output is a global workpiece scheduling evaluation value; the input of the machine global evaluation network is the real-time status of the machine in all processing stages, and the output is a global machine scheduling evaluation value.

[0033] Furthermore, the job scheduling network adopts a pointer network based on Long Short-Term Memory (LSTM) and attention mechanism. This network mainly consists of three parts: 1) an LSTM-based encoding layer, which can be used to encode job input sequences of different lengths; 2) an attention-based decoding layer, which is used to decode the encoded state and assign different score values ​​to the input jobs; 3) a Softmax-based pointer layer, which generates corresponding probabilities based on different job score values ​​and uses the sampling method to sample, and finally outputs the selected job number.

[0034] Furthermore, the machine scheduling network adopts a Transformer-based weight allocation network, which mainly consists of three parts: 1) a state expansion layer, which expands the dimensions of the input machine state through MLP; 2) a Transformer-based encoding layer, which encodes the expanded machine state using Transformer; and 3) an MLP-based decoding layer, which assigns corresponding weights to the machine based on the encoded state using the MLP network, thereby generating the machine selection probability, sampling in the form of samples, and finally outputting the machine number.

[0035] (2) State space: Since the stage agents are in a cooperative relationship and their basic structure is that of isomorphic agents, the general state space of the stage agents mainly includes two parts: the real-time state of the workpiece and the real-time state of the machine.

[0036] To address the real-time status of workpieces, eight workpiece status features were designed: the current stage number of the workpiece, the average processing time of the current process, the remaining completion time of the workpiece, the average processing cost of the current process, the remaining processing cost, the current estimated completion time of the workpiece, the fastest completion time of the current process, and the minimum processing cost of the current process. Based on this, the input layer dimension of the workpiece scheduling network is djn×8, where djn represents the number of workpieces to be decided, and the output layer dimension is 1 (workpiece number).

[0037] To assess the real-time status of machines, eight machine state features were designed: current machine state (0 for idle, 1 for otherwise), machine processing time, remaining processing time, time for processing the selected workpiece, cost of processing the selected workpiece, current machine health factor, current machine maintenance cost, and number of machine maintenance cycles. Based on this, the input layer dimension of the machine scheduling network is mn×8, where mn represents the number of machines in the stage agent, and the output layer dimension is 1 (machine number).

[0038] (3) Action masking mechanism: For a job scheduling network, the number of jobs to be decided varies for each decision step at each scheduling point. Therefore, jobs that have already been decided are masked by filling all states corresponding to those jobs with NaN. For a machine scheduling network, when a machine is occupied or unavailable, it is masked by filling all states corresponding to that machine with NaN.

[0039] (4) Global Differential Multi-Agent Reward Function: The first part of the reward function is about minimizing the completion time, which is achieved by minimizing the single-step transition time to minimize the total completion time. The second part of the reward function is about minimizing the total processing cost, which is achieved by minimizing the single-step transition time. The third part of the reward function is about minimizing the total maintenance cost, which is achieved by minimizing the predictive maintenance cost and opportunistic maintenance cost of all machines.

[0040] The three parts of the reward function are combined by assigning weights to form the final reward function:

[0041] In this embodiment pr The bonus value of 1 is 0.6 for completion time. pr 2 refers to a reward value of 0.2 related to processing costs. pr 3 represents a reward value of 0.2 for machine maintenance costs.

[0042] (5) Joint decision-making point: The joint decision-making point includes the operation and maintenance point and the scheduling point.

[0043] First, identify maintenance points. Based on the historical relationship between machine processing time and maintenance, set two key thresholds for the machine: a predictive maintenance threshold W1 and an opportunistic maintenance threshold W2, where W1 > W2. During production, obtain the machine's actual processing time and calculate its health factors accordingly. RM c And compare it with thresholds W1 and W2. If RM c If W1 ≥ W2, then predictive maintenance is performed on the machine; if W2 ≤ W1, then predictive maintenance is performed on the machine. RM c If W1 < W1, the machine will undergo opportunistic maintenance; otherwise, maintenance will not be performed. If maintenance is required for a machine in a certain processing stage, all machines in that stage must be shut down until the maintenance activity is completed; otherwise, normal workpiece processing activities will continue. When a machine's service life changes after maintenance, the subsequent machine time accumulation begins from the improved service life.

[0044] Then, the scheduling point is determined. The determination of the scheduling point follows the determination of the maintenance point; that is, the maintenance point is determined first. If there are machines that need maintenance, the maintenance phase begins, the maintenance strategy is executed, and after the maintenance activity ends, it is determined whether it is a scheduling point. If there are no machines that need maintenance, it is determined whether it is a scheduling point. First, the concept of a scheduling job pool (job_buffer) is defined. The scheduling job pool is a virtual pool used to store jobs that need to be scheduled. If the number of jobs in the scheduling job pool is greater than zero and there are available machines, then this point in time is considered a scheduling point. Additionally, when a job is inserted at a certain point in time, it is also considered a scheduling point. If it is a scheduling point, the job scheduling network and machine scheduling network in the stage agent are invoked for scheduling. Specifically, each scheduling point's scheduling job pool contains multiple jobs to be decided. Each job to be decided represents a decision step. In the decision step, the job scheduling network is first invoked to select a job, and then the machine scheduling network is invoked to select a machine for that job.

[0045] Specifically, at each joint decision-making point, the machine's health factor for that processing phase is calculated based on the machine's operating time (i.e., service life). RM c :

[0046]

[0047] in Represents the machine c Working hours are pt An intermediate variable value at time, x 1 and x 2 represents two operation and maintenance calculation factors, RM c Representative machine c Working hours are pt Health factors at that time C l Representative processing stage l A collection of machines.

[0048] A dynamic hybrid flow shop environment model considering machine predictive maintenance was built using a Python simulation platform.

[0049] Step 3: Based on the Markov decision process, use a multi-agent proximal policy training algorithm based on centralized training and decentralized execution to perform reinforcement learning training on the agents.

[0050] We acquire real-world case studies and process the data to obtain a training set. Using this training set, we train the stage-based agent scheduling network using a multi-agent proximal policy training algorithm based on centralized training and distributed execution. This allows the network to continuously learn and update its parameters, ultimately minimizing total working time, maintenance costs, and processing costs, while also handling predictive maintenance and other disruptive events. Specifically: (1) Construction of training and test sets: The training set is a large-scale set of concrete cases containing numerous moments requiring joint decision-making (scheduling). By designing these concrete cases, they are transformed into production states. This data is primarily in matrix form, such as the processing time of each process for each workpiece on different machines. Specifically, the production information includes: the number of available processing machines for each process of each workpiece, the processing time of each process on each available machine, the cost of each process on each available machine, the actual processing time (service life) of the machines, and the maintenance costs of the machines.

[0051] The test set contains 20 scheduling instances. The generation of each instance follows the same process as described above. The generated production data includes: the number of workpieces, the number of processing machines, the number of machine stages, the number of machines corresponding to each stage, the number of selectable processing machines for each workpiece's process, the processing time for each workpiece's process on each selectable machine, the cost of each workpiece's process on each selectable machine, the actual processing time (service life) of the machines, and the machine maintenance cost. Different processing times, processing costs, and machine maintenance costs result in different processing times and costs.

[0052] (2) The agent is trained by reinforcement learning through the training set.

[0053] Initialize the hyperparameters of I-MAPPO-CTDE, the number of training rounds, the memory pool capacity of the agent at each stage, and the workshop environment.

[0054] Training begins with a training set. At each stage, the agent interacts with the environment at a joint decision point, completing operational activities and resource scheduling, and storing decision data in a memory pool. Normal production activities continue at other times.

[0055] Determine if there are any unfinished workpieces. If so, it indicates the existence of a joint decision point in the environment, and continue with the above steps. If not, complete the processing of all workpieces, and the training round ends. Determine whether the total amount of data in the workpiece scheduling network and machine scheduling network in the memory pool of the agent at each stage reaches the total memory pool capacity. If it does, update each workpiece / machine scheduling network and the corresponding global evaluation network using historical data. The system checks if the set number of training iterations has been reached. If not, a new round of training begins; otherwise, training is complete. After training, the global evaluation network is removed, resulting in the trained agent scheduling network.

[0056] Specifically, during the training phase, the agent selects appropriate scheduling actions based on local states. Each agent has two networks (a job scheduling network and a machine scheduling network) to output scheduling actions (job number and machine number), and the input states are the state of the job to be scheduled and the machine state in the phase. Initially, scheduling instances are imported into the environment. The scheduling process is the interaction between the phase agent and the environment. The joint decision point is the moment when the scheduling network in the phase agent interacts with the environment. The data of each interaction is recorded as { a i , s i , r i (State, Action, Reward). To evaluate the scheduling actions of the two scheduling networks in the stage agent, two global evaluation networks are designed. The input of the workpiece global evaluation network is the real-time state of the workpiece to be processed in all processing stages, and the output is the global workpiece scheduling evaluation value. The input of the machine global evaluation network is the real-time state of the machine in all processing stages, and the output is the global machine scheduling evaluation value. Then, the loss values ​​of the workpiece / machine global evaluation values ​​and the joint reward are used to calculate the loss values ​​of the workpiece scheduling network and the machine scheduling network in each stage agent, which are used to update the network parameters.

[0057] After training, the high-performing scheduling network and its parameters are saved. This scheduling network enables the stage agent to select appropriate scheduling actions and machine maintenance activities based on the production status of each joint decision point.

[0058] Furthermore, the scheduling network parameters of each agent are determined through the test set: After training, the final set of scheduling networks may not be optimal. Multiple sets of scheduling networks can be selected during the reward function fluctuation phase in the later stage of training. The set of scheduling networks corresponding to the optimal reward function value is selected from the test set and used as the final scheduling network for each agent.

[0059] Step 4: Implement dynamic scheduling of the hybrid production line based on the scheduling network trained by each agent.

[0060] After training, the two global evaluation networks are removed. Each stage of the agent relies on its own two scheduling networks to make decisions, achieving dynamic scheduling of the hybrid assembly line workshop. When the workshop receives a new order, the agent can obtain the order information in real time, thereby obtaining the real-time production status, i.e., the designed state space. Subsequently, based on the real-time production status, it performs predictive machine maintenance and outputs scheduling rules, guiding production according to the output maintenance actions and scheduling rules.

[0061] The following are specific examples: Obtain a large-scale case, for example: the number of workpieces is 60 (workpiece 1, ..., workpiece 60), the number of processing machines is 26 (machine 1, ..., machine 26), the number of machine stages is 5, and the number of machines corresponding to each stage is 6-5-5-4-6; transform this case into a production state and build a training set. The test set contains 20 scheduling instances, and the generation of each scheduling instance is similar to that of the training set.

[0062] Based on a Python simulation platform, five job scheduling networks and one job global evaluation network were constructed, with the following structures: (din×8) - LSTM (64 hidden layers) - attention mechanism (64 hidden layers, 2 attention heads) - (din×64) - Softmax-1 (output); (number of jobs × 8) - average pooling (32 hidden layers) - 8 - fully connected layer (8-64-64-32-32-1) - 1 (output). Five machine scheduling networks and one machine global evaluation network were also constructed, with the following structures: (mn×8) - fully connected layer - mn×8 - Transformer (64 hidden layers, 2 attention heads) - fully connected layer - Softmax-1 (output); (number of all machines × 8) - average pooling (32 hidden layers) - 8 - fully connected layer (8-64-64-32-32-1) - 1 (output).

[0063] Initialize hyperparameters. The hyperparameters for each job scheduling network include: encoding layer learning rate: 3e-5, decoding layer learning rate: 5e-5, job global evaluation network learning rate: 1e-5, GAE: 0.98, loss entropy: 0.01, memory pool capacity: 1024, training batches: 128, usage count: 8, truncation factor: 0.2, training epochs: 10000. The hyperparameters for each machine scheduling network include: encoding layer learning rate: 4e-5. The learning rate for the decoding layer is 6e-5, the learning rate for the global evaluation network of the artifact is 2e-5, GAE is 0.98, the loss entropy is 0.01, the memory pool capacity is 1024, the training batches are 128, the number of uses is 8, the cutoff factor is 0.2, and the number of training epochs is 10000. The predictive maintenance threshold and the opportunistic maintenance threshold are 0.6 and 0.8, respectively. The three weights of the objective function are x1 = 0.6, x2 = 0.2, and x3 = 0.2.

[0064] At the joint decision point, for each decision step, the stage agent obtains the real-time production status (workpiece status and machine status), calls the workpiece scheduling network to complete the workpiece selection, and outputs the workpiece sequence number. Subsequently, for the selected workpiece, it calls the machine scheduling network to select a machine for it and outputs the machine sequence number. At each joint decision point, the decision process of the workpiece / machine scheduling network, including actions, states, and rewards {s, a, r}, will be stored in their respective memory pools.

[0065] If there are unfinished workpieces in the environment, repeat the above steps; otherwise, the cycle ends.

[0066] If the total amount of data in all memory pools does not exceed the total memory pool capacity, repeat the above steps; otherwise, begin the update. Each scheduling network obtains its corresponding loss value by calculating its advantage value, as shown in the following formula. During updates, a batch of 128 data points is randomly selected from the memory pool for each update, with a total of 8 updates. The loss value of the global evaluation network is obtained by calculating the average of the loss values ​​of all scheduling networks, i.e., using an average credit allocation method.

[0067]

[0068]

[0069]

[0070] in, A t Indicates the dominance value. Represents the final loss value. Represents time difference error. r t Represents the reward value. Represents the discount factor. Represents the attenuation coefficient. v Represents the evaluation value. Represents the cross-loss entropy weighting factor. Represents the current scheduling network policy. The strategy representing the old scheduling network, This represents the difference between the old and new strategies, where t represents the time sequence number of the decision sequence. Represents the state at time t. Representative strategy The cross-loss entropy, Represents expectations, Network layer parameters representing the scheduling network, Network layer parameters representing the old scheduling network, Represents the action at time t. Represents the truncation function. This represents the cutoff factor.

[0071] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic scheduling method for hybrid flow shop considering predictive machine maintenance, characterized in that, Includes the following steps: With the goal of minimizing total completion time, maintenance cost, and processing cost, the hybrid flow shop scheduling problem is established as a multi-objective joint optimization model; The system sets up an agent with the same number of processing stages. Each agent makes decisions for the corresponding processing stage, and a Markov decision process is constructed by combining a multi-objective joint optimization model. Each agent has an independent scheduling network, including a workpiece scheduling network and a machine scheduling network. All agents share a global evaluation network, including a workpiece global evaluation network and a machine global evaluation network, which are used to evaluate the scheduling quality of the workpiece scheduling network and the machine scheduling network, respectively. Based on the Markov decision process, the agent is trained by reinforcement learning. During the training process, the workshop is maintained at the operation and maintenance point, and at the scheduling point, the workpiece scheduling network is called to select workpieces according to the current state, and then the machine scheduling network is called to select machines for the selected workpieces. After training, the global evaluation network is removed, and dynamic scheduling of the hybrid assembly line is achieved based on the scheduling network trained by each agent.

2. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 1, characterized in that, The job scheduling network includes: an LSTM-based encoding layer for encoding job input sequences of different lengths; an attention-based decoding layer for decoding the encoded sequences and assigning different score values ​​to the jobs; and a Softmax-based pointer layer for generating corresponding selection probabilities based on the different score values ​​of the jobs, and then outputting the selected job sequence number.

3. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 1, characterized in that, The machine scheduling network includes: a state expansion layer, which expands the dimensions of the input machine state using an MLP; a Transformer-based encoding layer, which encodes the expanded machine state; and an MLP-based decoding layer, which assigns corresponding weights to the machine based on the encoded state using an MLP, generates the machine selection probability, and outputs the selected machine number.

4. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 1, characterized in that, Maintenance is performed on the workshop at the operation and maintenance point, including: Obtain the machine's actual processing time, and then calculate the machine's health factors. RM c And compare it with the predictive maintenance threshold W1 and the opportunistic maintenance threshold W2; if RM c If W1 ≥ W2, then predictive maintenance is performed on the machine; if W2 ≤ W1, then predictive maintenance is performed on the machine. RM c If W1 < W1, then the machine will undergo opportunistic maintenance; otherwise, the machine will not be maintained. If a machine in a certain processing stage requires maintenance, then all machines in that stage must be shut down until the maintenance activity is completed.

5. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 4, characterized in that, After maintenance is completed, the scheduling point is determined: if there are workpieces that need to be scheduled and there are available machines, then the current time point is the scheduling point; in addition, when a workpiece is inserted at a certain time point, it is also considered a scheduling point.

6. The hybrid flow shop dynamic scheduling method considering predictive machine maintenance as described in claim 1, characterized in that, The input to the global workpiece evaluation network is the real-time status of the workpiece to be processed in all processing stages, and the output is the global workpiece scheduling evaluation value. The workpiece scheduling network is optimized based on the global workpiece scheduling evaluation value. The input to the global machine evaluation network is the real-time status of the machine in all processing stages, and the output is the global machine scheduling evaluation value. The machine scheduling network is optimized based on the global machine scheduling evaluation value.

7. The hybrid flow shop dynamic scheduling method considering predictive machine maintenance as described in claim 6, characterized in that, The real-time status of the workpiece includes eight workpiece status features: the current stage number of the workpiece, the average processing time of the current process of the workpiece, the remaining completion time of the workpiece, the average processing cost of the current process of the workpiece, the remaining processing cost, the current estimated completion time of the workpiece, the fastest completion time of the current process of the workpiece, and the minimum processing cost of the current process of the workpiece. The real-time machine status includes eight machine status features: current machine status, machine processing time, remaining machine processing time, time for processing the selected workpiece, cost of processing the selected workpiece, current machine health factor, current machine maintenance cost, and number of machine maintenance operations.

8. The hybrid flow shop dynamic scheduling method considering predictive machine maintenance as described in any one of claims 1-7, characterized in that, Based on Markov decision processes, the agent is trained using reinforcement learning through a training set. After training, during the reward function fluctuation phase, multiple scheduling networks are selected, and the optimal scheduling network is chosen from them using the test set as the final scheduling network for each agent.

9. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 8, characterized in that, The reward function used during training is a weighted combination of total completion time, maintenance cost, and processing cost.

10. A dynamic scheduling system for a hybrid assembly line workshop considering predictive machine maintenance, characterized in that, Includes a processor for executing the hybrid flow shop dynamic scheduling method that takes into account predictive machine maintenance as described in any one of claims 1-9.

Images