Method and system for dynamic scheduling of hybrid flow shop considering machine predictive maintenance

Abstract

The application belongs to the field of workshop scheduling, and particularly discloses a mixed flow shop dynamic scheduling method and system considering machine predictive maintenance, which comprises the following steps: taking minimizing total completion time, maintenance cost and processing cost as the target, establishing a mixed flow shop scheduling problem as a multi-objective joint optimization model, setting the same number of intelligent agents as the number of processing stages, and constructing a Markov decision process; each intelligent agent has an independent scheduling network, including a workpiece scheduling network and a machine scheduling network; based on the Markov decision process, the intelligent agents are trained, the workshop is maintained at the operation and maintenance point, and the workpiece and machine selection are respectively performed by calling the workpiece scheduling network and the machine scheduling network at the scheduling point; after the training is completed, the trained intelligent agents are used to realize the dynamic scheduling of the workshop. The application effectively overcomes the mixed flow shop dynamic scheduling problem considering machine predictive maintenance by integrating the workshop scheduling of machine operation and maintenance, and has good dynamic and adaptability.

Description

Dynamic scheduling method and system for hybrid flow workshops considering predictive maintenance of machines Technical Field

[0001] This invention belongs to the field of workshop scheduling, and more specifically, relates to a dynamic scheduling method and system for hybrid assembly line workshops that takes into account predictive maintenance of machines. Background Technology

[0002] Under intense market competition, the manufacturing industry faces enormous economic pressure and environmental challenges. To meet diverse customer needs and save costs, companies continuously strive to improve the scalability, flexibility, and reliability of their manufacturing systems. In actual production, machine reliability gradually decreases over time. Predictive maintenance can significantly increase the production reliability of manufacturing systems. However, integrating predictive maintenance with shop floor scheduling to achieve joint scheduling presents a challenge. Furthermore, actual manufacturing systems are prone to various disruptive events, increasing the complexity and uncertainty of the shop floor.

[0003] Therefore, considering predictive maintenance of machines, researching dynamic and real-time hybrid flow shop scheduling methods to rationally arrange machine maintenance and production in the shop, thereby improving enterprise operational efficiency, has significant theoretical and practical value. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a dynamic scheduling method and system for hybrid assembly line workshops that takes into account predictive maintenance of machines. Its purpose is to effectively handle interference events and improve the efficiency and reliability of dynamic scheduling of hybrid assembly line workshops.

[0005] To achieve the above objectives, according to one aspect of the present invention, a dynamic scheduling method for hybrid flow shop considering predictive maintenance of machines is proposed, comprising the following steps:

[0006] With the goal of minimizing total completion time, maintenance cost, and processing cost, the hybrid flow shop scheduling problem is established as a multi-objective joint optimization model;

[0007] The system sets up an agent with the same number of processing stages. Each agent makes decisions for the corresponding processing stage, and a Markov decision process is constructed by combining a multi-objective joint optimization model. Each agent has an independent scheduling network, including a workpiece scheduling network and a machine scheduling network. All agents share a global evaluation network, including a workpiece global evaluation network and a machine global evaluation network, which are used to evaluate the scheduling quality of the workpiece scheduling network and the machine scheduling network, respectively.

[0008] Based on the Markov decision process, the agent is trained by reinforcement learning. During the training process, the workshop is maintained at the operation and maintenance point, and at the scheduling point, the workpiece scheduling network is called to select workpieces according to the current state, and then the machine scheduling network is called to select machines for the selected workpieces.

[0009] After training, the global evaluation network is removed, and dynamic scheduling of the hybrid assembly line is achieved based on the scheduling network trained by each agent.

[0010] As a further preferred embodiment, the job scheduling network includes: an LSTM-based encoding layer for encoding job input sequences of different lengths; an attention-based decoding layer for decoding the encoded sequences and assigning different score values ​​to the jobs; and a Softmax-based pointer layer for generating corresponding selection probabilities based on the different score values ​​of the jobs, and then outputting the selected job sequence number.

[0011] As a further preferred embodiment, the machine scheduling network includes: a state expansion layer, which expands the dimensions of the input machine state through MLP; a Transformer-based encoding layer, which encodes the expanded machine state; and an MLP-based decoding layer, which assigns corresponding weights to the machine according to the encoded state through MLP, generates the machine selection probability, and outputs the selected machine number.

[0012] As a further preferred option, maintenance of the workshop is carried out at the operation and maintenance point, including:

[0013] Obtain the machine's actual processing time, and then calculate the machine's health factor RM. c And compare it with the predictive maintenance threshold W1 and the opportunistic maintenance threshold W2; if RM c If W1 ≥ RM, then predictive maintenance will be performed on the machine; if W2 ≤ RM c If W1 < W1, then the machine will undergo opportunistic maintenance; otherwise, the machine will not be maintained. If a machine in a certain processing stage requires maintenance, then all machines in that stage must be shut down until the maintenance activity is completed.

[0014] As a further preferred option, after maintenance is completed, the scheduling point is determined: if there are workpieces that need to be scheduled and there are available machines, then the current time point is the scheduling point; in addition, when a workpiece is inserted at a certain time point, it is also considered a scheduling point.

[0015] As a further preferred embodiment, the input of the workpiece global evaluation network is the real-time status of the workpiece to be processed in all processing stages, and the output is the global workpiece scheduling evaluation value. The workpiece scheduling network is optimized based on the global workpiece scheduling evaluation value. The input of the machine global evaluation network is the real-time status of the machine in all processing stages, and the output is the global machine scheduling evaluation value. The machine scheduling network is optimized based on the global machine scheduling evaluation value.

[0016] As a further preferred embodiment, the real-time status of the workpiece includes eight workpiece status features: the current stage number of the workpiece, the average processing time of the current process of the workpiece, the remaining completion time of the workpiece, the average processing cost of the current process of the workpiece, the remaining processing cost, the current estimated completion time of the workpiece, the fastest completion time of the current process of the workpiece, and the minimum processing cost of the current process of the workpiece.

[0017] The real-time machine status includes eight machine status features: current machine status, machine processing time, remaining machine processing time, time for processing the selected workpiece, cost of processing the selected workpiece, current machine health factor, current machine maintenance cost, and number of machine maintenance operations.

[0018] As a further optimization, based on the Markov decision process, the agent is trained by reinforcement learning using the training set; after training, multiple scheduling networks are selected during the reward function fluctuation phase, and the optimal scheduling network is selected from them using the test set as the final scheduling network for each agent.

[0019] As a further optimization, the reward function used during training is a weighted combination of total completion time, maintenance cost, and processing cost.

[0020] According to another aspect of the present invention, a hybrid flow shop dynamic scheduling system considering predictive machine maintenance is provided, including a processor for executing the above-described hybrid flow shop dynamic scheduling method considering predictive machine maintenance.

[0021] In summary, compared with the prior art, the above-described technical solutions conceived by this invention mainly possess the following technical advantages:

[0022] 1. This invention divides intelligent agents into multiple stages and constructs an end-to-end scheduling network. It improves the previous centralized decision-making into decentralized decision-making at each stage, which can reduce the computational pressure on each intelligent agent and avoid the curse of dimensionality. At the same time, it sets up a workpiece scheduling network and a machine scheduling network, which can adapt to workpiece / machine input sequences of different lengths, perceive the production status of the corresponding processing stage in real time, and output scheduling actions. By training the heterogeneous network in each intelligent agent, it can learn excellent scheduling strategies to optimize the scheduling goal. Furthermore, it combines machine operation and maintenance with the scheduling process, which can effectively handle interference events and ensure the reliability of the manufacturing system while minimizing the total completion time, workpiece processing cost, and machine maintenance cost.

[0023] 2. This invention designs a predictive maintenance strategy for machines. Specifically, by comparing the machine's health factors with predictive maintenance thresholds and opportunistic maintenance thresholds, effective predictive maintenance of equipment is performed before resource scheduling. This effectively combines machine maintenance with production resource scheduling, overcoming problems such as interference events and unforeseen events in the workshop production process. It has strong versatility and superiority for production systems of different sizes and can provide excellent scheduling solutions based on the real-time status of the workshop. Attached Figure Description

[0024] Figure 1 is a flowchart of the dynamic scheduling method for hybrid assembly line workshops that considers predictive maintenance of machines according to an embodiment of the present invention.

[0025] Figure 2 is a diagram of the multi-agent collaborative scheduling framework based on MADRL in an embodiment of the present invention.

[0026] Figure 3 is a flowchart of the joint decision point judgment in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0028] The present invention provides a dynamic scheduling method for hybrid flow workshops that considers predictive maintenance of machines, as shown in Figures 1 to 3, and includes the following steps:

[0029] Step 1: With the goal of minimizing total completion time, maintenance cost and processing cost, the unrelated parallel machine scheduling problem of hybrid flow workshop considering predictive maintenance is established as a multi-objective joint optimization model.

[0030] Specifically, the first optimization objective is to minimize the total completion time:

[0031]

[0032] Among them, C T J represents the total completion time of workpiece processing, C represents the set of workpieces, and F represents the set of machines. j,c This represents the completion time of workpiece j on machine c.

[0033] The second optimization objective is to minimize the total processing cost:

[0034]

[0035] Where C m PC represents the total cost of machining a workpiece. j,c A represents the processing cost of workpiece j on machine c. j,c This indicates whether workpiece j is processed on machine c, where J represents the set of workpieces and C represents the set of machines.

[0036] The third optimization objective is to minimize the total maintenance cost:

[0037]

[0038] Where C n CPM represents the total maintenance cost of a machine. c NPM represents the cost required for a single predictive maintenance of machine c. c CCM represents the number of times machine C performs predictive maintenance. c NCM represents the cost required for a single opportunistic maintenance of machine c. c This represents the number of times machine c undergoes opportunistic maintenance, where C represents the set of machines.

[0039] After obtaining three objectives C T C m and C n Then, by assigning weights, the three optimization objectives are transformed into a single objective f, as follows:

[0040]

[0041] Where ω1, ω2 and ω3 represent C respectively. T C m and C n The weights are set to 0.6, 0.2, and 0.2 in this embodiment.

[0042] Step 2: Construct a multi-agent collaborative scheduling model based on multi-agent deep reinforcement learning, and combine it with a multi-objective joint optimization model to construct a Markov decision process.

[0043] (1) Multi-agent partitioning and end-to-end scheduling network construction:

[0044] A hybrid assembly line workshop has multiple processing stages, through which workpieces sequentially complete their processes. Essentially, different processing stages collaborate to achieve the same production goal: completing all workpiece processes. The production environment can be viewed as a fully collaborative one, with each processing stage acting as a stage agent capable of allocating processing resources and responding to various dynamic events. Each stage agent's scheduling actions involve workpiece sorting and machine selection. Therefore, each stage agent employs two heterogeneous scheduling networks: a workpiece scheduling network and a machine scheduling network, used for workpiece sorting and machine selection, respectively. In total, there are n workpiece scheduling networks and n machine scheduling networks, the number depending on the number of processing stages, n.

[0045] Meanwhile, to evaluate the quality of the output actions of each scheduling network, two shared global evaluation networks are constructed: a workpiece global evaluation network and a machine global evaluation network. The input of the workpiece global evaluation network is the real-time status of the workpiece to be processed in all processing stages, and the output is a global workpiece scheduling evaluation value; the input of the machine global evaluation network is the real-time status of the machine in all processing stages, and the output is a global machine scheduling evaluation value.

[0046] Furthermore, the job scheduling network adopts a pointer network based on Long Short-Term Memory (LSTM) and attention mechanism. This network mainly consists of three parts: 1) an LSTM-based encoding layer, which can be used to encode job input sequences of different lengths; 2) an attention-based decoding layer, which is used to decode the encoded state and assign different score values ​​to the input jobs; 3) a Softmax-based pointer layer, which generates corresponding probabilities based on different job score values ​​and uses the sampling method to sample, and finally outputs the selected job number.

[0047] Furthermore, the machine scheduling network adopts a Transformer-based weight allocation network, which mainly consists of three parts: 1) a state expansion layer, which expands the dimensions of the input machine state through MLP; 2) a Transformer-based encoding layer, which encodes the expanded machine state using Transformer; and 3) an MLP-based decoding layer, which assigns corresponding weights to the machine based on the encoded state using the MLP network, thereby generating the machine selection probability, sampling in the form of samples, and finally outputting the machine number.

[0048] (2) State space:

[0049] Since the stage agents are in a cooperative relationship and their basic structure is that of isomorphic agents, the general state space of the stage agents mainly includes two parts: the real-time state of the workpiece and the real-time state of the machine.

[0050] To address the real-time status of workpieces, eight workpiece status features were designed: the current stage number of the workpiece, the average processing time of the current process, the remaining completion time of the workpiece, the average processing cost of the current process, the remaining processing cost, the current estimated completion time of the workpiece, the fastest completion time of the current process, and the minimum processing cost of the current process. Based on this, the input layer dimension of the workpiece scheduling network is djn×8, where djn represents the number of workpieces to be decided, and the output layer dimension is 1 (workpiece number).

[0051] To assess the real-time status of machines, eight machine state features were designed: current machine state (0 for idle, 1 for otherwise), machine processing time, remaining processing time, time for processing the selected workpiece, cost of processing the selected workpiece, current machine health factor, current machine maintenance cost, and number of machine maintenance cycles. Based on this, the input layer dimension of the machine scheduling network is mn×8, where mn represents the number of machines in the stage agent, and the output layer dimension is 1 (machine number).

[0052] (3) Action masking mechanism:

[0053] For a job scheduling network, the number of jobs to be decided varies for each decision step at each scheduling point. Therefore, jobs that have already been decided are masked by filling all states corresponding to those jobs with NaN. For a machine scheduling network, when a machine is occupied or unavailable, it is masked by filling all states corresponding to that machine with NaN.

[0054] (4) Global Differential Multi-Agent Reward Function:

[0055] The first part of the reward function is about minimizing the completion time, which is achieved by minimizing the single-step transition time to minimize the total completion time. The second part of the reward function is about minimizing the total processing cost, which is achieved by minimizing the single-step transition time. The third part of the reward function is about minimizing the total maintenance cost, which is achieved by minimizing the predictive maintenance cost and opportunistic maintenance cost of all machines.

[0056] The three parts of the reward function are combined by assigning weights to form the final reward function:

[0057]

[0058] In this embodiment, pr1 is a reward value of 0.6 for completion time, pr2 is a reward value of 0.2 for processing cost, and pr3 is a reward value of 0.2 for machine maintenance cost.

[0059] (5) Joint decision-making point:

[0060] The joint decision-making point includes the operation and maintenance point and the scheduling point.

[0061] First, identify maintenance points. Based on the historical relationship between machine processing time and maintenance, set two key thresholds for the machine: a predictive maintenance threshold W1 and an opportunistic maintenance threshold W2, where W1 > W2. During production, obtain the machine's actual processing time and calculate the machine's health factor RM based on this. c And compare it with thresholds W1 and W2. If RM c If W2 ≥ RM, then the machine will undergo predictive maintenance; if W2 ≤ RM, then predictive maintenance will be performed. c If W1 < W1, the machine will undergo opportunistic maintenance; otherwise, maintenance will not be performed. If maintenance is required for a machine in a certain processing stage, all machines in that stage must be shut down until the maintenance activity is completed; otherwise, normal workpiece processing activities will continue. When a machine's service life changes after maintenance, the subsequent machine time accumulation begins from the improved service life.

[0062] Then, the scheduling point is determined. The determination of the scheduling point follows the determination of the maintenance point; that is, the maintenance point is determined first. If there are machines that need maintenance, the maintenance phase begins, the maintenance strategy is executed, and after the maintenance activity ends, it is determined whether it is a scheduling point. If there are no machines that need maintenance, it is determined whether it is a scheduling point. First, the concept of a scheduling job pool (job_buffer) is defined. The scheduling job pool is a virtual pool used to store jobs that need to be scheduled. If the number of jobs in the scheduling job pool is greater than zero and there are available machines, then this point in time is considered a scheduling point. Additionally, when a job is inserted at a certain point in time, it is also considered a scheduling point. If it is a scheduling point, the job scheduling network and machine scheduling network in the stage agent are invoked for scheduling. Specifically, each scheduling point's scheduling job pool contains multiple jobs to be decided. Each job to be decided represents a decision step. In the decision step, the job scheduling network is first invoked to select a job, and then the machine scheduling network is invoked to select a machine for that job.

[0063] Specifically, at each joint decision point, the machine's health factor (RM) is calculated based on the machine's operating time (i.e., service life) for that processing phase. c :

[0064]

[0065]

[0066] in This represents an intermediate variable value when machine c's working time is pt. x1 and x2 represent two maintenance calculation factors, RM. c C represents the health factor when machine c operates for pt. l The set of machines representing processing stage l.

[0067] A dynamic hybrid flow shop environment model considering machine predictive maintenance was built using a Python simulation platform.

[0068] Step 3: Based on the Markov decision process, use a multi-agent proximal policy training algorithm based on centralized training and decentralized execution to train the agents through reinforcement learning.

[0069] We acquire real-world case studies and process the data to obtain a training set. Using this training set, we train the stage-based agent scheduling network using a multi-agent proximal policy training algorithm based on centralized training and distributed execution. This allows the network to continuously learn and update its parameters, ultimately minimizing total working time, maintenance costs, and processing costs, while also handling predictive maintenance and other disruptive events. Specifically:

[0070] (1) Construction of training and test sets:

[0071] The training set is a large-scale set of concrete cases containing numerous moments requiring joint decision-making (scheduling). By designing these concrete cases, they are transformed into production states. This data is primarily in matrix form, such as the processing time of each process for each workpiece on different machines. Specifically, the production information includes: the number of available processing machines for each process of each workpiece, the processing time of each process on each available machine, the cost of each process on each available machine, the actual processing time (service life) of the machines, and the maintenance costs of the machines.

[0072] The test set contains 20 scheduling instances. The generation of each instance follows the same process as described above. The generated production data includes: the number of workpieces, the number of processing machines, the number of machine stages, the number of machines corresponding to each stage, the number of selectable processing machines for each workpiece's process, the processing time for each workpiece's process on each selectable machine, the cost of each workpiece's process on each selectable machine, the actual processing time (service life) of the machines, and the machine maintenance cost. Different processing times, processing costs, and machine maintenance costs result in different processing times and costs.

[0073] (2) The agent is trained by reinforcement learning through the training set.

[0074] Initialize the hyperparameters of I-MAPPO-CTDE, the number of training rounds, the memory pool capacity of the agent at each stage, and the workshop environment.

[0075] Training begins with a training set. At each stage, the agent interacts with the environment at a joint decision point, completing operational activities and resource scheduling, and storing decision data in a memory pool. Normal production activities continue at other times.

[0076] Determine if there are any unfinished workpieces. If so, it indicates the existence of a joint decision point in the environment, and continue with the above steps. If not, complete the processing of all workpieces, and the training round ends.

[0077] Determine whether the total amount of data in the workpiece scheduling network and machine scheduling network in the memory pool of the agent at each stage reaches the total memory pool capacity. If it does, update each workpiece / machine scheduling network and the corresponding global evaluation network using historical data.

[0078] The system checks if the set number of training iterations has been reached. If not, a new round of training begins; otherwise, training is complete. After training, the global evaluation network is removed, resulting in the trained agent scheduling network.

[0079] Specifically, during the training phase, the agent selects appropriate scheduling actions based on local states. Each agent has two networks (a job scheduling network and a machine scheduling network) to output scheduling actions (job number and machine number), and the input states are the state of the job to be scheduled and the machine state in the phase. Initially, scheduling instances are imported into the environment. The scheduling process is the interaction between the phase agent and the environment. The joint decision point is the moment when the scheduling network in the phase agent interacts with the environment. The data of each interaction is recorded as {a}. i s i r i (State, Action, Reward). To evaluate the scheduling actions of the two scheduling networks in the stage agent, two global evaluation networks are designed. The input of the workpiece global evaluation network is the real-time state of the workpiece to be processed in all processing stages, and the output is the global workpiece scheduling evaluation value. The input of the machine global evaluation network is the real-time state of the machine in all processing stages, and the output is the global machine scheduling evaluation value. Then, the loss values ​​of the workpiece / machine global evaluation values ​​and the joint reward are used to calculate the loss values ​​of the workpiece scheduling network and the machine scheduling network in each stage agent, which are used to update the network parameters.

[0080] After training, the high-performing scheduling network and its parameters are saved. This scheduling network enables the stage agent to select appropriate scheduling actions and machine maintenance activities based on the production status of each joint decision point.

[0081] Furthermore, the scheduling network parameters of each agent are determined through the test set: After training, the final set of scheduling networks may not be optimal. Multiple sets of scheduling networks can be selected during the reward function fluctuation phase in the later stage of training. The set of scheduling networks corresponding to the optimal reward function value is selected from the test set and used as the final scheduling network for each agent.

[0082] Step 4: Implement dynamic scheduling of the hybrid production line based on the scheduling network trained by each agent.

[0083] After training, the two global evaluation networks are removed. Each stage of the agent relies on its own two scheduling networks to make decisions, achieving dynamic scheduling of the hybrid assembly line workshop. When the workshop receives a new order, the agent can obtain the order information in real time, thereby obtaining the real-time production status, i.e., the designed state space. Subsequently, based on the real-time production status, it performs predictive machine maintenance and outputs scheduling rules, guiding production according to the output maintenance actions and scheduling rules.

[0084] The following are specific examples:

[0085] Obtain a large-scale case, for example: the number of workpieces is 60 (workpiece 1, ..., workpiece 60), the number of processing machines is 26 (machine 1, ..., machine 26), the number of machine stages is 5, and the number of machines corresponding to each stage is 6-5-5-4-6; transform this case into a production state and build a training set. The test set contains 20 scheduling instances, and the generation of each scheduling instance is similar to that of the training set.

[0086] Based on a Python simulation platform, five job scheduling networks and one job global evaluation network were constructed, with the following structures: (din×8) - LSTM (64 hidden layers) - attention mechanism (64 hidden layers, 2 attention heads) - (din×64) - Softmax-1 (output); (number of jobs × 8) - average pooling (32 hidden layers) - 8 - fully connected layer (8-64-64-32-32-1) - 1 (output). Five machine scheduling networks and one machine global evaluation network were also constructed, with the following structures: (mn×8) - fully connected layer - mn×8 - Transformer (64 hidden layers, 2 attention heads) - fully connected layer - Softmax-1 (output); (number of all machines × 8) - average pooling (32 hidden layers) - 8 - fully connected layer (8-64-64-32-32-1) - 1 (output).

[0087] Initialize hyperparameters. The hyperparameters for each job scheduling network include: encoding layer learning rate: 3e-5, decoding layer learning rate: 5e-5, job global evaluation network learning rate: 1e-5, GAE: 0.98, loss entropy: 0.01, memory pool capacity: 1024, training batches: 128, usage count: 8, truncation factor: 0.2, training epochs: 10000. The hyperparameters for each machine scheduling network include: encoding layer learning rate: 4e-5. The learning rate for the decoding layer is 6e-5, the learning rate for the global evaluation network of the artifact is 2e-5, GAE is 0.98, the loss entropy is 0.01, the memory pool capacity is 1024, the training batches are 128, the number of uses is 8, the cutoff factor is 0.2, and the number of training epochs is 10000. The predictive maintenance threshold and the opportunistic maintenance threshold are 0.6 and 0.8, respectively. The three weights of the objective function are x1 = 0.6, x2 = 0.2, and x3 = 0.2.

[0088] At the joint decision point, for each decision step, the stage agent obtains the real-time production status (workpiece status and machine status), calls the workpiece scheduling network to complete the workpiece selection, and outputs the workpiece sequence number. Subsequently, for the selected workpiece, it calls the machine scheduling network to select a machine for it and outputs the machine sequence number. At each joint decision point, the decision process of the workpiece / machine scheduling network, including actions, states, and rewards {s, a, r}, will be stored in their respective memory pools.

[0089] If there are unfinished workpieces in the environment, repeat the above steps; otherwise, the cycle ends.

[0090] If the total amount of data in all memory pools does not exceed the total memory pool capacity, repeat the above steps; otherwise, begin the update. Each scheduling network obtains its corresponding loss value by calculating its advantage value, as shown in the following formula. During updates, a batch of 128 data points is randomly selected from the memory pool for each update, with a total of 8 updates. The loss value of the global evaluation network is obtained by calculating the average of the loss values ​​of all scheduling networks, i.e., using an average credit allocation method.

[0091]

[0092]

[0093]

[0094] Among them, A t Indicates the dominance value. Represents the final loss value. Represents the time difference error, r t Represents the reward value. Represents the discount factor. The value represents the attenuation coefficient, and v represents the evaluation value. Represents the cross-loss entropy weighting factor. Represents the current scheduling network policy. The strategy representing the old scheduling network, This represents the difference between the old and new strategies, where t represents the time sequence number of the decision sequence. Represents the state at time t. Representative strategy The cross-loss entropy, Represents expectations, Network layer parameters representing the scheduling network, Network layer parameters representing the old scheduling network, Represents the action at time t. Represents the truncation function. This represents the cutoff factor.

[0095] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic scheduling method for hybrid flow shop considering predictive machine maintenance, characterized in that, The process includes the following steps: To minimize total completion time, maintenance cost, and processing cost, the hybrid flow shop scheduling problem is established as a multi-objective joint optimization model; an agent with the same number of processing stages is set up, each agent making decisions for its corresponding processing stage, and a Markov decision process is constructed based on the multi-objective joint optimization model; each agent has an independent scheduling network, including a workpiece scheduling network and a machine scheduling network; all agents share a global evaluation network, including a workpiece global evaluation network and a machine global evaluation network, used to evaluate the scheduling quality of the workpiece scheduling network and the machine scheduling network, respectively; the workpiece scheduling network includes: an LSTM-based encoding layer for encoding workpiece input sequences of different lengths; an attention-based decoding layer for decoding the encoded sequences and assigning different scores to the workpieces; and a Softmax-based indexing layer. The needle layer is used to generate corresponding selection probabilities based on different workpiece scores, and then output the selected workpiece number. The machine scheduling network includes: a state expansion layer, which expands the dimensions of the input machine state through MLP; a Transformer-based encoding layer, which encodes the expanded machine state; and an MLP-based decoding layer, which assigns corresponding weights to the machine based on the encoded state through MLP, generates the machine selection probability, and outputs the selected machine number. The agent is trained using reinforcement learning based on Markov decision processes. During training, the workshop is maintained at the operation and maintenance point, and at the scheduling point, the workpiece scheduling network is invoked to select workpieces based on the current state, and then the machine scheduling network is invoked to select machines for the selected workpieces. The workshop maintenance at the operation and maintenance point includes: obtaining the actual processing time of the machine and then calculating the machine's health factor RM. c And compare it with the predictive maintenance threshold W1 and the opportunistic maintenance threshold W2; if RM c If W1 ≥ RM, then predictive maintenance will be performed on the machine; if W2 ≤ RM c If W < W1, then the machine will be subject to opportunistic maintenance; otherwise, the machine will not be maintained. If a machine in a certain processing stage needs maintenance, then all machines in that stage need to be shut down until the maintenance activity is completed. After training, the global evaluation network is removed, and dynamic scheduling of the hybrid assembly line is realized based on the scheduling network trained by each agent.

2. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 1, characterized in that, After maintenance is completed, the scheduling point is determined: if there are workpieces that need to be scheduled and there are available machines, then the current time point is the scheduling point; in addition, when a workpiece is inserted at a certain time point, it is also considered a scheduling point.

3. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 1, characterized in that, The input to the global workpiece evaluation network is the real-time status of the workpiece to be processed in all processing stages, and the output is the global workpiece scheduling evaluation value. The workpiece scheduling network is optimized based on the global workpiece scheduling evaluation value. The input to the global machine evaluation network is the real-time status of the machine in all processing stages, and the output is the global machine scheduling evaluation value. The machine scheduling network is optimized based on the global machine scheduling evaluation value.

4. The dynamic scheduling method for hybrid assembly line workshops considering predictive machine maintenance as described in claim 3, characterized in that, The real-time status of the workpiece includes eight workpiece status features: the current stage number of the workpiece, the average processing time of the current process of the workpiece, the remaining completion time of the workpiece, the average processing cost of the current process of the workpiece, the remaining processing cost of the workpiece, the current estimated completion time of the workpiece, the fastest completion time of the current process of the workpiece, and the minimum processing cost of the current process of the workpiece; the real-time status of the machine includes eight machine status features: the current status of the machine, the processing time already completed by the machine, the remaining processing time of the machine, the processing time of the selected workpiece by the machine, the processing cost of the selected workpiece by the machine, the current health factor of the machine, the current maintenance cost of the machine, and the number of times the machine has been maintained.

5. The hybrid flow shop dynamic scheduling method considering predictive machine maintenance as described in any one of claims 1-4, characterized in that, Based on Markov decision processes, the agent is trained using reinforcement learning through a training set. After training, during the reward function fluctuation phase, multiple scheduling networks are selected, and the optimal scheduling network is chosen from them using the test set as the final scheduling network for each agent.

6. The hybrid flow shop dynamic scheduling method considering predictive machine maintenance as described in claim 5, characterized in that, The reward function used during training is a weighted combination of total completion time, maintenance cost, and processing cost.

7. A dynamic scheduling system for a hybrid assembly line workshop that considers predictive machine maintenance, characterized in that, Includes a processor for executing the hybrid flow shop dynamic scheduling method that takes into account predictive machine maintenance as described in any one of claims 1-6.

Images