A Hybrid Flow Shop Scheduling Method Based on Temporal Difference

A workshop scheduling and pipeline technology, applied in data processing applications, instruments, biological neural network models, etc., can solve problems such as poor real-time performance, limited functions of function generalizers, difficulty in dealing with large-scale complex and changeable, and avoid excessive estimated effect

Active Publication Date: 2022-04-01
NANJING UNIV OF SCI & TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] (1) The traditional scheduling algorithm cannot effectively use historical data for learning, and its real-time performance is poor, so it is difficult to cope with the large-scale, complex and changeable actual production scheduling environment
[0006] (2) At present, although the research on traditional HFSP is very mature, there are few studies on the use of reinforcement learning to solve the mixed flow shop problem, and there are problems such as difficult to characterize the processing environment and limited functions of the function generalizer
[0007] (3) The deep reinforcement learning algorithm can solve the problem of limited functions of the function generalizer. The weight sharing strategy of the convolutional neural network reduces the parameters that need to be trained. The same weight allows the filter to detect the signal without being affected by the position of the signal. The characteristics of the trained model make the generalization ability of the trained model stronger, but there are few researches on the deep reinforcement learning algorithm to solve the workshop scheduling problem at home and abroad.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Hybrid Flow Shop Scheduling Method Based on Temporal Difference
  • A Hybrid Flow Shop Scheduling Method Based on Temporal Difference
  • A Hybrid Flow Shop Scheduling Method Based on Temporal Difference

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0083] Parameter selection may affect the solution quality, and there are general principles to follow. The discount factor γ measures the weight of the subsequent state value on the total return, so the general value is close to 1, set γ=0.95; in the ε-greedy strategy, ε should be changed from large to small first, so as to fully explore the strategy space in the initial stage, and the end stage Use the obtained optimal strategy, so the initial ε=1, and decay exponentially with a discount rate of 0.995; set the learning rate α=0.02, the maximum number of interactions MAX_EPISODE=1000; memory D capacity N=6000, sampling batch BATCH_SIZE=256; intelligent Volume convolutional neural network structure such as image 3 As shown, the network parameters adopt a random initialization strategy.

[0084] (1) Small-scale problems

[0085] Small-scale problems Take a 10×8×6 scheduling problem as an example to test the feasibility of the algorithm. The example contains 10 workpieces an...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a deep reinforcement learning algorithm based on temporal difference, which is used to solve the mixed flow shop scheduling problem of related parallel machines. The algorithm combines convolutional neural network with TD learning in reinforcement learning, and according to the input state characteristics Behavior selection is more in line with the scheduling decision-making process of the actual order-responsive manufacturing system. By transforming the scheduling problem into a multi-stage decision-making problem, using the convolutional neural network model to fit the state-value function, inputting the processing state characteristic data of the manufacturing system into the model, using the temporal difference method to train the model, and using the heuristic algorithm or allocation rule as the scheduling decision Candidate behavior, combined with reinforcement learning reward and punishment mechanism, selects the optimal combined behavior strategy for each scheduling decision. Compared with the prior art, the algorithm proposed by the invention has the advantages of strong real-time performance and high flexibility.

Description

technical field [0001] The invention belongs to the scheduling control technology of mixed flow shop, in particular to a scheduling method of mixed flow shop based on time sequence difference. Background technique [0002] Hybrid flow-shop scheduling problem (Hybrid flow-shop scheduling problem, HFSP), also known as flexible flow-shop scheduling problem, was first proposed by Salvador in 1973. This problem can be regarded as a combination of classic flow-shop scheduling problem and parallel machine scheduling problem. , which is characterized in that there is a parallel machine stage in the processing of the workpiece, and the machine allocation is carried out while the processing sequence of the workpiece is determined. In the HFSP problem, the number of processors in at least one stage is greater than 1, which greatly increases the difficulty of solving HFSP. It has been proved that the two-stage HFSP with 2 and 1 processors is an NP-hard problem. [0003] At present, exa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06Q10/06G06N3/04
CPCG06Q10/06316G06N3/045
Inventor 陆宝春陈志峰顾钱翁朝阳张卫张哲
Owner NANJING UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products