Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Single-piece job shop scheduling method based on Deep Q-network deep reinforcement learning

A job shop, reinforcement learning technology, applied in neural learning methods, machine learning, biological neural network models, etc.

Pending Publication Date: 2021-12-14
ZHENGZHOU UNIVERSITY OF LIGHT INDUSTRY
View PDF0 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The heuristic rule is an approximate scheduling algorithm, which has been widely used in practical problems because of its insensitivity to NP characteristics and good real-time performance. However, studies have shown that the overall performance of a single dispatch rule depends on Due to the characteristics of the system, operating condition parameters and production goals, when the state of the manufacturing system changes, the previously effective scheduling rules may not be applicable anymore
The meta-heuristic algorithm performs iterative optimization through a certain evolution or moving mechanism, and forms a new scheduling scheme. Its advantage is that it has high solution accuracy and can obtain a better rescheduling scheme. "requirement" is not applicable in the case of frequent dynamic events
[0004] In the production process, it is necessary to deal with the job-shop scheduling problem (job-shop scheduling problem, JSP). However, the existing job-shop scheduling algorithm cannot quickly deal with the dynamic problems in the job shop, and its practicability is poor.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Single-piece job shop scheduling method based on Deep Q-network deep reinforcement learning
  • Single-piece job shop scheduling method based on Deep Q-network deep reinforcement learning
  • Single-piece job shop scheduling method based on Deep Q-network deep reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] The present invention will be described in more detail below in conjunction with the accompanying drawings and through specific embodiments.

[0060] The present invention is a job shop scheduling method based on Deep Q-network (DQN) deep reinforcement learning, and its flow is as follows figure 1 shown, including the following steps:

[0061] (1) Taking the ft06 benchmark case as the specific implementation description, and taking the minimum and maximum completion time as the performance evaluation index, the information of the ft06 benchmark case is shown in Table 1, and the disjunctive graph is used to establish the scheduling environment of the benchmark case, as shown in figure 2 shown;

[0062] Table 1 ft06 benchmark example

[0063]

[0064] (2) Extract state features from the disjunctive graph as the input of the convolutional neural network;

[0065] The principle of state design: From the perspective of task goals, the selection of state features shoul...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of production plans, and particularly relates to a single-piece job shop scheduling method based on Deep Q-network deep reinforcement learning, which comprises the following steps of: (1) modeling a job shop scheduling environment by adopting a disjunction graph method, converting a scheduling decision problem into a sequential decision problem, establishing a Markov quintuple model, solving the model by using deep reinforcement learning; (2) extracting a current state from a disjunction graph environment scheduled by the job shop; (3) fitting the action value function and the target value function by adopting a convolutional neural network; (4) adopting 18 heuristic scheduling rules as proxy actions of reinforcement learning; (5) designing a reward function to evaluate the whole scheduling decision, and updating a weight parameter of an action value function by using a DQN algorithm; (6) performing state transition; and (7) updating network parameters of the target value function; the scheduling problem of the job shop can be quickly processed, and the method has the advantages of being high in real-time performance and high in flexibility.

Description

technical field [0001] The invention belongs to the technical field of production planning and is used for optimizing the production plan of a job shop, in particular to a scheduling method for a single job shop based on Deep Q-network (DQN) deep reinforcement learning. Background technique [0002] In the intelligent manufacturing environment, "everything is aware, everything is connected, and everything is intelligent" has become a new goal for the development of the manufacturing industry. The workshop produces a large amount of data in the actual production process, which contains a large amount of scheduling knowledge. The traditional meta-heuristic algorithm cannot effectively use these data, resulting in waste of data. Under the modern manufacturing mode with the rapid development of intelligence and informatization, it is of great significance to study intelligent scheduling algorithms with self-perception, data analysis and self-decision. [0003] Traditional sched...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q10/04G06Q50/04G06N3/04G06N3/08G06N20/00
CPCG06Q10/04G06Q50/04G06N3/08G06N20/00G06N3/047G06N3/045Y02P90/30
Inventor 乔东平段绿旗王雅静肖艳秋文笑雨李浩罗国富李立伟孙春亚张玉彦王昊琪
Owner ZHENGZHOU UNIVERSITY OF LIGHT INDUSTRY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products