Hot rolling production scheduling method, system and electronic equipment based on dual DQN network

Abstract

The present invention provides a hot rolling production scheduling method, system, and electronic device based on a dual DQN network, relating to the technical field of production scheduling. The method comprises the following steps: collecting and preprocessing historical hot rolling production datasets; dividing the preprocessed historical hot rolling production datasets into training and test sets according to a predetermined ratio; constructing a dual DQN network, and designing a comprehensive reward function based on the three objectives of minimizing the number of rolling units, minimizing the cost of specification jumps, and minimizing the penalty for not scheduling slabs; training and validating the dual DQN network using the training and test sets, respectively, to obtain a scheduling optimization model; inputting slab specification information, production demand information, and constraint information for the current production batch into the scheduling optimization model, which outputs an optimal action strategy; and solving the optimal action strategy to obtain an optimal production scheduling solution. This method can quickly obtain high-quality hot rolling production scheduling solutions even with small sample data.

Description

Technical Field

[0001] The present invention relates to the technical field of production scheduling, and in particular to a hot rolling production scheduling method, system and electronic equipment based on a dual DQN network. Background Art

[0002] As a key link in the steel production process, the main research direction of hot rolling production scheduling is to achieve intelligent and green, which has a direct impact on the quality of the final product, production efficiency and resource consumption. In the context of intensified market competition, customer demands tend to be diversified, prompting product production decisions to consider multiple objectives comprehensively. In essence, hot rolling production scheduling is to determine the rolling unit to which candidate slabs belong and to determine the order of slabs within the unit (see Figure 1 ), is a complex NP-hard combinatorial optimization problem involving multiple conflicting objectives.

[0003] Traditional empirical or rule-based approaches rely heavily on the experience and intuition of production managers, lacking systematicity and scientificity. Mathematical modeling methods are complex and difficult to adapt to real-time demands. Heuristic algorithms such as genetic algorithms and simulated annealing often rely on the selection of initial solutions, making convergence speed and stability difficult to guarantee. In recent years, deep reinforcement learning (DRL) has demonstrated great potential for solving complex optimization problems due to its powerful decision-making capabilities and adaptive learning properties. However, directly applying DRL to hot-rolling scheduling still faces challenges, particularly in effectively building and training models in small sample sizes and integrating multi-objective optimization strategies to achieve a global optimal solution. Summary of the Invention

[0004] Purpose of the invention: The present invention combines dual DQN reinforcement learning and NSGAIII to propose a hot rolling production scheduling method, system and electronic equipment based on dual DQN network to solve the above-mentioned problems existing in the prior art.

[0005] In a first aspect of the present invention, a hot rolling production scheduling method based on a dual DQN network is proposed, comprising the following steps:

[0006] Collect historical hot rolling production data sets, including slab specifications, production requirements, and constraints;

[0007] Preprocessing the historical hot rolling production data set so that the slab specification information, production demand information, and constraint information are on the same scale;

[0008] The preprocessed historical hot rolling production data set is divided into a training set and a test set according to a predetermined ratio;

[0009] A dual DQN network was constructed, and a comprehensive reward function was designed by combining the three objectives of minimizing the number of rolling units, minimizing the cost of gauge jumps, and minimizing the penalty for not being included in the slab.

[0010] Using the training set and the test set to train and verify the dual DQN network, respectively, to obtain a scheduling optimization model;

[0011] Inputting slab specification information, production demand information, and constraint information of the current production batch into the scheduling optimization model, and outputting the optimal action strategy from the scheduling optimization model;

[0012] Solve the optimal action strategy to obtain the optimal production scheduling solution.

[0013] In a further embodiment of the first aspect, the slab specification information records the width, thickness, and hardness of the slab;

[0014] The production demand information records the number of rolling batches , the number of rolling units in each rolling batch ;

[0015] The constraint condition information records the maximum and minimum specification differences between slabs.

[0016] In a further embodiment of the first aspect, the dual DQN network consists of a Q network and a target network;

[0017] The Q network and the target network both include an input layer, a hidden layer and an output layer; wherein the input layer receives the state vector of the current rolling batch; ; n is the number of features; Represents the status characteristics of the current i-th sample, including the number of rolling units, the specifications of the slabs that have been discharged, and the remaining slabs;

[0018] The hidden layer is composed of a multi-layer fully connected neural network, which is used to extract state features and calculate the expected probability Q of taking each possible action in the current state. The output layer outputs the action space A, which has actions, the output layer is neurons, output vector .

[0019] In a further embodiment of the first aspect, the comprehensive reward function R is expressed as follows:

[0020]

[0021] Where, is the number of rolling units; is the number of slabs that have been discharged; is the weight coefficient, non-negative and ;

[0022] is the cost of specification jump between the i-th and i+1-th slabs:

[0023]

[0024] Where, and 、 and 、 and are the width, thickness and hardness jump costs between the i-th and i+1-th slabs respectively; is the weight of the three specification attributes of width, thickness and hardness, which is non-negative and ;

[0025] The total penalty for not being discharged into the slab is:

[0026]

[0027] Where, is the quantity not discharged into the slab; is the penalty value for a single slab not being placed in the slab.

[0028] In a further embodiment of the first aspect, the weighted TD error is used to train the weight parameters of the Q network and the target network, and the calculation formula is as follows:

[0029]

[0030]

[0031] Where, is the weighted TD error of the sample; 、 Represent the weight parameters of the Q network and the target network respectively; s, are the current state and the next state respectively; a, are the current action and the next possible action respectively; r is the reward obtained after executing action a from state s; A is the action space; is the discount factor; D is the experience replay pool; is the predicted value of the Q network; is the predicted value of the target network; is a hyperparameter that amplifies the weight of a state-action pair that receives a reward.

[0032] In a further embodiment of the first aspect, the loss function of the dual DQN network is as follows:

[0033]

[0034] Where, is the number of samples for each training batch; is the current action of sample i; is the next possible action of sample i; 、 Represent the weight parameters of the Q network and the target network respectively; is the predicted value of Q network for sample i; For sample i in action The maximum predicted value at ; From the state characteristics Execute an action Rewards received after is the importance sampling weight of sample i; A is the action space; is the discount factor.

[0035] In a further embodiment of the first aspect, solving the optimal action strategy to obtain an optimal production scheduling solution specifically includes:

[0036] Using the trained dual DQN network, an action a is selected based on the current state s, and a rolling batch is gradually constructed until all slabs are placed or a preset termination condition is reached. The generated strategy serves as the initial population for NSGA-III. Each solution represents a rolling batch plan, which contains a series of slab arrangement orders and the number of rolling units.

[0037] The optimization objectives are to minimize the number of rolling units, minimize the cost of specification jump and minimize the penalty for not arranging slabs, and use the comprehensive reward function R as the fitness function. According to the standard deviation of the distribution in each dimension of the target space, Dynamically adjust the distribution of reference points.

[0038] In a further embodiment of the first aspect, the adaptive function is designed Dynamically adjust the distribution of reference points, adaptive function The expression is as follows:

[0039]

[0040]

[0041] Where, is the standard deviation of the population distribution in the i-th target dimension; is the value of the jth individual on the i-th target dimension; is the mean value on the i-th target dimension; N is the number of individuals in the population; 、 、 are the mean, maximum, and minimum values of the standard deviation of the distribution in all target dimensions; k is the adjustment coefficient.

[0042] A second aspect of the present invention provides a hot rolling production scheduling optimization system, the system comprising:

[0043] The acquisition module is used to collect historical hot rolling production data sets, including slab specification information, production demand information, and constraint information;

[0044] A first data processing module is used to pre-process the historical hot rolling production data set so that the slab specification information, production demand information, and constraint information are on the same scale;

[0045] The second data processing module is used to divide the pre-processed historical hot rolling production data set into a training set and a test set according to a predetermined ratio;

[0046] The first building block is used to construct a dual DQN network, combining the three objectives of minimizing the number of rolling units, minimizing the cost of specification jumps, and minimizing the penalty for not being queued into the slab, to design a comprehensive reward function;

[0047] The second building module is used to train and verify the dual DQN network using the training set and the test set respectively to obtain a scheduling optimization model;

[0048] A first output module is used to input slab specification information, production demand information, and constraint information of the current production batch into the scheduling optimization model, and the scheduling optimization model outputs an optimal action strategy;

[0049] The second output module is used to solve the optimal action strategy and obtain the optimal production scheduling solution.

[0050] In a third aspect of the present invention, an electronic device is proposed, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the hot rolling production scheduling method based on the dual DQN network as described in the first aspect and its further embodiments is implemented.

[0051] Compared with the prior art, the present invention has at least the following beneficial effects:

[0052] (1) Dual DQN can learn a better strategy to generate the initial production batch solution through reinforcement learning with limited training samples, providing a good starting point for NSGAIII optimization solution.

[0053] (2) Based on the initial solution generated by the dual DQN, NSGAIII can further search for a better solution, thereby improving the solution efficiency and solution quality.

[0054] (3) It combines the advantages of reinforcement learning and multi-objective optimization, which can not only solve the problem quickly but also handle the trade-offs between multiple optimization objectives.

[0055] (4) It can provide a novel solution method for other similar production scheduling problems and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0057] Figure 1 This is a schematic diagram of hot rolling production scheduling.

[0058] Figure 2 This is a flow chart of the algorithm proposed in the present invention. DETAILED DESCRIPTION

[0059] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art have not been described to avoid confusion with the present invention.

[0060] The embodiment of the present invention combines dual DQN reinforcement learning and NSGAIII to propose a novel hot rolling production scheduling method based on dual DQN network, which enables to quickly obtain high-quality hot rolling production scheduling solutions with small sample data.

[0061] First, a five-layer dual DQN network is constructed, and the state space is defined as a vector space containing the current number of rolling units, the specifications (width, thickness, hardness) of the slabs that have been queued, and the specifications of the remaining slabs. The action space is a discrete space of the expected probability of taking each possible action in the current state. Secondly, with the goal of minimizing the number of rolling units, minimizing the cost of specification jumps, and minimizing the penalty for not slabs being queued, a weighted comprehensive reward function for dual DQN training is designed. Then, the dual DQN network is trained and an adaptive A greedy strategy is used to select actions. A weighted TD error (Temporal Difference Error) is designed based on reward sparsity by introducing importance sampling weights. Samples are drawn from the experience replay pool according to their priority. After calculating the target Q value, a new loss function is designed in combination with the importance sampling weights. The network weights are optimized and updated. After training, a strategy for selecting the optimal action (i.e., production scheduling decision) under different states is obtained. Finally, the trained DQN network is used to generate an initial solution for hot-rolled production scheduling. An adaptive function is designed to dynamically adjust the reference point to improve NSGAIII and iteratively optimize the initial solution to obtain the optimal production scheduling solution.

[0062] For detailed process, see Figure 2 As shown, the steps are as follows:

[0063] (1) Step 1: Collect historical hot rolling production data, including slab specifications (width, thickness, hardness), production demand information (number of rolling batches (denoted as ), the number of rolling units in each batch (denoted as ), constraint information (maximum and minimum specification differences between slabs), the number of rolling units (valued as ), the specifications (width, thickness, hardness) data of the slabs that have been arranged and the slabs that have not been arranged are taken as the data set.

[0064] (2) Step 2: Data preprocessing of the collected slab specification information, production demand information and constraint information. For all numerical missing values, use ( is the value of the jth feature of the i-th sample after filling, is the value of the jth characteristic data of the kth sample, N is the number of non-missing samples) for mean filling; the slab specification information is filled using ( is the value of the jth feature data of the i-th sample, 、 are the mean and variance of the jth feature) to remove outliers; for all features, ( After standardization The value of 、 Represent the maximum and minimum values of the j-th feature respectively) to standardize the data of different dimensions so that they are on the same scale.

[0065] (3) Step 3: Divide the processed dataset into a training set and a test set in a ratio of 80% and 20%, respectively, for training and verifying the generalization performance of the dual DQN.

[0066] (4) Step 4: Construct a dual DQN based on the input feature dimension and the output action space size. The dual DQN consists of a Q network and a target network, which have the same structure but different weights. The Q network is used to estimate the value of the current state-action pair, and the target network is used to stabilize the training process and periodically copy parameters from the Q network.

[0067] First, both the Q network and the target network consist of an input layer, three hidden layers, and an output layer. The input layer receives the state vector of the current rolling batch. , where n is the number of features, Indicates the number of rolling units, the size of the slabs that have been queued, and the remaining slabs contained in the current i-th sample. The hidden layer consists of a multi-layer fully connected neural network, which is used to extract state features and calculate the Q value. Design three hidden layers, each with m neurons, then the outputs of the first, second, and third hidden layers are expressed as 、 and ,in is the ReLU activation function, and 、 and 、 and are the weights and biases of the first, second, and third hidden layers, respectively. The output layer activation function is softmax, and the output is an action value vector, which represents the expected probability (Q value) of taking each possible action (corresponding to different possible slab selections or ending the current rolling unit) in the current state. Action space Assume that the discrete space actions, the output layer is neurons, output vector .

[0068] Secondly, the DQN hyperparameters are initialized and configured as shown in Table 1. Each of the three hidden layers contains 256 neurons and uses the ReLU activation function.

[0069] Table 1 DQN parameter initialization configuration

[0070]

[0071] (5) Step 5: Combining the three objectives of minimizing the number of rolling units, minimizing the cost of size jumps, and minimizing the penalty for not being included in the slab, a comprehensive reward function is designed. Specifically, a negative reward is given for each additional rolling unit, rewards or penalties are given based on the size difference between adjacent slabs, and negative rewards are given for slabs that are not included in any rolling unit. The overall reward function R is designed as follows:

[0072]

[0073]

[0074]

[0075] in, is the number of rolling units; is the number of slabs that have been discharged; is the total penalty for not being discharged into the slab; is the quantity not discharged into the slab; is the penalty value for a single slab not discharged; is the weight coefficient; non-negative and ; is the specification jump cost between the i-th and i+1-th slabs; and 、 and 、 and are the width, thickness and hardness jump costs between the i-th and i+1-th slabs, is the weight of the three specification attributes of width, thickness and hardness, which is non-negative and .

[0076] (6) Step 6: Train the dual DQN network to obtain the strategy for selecting the optimal action under different states.

[0077] Specifically, first, the He initialization method is used to change the weight values of the Q network and the target network from the normal distribution where n is the number of neurons in the previous layer, Initialized to 10000, Initialized to 10000.

[0078] At the same time, in order to balance exploration and utilization, adaptive -Greedy strategy to choose actions:

[0079]

[0080] in, is the probability of exploring the strategy, 、 They are The initial maximum and minimum values of are 1.0 and 0.1 respectively; is the decay rate, control The speed at which the value decreases is set to 0.1; t is the current time step; and T is the total number of training rounds.

[0081] Importance sampling weights are introduced to improve experience replay to correct sampling bias, and a target network mechanism is used to stabilize network training. A weight factor is introduced based on reward sparsity. A weighted TD error method is designed to give higher weights to state-action pairs that receive rewards in a sparse reward environment to accelerate the learning process and encourage the exploration of state-action pairs that may lead to high rewards.

[0082] The weighted TD error is calculated as follows:

[0083]

[0084]

[0085] in, is the weighted TD error of the sample; 、 Represent the weight parameters of the Q network and the target network respectively; s, are the current state and the next state respectively; a, are the current action and the next possible action respectively; r is the reward obtained after executing action a from state s; A is the action space; is the discount factor; D is the experience replay pool; is the predicted value of the Q network; is the predicted value of the target network; Is a hyperparameter used to amplify the weight of the state-action pair that receives the reward, here we take , , is the total number of state-action pair samples, for The number of state-action pairs that receive non-zero rewards in , k is a tuning parameter used to control Sensitivity to changes in S, where k is 1.5.

[0086] Then, according to Update the priority of each sample ( is a small positive number used to avoid the situation where the priority is 0, here it is taken as 0.001), select a larger p value and extract a batch of samples from the experience replay buffer according to the sample importance each time (here it is taken as ), periodically (taken as every 5000 steps) copy parameters from the Q network to the target function as a parameter value.

[0087] The importance of the i-th sample is calculated as:

[0088]

[0089] in, is the importance sampling weight of sample i; is the size of the experience replay pool; Is an adjustment parameter used to control the weight size. When , all samples have the same weight; when When , high-priority samples will get greater weights, where Take it as 0.1.

[0090] DQN usually uses mean square error as the loss function. This paper designs a new loss function by combining importance sampling weights:

[0091]

[0092] in, is the number of samples for each training batch; is the current action of sample i; is the next possible action of sample i; 、 Represent the weight parameters of the Q network and the target network respectively; is the predicted value of Q network for sample i; For sample i in action The maximum predicted value at ; From the state characteristics Execute an action Rewards received after is the importance sampling weight of sample i; A is the action space; is the discount factor.

[0093] Finally, set the maximum number of training rounds or observe the changing trend of the reward function. When the preset maximum number of rounds is reached or the loss function tends to be stable, the model converges and training stops. The strategy of selecting the optimal action (i.e., production scheduling decision) under different states is learned.

[0094] (7) Step 7: Based on step 6, improve NSGAIII to optimize and solve the optimal rolling batch plan.

[0095] First, using the trained dual DQN network, an action a is selected based on the current state s, gradually building a rolling batch until all slabs are placed or a termination condition (such as the maximum number of rolling units) is reached. The resulting policy serves as the initial population for NSGA-III. Each solution represents a rolling batch plan, consisting of a series of slab arrangement orders and the number of rolling units.

[0096] Secondly, the parameters of NSGAIII are initialized as shown in Table 2.

[0097] Table 2 NSGAIII parameter initialization configuration

[0098]

[0099] Then, the improved NSGAIII optimization solves the optimal production scheduling scheme. The optimization objectives are to minimize the number of rolling units, minimize the cost of specification jumps, and minimize the penalty for not arranging slabs. The total reward function R in step 5 is used as the fitness function. According to the standard deviation of the distribution in each dimension of the target space To adjust the distribution of reference points to guide the population to explore the Pareto frontier more effectively.

[0100] Designing adaptive functions Dynamically adjust the reference point:

[0101]

[0102]

[0103] in, is the standard deviation of the population distribution in the i-th target dimension; is the value of the jth individual on the i-th target dimension; is the mean value on the i-th target dimension; N is the number of individuals in the population; 、 、 are the mean, maximum, and minimum values of the standard deviation of the distribution in all target dimensions, respectively; k is the adjustment coefficient, which is used to control the scaling and direction of the adaptive function output. A positive value of k usually indicates that more reference points are added when the standard deviation is large. Here it is taken as 1.5.

[0104] At the same time, non-dominated sorting is performed based on the dominance relationship between individuals. Assume that the population P, each individual , for each individual p, initialize its dominating set , dominated set ; For each pair of individuals p, q, if p dominates q, then , if q dominates p, then , and finally according to The values are sorted hierarchically.

[0105] The individual crowding degree is calculated as:

[0106]

[0107] in, is the crowding degree of the i-th individual, and are the values of individuals i+1 and i-1 on the jth target, and are the maximum and minimum values on the j-th target.

[0108] Next, the present invention uses real number coding to perform crossover and mutation operations on the individuals in the population to obtain the next generation population. For parent individuals and Perform crossover operation to obtain individual 、 , the operation is as follows:

[0109]

[0110]

[0111] Mutation probability Perform the mutation operation on the parent individual Parent to obtain the individual Child as follows:

[0112]

[0113] in 、 are the upper and lower bounds of the gene, respectively.

[0114] After performing crossover and mutation, the next generation population is obtained.

[0115] When the termination condition is reached, a set of Pareto optimal solutions is output, i.e., the optimal rolling batch configuration, including the number of rolling units, slab composition and sorting.

[0116] Based on the technical process of the hot rolling production scheduling method based on the dual DQN network disclosed in the above embodiment, a hot rolling production scheduling optimization system can be developed. The system can be composed of an acquisition module, a first data processing module, a second data processing module, a first construction module, a second construction module, a first output module, and a second output module.

[0117] The acquisition module is used to collect historical hot rolling production data sets, including slab specification information, production demand information, and constraint information. The first data processing module is used to preprocess the historical hot rolling production data sets so that the slab specification information, production demand information, and constraint information are on the same scale. The second data processing module is used to divide the preprocessed historical hot rolling production data sets into training sets and test sets according to a predetermined ratio. The first construction module is used to construct a dual DQN network, and design a comprehensive reward function by combining the three objectives of minimizing the number of rolling units, minimizing the specification jump cost, and minimizing the penalty for not including slabs. The second construction module is used to use the training set and the test set to train and verify the dual DQN network respectively to obtain a scheduling optimization model. The first output module is used to input the slab specification information, production demand information, and constraint information under the current production batch into the scheduling optimization model, and the scheduling optimization model outputs the optimal action strategy. The second output module is used to solve the optimal action strategy to obtain the optimal production scheduling plan.

[0118] The technical process of the hot rolling production scheduling method based on the dual DQN network disclosed in the above embodiment can be implemented in whole or in part through software, hardware, firmware or any other combination.

[0119] When implemented in hardware, the aforementioned embodiments can be run on an electronic device by compiling all or part of the operating logic and computational processes into software. The electronic device includes a processor, memory, a communication interface, and a communication bus. The processor, memory, and communication interface communicate with each other via the communication bus. The memory stores at least one executable instruction that causes the processor to execute the technical process of the hot rolling production scheduling method based on a dual DQN network disclosed in the aforementioned embodiments.

[0120] When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. If the above methods are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present application. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), magnetic disks, or optical disks. Thus, the embodiments of the present application are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0121] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A hot rolling production scheduling method based on a dual DQN network, characterized in that: The steps include: Collect historical hot rolling production data sets, including slab specifications, production requirements, and constraints; Preprocessing the historical hot rolling production data set so that the slab specification information, production demand information, and constraint information are on the same scale; The preprocessed historical hot rolling production data set is divided into a training set and a test set according to a predetermined ratio; A dual DQN network was constructed, and a comprehensive reward function was designed by combining the three objectives of minimizing the number of rolling units, minimizing the cost of specification jumps, and minimizing the penalty for not being included in the slab. The expression of the comprehensive reward function R is as follows: Where, is the number of rolling units, Indicates the number of rolling batches, Indicates the number of rolling units in each rolling batch; is the number of slabs that have been discharged; is the weight coefficient, non-negative and ; is the cost of specification jump between the i-th and i+1-th slabs: Where, and 、 and 、 and are the width, thickness and hardness jump costs between the i-th and i+1-th slabs respectively; is the weight of the three specification attributes of width, thickness and hardness, which is non-negative and ; The total penalty for not being discharged into the slab is: Where, is the quantity not discharged into the slab; is the penalty value for a single slab not discharged; The dual DQN network consists of a Q network and a target network. The weighted TD error is used to train the weight parameters of the Q network and the target network. The calculation formula is as follows: Where, is the weighted TD error of the sample; 、 Represent the weight parameters of the Q network and the target network respectively; s, are the current state and the next state respectively; a, are the current action and the next possible action respectively; r is the reward obtained after executing action a from state s; A is the action space; is the discount factor; is the predicted value of the Q network; is the predicted value of the target network; is a hyperparameter used to amplify the weight of the state-action pair that receives the reward; Using the training set and the test set to train and verify the dual DQN network, respectively, to obtain a scheduling optimization model; Inputting slab specification information, production demand information, and constraint information of the current production batch into the scheduling optimization model, and outputting the optimal action strategy from the scheduling optimization model; Solve the optimal action strategy to obtain the optimal production scheduling solution.

2. The hot rolling production scheduling method based on the dual DQN network according to claim 1 is characterized in that: The slab specification information records the width, thickness and hardness of the slab; The production demand information records the number of rolling batches , the number of rolling units in each rolling batch ; The constraint condition information records the maximum and minimum specification differences between slabs.

3. The hot rolling production scheduling method based on the dual DQN network according to claim 1 is characterized in that: The Q network and the target network both include an input layer, a hidden layer and an output layer; wherein the input layer receives the state vector of the current rolling batch; ; n is the number of features; The hidden layer is composed of a multi-layer fully connected neural network, which is used to extract state features and calculate the expected probability Q of taking each possible action in the current state. The output layer outputs the action space A, which has actions, the output layer is neurons, output vector .

4. The hot rolling production scheduling method based on the dual DQN network according to claim 1 is characterized in that: The loss function of the dual DQN network is as follows: Where, is the number of samples for each training batch; is the current action of sample i; is the next possible action of sample i; 、 Represent the weight parameters of the Q network and the target network respectively; Represents the status characteristics of the current i-th sample, including the number of rolling units, the specifications of the slabs that have been discharged, and the remaining slabs; is the predicted value of Q network for sample i; For sample i in action The maximum predicted value at ; From the state characteristics Execute an action Rewards received after is the importance sampling weight of sample i; A is the action space; is the discount factor.

5. The hot rolling production scheduling method based on the dual DQN network according to claim 1, characterized in that: Solving the optimal action strategy to obtain the optimal production scheduling solution specifically includes: Using the trained dual DQN network, an action a is selected based on the current state s, and a rolling batch is gradually constructed until all slabs are placed or a preset termination condition is reached. The generated strategy serves as the initial population for NSGA-III. Each solution represents a rolling batch plan, which contains a series of slab arrangement orders and the number of rolling units. With minimizing the number of rolling units, minimizing the cost of specification jumps and minimizing the penalty for not arranging slabs as the optimization objectives, the comprehensive reward function R is used as the fitness function, and the distribution of reference points is dynamically adjusted according to the standard deviation of the distribution in each dimension of the target space.

6. The hot rolling production scheduling method based on the dual DQN network according to claim 5 is characterized in that: Designing adaptive functions Dynamically adjust the distribution of reference points, adaptive function The expression is as follows: Where, is the standard deviation of the population distribution in the i-th target dimension; is the value of the jth individual on the i-th target dimension; is the mean value on the i-th target dimension; N is the number of individuals in the population; 、 、 are the mean, maximum, and minimum values of the standard deviation of the distribution in all target dimensions; k is the adjustment coefficient.

7. A hot rolling production scheduling optimization system, used to execute the hot rolling production scheduling method based on a dual DQN network according to any one of claims 1 to 6, characterized in that: The hot rolling production scheduling optimization system includes: The acquisition module is used to collect historical hot rolling production data sets, including slab specification information, production demand information, and constraint information; A first data processing module is used to pre-process the historical hot rolling production data set so that the slab specification information, production demand information, and constraint information are on the same scale; The second data processing module is used to divide the preprocessed historical hot rolling production data set into a training set and a test set according to a predetermined ratio; The first building block is used to construct a dual DQN network, combining the three objectives of minimizing the number of rolling units, minimizing the cost of specification jumps, and minimizing the penalty for not being queued into the slab, to design a comprehensive reward function; The second building module is used to train and verify the dual DQN network using the training set and the test set respectively to obtain a scheduling optimization model; A first output module is used to input slab specification information, production demand information, and constraint information of the current production batch into the scheduling optimization model, and the scheduling optimization model outputs an optimal action strategy; The second output module is used to solve the optimal action strategy and obtain the optimal production scheduling solution.

8. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the hot rolling production scheduling method based on the dual DQN network according to any one of claims 1 to 6 is implemented.

Images