Deep reinforcement learning-based self-learning deviation correction control method for finish rolling threading process
By constructing a deep reinforcement learning model and a closed-loop feedback mechanism, the problem of precise adjustment of rolling force during the strip head through the strip is solved, and the precise control of the strip outlet deviation is achieved, and the material yield and equipment stability of the hot continuous rolling finishing mill group are improved.
Patent Information
- Application Number
- CN202510434706.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2025-08-26
AI Technical Summary
The prior art cannot accurately adjust the rolling force of the strip head through the strip in the actual production process, resulting in the strip outlet deviation, affecting the material yield and equipment stability.
A self-learning correction control method for fine rolling and stranding processes based on deep reinforcement learning is constructed. By constructing state space, designing action space and reward functions, the optimal strategy is generated using the deep Q network algorithm, and combining the closed-loop feedback mechanism to compensate the strip entrance deviation in real time to optimize the rolling force allocation.
It realizes precise control of strip outlet deviation, improves the yield rate and equipment stability, and is suitable for high-precision rolling of multi-special strip steel in hot continuous rolling and finishing mills.
Smart Images

Figure CN120532873A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of steel rolling automation control, and in particular to a self-learning and deviation-correcting control method for a finishing rolling threading process based on deep reinforcement learning. Background Art
[0002] During the hot rolling process, the strip head threading process is prone to strip deviation at the exit due to factors such as uneven rolling force distribution and dynamic changes in rolling mill stiffness, affecting the yield rate and equipment stability. Traditional control methods rely on mechanism models and manual experience adjustments, and have the following defects:
[0003] 1. Response lag: Manual adjustments rely on post-event feedback, making it difficult to compensate for dynamic operating condition changes in a timely manner.
[0004] 2. Strong dependence on experience: Parameters need to be repeatedly adjusted for different specifications of strip steel, lacking adaptive capabilities;
[0005] 3. Multivariable coupling: Nonlinear parameters such as rolling mill stiffness difference and plasticity coefficient are difficult to accurately represent using fixed models.
[0006] In recent years, deep reinforcement learning (DRL) has demonstrated its advantages in complex industrial control. Through interactive learning between an agent and its environment, it can achieve real-time optimization of multivariable systems. However, existing research has mostly focused on theoretical simulations and has not addressed the following core issues in rolling scenarios:
[0007] 1. How to construct a state space and action space that are strongly related to the physical characteristics of the rolling mill;
[0008] 2. How to design a reward function that directly links to process indicators (such as exit deviation);
[0009] 3. How to implement the embedded application of the strategy in the real rolling mill control system.
[0010] In summary, the existing technology cannot achieve precise adjustment of the rolling force during the strip head threading process in actual production. Summary of the Invention
[0011] The present invention provides a self-learning and correction control method for the finishing rolling threading process based on deep reinforcement learning, so as to solve the technical problem that the existing technology cannot achieve precise adjustment of the rolling force during the strip head threading process in actual production.
[0012] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0013] On the one hand, the present invention provides a self-learning and correcting control method for a finishing strip threading process based on deep reinforcement learning, the self-learning and correcting control method for a finishing strip threading process based on deep reinforcement learning comprises:
[0014] A deep reinforcement learning model is constructed using the finishing mill stand during the finishing strip threading process as the intelligent agent.
[0015] The deep reinforcement learning model is used to train a state-action strategy, and based on the deep reinforcement learning model, an optimal strategy is generated through interaction between the intelligent agent and the rolling environment;
[0016] The optimal strategy is embedded in the self-learning system of the finishing rolling threading process for real-time correction, so as to compensate for the strip entry deviation in real time and control the strip exit deviation within the process allowable range.
[0017] Furthermore, the finishing mill stand in the finishing strip threading process is used as an intelligent agent to construct a deep reinforcement learning model, including:
[0018] The state space is constructed using the dynamic parameters of the strip head entering the rolling mill during the finishing rolling threading process. The dynamic parameters of the strip head entering the rolling mill during the finishing rolling threading process include: set rolling force, rolling mill stiffness difference, rolling mill plasticity coefficient, finishing exit set width, finishing exit set thickness, entrance deviation, rolling roll bending force, and rolling speed. The rolling mill stiffness difference is defined as the stiffness difference between the operating side and the transmission side of the rolling mill, and the rolling mill plasticity coefficient is a quantitative indicator of the strip's plastic deformation capacity during the rolling mill roll gap leveling process.
[0019] The core process parameters of the rolling mill operation are used as actions to construct an action space; wherein the core process parameters of the rolling mill operation include: the adjustment amount of the rolling force on the operating side and the adjustment amount of the rolling force on the transmission side;
[0020] Construct a reward function and use the negative absolute value of the strip exit deviation as the objective function for reinforcement learning optimization;
[0021] Based on the state space, action space, and reward function, a deep reinforcement learning model is constructed.
[0022] Furthermore, the reward function uses the strip entrance deviation and process parameters as inputs of a preset model, predicts the strip exit deviation through the preset model, and uses the negative absolute value of the predicted strip exit deviation as the objective function of reinforcement learning optimization.
[0023] Furthermore, the process parameters include: set rolling force, rolling mill stiffness difference, rolling mill plasticity coefficient, rolling bending roll force, rolling speed, rolling force difference between the operating side and the transmission side, and total strip threading rolling force.
[0024] Furthermore, the preset model is an SVR model.
[0025] Furthermore, the preset model uses mean square error as the loss function.
[0026] Furthermore, the deep reinforcement learning model is a deep Q network model.
[0027] Furthermore, the deep reinforcement learning model adopts an ε-greedy strategy for dynamic exploration when training the state-action strategy; uses the rectified linear unit (ReLU) as the activation function; calculates the loss function based on the temporal difference error; and optimizes the network parameters through gradient descent.
[0028] Furthermore, the optimal strategy is embedded in the self-learning system of the finishing strip threading process for real-time correction, so as to compensate for the strip entry deviation in real time and control the strip exit deviation within the process allowable range, including:
[0029] According to the historical deviation data of the rolling forces on both sides of the rolling mill, based on the optimal strategy, the distribution ratio of the rolling forces on the operating side and the transmission side is dynamically adjusted;
[0030] The closed-loop feedback mechanism compensates for the strip entry deviation in real time, ensuring that the strip exit deviation is within the process allowable range.
[0031] The response speed and accuracy of roll gap leveling control are optimized by combining the rolling mill stiffness difference and plasticity coefficient.
[0032] Furthermore, the closed-loop feedback mechanism specifically includes:
[0033] Real-time collection of rolling force sensor data on both sides of the rolling mill to calculate the deviation between the actual rolling force and the target value;
[0034] Based on the deviation between the actual rolling force and the target value, the rolling force adjustment amount on the operating side and the transmission side in the action strategy output by the deep reinforcement learning model is dynamically adjusted;
[0035] Through iterative correction, the strip exit deviation is reduced to meet the rolling requirements.
[0036] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.
[0037] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the above method.
[0038] The beneficial effects brought about by the technical solution provided by the present invention include at least:
[0039] The present invention constructs a state space that integrates the dynamic characteristics of the rolling mill and the process parameters of the strip steel, designs a discretized rolling force adjustment action space, and uses a support vector regression (SVR) model to predict the exit deviation to generate a reward function. The deep Q network (DQN) algorithm is used, combined with the ε-greedy dynamic exploration strategy and the Relu activation function, to train the intelligent agent to interact with the rolling environment to generate the optimal control strategy. Thus, by adjusting the rolling force distribution ratio between the operating side and the transmission side of the rolling mill in real time, the roll gap leveling response is optimized in combination with a closed-loop feedback mechanism. This method is simple and has good performance. It solves the problems of traditional control relying on manual experience and delayed response, can significantly improve the strip steel exit centering accuracy, and is suitable for the high-precision rolling requirements of multiple specifications of strip steel in hot rolling finishing mills. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0041] Figure 1 1 is a schematic diagram of the execution flow of the self-learning deviation correction control method for the finishing strip threading process based on deep reinforcement learning provided by an embodiment of the present invention;
[0042] Figure 2 2 is a schematic diagram of the implementation effect of the self-learning deviation correction control method for the finishing strip threading process based on deep reinforcement learning provided by an embodiment of the present invention;
[0043] Figure 3 This is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0044] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
[0045] First, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present concepts in a concrete manner. In addition, in the embodiments of the present invention, the meaning of "and / or" can be both or either of the two.
[0046] First embodiment
[0047] This embodiment provides a self-learning deviation correction control method for the finishing rolling process based on deep reinforcement learning. The core lies in the deep integration of the deep reinforcement learning framework with the rolling process. By constructing a process-driven state space, designing a discretized action space, integrating the reward function of support vector regression (SVR), and combining the deep Q network (DQN) algorithm, real-time closed-loop optimization of the strip deviation is achieved, and the rolling accuracy and adaptive ability are improved. This method can be implemented by electronic equipment, and its execution process is as follows: Figure 1 As shown, the following steps are included:
[0048] S1, design three elements of reinforcement learning;
[0049] Specifically, the three elements of reinforcement learning include state space, action space and reward function. Among them, it should be noted that the self-learning correction control model of the finishing strip threading process is to adjust the adjustment amount of the previous roll of strip in the threading process to the pre-controlled roll gap value of the next roll of strip according to a certain proportion. The deep reinforcement learning model can be used to adaptively adjust the adjustment amount of the self-learning of threading according to the running state of the strip. The self-learning process of strip threading can be defined by a Markov decision process, which is expressed by a five-tuple (S, A, P, R, γ). Among them, S is the state space, A is the action space, R is all the reward values, P is the state transfer function, and γ is the discount factor. In this regard, in order to construct the subsequent reinforcement learning model, the three elements of reinforcement learning need to be designed first.
[0050] The state space is constructed based on the dynamic parameters of the strip head entering the rolling mill during the finishing mill threading process. This embodiment selects key parameters to design the strip state space based on the dynamic characteristics of the rolling mill and the strip process parameters. These parameters are the values that affect the strip exit deviation after passing through the rolls. These parameters include: set rolling force, mill stiffness difference, mill plasticity coefficient, set width at the finishing mill exit, set thickness at the finishing mill exit, entry deviation, rolling roll bending force, and rolling speed. The mill stiffness difference is defined as the difference in stiffness between the operating and transmission sides of the rolling mill, and the mill plasticity coefficient is a quantitative indicator of the strip's plastic deformation capacity during the mill roll gap leveling process.
[0051] The state space parameters are selected based on:
[0052] a) setting the rolling force, the width and thickness of the finishing exit to represent the target geometric dimensions of the strip;
[0053] b) Mill stiffness difference and plasticity coefficient are used to quantify the asymmetry of mechanical properties on both sides of the mill;
[0054] c) Entry deviation, rolling roll bending force and rolling speed are used to evaluate the dynamic behavior of the strip in real time;
[0055] The combination of the above parameters can fully describe the interaction between the rolling mill and the strip during the strip threading process.
[0056] The action space is defined as the core process parameters of the rolling mill operation, including the operating side (OS) rolling force adjustment and the drive side (DS) rolling force adjustment, which are used to control the rolling mill; it should be noted that the discrete action combination of the action space designed in this embodiment is (a1, a2), wherein a1 is the rolling force change on the OS side, and a2 is the rolling force change on the DS side. The rolling force adjustment resolution of modern rolling mill hydraulic cylinders is generally ±200 to 1000 kN, with a typical adjustment step of 500 kN. This can effectively compensate for instantaneous deviations (such as rolling force differences on the OS / DS side caused by mill stiffness differences) in a single-step action. The adjustment steps include: (+a1, +a2), (+a1, -a2), (-a1, +a2), and (-a1, -a2). In this embodiment, the change in each action value is defined as 500. Therefore, based on the set rolling force, four action states can be defined, specifically: (+500 kN, +500 kN), (+500 kN, -500 kN), (-500 kN, +500 kN), and (-500 kN, -500 kN). The design basis is:
[0057] a) The 500kN adjustment can effectively compensate for the instantaneous deviation of the rolling force on both sides of the rolling mill;
[0058] b) Discrete action combinations can reduce the dimension of the strategy search space and improve the convergence efficiency of the DQN algorithm;
[0059] c) The action amplitude matches the control accuracy of the actual rolling mill hydraulic system.
[0060] d) The discretization strategy reduces the search dimension of the DQN algorithm and improves convergence efficiency.
[0061] The reward function uses the radial basis function (RBF) kernel algorithm in the support vector regression (SVR) model to predict the exit deviation based on the entry deviation and process parameters, and uses the negative absolute value of the predicted exit deviation as the objective function of the reinforcement learning optimization. The specific design steps are as follows:
[0062] 1) An exit runout predictor is constructed based on the SVR model. The input parameters of the runout predictor include: set rolling force, mill stiffness difference, mill plasticity coefficient, entry runout, rolling roll bending force, rolling speed, OS / DS side rolling force difference, and total strip threading rolling force; the output parameter is the exit runout;
[0063] 2) Using mean square error (MSE) as the loss function of the SVR model, the predictor is trained using historical rolling data;
[0064] 3) The negative absolute value of the predicted outlet deviation is used as the reward value, so that the reinforcement learning strategy can directly optimize the strip alignment performance.
[0065] The deep reinforcement learning model can be built based on the three designed reinforcement learning elements.
[0066] S2, training state-action strategy;
[0067] Specifically, this embodiment uses the Deep Q Network (DQN) algorithm to train the state-action strategy. Its network architecture is: a two-hidden layer fully connected network with ReLU activation function and the output layer as the Q value of each action. Based on this, the process of implementing the state-action strategy training includes:
[0068] a) Initialize DQN network parameters and construct state-action value function mapping;
[0069] b) Dynamic exploration is performed using an ε-greedy strategy. Given an initial exploration rate ε, in the early stages of reinforcement learning, the agent lacks prior knowledge of the rolling environment and needs to rapidly accumulate experience with state-action pairs through a high exploration rate (nearly completely random). The mill threading process is highly dynamic (e.g., due to poor mill stiffness and strip plastic deformation). A high exploration rate prevents the strategy from prematurely falling into local optima, ensuring comprehensive coverage of complex operating conditions. The exploration rate is also ensured to gradually decrease over thousands to tens of thousands of steps to avoid insufficient exploration later due to rapid decay. A 1% random exploration probability is retained to prevent control instability due to environmental noise after the strategy fully converges.
[0070] Specifically, in this embodiment, the initial value of ε is 0.95 and decays to a minimum value of 0.01 at a rate of 0.001 to balance exploration and utilization.
[0071] c) Use rectified linear unit (ReLU) as the activation function to ensure network training stability;
[0072] d) Calculate the loss function based on the temporal difference (TD) error and optimize the network parameters by gradient descent.
[0073] S3, the agent generates the optimal strategy;
[0074] Specifically, this embodiment generates an optimal strategy through the interaction between the intelligent agent and the rolling environment, and uses it for self-learning and correction control of finishing strip threading. The state-action strategy is obtained by using the existing state space, action space, and reward function, including the following steps:
[0075] a) Initialize the network, input the state st, and output the Q value of all actions;
[0076] b) At each time step t, based on the current state st, an action at is selected using the strategy (ε-greedy), at is input into the environment, and after execution, a new state st+1 and a reward value r are obtained;
[0077] c) Accumulate single-step reward values and update the strategy based on the principle of maximizing the total reward value; including: 1) Calculate the TD target: yt = rt + γ * maxaQ (st + 1, a; w); 2) Calculate the loss function: L = 1 / 2 * [yt - Q (s, a; w)] 2 3) Update the Q parameter so that Q(st, at) is as close to yt as possible, treating it as a regression problem and using gradient descent to update it; 4) From the above steps, we get a four-tuple transition: (st, at, rt, st+1); 5)
[0078] Enter the new status and repeat the update work;
[0079] d) Generate an optimal control strategy that adapts to the dynamic operating conditions of the rolling mill through iterative optimization using a Markov decision process (MDP). The five-tuple of the Markov decision process is defined as (S, A, P, R, γ), which includes: 1) Determining the initial value of the discount factor γ (in this embodiment, the initial value of the γ discount factor is 0.95, the minimum value is 0.01, and the exploration rate is set to 0.001 each time). In the rolling scenario, the strip threading process is a short-term task that needs to maximize the cumulative reward within a limited number of steps. The initial value of the discount factor must ensure that the rewards for the next 5 to 10 steps significantly affect the current decision, balancing immediate rewards with long-term benefits; 2) The state transition function P is jointly characterized by the rolling mill dynamics model and the strip plastic deformation model; 3) The reward function R is strongly correlated with the process requirements for the strip exit deviation, ensuring that the control objectives are consistent with the production indicators.
[0080] By accumulating single-step rewards and maximizing the total reward value, the optimal control strategy that adapts to dynamic working conditions is generated.
[0081] S4, correct the threading process;
[0082] Specifically, after obtaining the optimal control strategy, it can be embedded into the finishing mill threading process self-learning system. This optimal strategy can then be applied to the strip threading process data, correcting the threading self-learning process. Specifically, the rolling force is adjusted based on the rolling force adjustment value in the optimal strategy to adjust the strip deviation. After adjusting the strip deviation, the rolling force can be dynamically adjusted in real time through a closed-loop regulation mechanism to further optimize the strip deviation, ultimately ensuring that the strip deviation meets process requirements.
[0083] In this embodiment, the specific implementation process of correcting the tape threading process is as follows:
[0084] a) Dynamically adjust the distribution ratio of rolling force on the OS side and DS side based on the historical deviation data of rolling force on both sides of the rolling mill;
[0085] b) A closed-loop feedback mechanism is used to compensate for strip entry deviation in real time, ensuring that exit deviation is within the process tolerance. This closed-loop feedback mechanism specifically includes: a time-correction mechanism that dynamically adjusts the OS / DS side rolling force distribution ratio to compensate for historical deviations; closed-loop feedback based on sensor data to correct the action strategy in real time; and roll gap leveling optimization: combining the mill's stiffness differential and plasticity coefficient to improve the leveling response speed and accuracy, ensuring that exit deviation is within the process tolerance. The implementation process involves real-time acquisition of rolling force sensor data from both sides of the mill, calculating the deviation between the actual rolling force and the target value; dynamically adjusting the OS / DS side rolling force adjustment in the action strategy based on the deviation; and iterative corrections to reduce strip exit deviation, meeting high-precision rolling requirements.
[0086] c) Combine the rolling mill stiffness difference and plasticity coefficient to optimize the response speed and accuracy of roll gap leveling control.
[0087] In summary, this embodiment provides a self-learning correction control method for the finishing strip threading process based on deep reinforcement learning, and its implementation effect is as follows: Figure 2 As shown in the figure, the self-learning and deviation-correcting control method for the finishing strip threading process is applicable to the strip threading process of the hot rolling finishing mill. Its specific application scenarios include but are not limited to:
[0088] a) The strip head runs off the mill due to uneven distribution of rolling force when entering the finishing mill;
[0089] b) hysteresis effect of roll gap leveling caused by dynamic changes in mill stiffness;
[0090] c) Correction of cumulative errors caused by fluctuations in process parameters during continuous rolling of multiple coils of steel strip.
[0091] Second embodiment
[0092] This embodiment provides an electronic device, such as Figure 3 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment described above. In addition, the electronic device may also include a transceiver; the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.
[0093] Next, combine Figure 3 A detailed introduction to the various components of the electronic device is given below:
[0094] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor can perform various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.
[0095] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 3 The CPU0 and CPU1 shown in FIG are, of course, only exemplary.
[0096] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.
[0097] Optionally, the memory may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.
[0098] The transceiver may include a receiver and a transmitter ( Figure 3 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 3 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.
[0099] In addition, it should be noted that Figure 3 The structure of the electronic device shown in the figure does not constitute a limitation on the device. The actual device may include more or fewer components than shown, or may combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment can refer to the technical effects described in the first embodiment above, and therefore will not be repeated here.
[0100] Third embodiment
[0101] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device. The instructions stored therein can be loaded by a processor in a terminal to execute the method described above.
[0102] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention may take the form of a fully or partially hardware embodiment, a fully or partially software embodiment, or an embodiment combining software and hardware aspects. Furthermore, when implemented using software, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired connection (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state drive.
[0103] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0104] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0105] It should also be noted that, in this document, relational terms such as first and second are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. The terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, article, or terminal device comprising the element. In addition, the term "and / or" is merely a description of an associative relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: the presence of A alone, the presence of A and B simultaneously, or the presence of B alone, where A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "more" means two or more. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.
[0106] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0107] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.
[0108] In the several embodiments provided herein, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of functional modules / units is merely a logical functional division. In actual implementation, other division methods may be used, such as multiple units or components being combined or integrated into another device, or some features being ignored or not implemented. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interface, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs. In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.
[0109] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0110] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. It should be noted that, although preferred embodiments of the present invention have been described, those skilled in the art, once understanding the basic inventive concepts of the present invention, may make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all variations and modifications that fall within the scope of the embodiments of the present invention.
Claims
1. A self-learning and correction control method for the finishing strip threading process based on deep reinforcement learning, characterized in that: The self-learning and deviation-correcting control method for the finishing strip threading process based on deep reinforcement learning includes: A deep reinforcement learning model is constructed using the finishing mill stand during the finishing strip threading process as the intelligent agent. The deep reinforcement learning model is used to train a state-action strategy, and based on the deep reinforcement learning model, an optimal strategy is generated through interaction between the intelligent agent and the rolling environment; The optimal strategy is embedded in the self-learning system of the finishing rolling threading process for real-time correction, so as to compensate for the strip entry deviation in real time and control the strip exit deviation within the process allowable range.
2. The self-learning and error-correcting control method for the finishing strip threading process based on deep reinforcement learning according to claim 1 is characterized in that: The method uses the finishing mill stand in the finishing strip threading process as an intelligent agent to construct a deep reinforcement learning model, including: The state space is constructed using the dynamic parameters of the strip head entering the rolling mill during the finishing rolling threading process. The dynamic parameters of the strip head entering the rolling mill during the finishing rolling threading process include: set rolling force, rolling mill stiffness difference, rolling mill plasticity coefficient, finishing exit set width, finishing exit set thickness, entrance deviation, rolling roll bending force, and rolling speed. The rolling mill stiffness difference is defined as the stiffness difference between the operating side and the transmission side of the rolling mill, and the rolling mill plasticity coefficient is a quantitative indicator of the strip's plastic deformation capacity during the rolling mill roll gap leveling process. The core process parameters of the rolling mill operation are used as actions to construct an action space; wherein the core process parameters of the rolling mill operation include: the adjustment amount of the rolling force on the operating side and the adjustment amount of the rolling force on the transmission side; Construct a reward function and use the negative absolute value of the strip exit deviation as the objective function for reinforcement learning optimization; Based on the state space, action space, and reward function, a deep reinforcement learning model is constructed.
3. The self-learning and error-correcting control method for the finishing strip threading process based on deep reinforcement learning according to claim 2, characterized in that: The reward function uses the strip entry deviation and process parameters as inputs of a preset model, predicts the strip exit deviation through the preset model, and uses the negative absolute value of the predicted strip exit deviation as the objective function of reinforcement learning optimization.
4. The self-learning and error-correcting control method for the finishing strip threading process based on deep reinforcement learning according to claim 3 is characterized in that: The process parameters include: set rolling force, rolling mill stiffness difference, rolling mill plasticity coefficient, rolling roll bending force, rolling speed, rolling force difference between the operating side and the transmission side, and total strip threading rolling force.
5. The self-learning and error-correcting control method for the finishing strip threading process based on deep reinforcement learning according to claim 3 is characterized in that: The preset model is the SVR model.
6. The self-learning and error-correcting control method for the finishing strip threading process based on deep reinforcement learning according to claim 3, characterized in that: The preset model uses mean square error as the loss function.
7. The self-learning and error-correcting control method for the finishing strip threading process based on deep reinforcement learning according to claim 1, characterized in that: The deep reinforcement learning model is a deep Q network model.
8. The self-learning and error-correcting control method for the finishing strip threading process based on deep reinforcement learning according to claim 1, characterized in that: The deep reinforcement learning model adopts the ε-greedy strategy for dynamic exploration when training the state-action strategy; uses the rectified linear unit (ReLU) as the activation function; calculates the loss function based on the temporal difference error; and optimizes the network parameters through gradient descent.
9. The self-learning deviation correction control method for the finishing strip threading process based on deep reinforcement learning according to claim 1, characterized in that: The optimal strategy is embedded in the self-learning system of the finishing strip threading process for real-time correction to compensate for the strip entry deviation in real time and control the strip exit deviation within the process allowable range, including: According to the historical deviation data of the rolling forces on both sides of the rolling mill, based on the optimal strategy, the distribution ratio of the rolling forces on the operating side and the transmission side is dynamically adjusted; The closed-loop feedback mechanism compensates for the strip entry deviation in real time, ensuring that the strip exit deviation is within the process allowable range. The response speed and accuracy of roll gap leveling control are optimized by combining the rolling mill stiffness difference and plasticity coefficient.
10. The self-learning deviation correction control method for the finishing strip threading process based on deep reinforcement learning according to claim 9, characterized in that: The closed-loop feedback mechanism specifically includes: Real-time collection of rolling force sensor data on both sides of the rolling mill to calculate the deviation between the actual rolling force and the target value; Based on the deviation between the actual rolling force and the target value, the rolling force adjustment amount on the operating side and the transmission side in the action strategy output by the deep reinforcement learning model is dynamically adjusted; Through iterative correction, the strip exit deviation is reduced to meet the rolling requirements.
Citation Information
Cited By
Pre-swing roll gap dynamic correction method for strip head and tail deviation characteristics
CN121017251A
A finishing strip head offset control method based on deep reinforcement learning
CN122500050A
A finishing strip head offset control method based on deep reinforcement learning
CN122500050B
