A method for setting and realizing optimal quenching time of steel pipe

Abstract

This invention discloses a method for setting and implementing the optimal quenching time for steel pipes, belonging to the field of metallurgical heat treatment technology. The method includes the following steps: S1, constructing a two-dimensional unsteady-state heat conduction model of the water quenching process of the steel pipe and setting staged heat transfer boundary conditions to obtain the theoretical quenching time; S2, obtaining the theoretical quenching time through discrete solution using the finite difference method and Newton iteration update; S3, constructing a neural network deviation correction model and fusing deviation prediction values ​​to obtain the initial optimal quenching time for the steel pipe; S4, obtaining the final optimal quenching time for the steel pipe by constructing a reinforcement learning framework of a Markov decision process and executing closed-loop interactive optimization using a deep Q-network. Using the above method, the mechanism of steel pipe quenching time can be accurately calculated, data deviation compensated, and dynamically adaptively corrected, significantly improving the accuracy of final cooling temperature control and adaptability to complex working conditions, achieving high-precision, high-robustness, and intelligent closed-loop optimization of the quenching process.

Description

Technical Field

[0001] This invention relates to the field of metallurgical heat treatment technology, and in particular to a method for setting and implementing the optimal quenching time for steel pipes. Background Technology

[0002] Steel is an indispensable basic material for modern industry. Seamless steel pipes, with their excellent load-bearing and pressure-resistant properties, are widely used in petroleum, military, aerospace, nuclear energy, and automobile manufacturing. The precise control of the quenching process directly determines the mechanical properties of steel pipes and is the core link in steel pipe quality control. At present, a large number of studies have been carried out at home and abroad on steel pipe quenching, covering cooling medium optimization, temperature field and phase transformation stress modeling, thermo-mechanical-phase transformation finite element simulation, experimental analysis, and data-driven methods such as BP neural networks. Many research results have been achieved in the direction of quenching process optimization.

[0003] However, existing technologies still have prominent technical defects and problems: traditional steel pipe quenching process parameters are highly dependent on human experience, lack real-time feedback and closed-loop adjustment mechanisms, data-driven models lack physical constraints, and have poor generalization ability and interpretability. At the same time, most control strategies are based on fixed mechanism models or static fitting methods, which cannot adapt to dynamic working condition fluctuations such as ambient temperature, cooling water temperature, and spray intensity. The overall system has a low level of intelligence, lacks the ability to autonomously optimize and adaptively adjust under complex working conditions, and it is difficult to achieve precise and stable control of quenching temperature. Summary of the Invention

[0004] The purpose of this invention is to provide a method for setting and implementing the optimal quenching time for steel pipes, thereby solving the aforementioned technical problems.

[0005] To achieve the above objectives, the present invention provides a method for setting and implementing the optimal quenching time for steel pipes, comprising the following steps: S1. Based on the heat conduction theory, a two-dimensional unsteady-state heat conduction model of the water quenching process of steel pipe is constructed and a staged heat transfer boundary condition is set to obtain the theoretical quenching time of steel pipe quenching. S2. Based on the two-dimensional unsteady heat conduction model of S1, the theoretical quenching time, which has been refined through numerical verification, is obtained by discretization and Newton iteration update using the finite difference method. S3. Based on historical production data and the theoretical quenching time of S2, a neural network deviation correction model is constructed and the deviation prediction value is fused to obtain the initial optimal time for quenching steel pipes. S4. Based on the initial optimal quenching time of steel pipe in S3, the online dynamic correction of the quenching time is achieved by constructing a reinforcement learning framework of Markov decision process and performing closed-loop interactive optimization of deep Q network, so as to obtain the final optimal quenching time of steel pipe.

[0006] Preferably, in S1, the two-dimensional unsteady-state heat conduction equation constructed based on heat conduction theory is as follows: ; in, Density of the steel pipe; The specific heat capacity of the steel pipe; The thermal conductivity of the steel pipe; For temperature; For time; The axial direction of the steel pipe; This refers to the direction of the steel pipe wall thickness.

[0007] Preferably, the input parameters of the two-dimensional unsteady heat conduction model are derived from the MES system and the quality system. The input parameters include the steel pipe card number, steel type, wall thickness, outer diameter, internal / external spray water flow rate and water temperature, steel temperature before quenching, and ambient temperature. The output of the numerical solution is the spraying time that meets the target final cooling temperature.

[0008] Preferably, in S1, the staged heat transfer boundary conditions are as follows: For the four stages of air cooling, internal spray cooling, internal spray external shower cooling, and secondary air cooling, radiative heat transfer and third-type convective heat transfer boundary conditions are set respectively, where: During the air-cooling stage, the steel pipe is not in contact with the cooling water, and its surface mainly dissipates heat to the environment through thermal radiation. The heat transfer through radiation is expressed by the formula: ; in, For radiative heat exchange; Surface emissivity; This represents the wall area of ​​the steel pipe. It is the Stefan-Boltzmann constant; and The temperatures are, in order, the surface temperature of the steel pipe and the ambient temperature; Radiative heat flux density The calculation formula is: ; When the steel pipe enters the water quenching stage, surface heat transfer is dominated by water or air convection, represented by the third type of boundary condition, and includes both internal spray cooling and internal spray-external spray cooling stages. In the internal spray cooling stage, only the inner surface of the steel pipe is in contact with water, while the outer surface remains cooled by air. The formula is: ; ; in, The heat flux density on the inner surface of the steel pipe; The heat flux density on the outer surface of the steel pipe; The coefficient of thermal transfer; , These are the inner surface temperature and the outer surface temperature of the steel pipe, respectively. , The temperature of the medium (water) and the ambient temperature; During the internal spray and external cooling stage, the inner and outer surfaces of the steel pipe are simultaneously cooled by water, as shown in the formula: ; ; In the second air-cooling stage, cooling is achieved through air cooling.

[0009] Preferably, based on the internal spray and external spray cooling stages in S1, a numerical solution strategy with spray time as the optimization variable is introduced, and the objective function is defined as: ; in, This refers to the spraying time during the internal spraying and external spraying stages; For the spraying time is The final temperature obtained from the simulation under the given conditions; The target final cooling temperature is set.

[0010] Preferably, the specific steps of S2 include: S21. Simplify the three-dimensional heat transfer of steel pipes into two-dimensional planar heat transfer, forming a regular two-dimensional grid; S22. Using the control volume energy balance method, the two-dimensional unsteady heat conduction equation is discretized, divided into straight boundary nodes, external corner nodes, and internal boundary nodes, and corresponding difference expressions are established for each. S23. The objective function for the internal spray and external cooling stage is solved using the Newton-Raphson iteration method. The spray time update formula for the Newton-Raphson iteration is as follows: ; in, For numerical difference approximation, and , This represents the time increment for Newton's iterations; Each iteration requires re-execution of the complete internal spray and external cooling stage and the second air cooling stage to obtain the refined theoretical quenching time that meets the termination condition. .

[0011] Preferably, the iteration termination condition in S23 includes: the temperature error satisfies... , Tolerance; reaching the maximum number of iterations; numerical derivative. If the value is too small, the update range will become invalid; the iteration will terminate once any condition is met.

[0012] Preferably, the specific steps of S3 include: S31. Based on the historical records of the MES system and the quality traceability system, abnormal and missing samples are removed through data preprocessing, and continuous variables are standardized to obtain a feature vector containing the working condition status. ; S32. Based on the eigenvectors and theoretical quenching time, a quenching time deviation term is introduced to characterize the systematic difference between the prediction results of the mechanism model and actual production. And define the deviation function formula as follows: ; in, The actual quenching time recorded in historical production records; This is the feature vector of the current operating condition. S33, Using an artificial neural network to calculate the deviation function of S32 Modeling is performed using a neural network with standardized operating condition feature vectors. As input, the corresponding predicted value of the quenching time deviation is output. It is used to characterize the systematic errors of the mechanism model under complex working conditions; S34, Based on theoretical quenching time Deviation Predicted Value Through fusion calculation, the initial optimal quenching time for the steel pipe is obtained, and the calculation formula is as follows: ; in, The initial optimal quenching time for steel pipes is determined. This fusion strategy, while maintaining the physical consistency of the mechanistic model, introduces a data-driven model to correct for the impact of complex working conditions, enabling the predicted quenching time to better adapt to the dynamic changes in the actual production environment.

[0013] Preferably, the specific steps of S4 include: S41. Based on S34, the initial optimal time for steel pipe quenching is determined by extracting observable process parameters during quenching. A state space is constructed that includes initial temperature, initial optimal time, ambient temperature, water temperature, and disturbance characteristics. The parameters for... The action space of the finite set of discrete actions to be added or subtracted, and the reward function with temperature deviation as its core. Thus, a Markov decision process model for the quenching process of steel pipes was obtained; S42. Based on the Markov decision process model, the initial optimal quenching time is determined... With the Action correction quantity for the next decision choice Combining, we obtain the first The actual execution time for the next reinforcement learning decision is calculated using the following formula: ; in, For the first The actual execution time corresponding to the reinforcement learning decision; Number the decision steps; For motion correction, and , It is a finite set of discrete actions used to describe the increase or decrease of the reference quenching time. Adjustment amount for time; S43, Based on actual quenching time Substituting the two-dimensional unsteady heat conduction model of S1, a complete simulation of the steel pipe quenching process is performed to obtain the final temperature, as shown in the formula: ; in, A temperature evolution calculation model was constructed for the preceding steps; For the first The working condition state vector corresponding to each decision; S44, Final Temperature Based on S43 Substituting the reward function of S41 into the reward feedback, and then implicitly updating to the next state through the final temperature and related ambient and water temperature feedback, the state transition relationship formula is as follows: ; in, Update the mapping for the state; For the first The current state of this decision; The next state after the transition; Current state ,action ,award and the next state As basic training samples for reinforcement learning; S45. Based on the Markov decision process model, a deep Q-network is used to implement a reinforcement learning strategy. The action value function characterizes the long-term benefit of taking an adjustment action at a certain time under a given working condition. A greedy strategy selects actions to balance exploration and exploitation; the formula is: ; in, To explore probability; The state-action value function; S46. Construct the objective value for updating the value function based on the Bellman optimality principle. The calculation formula is as follows: ; in, Discount factor; For target network parameters; To reinforce the target values ​​constructed during the learning and training process; Candidate actions for the next state; For the next state All possible candidate actions Take the maximum value of the corresponding state-action value function; S47. An experience replay mechanism and a dual-network structure are introduced to improve training stability and convergence. The experience replay mechanism is used to mitigate the problem of sample correlation, and the dual-network structure is used to mitigate the problem of overestimation. The target network parameters are iteratively updated using a soft update method, and the calculation formula is as follows: ; in, To update the coefficients; Main network parameters; S48. Through continuous interactive learning between the agent and the quenching environment, the quenching time correction amount is dynamically adjusted using an iteratively optimized deep Q-network strategy to achieve online correction of the quenching time and obtain the final optimal quenching time for the steel pipe.

[0014] Therefore, the present invention employs the above-mentioned method for setting and implementing the optimal quenching time for steel pipes, which has the following beneficial effects: 1. A two-dimensional unsteady-state heat conduction model of the entire water quenching process of steel pipe is constructed based on heat conduction theory. Matching radiative heat transfer and third-type convective heat transfer boundary conditions are set for the four stages of air cooling, internal spray cooling, internal spray and external spray cooling, and second air cooling. This model can accurately reproduce the temperature transfer law of steel pipe quenching. Compared with the traditional single heat transfer model, it greatly improves the physical reality and theoretical reliability of temperature field simulation, and lays a solid mechanistic foundation for the accurate calculation of quenching time.

[0015] 2. The three-dimensional heat transfer of steel pipe is simplified to two-dimensional planar heat transfer. The heat conduction equation is solved discretely using the control volume energy balance method, and the objective function with spraying time as the core is optimized by Newton's iteration method. This not only reduces the complexity of numerical calculation and improves the solution efficiency, but also ensures the calculation accuracy of theoretical quenching time through a multi-condition iteration termination mechanism, thus achieving a dual improvement in efficiency and accuracy of numerical optimization of quenching time.

[0016] 3. By combining historical production data from the MES and quality traceability system, a deviation compensation model is constructed using a neural network. The theoretical quenching time and the deviation prediction value are integrated to obtain the initial optimal quenching time. While maintaining the physical consistency of the mechanism model, it effectively makes up for the deficiency of the pure mechanism model in being unable to characterize the implicit influencing factors of complex working conditions. This makes the quenching time prediction more in line with the actual production scenario and greatly improves the engineering adaptability of the solution.

[0017] 4. By introducing deep Q-network reinforcement learning to construct a Markov decision process framework, the online dynamic correction of the spray quenching time is realized based on the initial optimal quenching time. Through the action strategy of balancing exploration and utilization and the optimization mechanism of stable training, the influence of dynamic disturbances such as temperature fluctuation, water temperature change, and uneven spraying can be adaptively offset, forming a closed-loop optimization of "prediction-feedback-correction", which significantly improves the control accuracy of the final cooling temperature of quenching under dynamic working conditions and the robustness of the system.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] Figure 1 A flowchart of a method for setting and implementing the optimal quenching time for steel pipes provided by the present invention; Figure 2 A diagram illustrating the quenching process of a steel pipe, provided by the present invention, for a method of setting and implementing the optimal quenching time for steel pipes; Figure 3 The overall framework diagram of steel pipe quenching time prediction and optimization control for a method of setting and implementing optimal quenching time for steel pipes provided by the present invention; Figure 4 The flowchart shows the calculation process of the water quenching mechanism model for a method of setting and implementing the optimal quenching time for steel pipes provided by this invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0021] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0022] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0023] Current research on quenching control of seamless steel pipes covers areas such as heat transfer modeling, finite element simulation, experimental analysis, and data-driven methods, and has made some progress in process optimization. However, in actual production, quenching parameters are still generally set based on manual experience, lacking effective feedback and closed-loop adjustment mechanisms. At the same time, pure data-driven models lack physical constraints, have insufficient generalization and interpretability, and most control strategies adopt fixed mechanisms or static fitting methods, which are difficult to adapt to dynamic working condition disturbances such as initial temperature fluctuations, cooling water temperature changes, and uneven spraying. Overall, the intelligent and adaptive control capabilities are low, and it is impossible to guarantee accurate and stable control of quenching temperature under complex production scenarios.

[0024] Based on the above analysis, this invention is designed, see appendix. Figure 1-4 A method for setting and implementing the optimal quenching time for steel pipes includes the following steps: S1. Based on the heat conduction theory, a two-dimensional unsteady-state heat conduction model of the water quenching process of steel pipe is constructed and a staged heat transfer boundary condition is set to obtain the theoretical quenching time of steel pipe quenching. In S1, based on the theory of heat conduction, the constructed two-dimensional unsteady-state heat conduction equation is as follows: ; in, Density of the steel pipe; The specific heat capacity of the steel pipe; The thermal conductivity of the steel pipe; For temperature; For time; The axial direction of the steel pipe; In the direction of steel pipe wall thickness; The input parameters of the two-dimensional unsteady heat conduction model come from the MES system and the quality system. The input parameters include steel pipe card number, steel type, wall thickness, outer diameter, internal / external spray water flow rate and water temperature, steel temperature before quenching, and ambient temperature. The output of the numerical solution is the spraying time that meets the target final cooling temperature.

[0025] In S1, the specific phased heat transfer boundary conditions are as follows: For the four stages—air cooling stage 1, internal spray cooling stage 2, internal spray external shower cooling stage 3, and second air cooling stage 4—radiative heat transfer and third-type convective heat transfer boundary conditions are set respectively, where: In the air-cooling stage 1, the steel pipe is in a natural cooling state from the outlet of the heat treatment furnace to its entry into the quenching machine. The surface mainly exchanges heat with the environment through radiation and convection. That is, the steel pipe is not in contact with the cooling water, and the surface mainly dissipates heat to the environment through thermal radiation. The heat transfer through radiation is expressed by the formula: ; in, For radiative heat exchange; Surface emissivity; This represents the wall area of ​​the steel pipe. The value is the Stefan-Boltzmann constant. ; and The temperatures are, in order, the surface temperature of the steel pipe and the ambient temperature; Radiative heat flux density The calculation formula is: ; Radiative heat flux density As a boundary input to the outer surface of the steel pipe, it is used to drive the temperature field update; After the steel pipe enters the water quenching stage, it enters the quenching machine. Heat exchange on the steel pipe surface is dominated by water or air convection, represented by a third type of boundary condition. This includes an internal spray cooling stage 2 and an internal spray-external cooling stage 3. In the internal spray cooling stage 2, the internal spray device is triggered first, and only the inner surface is cooled by water, while the outer surface remains air-cooled. The formula is: ; ; in, The heat flux density on the inner surface of the steel pipe; The heat flux density on the outer surface of the steel pipe; The coefficient of thermal transfer; , These are the inner surface temperature and the outer surface temperature of the steel pipe, respectively. , The temperature of the medium (water) and the ambient temperature; In stage 3 of the internal spray and external spray cooling process, the external spray system is activated, and the inner and outer surfaces of the steel pipe are simultaneously subjected to water cooling, further enhancing the cooling intensity. The formula is: ; ; In the second air cooling stage 4, the steel pipe is in a natural cooling state again between leaving the quenching machine and the final temperature measurement point, that is, cooled by air.

[0026] In actual production, the cooling times for stages 1, 2, and 4 are usually set based on historical data and experience. For stage 3, given the outer diameter of the steel pipe, wall thickness, steel grade, initial temperature, ambient temperature, and water temperature, the formula for calculating the quenching time is: ; in, The theoretical quenching time is calculated based on the mechanism model. This indicates a systematic deviation in actual production caused by unmodeled factors; This is a predicted value for quenching time; This is the initial temperature.

[0027] Based on the internal spray and external spray cooling stage 3 in S1, a numerical solution strategy with spray time as the optimization variable is introduced, and the objective function is defined as: ; in, This refers to the spraying time during the internal spraying and external spraying stages; For the spraying time is The final temperature obtained from the simulation under the given conditions; The target final cooling temperature is set.

[0028] S2. Based on the two-dimensional unsteady heat conduction model of S1, the theoretical quenching time, which has been refined through numerical verification, is obtained by discretization and Newton iteration update using the finite difference method. The specific steps of S2 include: S21. Simplify the three-dimensional heat transfer of steel pipes into two-dimensional planar heat transfer, forming a regular two-dimensional grid; Specifically, the computational domain is divided into m and n nodes along the axial direction (x-direction) and wall thickness direction (y-direction) of the steel pipe, respectively, forming a regular two-dimensional mesh. Any node is denoted as m. Its temperature is expressed as The corresponding spatial step sizes are as follows: ; in, To calculate the length; For the steel pipe wall thickness; S22. Using the control volume energy balance method, the two-dimensional unsteady heat conduction equation is discretized, divided into straight boundary nodes, external corner nodes, and internal boundary nodes, and corresponding difference expressions are established for each. Specifically, the control volume energy balance method assumes that heat is conducted only through adjacent nodes or exchanged with the outside through the boundary, and that thermal properties remain constant within the element. For internal nodes, temperature updates are determined solely by the thermal conductivity term. However, for boundary nodes, convective heat transfer with the cooling medium must be considered. Based on their geometric location within the computational domain, boundary nodes can be categorized into three types: straight boundary nodes, external corner nodes, and internal boundary nodes, with specific limitations. Given the conditions, provide the difference expressions respectively; For nodes located on straight boundaries Its control volume has one side in contact with the cooling medium and the other three sides adjacent to the solid. Based on energy balance, the finite difference form of the energy balance equation for the straight boundary nodes can be obtained: ; in, for Spatial step size in direction; The area is The contact surface cooling coefficient, and , The surface heat transfer coefficient, The surface area of ​​the object in contact with the fluid; The thermal conductivity of the steel pipe; Temperature of the target node with a straight boundary; Temperature of the cooling medium; It reflects the balance between the heat a node gains from adjacent solids through conduction and the heat it loses through convection to the fluid medium; For a node located at an outer corner of the computational domain, it is adjacent to the solid in only two directions, with the other two sides exposed to the cooling medium. In this case, the finite difference form of the energy balance equation for the outer corner node is: ; For an internal boundary node located at the boundary but not geometrically a corner point, whose control volume is in contact with three solid elements and one fluid element simultaneously, the finite difference form of the energy balance equation for the internal boundary node is: ; S23. The objective function for the internal spray and external cooling stage is solved using the Newton-Raphson iteration method. The spray time update formula for the Newton-Raphson iteration is as follows: ; in, For numerical difference approximation, and , This represents the time increment for Newton's iterations; Each iteration requires re-execution of the complete internal spray and external cooling stage and the second air cooling stage to obtain the refined theoretical quenching time that meets the termination condition. .

[0029] The iteration termination condition in S23 includes: the temperature error meets the following condition. , Tolerance; reaching the maximum number of iterations; numerical derivative. If the value is too small, the update range will become invalid; the iteration will terminate once any condition is met.

[0030] Specifically, the optimization algorithm for steel pipe quenching time based on finite difference and Newton's method is shown in Table 1, as follows: Table 1 Optimization algorithm for quenching time of steel pipe

[0031] S3. Based on historical production data and the theoretical quenching time of S2, a neural network deviation correction model is constructed and the deviation prediction value is fused to obtain the initial optimal time for quenching steel pipes. The specific steps of S3 include: S31. Based on the historical records of the MES system and the quality traceability system, abnormal and missing samples are removed through data preprocessing, and continuous variables are standardized to obtain a feature vector containing the working condition status. ; S32. Based on eigenvectors and theoretical quenching time, a quenching time deviation term is introduced to characterize the systematic difference between the prediction results of the mechanism model and actual production. And define the deviation function formula as follows: ; in, The actual quenching time recorded in historical production records; The feature vector for the current operating condition includes the steel pipe's geometric dimensions, initial temperature, and cooling-related process parameters; the goal of deviation modeling is to establish a feature vector based on the operating condition parameters. Time deviation The mapping relationship is used to compensate for the complex influencing factors that are not explicitly characterized in the mechanism model; S33, Using an artificial neural network to calculate the deviation function of S32 Modeling is performed using a neural network with standardized operating condition feature vectors. As input, the corresponding predicted value of the quenching time deviation is output. It is used to characterize the systematic errors of the mechanism model under complex working conditions; S34, Based on theoretical quenching time Deviation Predicted Value Through fusion calculation, the initial optimal quenching time for the steel pipe is obtained, and the calculation formula is as follows: ; in, The initial optimal quenching time for steel pipes is determined by this fusion strategy. While maintaining the physical consistency of the mechanism model, the strategy introduces a data-driven model to correct the impact of complex working conditions, enabling the quenching time prediction results to better adapt to the dynamic changes in the actual production environment.

[0032] S4. Based on the initial optimal time for steel pipe quenching in S3, the online dynamic correction of the quenching time is achieved by constructing a reinforcement learning framework of Markov decision process and executing closed-loop interaction optimization of deep Q network, so as to obtain the final optimal time for steel pipe quenching. The specific steps of S4 include: S41. Based on S34, the initial optimal time for steel pipe quenching is determined by extracting observable process parameters during quenching. A state space is constructed that includes initial temperature, initial optimal time, ambient temperature, water temperature, and disturbance characteristics. The parameters for... The action space of the finite set of discrete actions to be added or subtracted, and the reward function with temperature deviation as its core. Thus, a Markov decision process model for the quenching process of steel pipes was obtained; S42. Based on the Markov decision process model, the initial optimal quenching time is determined... With the Action correction quantity for the next decision choice Combining, we obtain the first The actual execution time for the next reinforcement learning decision is calculated using the following formula: ; in, For the first The actual execution time corresponding to the reinforcement learning decision; Number the decision steps; For motion correction, and , It is a finite set of discrete actions used to describe the increase or decrease of the reference quenching time. This is the time adjustment amount, used to represent the difference between adjusting the initial optimal time and the actual execution time; S43, Based on actual quenching time Substituting the two-dimensional unsteady heat conduction model of S1, a complete simulation of the steel pipe quenching process is performed to obtain the final temperature, as shown in the formula: ; in, A temperature evolution calculation model was constructed for the preceding steps; For the first The working state vector corresponding to the next decision, and ; S44, Final Temperature Based on S43 Substituting the reward function of S41 into the reward feedback, and then implicitly updating to the next state through the final temperature and related ambient and water temperature feedback, the state transition relationship formula is as follows: ; in, Update the mapping for the state; For the first The current state of this decision; The next state after the transition; Current state ,action ,award and the next state As basic training samples for reinforcement learning; S45. Based on the Markov decision process model, a deep Q-network is used to implement a reinforcement learning strategy. The action value function characterizes the long-term benefit of taking an adjustment action at a certain time under a given working condition. A greedy strategy selects actions to balance exploration and exploitation; the formula is: ; in, To explore probability; The state-action value function; S46. Construct the objective value for updating the value function based on the Bellman optimality principle. The calculation formula is as follows: ; in, Discount factor; For target network parameters; To reinforce the target values ​​constructed during the learning and training process, which are used to guide the updating of network parameters; Candidate actions for the next state; For the next state All possible candidate actions Take the maximum value of the corresponding state-action value function; S47. An experience replay mechanism and a dual-network structure are introduced to improve training stability and convergence. The experience replay mechanism is used to mitigate the problem of sample correlation, and the dual-network structure is used to mitigate the problem of overestimation. The target network parameters are iteratively updated using a soft update method, and the calculation formula is as follows: ; in, To update the coefficients; Main network parameters; S48. Through continuous interactive learning between the agent and the quenching environment, the quenching time correction amount is dynamically adjusted using an iteratively optimized deep Q-network strategy to achieve online correction of the quenching time and obtain the final optimal quenching time for the steel pipe.

[0033] Specifically, the quenching time optimization algorithm based on deep reinforcement learning is shown in Table 2, as follows: Table 2. Optimization algorithm for quenching time

[0034] In summary, this invention constructs a full-process quenching time setting scheme that integrates heat conduction mechanism modeling, historical data deviation compensation, and online optimization using deep reinforcement learning. This scheme preserves the physical consistency of the mechanism model, compensates for the influence of unmodeled factors through data compensation, and utilizes reinforcement learning to achieve online dynamic closed-loop optimization of the quenching time. This effectively improves the control accuracy, stability, and adaptability of the final cooling temperature under complex working conditions, significantly reduces the quality risks caused by process fluctuations, and provides a reliable solution for intelligent, high-precision, and robust control of steel pipe quenching.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for setting and implementing the optimal quenching time for steel pipes, characterized in that: Includes the following steps: S1. Based on the heat conduction theory, a two-dimensional unsteady-state heat conduction model of the water quenching process of steel pipe is constructed and a staged heat transfer boundary condition is set to obtain the theoretical quenching time of steel pipe quenching. S2. Based on the two-dimensional unsteady heat conduction model of S1, the theoretical quenching time, which has been refined through numerical verification, is obtained by discretization and Newton iteration update using the finite difference method. S3. Based on historical production data and the theoretical quenching time of S2, a neural network deviation correction model is constructed and the deviation prediction value is fused to obtain the initial optimal time for quenching steel pipes. S4. Based on the initial optimal quenching time of steel pipe in S3, the online dynamic correction of the quenching time is achieved by constructing a reinforcement learning framework of Markov decision process and performing closed-loop interactive optimization of deep Q network, so as to obtain the final optimal quenching time of steel pipe.

2. The method for setting and implementing the optimal quenching time for steel pipes according to claim 1, characterized in that: In S1, based on the theory of heat conduction, the constructed two-dimensional unsteady-state heat conduction equation is as follows: ； in, Density of the steel pipe; The specific heat capacity of the steel pipe; The thermal conductivity of the steel pipe; For temperature; For time; The axial direction of the steel pipe; This refers to the direction of the steel pipe wall thickness.

3. The method for setting and implementing the optimal quenching time for steel pipes according to claim 2, characterized in that: The input parameters of the two-dimensional unsteady heat conduction model come from the MES system and the quality system. The input parameters include steel pipe card number, steel type, wall thickness, outer diameter, internal / external spray water flow rate and water temperature, steel temperature before quenching, and ambient temperature. The output of the numerical solution is the spraying time that meets the target final cooling temperature.

4. The method for setting and implementing the optimal quenching time for steel pipes according to claim 3, characterized in that: In S1, the specific phased heat transfer boundary conditions are as follows: For the four stages of air cooling, internal spray cooling, internal spray external shower cooling, and secondary air cooling, radiative heat transfer and third-type convective heat transfer boundary conditions are set respectively, where: During the air-cooling stage, the steel pipe is not in contact with the cooling water, and its surface mainly dissipates heat to the environment through thermal radiation. The heat transfer through radiation is expressed by the formula: ； in, For radiative heat exchange; Surface emissivity; This represents the wall area of ​​the steel pipe. It is the Stefan-Boltzmann constant; and The temperatures are, in order, the surface temperature of the steel pipe and the ambient temperature; Radiative heat flux density The calculation formula is: ； When the steel pipe enters the water quenching stage, surface heat transfer is dominated by water or air convection, represented by the third type of boundary condition, and includes both internal spray cooling and internal spray-external spray cooling stages. In the internal spray cooling stage, only the inner surface of the steel pipe is in contact with water, while the outer surface remains cooled by air. The formula is: ； ； in, The heat flux density on the inner surface of the steel pipe; The heat flux density on the outer surface of the steel pipe; The coefficient of thermal transfer; , These are the inner surface temperature and the outer surface temperature of the steel pipe, respectively. , The temperature of the medium (water) and the ambient temperature; During the internal spray and external cooling stage, the inner and outer surfaces of the steel pipe are simultaneously cooled by water, as shown in the formula: ； ； In the second air-cooling stage, cooling is achieved through air cooling.

5. The method for setting and implementing the optimal quenching time for steel pipes according to claim 4, characterized in that: Based on the internal spray and external spray cooling stages in S1, a numerical solution strategy with spray time as the optimization variable is introduced, and the objective function is defined as: ； in, This refers to the spraying time during the internal spraying and external spraying stages; For the spraying time is The final temperature obtained from the simulation under the given conditions; The target final cooling temperature is set.

6. The method for setting and implementing the optimal quenching time for steel pipes according to claim 5, characterized in that: The specific steps of S2 include: S21. Simplify the three-dimensional heat transfer of steel pipes into two-dimensional planar heat transfer, forming a regular two-dimensional grid; S22. Using the control volume energy balance method, the two-dimensional unsteady heat conduction equation is discretized, divided into straight boundary nodes, external corner nodes, and internal boundary nodes, and corresponding difference expressions are established for each. S23. The objective function for the internal spray and external cooling stage is solved using the Newton-Raphson iteration method. The spray time update formula for the Newton-Raphson iteration is as follows: ； in, For numerical difference approximation, and , This represents the time increment for Newton's iterations; Each iteration requires re-execution of the complete internal spray and external cooling stage and the second air cooling stage to obtain the refined theoretical quenching time that meets the termination condition. .

7. The method for setting and implementing the optimal quenching time for steel pipes according to claim 6, characterized in that: The iteration termination condition in S23 includes: the temperature error meets the following condition. , Tolerance; reaching the maximum number of iterations; numerical derivative. If the value is too small, the update range will become invalid; the iteration will terminate once any condition is met.

8. The method for setting and implementing the optimal quenching time for steel pipes according to claim 7, characterized in that: The specific steps of S3 include: S31. Based on the historical records of the MES system and the quality traceability system, abnormal and missing samples are removed through data preprocessing, and continuous variables are standardized to obtain a feature vector containing the working condition status. ; S32. Based on eigenvectors and theoretical quenching time, a quenching time deviation term is introduced to characterize the systematic difference between the prediction results of the mechanism model and actual production. And define the deviation function formula as follows: ； in, The actual quenching time recorded in historical production records; This is the feature vector of the current operating condition. S33, Using an artificial neural network to calculate the deviation function of S32 Modeling is performed using a neural network with standardized operating condition feature vectors. As input, the corresponding predicted value of the quenching time deviation is output. It is used to characterize the systematic errors of the mechanism model under complex working conditions; S34, Based on theoretical quenching time Deviation Predicted Value Through fusion calculation, the initial optimal quenching time for the steel pipe is obtained, and the calculation formula is as follows: ； in, The optimal quenching time for the initial steel pipe.

9. The method for setting and implementing the optimal quenching time for steel pipes according to claim 8, characterized in that: The specific steps of S4 include: S41. Based on S34, the initial optimal time for steel pipe quenching is determined by extracting observable process parameters during quenching. A state space is constructed that includes initial temperature, initial optimal time, ambient temperature, water temperature, and disturbance characteristics. The parameters for... The action space of the finite set of discrete actions to be added or subtracted, and the reward function with temperature deviation as its core. Thus, a Markov decision process model for the quenching process of steel pipes was obtained; S42. Based on the Markov decision process model, the initial optimal quenching time is determined... With the Action correction quantity for the next decision choice Combining, we obtain the first The actual execution time for the next reinforcement learning decision is calculated using the following formula: ； in, For the first The actual execution time corresponding to the reinforcement learning decision; Number the decision steps; For motion correction, and , It is a finite set of discrete actions used to describe the increase or decrease of the reference quenching time. Adjustment amount for time; S43, Based on actual quenching time Substituting the values ​​into a two-dimensional unsteady-state heat conduction model, a complete simulation of the steel pipe quenching process is performed to obtain the final temperature, as shown in the formula: ； in, A temperature evolution calculation model was constructed for the preceding steps; For the first The working condition state vector corresponding to each decision; S44, Final Temperature Based on S43 Substituting the reward function of S41 into the reward feedback, and then implicitly updating to the next state through the final temperature and related ambient and water temperature feedback, the state transition relationship formula is as follows: ； in, Update the mapping for the state; For the first The current state of this decision; The next state after the transition; Current state ,action ,award and the next state As basic training samples for reinforcement learning; S45. Based on the Markov decision process model, a deep Q-network is used to implement a reinforcement learning strategy. The action value function characterizes the long-term benefit of taking an adjustment action at a certain time under a given working condition. A greedy strategy selects actions to balance exploration and exploitation; the formula is: ； in, To explore probability; The state-action value function; S46. Construct the objective value for updating the value function based on the Bellman optimality principle. The calculation formula is as follows: ； in, Discount factor; For target network parameters; To reinforce the target values ​​constructed during the learning and training process; Candidate actions for the next state; For the next state All possible candidate actions Take the maximum value of the corresponding state-action value function; S47. An experience replay mechanism and a dual-network structure are introduced to improve training stability and convergence. The experience replay mechanism is used to mitigate the problem of sample correlation, and the dual-network structure is used to mitigate the problem of overestimation. The target network parameters are iteratively updated using a soft update method, and the calculation formula is as follows: ； in, To update the coefficients; Main network parameters; S48. Through continuous interactive learning between the agent and the quenching environment, the quenching time correction amount is dynamically adjusted using an iteratively optimized deep Q-network strategy to achieve online correction of the quenching time and obtain the final optimal quenching time for the steel pipe.

Images