Continuous casting and rolling intelligent blank cutting and sizing method based on deep reinforcement learning
By using a deep reinforcement learning-based approach, the continuous casting and rolling process is deconstructed, a process mechanism model is established, and billet cutting length decisions are optimized. This solves the problem of lack of dynamic response in the continuous casting and rolling process of traditional methods, achieves efficient billet cutting control, and improves the length cutting rate and production efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-22
- Publication Date
- 2026-04-07
AI Technical Summary
Traditional manual experience-driven judgment of billet weight and cutting decisions lack the ability to quantitatively perceive, dynamically respond to and match high-precision production disturbances from multiple sources in the continuous casting and rolling process. This makes it difficult to achieve refined control of the integrated continuous casting and rolling process. Existing model-driven methods have poor migration and deployment capabilities and lack online disturbance perception and adaptive adjustment.
Based on deep reinforcement learning, this method deconstructs the continuous casting and rolling process, establishes a process mechanism model for full-length cutting, constructs a Markov decision process, optimizes billet cutting length decision using the PPO algorithm, and designs a reward function by combining mass conservation and thermal expansion effects to achieve precise control of billet cutting length.
The fixed-size rate was increased to 99.994%, which is 0.5% higher than the existing process, saving production costs. The strategy converges quickly under complex perturbations, with excellent generalization and robustness. The decrease in fixed-size rate did not exceed 0.05%, avoiding strategy instability.
Smart Images
Figure CN121797754A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of continuous casting and rolling technology, and more specifically, relates to an intelligent billet cutting and length setting method for continuous casting and rolling based on deep reinforcement learning. Background Technology
[0002] Currently, global annual steel production is approaching 2 billion tons, with approximately 97% of the product completed through the two key processes of continuous casting and continuous rolling. As core links in the steel manufacturing process, continuous casting and continuous rolling not only constitute the basic chain for product forming and performance evolution, but also directly determine the dimensional accuracy, surface quality, and material utilization rate of the final steel product, having a decisive impact on product quality and resource efficiency. However, for a long time, continuous casting units have typically used fixed-length shearing to segment the cast billets before transferring them to the rolling mill, while continuous rolling has implemented production planning and material allocation based on a fixed-weight strategy. This typical segmented production organization structure leads to a fragmentation of process information and control strategies, making it difficult to achieve full-process optimization for end-product dimensional accuracy and resource utilization.
[0003] Against this backdrop, the integrated continuous casting and rolling process has emerged, making sizing control possible throughout the entire process from billet formation to finished product rolling. Precise control of billet cutting length is crucial for ensuring both yield and sizing success rate. However, traditional manual experience-driven billet weight determination and cutting decisions lack real-time quantitative perception, dynamic response, and high-precision matching capabilities for multi-source coupled production disturbances during continuous casting and rolling. Furthermore, they are susceptible to the lag in manual decision-making and individual differences, making it difficult to adapt to the refined control requirements of the integrated process for cutting length. Consequently, some researchers have proposed model-driven and data-driven sizing control methods, but their deployment capabilities are limited by the massive data training requirements of single neural network models. Moreover, these methods focus on weight cutting during the continuous casting stage, lacking the ability to perceive and adaptively adjust to online disturbances. This makes it difficult to achieve optimal control in the dynamic and complex continuous casting and rolling process, and also fails to fully leverage the potential of dynamic feedback control and adaptive optimization in the integrated continuous casting and rolling process.
[0004] In summary, solutions to this problem are already limited, and control strategies with high efficiency and dynamic response capabilities are even scarcer. Therefore, a more intelligent and adaptive method for intelligent billet cutting and sizing based on deep reinforcement learning is proposed. Summary of the Invention
[0005] This invention provides an intelligent billet cutting and length determination method based on deep reinforcement learning for continuous casting and rolling, which aims to solve the problem that relying on traditional manual experience decision-making or single model control has limitations in the case of multi-source coupled disturbances during the integrated production process of continuous casting and rolling.
[0006] To address the aforementioned issues, this invention first deconstructs the actual production line of continuous casting and rolling, establishes a process mechanism model of continuous casting and rolling based on full fixed length, further refines and optimizes the objective function based on the comprehensive model, and then models the task decision problem as a Markov decision process based on the optimized objective function. Finally, the PPO algorithm is used to solve the optimization problem and obtain the optimal fixed length billet cutting decision.
[0007] The above method targets the continuous casting and rolling process as follows: First, molten steel solidifies into a continuous casting billet in the continuous casting stage, and is then cut to length to form a steel billet; subsequently, it is conveyed by roller conveyor to a heating furnace for heat preservation and heating, and after high-pressure water descaling, the steel billet enters the roughing, intermediate rolling and finishing rolling stages in sequence, and the ends of the rolled piece are sheared and adjusted by flying shear, while the rolling temperature is controlled by a water cooling system; finally, the rolled multiple-length pieces are conveyed to a cooling bed for cooling after being adjusted by flying shear and multiple-length shear, and then the finished products are obtained by length shearing, and then bundled and stored.
[0008] Specifically, the technical solution adopted in this invention is as follows: Step 1: Deconstruct the actual continuous casting and rolling production line and divide the overall process into four stages: finished product length → multiple length, multiple length → post-rolling quality, post-rolling quality → pre-rolling quality, pre-rolling quality → billet cutting to length. Step Two: Using quality conservation as the main principle, and combining the four stages of the actual continuous casting and rolling production line from Step One, reverse modeling is performed based on the finished product target orientation to construct a process mechanism model. The modeling process for the above four stages is detailed below: The modeling process for the finished bar length to multiple length stage is as follows: the process target is usually specified by the finished bar requirements, i.e., the final bar length. L f Because the production process typically employs a "multiple-length shearing" method, which involves cutting multiple finished products to length from a single, longer bar (multiple length), the final length of the bar depends on the length of the finished product. L f Reverse calculation of multiple length L b In the derivation process, it is also necessary to consider the thermal expansion effect caused by the high temperature of the steel at the finish rolling exit, and calculate the length increase Δ of each section of finished product due to thermal expansion. L e The resulting multiple-scale length L b The expression is as follows:
[0009]
[0010] in, N f The number of finished products of a fixed length. Lc is the length of the cooling bed, Δ L e max is the maximum length increment generated by the thermal expansion of each section of finished products. To conform to the actual process equipment, L b the length should be less than the length of the cooling bed L c so that it can completely enter it for cooling.
[0011] As a further improvement of the present invention, Δ L e The calculation formula is as follows:
[0012] E ( T ) is the thermal expansion function, which is based on the actual temperature of the steel after finish rolling (usually 850 °C < T < 1100 °C), and conducts piecewise modeling and characterization of the specific thermal expansion coefficient in each temperature range, and the calculation is as follows:
[0013] The temperature range activation function is calculated as follows:
[0014] When , the value is 1, otherwise the value is 0.
[0015]
[0016] Among them, k i is the thermal expansion coefficient in the corresponding range, T i1 , T i2 represents the boundary of each temperature range from the temperature after rolling to room temperature. The specific calculation principle can be simply described as: the activation function judges the temperature T in the range, and at the same time T - T i1 calculates the temperature offset of the current temperature T in the range, then accumulates the temperature spans of all previous ranges, then adapts the corresponding thermal expansion coefficient for each temperature range.
[0017] The modeling process of the multiple-length → post-rolling quality process stage: When cut into N b multiple lengths by the multiple-length shear, the head and tail lengths cut off after finish rolling are supplementedL After step 1, the calculated total length of the rolled piece after rough rolling can be obtained. L t The calculation is as follows:
[0018]
[0019] Combined with steel density With the cross-sectional area of steel S f Calculate the quality after rolling M a The formula is as follows:
[0020] In actual rolling processes, numerous uncertainties, such as mill pressure fluctuations, roll pass wear, and oxide scale formation, collectively affect the final product. S f Such influence relationships are usually difficult to express quantitatively, but this method cleverly uses the tolerance influence function. W ( x ) Perform empirical fitting (where x The total steel throughput X of the continuous rolling mill was used as the core variable for integrating multi-source data from actual production, and statistical methods were employed to complete the calculation. W ( x (functionalized representation), ultimately acting on S f The calculation is as follows:
[0021]
[0022] D f The standard diameter of the finished product cross-section. D f max The maximum allowable tolerance in the production standard is used to limit it. W ( x The parameters that meet the production standards are fitted.
[0023] The modeling process for the post-rolling quality → pre-rolling quality stage: Given the post-rolling quality... M a Based on this, the pre-rolling quality can be obtained by supplementing two aspects of loss. One aspect considers the head and tail trimming operation during the roughing stage to ensure smooth rolling, with a shear loss ratio of... c On the other hand, it compensates for the oxidation loss during the heating and heat preservation process of the heating furnace. O ( Lp The overall calculation is as follows:
[0024]
[0025] Among them, the oxidation loss takes into account the holding and heating time. t Furnace temperature T d The length of the continuously cast billet is calculated using classical thermodynamics formulas as follows:
[0026]
[0027] The units for length calculations are all in mm, and the constants α and β are obtained based on empirical experimental fitting. The furnace exit temperature of this production line is typically 970℃. T d <1100℃.
[0028] The modeling process for the pre-rolling quality → billet cutting and sizing stage is as follows: Pre-rolling quality is obtained. M b That is, after the continuous casting billet is cut, the billet should weigh as M b According to the principle of conservation of mass, the final cut length of the continuously cast billet is: L p The calculation is as follows:
[0029] in, Density of steel; S p The cross-sectional area of the continuously cast billet is given, and all values are known from actual production. This expression ultimately transfers the control target for the shearing length from the rolling stage to the continuous casting stage.
[0030] Step 3: Based on the aforementioned optimization objective function, the billet cutting length decision problem is transformed into a Markov decision process; Based on the rolling process model and thermal expansion disturbance mechanism proposed in this invention, the optimization objective function is extracted from step two, and the trimming optimization problem is further formalized and defined as follows:
[0031]
[0032] in, N c This refers to the maximum number of shearing steps allowed within one tapping cycle of the tundish in a continuous casting machine. L pTo determine the cutting length of continuously cast billets; S p This represents the cross-sectional area of the continuously cast billet. c This refers to the shear loss ratio; L 1 represents the length of the head and tail shearing in the precision rolling process. S f The cross-sectional area of the steel is... N b The number of multiples of a ruler. N f The number of finished products of a fixed length. L f For the fixed length of the terminal bar, Δ L e This refers to the length increase caused by thermal expansion of each section of timber.
[0033] To further explain, This indicates that for each trajectory from n =0 to the termination step n = N c The time steps are accumulated progressively, and their physical meaning corresponds to the maximum number of shearing steps allowed in one tapping cycle of the continuous casting machine, that is, the maximum length of the billet sequence that can be generated in one continuous casting cycle. Meanwhile, the above-mentioned optimization objective function measures the initial billet cutting length under the influence of disturbances. L p Starting from, passing through cross-sectional disturbance S f Thermal expansion Δ L e and burn-off loss O ( L p After that, the matching error between the actual total length of the multiple-length rolling and the fixed-length requirement. This error term reflects the final performance of the strategy's shearing capabilities and is a direct indicator of the strategy's quality.
[0034] In step three, to facilitate the introduction of deep reinforcement learning algorithms to solve the above optimization objective function, the decision problem is further formalized into a Markov decision process, specifically modeled as follows: State space: Environment state vector It encompasses five key physical and technological dimensions, comprehensively depicting the critical state information during the billet cutting process. Among them, for t The actual length of the steel billet to be cut in the continuous casting process directly determines the cutting step length and the multiple length reconstruction strategy, which is the core of the cutting decision control. for t The billet exit temperature during constant heat preservation significantly affects the degree of oxidation loss and thermal disturbance during subsequent rolling, and is an important thermal parameter affecting the dimensional accuracy of the finished product. for t The cross-sectional area after precision rolling is greatly affected by fluctuations in the continuous rolling process. It is a direct representation of the dynamic changes in the rolling process and determines the final cross-sectional dimensions of the finished product. for t The temperature of the double-length shearing after precision rolling reflects the hot processing characteristics of the material and is related to the final dimensions of the finished product after thermal shrinkage. for t The normalized index of the cumulative steel throughput of the entire rolling mill at any given time is used to reflect the load change trend of the equipment during long-term operation and plays an important role in simulating process degradation and equipment response deviation. The specific description is as follows:
[0035] Action space: The action space is defined as the set of discrete integers. This indicates a gradual adjustment of the blank cutting length. Considering the mechanical response characteristics of the flame cutting machine and the allowable control precision in actual operation, the motion space is designed in a parametric form, as shown below:
[0036] in, This indicates the specific action value. This indicates the maximum allowable adjustment range (usually set to 1000 mm). This represents the distance from the walk (usually set to 10 mm). The formula generates a symmetric set of actions centered at zero, containing... Individual discrete actions. This design ensures full coverage of the adjustment range common in industrial cutting operations, while maintaining a balance between control resolution and mechanical responsiveness.
[0037] Reward Design: The reward function not only constitutes the core driving mechanism of policy learning but also essentially determines the target direction of agent behavior optimization. Since the original task optimization function involves non-smooth terms and complex physical variables, it is not suitable for gradient updates directly used in deep reinforcement learning. To adapt to the numerical stability and differentiability of the policy optimization process, a normalized single-step negative reward function is further introduced during the training phase, as shown below:
[0038] in, This represents the length of the remainder resulting from the current clipping decision. L b It is twice the length of a foot. L f The final bar length is set. This approach guides the strategy towards minimizing shearing error by explicitly penalizing non-integer multiple shearing behavior. While ensuring convergence, it also effectively enhances the strategy's generalization ability and industrial applicability.
[0039] Step 4: Use the PPO algorithm to solve the objective function and obtain the billet cutting and length fixing strategy. The specific operation process of using the PPO algorithm is as follows: 1) Input the state information obtained from the interaction between each time slot and the environment into the new billet cutting and length setting strategy network of the PPO algorithm. With the old blank cutting and sizing network middle; 2) During each training cycle of the PPO algorithm, the current new billet cutting length strategy network is used. Interact with the environment to collect samples; 3) During each training cycle of the PPO algorithm, the billet cutting and length setting policy network is updated using the policy gradient algorithm. parameters and utilize The sampled data for the old edge computing and caching strategy network Conduct training, training The objective function for the network is as follows:
[0040] in, , which represents the ratio difference between the new billet sizing decision strategy and the old billet sizing decision strategy, is a hyperparameter used to control the range of the difference; A The dominant function is the state. s Middle Action a The value of the dominant function A The calculation combines the reward of the current time step. r t and the next state s τ+1 It is accomplished using the state value; This is the clipping value specified in the PPO algorithm, used for clipping based on the difference "ratio" between comparison values; For the action at time τ, The state at time τ; 4) After training, based on the new billet cutting and length setting strategy network after training. It provides an optimal task decision-making process based on the state information obtained from the environment.
[0041] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention applies a deep enhancement algorithm to the field of continuous casting and rolling billet cutting. By deconstructing the actual continuous casting and rolling production line, reverse modeling is performed based on the finished product target orientation to construct a process mechanism model. The constructed model considers the realization of precise control of billet cutting length, overcoming the problems of waste of labor, low degree of automation and intelligence, waste of finished products, low efficiency, and low fixed length rate in the existing process.
[0042] 2. Using the method of this invention, the optimal length-to-length ratio is as high as 99.994% (the length-to-length ratio is calculated by taking the total length of the final rolled piece as the benchmark, and then calculating the ratio of 1 - the residual value of the final length-to-length shearing to the total length of the final rolled piece). Compared with the existing process, this is up to 0.5% higher, which is equivalent to saving 200,000 yuan per 10,000 tons of steel produced.
[0043] 3. Under varying environments and complex disturbance conditions, the method of the present invention can quickly converge the strategy, and the output remains within an acceptable tolerance range. The decrease in the fixed-size rate does not exceed 0.05%, and no abnormal behavior such as strategy instability occurs. It has excellent generalization and robustness. Attached Figure Description
[0044] Figure 1 This is an overall flowchart of an intelligent billet cutting and length determination method for continuous casting and rolling based on deep reinforcement learning, according to the present invention. Figure 2 This is a schematic diagram of the process flow in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the PPO parameterization target strategy in Embodiment 1 of the present invention. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0046] Example 1 Reference Figure 2 This example demonstrates a single-machine, single-strand bar continuous casting and rolling production line of this embodiment. The maximum ladle output of this line is 110 tons. It uses a 160×160mm small square billet continuous casting machine, with an initial billet length of 12 meters. The maximum single-bill cutting quantity is 45 pieces, and the finished product length is... L f =12m, length of cooling bed L c =130m, flame cutting machine working range ±1m, combined Figure 1, the steps to achieve intelligent billet cutting and sizing in continuous casting and rolling using the method of the present invention are as follows: Step 1: Deconstruct the actual production line, divide it into four major process stages, and perform recursive modeling.
[0047] Stage 1: Finished product sizing → multiple-length stage: The process target is usually specified by the finished product requirements, that is, the fixed length of the terminal bar L f = 12m. First, it is necessary to reverse-derive the multiple-length from the final sizing L b , and this derivation needs to simultaneously consider the thermal expansion effect caused by the high-temperature state of the steel at the exit of the finishing rolling. The calculation is as follows:
[0048]
[0049] Among them, Δ L e is the length increment generated by thermal expansion for each section of the finished product. The calculation is as follows:
[0050] E ( T ) is the thermal expansion function, which is based on the actual temperature T of the steel after finishing rolling (850°C < T < 1100°C for this production line), and performs piecewise modeling and characterization of the specific thermal expansion coefficient for each temperature range. The calculation is as follows:
[0051] The temperature range activation function among them The calculation is as follows:
[0052]
[0053] Among them, k i is the thermal expansion coefficient within the corresponding range, T i1 , T i2 represents the boundary of each temperature range from the temperature after rolling to room temperature. Based on the above formula, taking the temperature of the rolled piece as 800°C in a certain period as an example, it is calculated that = (718 - 25) + (760 - 718) + (800 - 760) = 0.0105, Δ Le =12 0.0105 = 0.126 N f (12000 + 0.126) is obtained L b , N f This is the integer number of cuts to the final length (specifically set to 10 in this embodiment).
[0054] Phase Two, Multiple Length → Post-Rolling Quality Stage: When the material is cut to multiple length... N b The length is calculated as a multiple of 10 (in actual production, this is set to 10), plus the head and tail lengths cut off after finishing rolling. L 1 (In actual production) L After setting 1 to 300mm, the total length of the rolled piece after rough rolling can be calculated. L t The calculation formula is as follows: ,
[0055] Calculations show that, L t =10×(120001.26+300), combined with the density of steel (Value is 7850kg / m) 3 ) and cross-sectional area S f That is, to calculate the quality after rolling. M a The calculation formula is as follows:
[0056] Among them, the cross-sectional area of the rolled piece S f for:
[0057]
[0058] generally, W ( x The fitted value tends to [1, 1.025], and 1.025 can be taken here. D f The standard diameter of the finished cross-section is 20mm in this embodiment, calculated using the formula above. M a =7.85×10 -6 ×1203012.6×0.25π×(20×1.025) 2 =3117.0045502.
[0059] Phase 3: Post-rolling quality → Pre-rolling quality stage: Post-rolling quality M a =3117.0045502. Based on this, the pre-rolling mass can be obtained by considering the following two aspects of loss compensation. M b :
[0060] On the one hand, considering the head and tail trimming operation in the roughing stage to ensure smooth rolling, the shear loss ratio is: c ,0< c < c max Based on the actual production situation in this embodiment, c =3.4%; On the other hand, it compensates for the oxidation loss during the heating and heat preservation process of the heating furnace. O ( L p The oxidation loss should take into account the holding and heating time. t (In this embodiment) t =0.58min), furnace exit temperature T d Cutting billets to fixed lengths with continuously cast billets L p The fitting calculation is as follows:
[0061]
[0062] The unit of measurement for length is mm, with constants α=6.3 and β=9000. The furnace exit temperature of this production line is 970℃. T d <1100℃, specifically taken in this embodiment T d =1000℃, substituting into the above formula, we get... O ( L p ) = 0.0034, M b =3226.7162712.
[0063] Phase 4, Pre-rolling Quality → Billet Cutting and Length Setting Stage: Obtaining Pre-rolling Quality M b That is, after the continuous casting billet is cut, the billet should weigh as M b According to the principle of conservation of mass, the final cut length of the continuously cast billet is: L pThe calculation is as follows:
[0064] Ultimately obtain L p The length should be 16056.5mm.
[0065] This concludes the demonstration of the entire computational workflow for the simulation environment based on physical process mechanisms. Based on the simulation environment, you can now begin interacting with the environment.
[0066] First, construct the state space, and the state space vector. It includes five key physical and technological dimensions, comprehensively depicting the critical state information during the billet cutting process, as shown in the following formula:
[0067] in, for t The actual length of the steel billet currently to be cut in continuous casting; for t The billet exit temperature during constant heating and heat preservation; for t The cross-sectional area of multiple dimensions after precision rolling; for t The shearing temperature of the multiple length after precision rolling; for t The normalized index of the total cumulative steel output of the entire rolling mill at any given time.
[0068] Secondly, the reward function is used to calculate the length of the finished part under the current fixed-length cutting strategy. L f and multiple length of piece L b The modulus is calculated as follows:
[0069] Input the above state values and reward values into the PPO algorithm. The PPO algorithm's processing flow is as follows: Figure 3 As shown, this is a parameterized objective policy for PPO, using a neural network to fit the policy function. PPO updates the neural network parameters by sampling historical data and updating them according to the state. PPO uses two neural networks: an Actor network and a Critic network. The Actor network outputs the probability of each action in the state, while the Critic network evaluates the expected value of each action. If an action has a greater long-term value, its corresponding probability is also greater. The Actor network contains the new billet cutting and length setting policy network. parameters Used for parameterizing the policy function. During each training cycle, the current new billet cutting length policy network is used. The system interacts with the environment to collect samples. The PPO uses a policy gradient update algorithm to update the new billet cutting and sizing policy network. Network parameters To improve sample utilization and algorithm efficiency, it is necessary to utilize a new blank cutting length-fixing network. The sampling data is used for the old blank cutting length strategy network. Training is conducted to train the old billet cutting and length-setting strategy network. The objective function at that time is as follows:
[0070] in, This represents the ratio difference between the new billet cutting strategy and the old billet cutting strategy. It is a hyperparameter used to control the range of the difference. The dominant function represents the state. Middle Action The value of the dominant function The calculation combines the reward of the current time step. and the next state It is accomplished using the state value; It is the clipping value specified in the PPO algorithm, used to compare differences between values. Cut it.
[0071] Therefore, the new blank cutting length strategy after parameter update outputs new actions. a Perform fixed-length cutting.
[0072] By repeating the above steps, the fixed-length blank cutting strategy is updated repeatedly, thus constructing the system's strategy decision-making process and finally outputting the optimal fixed-length blank cutting length.
[0073] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the figures shown are only one embodiment of the present invention; the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present invention, such designs should fall within the protection scope of the present invention.
Claims
1. A method for intelligent billet cutting and length determination based on deep reinforcement learning in continuous casting and rolling, characterized in that: Includes the following steps: Step 1: Deconstruct the actual continuous casting and rolling production line and divide the overall process into four stages: finished product length → multiple length, multiple length → post-rolling quality, post-rolling quality → pre-rolling quality, pre-rolling quality → billet cutting to length. Step 2: Taking mass conservation as the main line, and combining the four stages of the actual continuous casting and rolling production line in Step 1, reverse modeling is carried out based on the finished product target orientation to construct a process mechanism model; considering thermal expansion disturbance, the model optimization objective function is refined. The optimization objective function is used to measure the matching error between the total length of multiples obtained by actual rolling and the fixed length requirement under the influence of disturbance. Step 3: Based on the aforementioned optimization objective function, the billet cutting length decision problem is transformed into a Markov decision process; Step 4: Use the PPO algorithm to solve the optimization objective function and obtain the billet cutting length strategy.
2. The intelligent billet cutting method based on deep reinforcement learning for continuous casting and rolling as described in claim 1, characterized in that, In step two, the modeling process for the finished material length to multiple length process includes: 1) Based on the fixed length of the terminal bar L f Reverse calculation of multiple length L b ; 2) Considering the thermal expansion effect caused by the high temperature of the steel at the finish rolling exit, calculate the length increase Δ of each section of finished steel due to thermal expansion. L e ; 3) Obtain the length of the multiple scale. L b The expression is as follows: in, N f The number of finished products of a fixed length. L c Δ is the length of the cooling bed. L e max This represents the maximum increase in length caused by the thermal expansion of each section of the timber.
3. The intelligent billet cutting method based on deep reinforcement learning for continuous casting and rolling as described in claim 1, characterized in that, Step two, the modeling process for the multiple-length → post-rolling quality process stage includes: 1) Considering the lengths of the head and tail cut off during finishing rolling, calculate the total length of the workpiece after rough rolling. L t ; 2) Calculate the post-rolling quality by combining the steel density and cross-sectional area. M a The formula is as follows: in, Density of steel; S f This represents the cross-sectional area of the steel.
4. The intelligent billet cutting method based on deep reinforcement learning for continuous casting and rolling as described in claim 3, characterized in that, When calculating the cross-sectional area of steel, a tolerance influence function is introduced. W ( x The cross-sectional area of the steel mentioned above. S f The calculation formula is as follows: D f The standard diameter of the finished product cross-section. D f max This represents the maximum tolerance allowed in the production standard.
5. The intelligent billet cutting method based on deep reinforcement learning for continuous casting and rolling as described in claim 3, characterized in that, In step two, the modeling process for the post-rolling quality → pre-rolling quality stage is as follows: Based on the post-rolling quality calculated in the previous stage, considering shear loss and oxidation loss during the heat preservation and heating process, the pre-rolling quality is calculated. M b The calculation formula is as follows: c This is the shear loss ratio. L p For the cut length of continuously cast billets, α and β are constants. T d This refers to the temperature at which the food exits the furnace. t For heat preservation and heating time, S p This represents the cross-sectional area of the continuously cast billet. l 1. l 2 represents the relevant length parameters of the continuously cast billet, with the unit of length being mm.
6. The intelligent billet cutting method based on deep reinforcement learning for continuous casting and rolling as described in claim 5, characterized in that, In step two, the modeling process for the pre-rolling quality → billet cutting and sizing stage includes: based on the pre-rolling quality calculated in the previous stage, and according to the principle of mass conservation, calculating the final continuous casting billet cutting and sizing length. L p The calculation is as follows: in, Density of steel; S p This represents the cross-sectional area of the continuously cast billet.
7. The intelligent billet cutting method based on deep reinforcement learning for continuous casting and rolling as described in claim 6, characterized in that, In step two, the extracted optimization objective function is as follows: in, N c This refers to the maximum number of shearing steps allowed within one tapping cycle of the tundish in a continuous casting machine. L p To determine the cutting length of continuously cast billets; S p This represents the cross-sectional area of the continuously cast billet. c This refers to the shear loss ratio; L 1 represents the shearing length at the beginning and end of the finishing roll. S f The cross-sectional area of the steel is... N b The number of multiples of a ruler. N f The number of finished products of a fixed length. L f For the fixed length of the terminal bar, Δ L e This refers to the length increase caused by thermal expansion of each section of timber.
8. The intelligent billet cutting method based on deep reinforcement learning for continuous casting and rolling as described in claim 6, characterized in that, Step three involves the following specific steps: 1) Created state space S t The formula is as follows: in, for t The actual length of the steel billet currently to be sheared in the continuous casting process. for t The billet exit temperature during constant heating and heat preservation; for t The cross-sectional area of multiple dimensions after precision rolling; for t Temperature of double-length shearing after precision rolling; for t Normalized index of the total cumulative steel throughput of the entire rolling mill at any given moment; 2) Action Space Recording A t The formula is as follows: in, a This indicates the specific action value. L max Indicates the maximum allowable adjustment range, Δ L For a long walk; 3) Set a single-step negative reward function r t The formula is as follows: in, This represents the length of the remainder resulting from the current clipping decision. L b It is twice the length of a foot. L f Set the length of the terminal bar.
9. A method for intelligent billet cutting and length determination based on deep reinforcement learning in continuous casting and rolling according to any one of claims 1-8, characterized in that, In step four, the specific operation process using the PPO algorithm is as follows: 1) Input the state information obtained from the interaction between each time slot and the environment into the new billet cutting and length setting strategy network of the PPO algorithm. With the old blank cutting and sizing network middle; 2) During each training cycle of the PPO algorithm, the current new billet cutting length strategy network is used. Interact with the environment to collect samples; 3) During each training cycle of the PPO algorithm, the billet cutting and length setting policy network is updated using the policy gradient algorithm. parameters and utilize The sampled data for the old edge computing and caching strategy network Conduct training; 4) After training, based on the new billet cutting and length setting strategy network after training. It provides an optimal task decision-making process based on the state information obtained from the environment.
10. A method for intelligent billet cutting and length determination based on deep reinforcement learning in continuous casting and rolling according to any one of claims 1-8, characterized in that, Training the cache policy network The objective function is as follows: in, , A For the dominant function, This is the clipping value. for τ Momentary actions for τ The state at any given moment.