Intelligent decision-making method and system for casting and forging based on high-low fidelity twin asynchronous correction
By constructing a high-fidelity and low-fidelity twin asynchronous correction method, and utilizing the collaborative work of the high-fidelity mechanistic twin model and the low-fidelity surrogate twin model, the problems of low training efficiency, poor safety, and low reliability of deep reinforcement learning in the casting and forging production process are solved, and efficient, safe, and reliable dynamic optimization decision-making in the casting and forging production process is realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CITIC HEAVY INDUSTRIES CO LTD
- Filing Date
- 2025-12-25
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies for deep reinforcement learning suffer from low training efficiency, poor safety, and low reliability on slow industrial mechanistic models. In particular, during the casting and forging production process, traditional methods cannot effectively address the nonlinear relationship between process parameters and final service performance. Production decisions are static and multi-objective optimization is difficult, and the training cost of deep reinforcement learning algorithms is high and the cycle is long.
A high-fidelity mechanistic twin asynchronous correction method is adopted to construct a high-fidelity mechanistic twin model and a low-fidelity surrogate twin model. Through the asynchronous correction mechanism, the high-speed characteristics of the surrogate twin model are used to generate exploration trajectory data, identify key state-action pairs, and use the high-fidelity mechanistic twin model for asynchronous verification and correction, so as to achieve collaborative optimization of the policy network and surrogate twin model parameters of the deep reinforcement learning agent.
It significantly improves the training efficiency, decision safety, and reliability of deep reinforcement learning in the casting and forging production process, realizes dynamic optimization of the casting and forging production process, improves quality consistency, reduces energy consumption and shortens the production cycle, and solves the problems of low training efficiency, poor safety and low reliability in traditional methods.
Smart Images
Figure CN122154380A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of intelligent manufacturing, industrial artificial intelligence and materials science and technology, and in particular to an intelligent decision-making training framework for complex industrial processes, specifically to an intelligent decision-making method and system for casting and forging based on high- and low-fidelity twin asynchronous correction. Background Technology
[0002] Casting and forging are fundamental core technologies in the equipment manufacturing industry. Their production processes are typical complex industrial processes, characterized by multi-physics coupling, nonlinearity, and strong time-varying properties.
[0003] The existing production model in the casting and forging industry suffers from several problems: First, the quality and performance are a "black box": there is a complex nonlinear relationship between process parameters (such as temperature and speed) and the final service performance (such as fatigue life), which is also greatly affected by the evolution of microstructure. Existing systems cannot open the "process-structure-performance" black box. Second, production decisions are static: production plans and process procedures are mostly executed statically. When disturbances occur in the production line, the system cannot respond dynamically, leading to low efficiency and high energy consumption. Third, multiple objectives are difficult to coordinate: objectives such as quality, efficiency, cost, and energy consumption (carbon emissions) are often conflicting, and traditional decision-making methods struggle to find dynamic optimal solutions for multiple objectives. Fourth, the fundamental problem of "untrainable" intelligent decision-making algorithms: advanced intelligent decision-making algorithms such as deep reinforcement learning (DRL) are considered effective ways to solve the above-mentioned dynamic multi-objective optimization problems. However, the convergence of DRL algorithms requires massive (usually millions of) interactive trial and error. In the casting and forging field: (a) Physical trial and error: This is extremely costly, time-consuming, and dangerous, making it absolutely unfeasible; (b) Simulation trial and error: Building high-fidelity mechanistic models (such as finite element method (FEA) or phase-field method (PF)) to simulate the entire "process-microstructure-property" chain is computationally extremely expensive, and a single simulation may take several hours or even days. If DRL is trained on such "slow" models, it would take hundreds of years to converge, which is unaffordable and results in low training efficiency.
[0004] Reducing model complexity and using a high-speed "surrogate model" for training can lead to DRL learning incorrect or even dangerous policies, i.e., "unsafe" and "untrustworthy". Therefore, how to enable data-driven agents such as DRL to be trained efficiently, safely, and reliably on high-cost, slow, and high-risk industrial mechanism models is the core technical challenge that this application urgently needs to solve. Summary of the Invention
[0005] The embodiments of this disclosure provide a casting and forging intelligent decision-making method and system based on high- and low-fidelity twin asynchronous correction, which at least solves the technical problems of low training efficiency, poor security and low reliability of deep reinforcement learning on slow industrial mechanism models in the prior art.
[0006] According to one aspect of the present disclosure, a casting and forging intelligent decision-making method based on high- and low-fidelity twin asynchronous correction is provided, comprising: constructing a mechanistic twin model and a surrogate twin model; wherein the mechanistic twin model is used to simulate the process, microstructure, and macroscopic performance evolution of the casting and forging production process; the surrogate twin model is used to simulate the casting and forging production process; and the simulation fidelity of the mechanistic twin model is greater than that of the surrogate twin model; interacting with the surrogate twin model using a pre-constructed deep reinforcement learning agent to generate exploration trajectory data; identifying key state-action pairs of the casting and forging production process based on the exploration trajectory data and according to preset gating rules; sending the key state-action pairs to the mechanistic twin model for asynchronous verification to obtain the truth reward and truth state transition output by the mechanistic twin model; asynchronously correcting the policy network of the deep reinforcement learning agent and the parameters of the surrogate twin model using the truth reward and truth state transition; collecting real-time state data of the casting and forging production process; and inputting the real-time state data into the deep reinforcement learning agent to output optimized decision actions.
[0007] According to another aspect of the present disclosure, a storage medium is also provided, the storage medium including a stored program, wherein the methods described above are executed by a processor when the program is running.
[0008] According to another aspect of the present disclosure, a casting and forging intelligent decision-making platform based on high-fidelity twin asynchronous correction is also provided, comprising: a data perception layer, a digital twin layer, an intelligent optimization layer, and an optimization decision layer; the data perception layer includes a data acquisition module, the digital twin layer includes a mechanistic twin model and a surrogate twin model, the intelligent optimization layer includes an asynchronous correction reinforcement learning framework, the asynchronous correction reinforcement learning framework is used to train a deep reinforcement learning agent, the asynchronous correction reinforcement learning framework includes: a deep reinforcement learning agent, a high-speed exploration module, a key decision gating module, an asynchronous verification module, and an asynchronous correction module, and the optimization decision layer includes an optimization decision module; wherein, the mechanistic twin model is used to simulate the process, microstructure, and macroscopic performance evolution of the casting and forging production process; the surrogate twin model is used to simulate the casting and forging production process; and the simulation of the mechanistic twin model... The fidelity is greater than that of the surrogate twin model; the high-speed exploration module is used to interact with the pre-built deep reinforcement learning agent and the surrogate twin model to generate exploration trajectory data; the key decision gating module is used to identify key state-action pairs in the casting and forging production process based on the exploration trajectory data and according to preset gating rules; the asynchronous verification module is used to send the key state-action pairs to the mechanistic twin model for asynchronous verification and obtain the truth value reward and truth value state transition output by the mechanistic twin model; the asynchronous correction module is used to asynchronously correct the policy network of the deep reinforcement learning agent and the parameters of the surrogate twin model using the truth value reward and truth value state transition; the data acquisition module is used to collect real-time state data of the casting and forging production process; the optimization decision module is used to input the real-time state data into the deep reinforcement learning agent and output optimization decision actions.
[0009] According to another aspect of the present disclosure, a casting and forging intelligent decision-making system based on high-fidelity twin asynchronous correction is also provided, comprising: an industrial intelligent decision-making body for executing the above-described casting and forging intelligent decision-making method; and a cyber-physical system interacting with the industrial intelligent decision-making body; wherein the industrial intelligent decision-making body deploys a policy network trained by the asynchronous correction reinforcement learning framework, and has the ability to request decision assistance and security verification from a mechanistic twin model and a proxy twin model in the cloud when executed on a physical device.
[0010] In the training phase, this application first constructs a mechanistic twin model with higher simulation fidelity and a faster surrogate twin model, allowing the high-fidelity mechanistic model and the high-speed surrogate model to perform their respective functions in subsequent processing, thus providing a foundation for resolving the contradiction between efficiency and accuracy. Then, by interacting with the deep reinforcement learning agent and the surrogate twin model, exploration trajectory data is generated. The high-speed characteristics of the surrogate twin model are utilized to achieve high-speed exploration of the decision space at low cost. Based on this, key state-action pairs are identified according to the exploration trajectory data and gating rules to accurately filter out high-value or high-uncertainty decision points. Subsequently, the key state-action pairs are sent to the mechanistic twin model for asynchronous verification, utilizing its high-fidelity characteristics to obtain reliable truth rewards and truth state transitions, ensuring the accuracy of the correction information. Afterwards, information fusion is achieved through an asynchronous correction mechanism. This truth is used to asynchronously correct the parameters of the agent's policy network and the surrogate twin model, simultaneously improving the reliability of the decision policy and the prediction accuracy of the surrogate twin model.
[0011] In the application phase, real-time state data is directly input into the deep reinforcement learning agent trained through the aforementioned process, outputting optimized decisions. Thus, through the collaborative work and asynchronous correction of the high-fidelity model and the high-speed agent model, the decoupling and unification of training efficiency and decision accuracy are fundamentally achieved. Without significantly increasing the computational burden, the training efficiency, decision security, and final reliability of deep reinforcement learning in the simulation of the typical slow-mechanism process of casting and forging are significantly improved. This solves the technical problems of low training efficiency, poor security, and low reliability of deep reinforcement learning on industrial slow-mechanism models in existing technologies. Attached Figure Description
[0012] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings: Figure 1 This is an application scenario diagram of the intelligent decision-making platform for casting and forging based on high- and low-fidelity twin asynchronous correction described in the embodiments of this application; Figure 2 This is a flowchart of the intelligent decision-making method for casting and forging based on high-fidelity twin asynchronous correction as described in the embodiments of this application; Figure 3 This is a schematic diagram of the intelligent decision-making platform for casting and forging described in the embodiments of this application. Detailed Implementation
[0013] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.
[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0015] Example According to this embodiment, a casting and forging intelligent decision-making platform based on high- and low-fidelity twin asynchronous correction is provided. Figure 1 The diagram illustrates an application scenario for the intelligent decision-making platform for casting and forging. Figure 1 As shown, this application scenario includes a terminal device 100 for technicians 110, a casting and forging intelligent decision-making platform 200, and a cyber-physical system (CPS) 300. The terminal device 100 provides technicians 110 with a visual interactive interface for monitoring production status, receiving optimization decision suggestions, and making manual interventions. The casting and forging intelligent decision-making platform 200, as the core processing unit, deploys a two-layer digital twin environment containing a high-fidelity mechanistic twin model (H-Twin) and a low-fidelity surrogate twin model (L-Twin), as well as an intelligent optimization agent based on the asynchronous corrected reinforcement learning (ACRL) framework. The cyber-physical system 300 includes actual casting and forging production line equipment, sensor networks, and actuators, responsible for executing optimization decisions and feeding back real-time production data.
[0016] Under the aforementioned operating environment, according to the first aspect of this embodiment, a casting and forging intelligent decision-making method based on high-fidelity twin asynchronous correction is provided, through... Figure 1 The intelligent decision-making platform for casting and forging shown in Figure 200 has been implemented. Figure 2 A flowchart illustrating the method is shown below. (Refer to...) Figure 2 As shown, the method includes: S201: Construct a mechanistic twin model and a surrogate twin model; wherein, the mechanistic twin model is used to simulate the process, microstructure and macroscopic performance evolution of the casting and forging production process; the surrogate twin model is used to simulate the casting and forging production process; and the simulation fidelity of the mechanistic twin model is greater than that of the surrogate twin model; S202: Interact the pre-built deep reinforcement learning agent with the agent twin model to generate exploration trajectory data; S203: Based on the exploration trajectory data and according to the preset gating rules, identify the key state-action pairs in the casting and forging production process; S204: Send the key state-action pair to the mechanism twin model for asynchronous verification, and obtain the truth value reward and truth value state transition output by the mechanism twin model; S205: Using the truth reward and truth state transition, asynchronously correct the parameters of the policy network of the deep reinforcement learning agent and the agent twin model; S206: Collect real-time status data of the casting and forging production process; S207: Input the real-time state data into the deep reinforcement learning agent and output an optimized decision action.
[0017] In the training phase of this invention, a mechanistic twin model and a surrogate twin model are constructed (corresponding to step S201). Specifically, a high-fidelity mechanistic twin model is constructed based on physical mechanisms such as the finite element method (FEA), computational fluid dynamics (CFD), and / or cellular automata (CA) / phase field method (PF) to simulate the temperature field, stress-strain field, grain evolution, and final performance during the casting and forging process with high precision but slow speed. That is, to simulate grain evolution, phase transformation, and defect formation under a thermo-mechanical coupling field. At the same time, a high-speed but low-fidelity surrogate twin model is constructed based on neural networks (such as graph neural networks (GNN), convolutional neural networks (CNN), or recurrent neural networks (RNN)). By learning from historical simulation data, a rapid approximate simulation of the process is achieved, providing a virtual-real combined environment foundation for subsequent intelligent decision-making training.
[0018] Then, the deep reinforcement learning agent and the surrogate twin model are subjected to large-scale, high-frequency interactive trial and error to initially explore the decision space and generate exploration trajectory data (corresponding to step S202). Specifically, the DRL agent executes a large number of process parameter adjustment actions, such as adjusting heating temperature and forging speed, at millisecond speeds within the surrogate twin model, and records the instantaneous reward and state transition for each state-action pair, forming a massive dataset of exploration trajectories. Thus, millions of trials and errors are completed in an extremely short time, quickly and initially grasping the feature distribution of the decision space.
[0019] Next, based on the exploration trajectory data and according to preset gating rules (such as uncertainty sampling or value gradient), key state-action pairs in the casting and forging production process are identified (corresponding to step S203). Specifically, by monitoring the output variance or Q-value confidence interval of the DRL agent's value network, when the uncertainty exceeds a preset threshold, the current state-action pair is determined to be a key decision point; for example, when the agent's estimate of the decision value of "using a certain forging speed at a specific temperature" fluctuates significantly, it is marked as a key sample. Thus, decision points that the agent is "confused" or have potential high value are accurately screened, avoiding high-cost verification of all data.
[0020] Next, the key state-action pairs are submitted to the mechanistic twin model for asynchronous verification to obtain the truth reward and truth state transition (corresponding to step S204). Specifically, the gated key decision points are added to the asynchronous task queue, and the mechanistic twin model performs high-fidelity simulations sequentially. Based on the transient temperature field, stress-strain field, and microstructure field output by the simulation, a comprehensive reward reflecting macroscopic performance, carbon emissions, and production time is calculated, and the next state data is obtained. Thus, physically reliable "gold standard" data is provided for subsequent calibration.
[0021] Subsequently, using truth value rewards and truth value state transitions, the parameters of the policy network of the DRL agent and the surrogate twin model are asynchronously updated and corrected simultaneously (corresponding to step S205). Specifically, the truth value quadruples are stored in a high-fidelity experience replay pool for priority sampling by the DRL agent to update the policy network; simultaneously, they are used as new samples to fine-tune the surrogate twin model online, improving its prediction accuracy in key decision regions. Thus, the co-evolution of the fidelity of the DRL policy and the surrogate model is achieved, forming a reinforcement learning closed loop of "exploration-verification-correction".
[0022] The aforementioned application phase (steps S201-S205) relies on the trained deep reinforcement learning agent to make real-time decisions, and solves the core pain point of industrial non-equilibrium samples through an asynchronous correction mechanism.
[0023] During the application phase, real-time data from the production line is collected and input into a trained agent, which directly outputs optimization decisions such as furnace temperature setting and forging press speed (corresponding to steps S206 and S207). After safety verification, these decisions are sent to physical equipment for execution. This achieves dynamic closed-loop optimization of the casting and forging production process, improving quality consistency, reducing energy consumption, and shortening the production cycle.
[0024] As described in the background section, traditional DRL methods suffer from slow simulation speed and extremely high training costs due to high-fidelity mechanism models. If surrogate models are used, there is a risk of unreliable strategies, which seriously restricts the application of intelligent decision-making in industrial scenarios.
[0025] In view of this, this application, during the training phase, first constructs a mechanistic twin model with higher simulation fidelity and a faster surrogate twin model, allowing the high-fidelity mechanistic model and the high-speed surrogate model to perform their respective functions in subsequent processing, thus providing a foundation for resolving the contradiction between efficiency and accuracy. Then, by interacting with the deep reinforcement learning agent and the surrogate twin model, exploration trajectory data is generated. The high-speed characteristics of the surrogate twin model are utilized to achieve high-speed exploration of the decision space at low cost. Based on this, key state-action pairs are identified according to the exploration trajectory data and gating rules to accurately filter out high-value or high-uncertainty decision points. Subsequently, the key state-action pairs are sent to the mechanistic twin model for asynchronous verification, utilizing its high-fidelity characteristics to obtain reliable truth rewards and truth state transitions, ensuring the accuracy of the correction information. Afterwards, information fusion is achieved through an asynchronous correction mechanism. This truth is used to asynchronously correct the parameters of the agent's policy network and the surrogate twin model, simultaneously improving the reliability of the decision policy and the prediction accuracy of the surrogate twin model.
[0026] In the application phase, real-time state data is directly input into the deep reinforcement learning agent trained through the aforementioned process, outputting optimized decisions. Thus, through the collaborative work and asynchronous correction of the high-fidelity model and the high-speed agent model, the decoupling and unification of training efficiency and decision accuracy are fundamentally achieved. Without significantly increasing the computational burden, the training efficiency, decision security, and final reliability of deep reinforcement learning in the simulation of the typical slow-mechanism process of casting and forging are significantly improved. This solves the technical problems of low training efficiency, poor security, and low reliability of deep reinforcement learning on industrial slow-mechanism models in existing technologies.
[0027] Optionally, based on the exploration trajectory data and according to preset gating rules, the operation of key state-action pairs in the casting and forging production process is identified, including: calculating the output variance of the state value network of the deep reinforcement learning agent for state value estimation; calculating the confidence interval of the action value network of the deep reinforcement learning agent for action value estimation; when the output variance or the confidence interval exceeds a preset threshold, the current state-action pair is determined to be a key state-action pair.
[0028] In this embodiment of the invention, the same state is processed through multiple parallel output heads (e.g., 5 heads) of the state value network. Each value was estimated separately, resulting in a set of value estimates. Then, the variance of this set of estimates is calculated: ; in, Let K be the mean of the K estimates.
[0029] The output variance of this state value network reflects the uncertainty of the DRL agent's long-term value assessment of the current casting and forging production state (such as workpiece temperature distribution and equipment operating parameters): the higher the variance, the more confused the agent is about the consequences of decisions under this state.
[0030] Simultaneously, a bootstrap method is employed to repeatedly sample training subsets with replacement from the experience replay pool to construct multiple action value networks. For a given state-action pair... Each network outputs a Q-value, forming a set of Q-values. ,That The confidence interval is calculated by determining the lower bound quantile of the set. Quantiles of the upper limit quantile Thus, the width of the confidence interval reflects the confidence level of the action. The uncertainty of value assessment. Therefore, the confidence interval of the action value network quantifies the reliability of the DRL agent's immediate value estimate of performing a specific process action (such as adjusting heating, adjusting forging speed, changing cooling rate, etc.). In other words, the wider the confidence interval, the more uncertain the value assessment of the action. By setting reasonable thresholds (e.g., a variance threshold of 0.1 and a confidence interval threshold of [Q]), this can be addressed. low Q high With a width threshold of 0.5, the system can automatically filter out decision-making situations where the agent has not yet grasped the rules or has great potential value.
[0031] When the output variance or confidence interval exceeds a preset threshold, the current state-action pair is identified as a critical state-action pair. This enables the precise capture of "high-value" and "high-uncertainty" decision points during the exploration process. It avoids the waste of computational resources caused by submitting a large number of ordinary decisions to a slow mechanistic model, while ensuring that critical decisions can be verified with high fidelity, thus improving the intelligence and economy of the entire training process from the source.
[0032] Optionally, sending the key state-action pair to the mechanistic twin model for asynchronous verification and obtaining the truth value reward and truth value state transition output by the mechanistic twin model includes: injecting the key state-action pair into an asynchronous task queue; having the mechanistic twin model sequentially obtain and process the key state-action pair from the asynchronous task queue and perform simulation; and, based on the simulation results, calculating a comprehensive reward reflecting macroscopic mechanical properties, total carbon emissions, and total production time as the truth value reward, and obtaining the next state of the forging and casting production process in the model as the truth value state transition.
[0033] In this embodiment of the invention, a first-in-first-out asynchronous task queue is pre-generated. Then, the identified key state-action pairs are encapsulated into standardized task units, appended with timestamps and unique identifiers, and injected into the asynchronous task queue. This asynchronous task queue acts as a buffer, ensuring that the high-speed exploration process of the DRL agent and the agent twin model is not blocked by the slow mechanistic twin model simulation, thus achieving asynchronous parallelization of the training process. The mechanistic twin model, as a high-fidelity verifier, performs full-process simulation of the key decisions injected into the asynchronous task queue based on physical mechanisms such as the finite element method and phase-field method, outputting complete physical field data including transient temperature field, stress-strain field, and microstructure field.
[0034] Based on these high-fidelity simulation data, the macroscopic yield strength and other mechanical properties of the workpiece are calculated through a cross-scale analysis model. The total carbon emissions of the process are accumulated through an energy consumption mapping model, and the total production time is recorded. Finally, these multi-objective indicators are integrated into a comprehensive reward value. At the same time, the complete state of the system at the end of the simulation (including workpiece geometry, temperature distribution, microstructure characteristics, etc.) is output as the true state transition.
[0035] This ensures that the verification process is not constrained by the speed of exploration, greatly improving the overall training efficiency of the system, while also ensuring that the "true value" data used for correction has high credibility at the physical mechanism level. This provides a reliable data foundation for DRL agents to learn safe and optimal strategies that conform to physical laws, fundamentally solving the inherent contradiction between training efficiency and model accuracy.
[0036] Optionally, the simulation results include the transient temperature field, transient stress field, transient strain field, microstructure field image, and total production time of the workpiece during the simulation process; and, based on the simulation results, the operation of calculating a comprehensive reward reflecting macroscopic mechanical properties, total carbon emissions, and total production time as the truth value reward includes: Based on the transient temperature field The transient stress field and the transient strain field The macroscopic yield strength of the workpiece is calculated by integrating the crystal plastic constitutive model. : ; in, The function is the expression for the crystal plastic constitutive model; Based on the transient temperature field Historical data and the microstructure field images The average grain size of the workpiece was statistically analyzed using a recrystallization kinetic model. : ; in, The function represents a dynamic recrystallization volume fraction and the microstructure field image. As input, with the average grain size The mapping relationship is for the output; and the dynamic recrystallization volume fraction is calculated by the recrystallization kinetics model. By using a preset energy consumption model, and based on the transient temperature field The mapping relationship between the energy consumption of the heating furnace is used to calculate the total carbon emissions of the process. : ; in, For the heating furnace power, As a carbon emission factor of the power grid, The total production time of the process; The truth value reward is calculated using the following formula based on the macroscopic yield strength, the average grain size, the total carbon emissions, and the total production time. : ; in, As a comprehensive quality indicator, it is defined as follows: ; The equivalent comprehensive cost is defined as follows: ; and During the training process The historical maximum and minimum values; This is an ideal reference value for yield strength; This is an ideal reference value for grain size; The target value is the equivalent comprehensive cost. For ideal production time; , , , , as well as The weights and adjustment factors are greater than zero.
[0037] In this embodiment of the invention, based on the transient temperature field Transient stress field and transient strain field The macroscopic yield strength of the workpiece is calculated by integrating the crystal plastic constitutive model. Specifically, the crystal plastic constitutive model first considers the transient temperature field. Determine the current material parameters (such as shear modulus, critical shear stress), and then based on the transient strain field. Calculate the shear strain rate on each slip system, then update the shear stress of each slip system according to constitutive relations (usually using rate-dependent flow and hardening rules). Finally, by averaging the stress field at all integration points throughout the entire workpiece volume, and based on the Mises yield criterion or by analyzing the macroscopic stress-strain curve, determine the overall macroscopic yield strength of the workpiece. This ensures that the prediction of yield strength is no longer a rough estimate based on empirical formulas, but is based on microscopic physical mechanisms (dislocation slip), thereby ensuring that the calculation results of this key quality indicator have a solid physical basis and high reliability, and avoiding the DRL agent learning incorrect process strategies based on distorted performance data.
[0038] The process of statistically averaging grain size based on transient temperature field history and microstructure field image is as follows: First, the recrystallization kinetic model is based on the transient temperature field... Historical data and associated strain field history are used to calculate the dynamic recrystallization volume fraction of each material point over time by solving constitutive models such as the Avrami equation. Subsequently, the function... Compare the calculated recrystallization volume fraction with the microstructure field image at the final time. This method combines various techniques. The image is processed using metallographic image processing techniques (such as grayscale thresholding and grain boundary identification) to identify the regions of each grain. The equivalent circle diameter or intercept length of each grain is calculated, and finally, the arithmetic mean of all identified grain sizes is taken to obtain the average grain size of the workpiece. Thus, the dynamic microstructure evolution process (recrystallization), which is difficult to measure directly, is quantified through a physical model and combined with the final static microstructure characterization, achieving accurate modeling of the causal relationship between "process history and microstructure". This makes the calculation of the key quality indicator of average grain size no longer a simple image post-processing, but a comprehensive result reflecting the influence of the entire process, greatly improving the accuracy and interpretability of the quality prediction model.
[0039] The process of calculating total carbon emissions using an energy consumption model is as follows: the energy consumption model establishes the power of the heating furnace. With the transient temperature field of the workpiece and furnace The mapping relationship can be determined by combining heat transfer principles (such as calculating heat loss based on temperature difference and thermal resistance) with the equipment efficiency curve. During simulation... Internally, the instantaneous power of the heating furnace Integrate over time to obtain the total power consumption (in kWh), then multiply by the grid's carbon emission factor. (Unit: kg CO2 / kWh) This gives the total carbon emissions of the process. This directly links the abstract "carbon emission" target with specific, calculable physical quantities (temperature field, time), enabling precise and dynamic quantification of the environmental impact of the process. This provides the DRL agent with actionable optimization targets and reliable evaluation criteria for learning green and low-carbon manufacturing processes, transforming energy conservation and emission reduction from a macro-level concept into specific, optimizable engineering indicators.
[0040] Finally, the comprehensive truth value reward The calculation process unifies all the above physical quantities into a single objective: quality index. Characterized by a weighted combination normalized strength and grain size (the reciprocal, because finer grains generally result in better performance); cost indicators Carbon emissions and time are unified into economic costs. The reward function is constructed using the product of the normalization and exponentiation of the quality term, the reciprocal quadratic form of the cost term, and the exponential decay function of the time term. A nonlinear multi-objective optimization model is built, whose parameters are used to precisely adjust the agent's preferences and tolerances for each objective. The comprehensive reward function represented by the multi-objective optimization model is as follows: ; in, As a comprehensive quality indicator, it is defined as follows: ; The equivalent comprehensive cost is defined as follows: ; and During the training process The historical maximum and minimum values; This is an ideal reference value for yield strength; This is an ideal reference value for grain size; The target value is the equivalent comprehensive cost. For ideal production time; , , , , as well as The weights and adjustment factors are greater than zero.
[0041] This comprehensive reward function design, through its inherent nonlinear structure and adjustable parameters, can effectively characterize the complex, even conflicting, trade-offs between multiple objectives such as quality, cost, and time. It forces the DRL agent to go beyond simply optimizing a single metric, but to seek a Pareto optimal solution that simultaneously considers performance, environmental friendliness, and efficiency, thereby guiding the agent to discover a balanced and superior process solution that truly meets the actual needs of industry.
[0042] Thus, by deeply integrating specific technologies from materials science, heat transfer, image processing, and multi-objective optimization, the raw physical field data output from high-fidelity simulation is systematically and reliably transformed into a comprehensive reward signal that drives the DRL agent to learn efficiently, safely, and reliably. This not only ensures the physical credibility of the agent's decisions, but more importantly, it structures complex industrial multi-objective optimization problems through precise mathematical modeling, enabling DRL to effectively explore the solution space and converge to the optimal strategy that meets the actual engineering needs.
[0043] Optionally, the asynchronous correction of the policy network of the deep reinforcement learning agent and the parameters of the surrogate twin model using the truth reward and truth state transition includes: constructing the key state-action pair, the truth reward, and the truth state transition into a quadruple and storing it in the experience replay pool; when training the deep reinforcement learning agent, extracting the corresponding quadruple from the experience replay pool with a preset sampling priority, and updating the policy network of the deep reinforcement learning agent according to the extracted quadruple; simultaneously, using the quadruple as a new training sample to perform online incremental training or fine-tuning of the surrogate twin model to update its model parameters.
[0044] In this embodiment of the invention, key state-action pairs, truth rewards, and truth state transitions are constructed as quadruples. Specifically, the quadruples are constructed in a standard format of (state, action, truth reward, truth next state), and then stored in a dedicated high-fidelity experience replay pool after being appended with a timestamp and source identifier. When training the DRL agent, a priority experience replay mechanism based on temporal difference error is adopted, that is, the sampling probability of each quadruple is dynamically adjusted according to the difference between the predicted value and the truth reward, and the sample with the larger the difference is sampled with higher priority. The policy network is updated by gradient backpropagation, which minimizes the loss function between the current policy output and the improvement target calculated based on the truth reward. At the same time, the online incremental training of the surrogate Siamese model adopts a sliding window mechanism, which mixes the newly added quadruples with historical data in a certain proportion. By minimizing the mean square error between the predicted state and the truth next state, the network weights of the surrogate model are fine-tuned, so that its prediction accuracy in the key decision region is continuously improved.
[0045] Thus, this dual correction mechanism ensures that the DRL agent prioritizes learning from high-value, high-uncertainty key decision-making experiences, significantly accelerating the policy convergence process and raising the performance ceiling of the final policy. Simultaneously, it enables the surrogate twin model to continuously optimize online using the "truth" data provided by the high-fidelity mechanistic model, gradually narrowing the prediction gap between it and the mechanistic model, forming a virtuous cycle of "exploration-verification-correction." This not only greatly improves the efficiency of training data utilization and the learning effect of the agent but also fundamentally enhances the adaptability, stability, and reliability of the entire intelligent decision-making system during long-term operation.
[0046] Optionally, after inputting the real-time state data into the trained deep reinforcement learning agent and outputting the optimized decision action, the method further includes: before sending the optimized decision action to the physical device for execution, calling the mechanistic twin model or preset mechanistic rules to perform a security verification on the optimized decision action; when the security verification is passed, the optimized decision action is sent to the physical device for execution.
[0047] In this embodiment of the invention, before the optimized decision action is sent to the physical equipment for execution, a mandatory safety verification is performed. Only actions that pass the verification are allowed to be executed. This safety verification employs a dual verification mechanism. Specifically, firstly, a rapid initial screening is performed based on preset mechanism rules to check whether the decision action exceeds the physical limits of the equipment (such as hard constraints like maximum pressure or maximum temperature). After passing the initial screening, a high-fidelity simulation verification is performed using a mechanism twin model to predict the evolution of key physical fields (such as stress and temperature fields) inside the workpiece after executing the decision action, and to determine whether risks such as material damage, equipment overload, or product quality defects may occur. The entire verification process uses ultra-real-time simulation technology, which sacrifices simulation accuracy for computational speed, ensuring safety while keeping the verification time within the allowable range of the production cycle.
[0048] Thus, a safe closed loop of "decision-verification-execution" is constructed, which not only makes full use of the optimization capabilities of DRL agents, but also effectively prevents potential risks through authoritative verification of the mechanism model. This solves the trust problem of "not daring to use" AI decision-making in industrial scenarios and significantly improves the reliability and acceptability of intelligent decision-making systems in actual production.
[0049] According to a second aspect of this embodiment, a casting and forging intelligent decision-making platform 200 based on high- and low-fidelity twin asynchronous correction is provided. Figure 3 A schematic diagram of the platform's structure is shown for reference. Figure 3As shown, the intelligent decision-making platform 200 for casting and forging includes: a data perception layer 210, a digital twin layer 220, an intelligent optimization layer 230, and an optimization decision layer 240. The data perception layer includes a data acquisition module; the digital twin layer includes a mechanistic twin model and a surrogate twin model; the intelligent optimization layer includes an asynchronous correction reinforcement learning framework, which is used to train a deep reinforcement learning agent. The asynchronous correction reinforcement learning framework includes: a deep reinforcement learning agent, a high-speed exploration module, a key decision gating module, an asynchronous verification module, and an asynchronous correction module; the optimization decision layer includes an optimization decision module. The mechanistic twin model is used to simulate the process, microstructure, and macroscopic performance evolution of the casting and forging production process; the surrogate twin model is used to simulate the casting and forging production process; and the simulation fidelity of the mechanistic twin model is greater than that of the digital twin model. The system comprises the following modules: a proxy twin model; a high-speed exploration module for interacting with a pre-built deep reinforcement learning agent to generate exploration trajectory data; a key decision gating module for identifying key state-action pairs in the casting and forging production process based on the exploration trajectory data and preset gating rules; an asynchronous verification module for sending the key state-action pairs to the mechanistic twin model for asynchronous verification, obtaining the truth reward and truth state transition output by the mechanistic twin model; an asynchronous correction module for asynchronously correcting the policy network of the deep reinforcement learning agent and the parameters of the proxy twin model using the truth reward and truth state transition; a data acquisition module for acquiring real-time state data of the casting and forging production process; and an optimization decision module for inputting the real-time state data into the deep reinforcement learning agent and outputting optimization decision actions.
[0050] In this embodiment of the invention, the data perception layer 210 is used to construct the data foundation of the platform, collecting process, equipment, quality, material, and environmental data. The digital twin layer 220 is the "two-layer twin" environment of this application. The high-fidelity mechanistic twin model is the source of "ground truth," but it is extremely slow. It consists of multi-scale mechanistic models, such as: (a) macroscopic models: forging thermo-coupling models based on FEA; (b) microscopic models: grain evolution and phase transition models based on cellular automata (CA) or phase-field method (PF); (c) performance models: strength prediction models based on the Hall-Petch formula. The low-fidelity surrogate twin model is the source of "sparring partners," but it is extremely fast. It is usually a lightweight "surrogate" or "alternative" of the mechanistic twin model, such as a GNN or CNN. It is built by learning historical simulation data of the mechanistic twin model offline, and can predict results in milliseconds, but it has a certain degree of error.
[0051] The intelligent optimization layer 230 is the core innovation of this application. For example... Figure 3 As shown, an asynchronous corrected reinforcement learning (ACRL) framework is deployed within the intelligent optimization layer 230. This framework is used to train deep reinforcement learning (DRL) agents (such as agents using PPO or SAC algorithms). Its training process is as follows: Step 1: In the high-speed exploration module, the DRL agent primarily interacts with the surrogate twin model. The millisecond-level response speed of the surrogate twin model allows the agent to complete millions of "cheap" trials within hours, quickly learning a basic decision-making strategy.
[0052] Step Two: Critical Decision Gating Module. During high-speed exploration, the DRL AI encounters decision points it is unsure about or deems important. The critical decision gating module monitors these points in real time, for example: Uncertainty Gating: When the value network of a DRL agent produces a high variance in its estimate of the value V(S) of the current state S (i.e., multiple evaluation heads output inconsistently), it indicates that the agent is "confused" about the current state.
[0053] Value gradient gating: When a DRL agent believes that an action A can bring a much higher-than-average expected reward (high Q value), it indicates that this may be a "critical opportunity". When gating is triggered, the (state S, action A) pair is marked as a "critical decision point".
[0054] Step 3: The asynchronous verification module sends these "critical decision points" (S, A) to an asynchronous task queue. The mechanistic twin model, acting as a background "truth server," independently retrieves tasks from this queue and performs slow but high-fidelity simulation calculations. For example, the mechanistic twin model may take 2 hours to complete the calculation, obtaining the "truth" next state S', the "truth" microorganism, and the "truth" reward R (e.g., a reward calculated based on real performance and real carbon emissions) resulting from the decision point.
[0055] Step 4: The asynchronous verification module sends the "truth values" (S, A, R, S') output by the mechanistic twin model to the asynchronous correction module. This asynchronous correction module performs dual correction: Correcting the DRL agent: The "truth" quadruple is stored in a high-fidelity experience replay pool. During subsequent training, the DRL agent will preferentially sample from this pool (Prioritized Experience Replay) to ensure that its policy is corrected based on erroneous data from the "truth" rather than the "surrogate model".
[0056] Correcting the surrogate twin model: The "truth value" quadruple is also considered a new, high-value training sample for the surrogate twin model. The surrogate twin model uses these samples for online incremental training (fine-tuning), making it increasingly accurate in the "confused" regions of the DRL agent.
[0057] The execution control layer 240 is used to send the instructions output by the trained DRL agent and verified by security to the physical devices in the cyber-physical system 300.
[0058] Service application layer 250 is used to provide intelligent quality diagnosis (inversion) and process recommendation based on mechanistic twin models and surrogate twin models.
[0059] Therefore, through the ACRL framework of this application, this platform solves the "training efficiency" problem of DRL by leveraging the high speed of the surrogate twin model, while simultaneously solving the "training reliability" problem of DRL by utilizing the high fidelity of the mechanistic twin model. This makes it possible to train DRL on slow industrial mechanistic models, achieving a decoupling and unification of efficiency and accuracy.
[0060] Optionally, the intelligent optimization layer further includes a security constraint module, which is used to call the mechanism twin model or preset mechanism rules to perform security verification on the optimization decision action before sending the optimization decision action to the physical device for execution; when the security verification is passed, the optimization decision action is sent to the physical device for execution.
[0061] Optionally, the intelligent decision-making platform 200 for casting and forging also includes an execution control layer 250 and a service application layer 260. The execution control layer 250 converts the instructions output by the optimization decision module and verified through security checks into control commands executable by specific equipment, coordinates the coordinated operation of various actuators (such as heating furnaces, forging presses, and cooling systems), and monitors equipment status in real time to ensure stable execution of the production process. The service application layer 260 provides human-machine interfaces for different user roles, including process parameter recommendations and quality diagnosis interfaces for process engineers, production status monitoring dashboards and energy efficiency analysis reports for production managers, and equipment health status warnings and maintenance suggestions for equipment maintenance personnel, realizing the scenario-based output of the platform's intelligent service capabilities.
[0062] According to a third aspect of this embodiment, a casting and forging intelligent decision-making system based on high-fidelity twin asynchronous correction is provided, comprising: an industrial intelligent decision-making body for executing the above-described casting and forging intelligent decision-making method; and a cyber-physical system interacting with the industrial intelligent decision-making body; wherein the industrial intelligent decision-making body deploys a policy network trained by the asynchronous correction reinforcement learning framework, and has the ability to request decision assistance and security verification from a mechanistic twin model and a proxy twin model in the cloud when executed on a physical device.
[0063] It should be noted that the intelligent decision-making platform and system for casting and forging based on high- and low-fidelity twin asynchronous correction provided in this embodiment can realize all the functions and steps in the above method embodiments, solve the same technical problems, and achieve the same technical effects. The similarities will not be repeated here.
[0064] In summary, this application constructs a complete technical architecture encompassing "data perception - digital twin - intelligent optimization - optimization decision-making - execution control - service application," deeply integrating the advantages of high- and low-fidelity twin models with asynchronous correction reinforcement learning mechanisms. This not only overcomes the core technical bottlenecks of low training efficiency and poor reliability of DRL on slow industrial mechanistic models, but also forms a closed-loop intelligent decision-making system from intelligent training in virtual space to safe execution in physical space. This provides comprehensive technical support for improving the intelligence level of casting and forging production and achieving the goals of quality improvement, efficiency enhancement, and green low-carbon development. Thus, it solves the challenges of training convergence and safe deployment of intelligent decision-making on slow industrial mechanistic models.
[0065] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0066] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A casting and forging intelligent decision-making method based on high- and low-fidelity twin asynchronous correction, characterized in that, include: A mechanistic twin model and a surrogate twin model are constructed; wherein, the mechanistic twin model is used to simulate the process, microstructure and macroscopic property evolution of the casting and forging production process; the surrogate twin model is used to simulate the casting and forging production process; and the simulation fidelity of the mechanistic twin model is greater than that of the surrogate twin model. The pre-built deep reinforcement learning agent interacts with the surrogate twin model to generate exploration trajectory data; Based on the exploration trajectory data and according to the preset gating rules, the key state-action pairs of the casting and forging production process are identified; The key state-action pair is sent to the mechanism twin model for asynchronous verification to obtain the truth value reward and truth value state transition output by the mechanism twin model; The parameters of the policy network of the deep reinforcement learning agent and the agent twin model are asynchronously corrected using the truth reward and truth state transition. Collect real-time status data of the casting and forging production process; The real-time state data is input into the deep reinforcement learning agent, which then outputs optimized decision-making actions.
2. The method according to claim 1, characterized in that, Based on the exploration trajectory data and according to preset gating rules, the operation of the key state-action pair in the casting and forging production process is identified, including: Calculate the output variance of the state value network of the deep reinforcement learning agent for the state value estimate; calculate the confidence interval of the action value network of the deep reinforcement learning agent for the action value estimate; When the output variance or the confidence interval exceeds a preset threshold, the current state-action pair is determined to be a critical state-action pair.
3. The method according to claim 1, characterized in that, The key state-action pair is sent to the mechanistic twin model for asynchronous verification, and the operations for obtaining the truth reward and truth state transition output by the mechanistic twin model include: Inject the key state-action pairs into the asynchronous task queue; The key state-action pairs are sequentially retrieved and processed from the asynchronous task queue by the mechanistic twin model, and the simulation is performed. Based on the simulation results, a comprehensive reward reflecting macroscopic mechanical properties, total carbon emissions, and total production time is calculated as the truth reward, and the next state of the casting and forging production process in the model is obtained as the truth state transition.
4. The method according to claim 3, characterized in that, The simulation results include the transient temperature field, transient stress field, transient strain field, microstructure field image, and total production time of the workpiece during the simulation process; Furthermore, based on the simulation results, the operation of calculating the comprehensive reward reflecting macroscopic mechanical performance, total carbon emissions, and total production time as the truth reward includes: Based on the transient temperature field The transient stress field and the transient strain field The macroscopic yield strength of the workpiece is calculated by integrating the crystal plastic constitutive model. : ； in, The function is the expression for the crystal plastic constitutive model; Based on the transient temperature field Historical data and the microstructure field images The average grain size of the workpiece was statistically analyzed using a recrystallization kinetic model. : ； in, The function represents a dynamic recrystallization volume fraction and the microstructure field image. As input, with the average grain size The mapping relationship is for the output; and the dynamic recrystallization volume fraction is calculated by the recrystallization kinetics model. By using a preset energy consumption model, and based on the transient temperature field The mapping relationship between the energy consumption of the heating furnace is used to calculate the total carbon emissions of the process. : ； in, For the heating furnace power, As a carbon emission factor of the power grid, The total production time of the process; The truth value reward is calculated using the following formula based on the macroscopic yield strength, the average grain size, the total carbon emissions, and the total production time. : ； in, As a comprehensive quality indicator, it is defined as follows: ; The equivalent comprehensive cost is defined as follows: ; and During the training process The historical maximum and minimum values; This is an ideal reference value for yield strength; This is an ideal reference value for grain size; The target value is the equivalent comprehensive cost. For ideal production time; , , , , as well as The weights and adjustment factors are greater than zero.
5. The method according to claim 1, characterized in that, The asynchronous correction operation of the policy network of the deep reinforcement learning agent and the parameters of the surrogate twin model using the truth reward and truth state transition includes: The key state-action pair, the truth reward, and the truth state transition are constructed into a quadruple and stored in the experience replay pool; When training the deep reinforcement learning agent, corresponding quadruplets are extracted from the experience replay pool according to a preset sampling priority, and the policy network of the deep reinforcement learning agent is updated according to the extracted quadruplets. At the same time, the quadruplets are used as new training samples to perform online incremental training or fine-tuning on the surrogate twin model to update its model parameters.
6. The method according to claim 1, characterized in that, After inputting the real-time state data into the trained deep reinforcement learning agent and outputting the optimized decision action, the method further includes: Before the optimization decision action is sent to the physical device for execution, the mechanism twin model or the preset mechanism rule is invoked to perform a security verification on the optimization decision action; When the security check is passed, the optimization decision action is sent to the physical device for execution.
7. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, a processor performs the method according to any one of claims 1 to 6.
8. A casting and forging intelligent decision-making platform based on high- and low-fidelity twin asynchronous correction, characterized in that, include: The system comprises a data perception layer, a digital twin layer, an intelligent optimization layer, and an optimization decision layer. The data perception layer includes a data acquisition module. The digital twin layer includes a mechanistic twin model and a surrogate twin model. The intelligent optimization layer includes an asynchronous correction reinforcement learning framework for training a deep reinforcement learning agent. The asynchronous correction reinforcement learning framework includes a deep reinforcement learning agent, a high-speed exploration module, a key decision gating module, an asynchronous verification module, and an asynchronous correction module. The optimization decision layer includes an optimization decision module. The mechanistic twin model is used to simulate the process, microstructure, and macroscopic performance evolution of the casting and forging production process; the surrogate twin model is used to simulate the casting and forging production process; and the simulation fidelity of the mechanistic twin model is greater than that of the surrogate twin model. The high-speed exploration module is used to interact with the pre-built deep reinforcement learning agent and the agent twin model to generate exploration trajectory data; The critical decision gating module is used to identify the critical state-action pairs in the casting and forging production process based on the exploration trajectory data and according to preset gating rules. The asynchronous verification module is used to send the key state-action pair to the mechanism twin model for asynchronous verification, and obtain the truth value reward and truth value state transition output by the mechanism twin model; The asynchronous correction module is used to asynchronously correct the parameters of the policy network of the deep reinforcement learning agent and the agent twin model by utilizing the truth reward and truth state transition. The data acquisition module is used to collect real-time status data of the casting and forging production process; The optimization decision module is used to input the real-time state data into the deep reinforcement learning agent and output optimization decision actions.
9. The intelligent decision-making platform for casting and forging according to claim 8, characterized in that, The intelligent optimization layer also includes a security constraint module, which is used to call the mechanism twin model or preset mechanism rules to perform security verification on the optimization decision action before sending the optimization decision action to the physical device for execution; when the security verification is passed, the optimization decision action is sent to the physical device for execution.
10. A casting and forging intelligent decision-making system based on high- and low-fidelity twin asynchronous correction, characterized in that, include: An industrial intelligent decision-making body, used to execute the intelligent decision-making method for casting and forging as described in any one of claims 1 to 6; And a cyber-physical system that interacts with the industrial intelligent decision-making body; wherein the industrial intelligent decision-making body deploys a policy network trained by an asynchronous correction reinforcement learning framework, and has the ability to request decision assistance and security verification from the mechanistic twin model and the agent twin model in the cloud when executed on the physical device.