A high-temperature forging real-time dynamic digital twin system and method based on structured light three-dimensional reconstruction

Abstract

The application discloses a high-temperature forging real-time dynamic digital twin system and method based on structured light three-dimensional reconstruction, and belongs to the field of precision forging intelligent manufacturing and digital twin technology. In view of the problems that the three-dimensional reconstruction of 800-1250 DEG C hot forging is easily interfered by high-temperature thermal radiation and strong light overexposure, the existing technology has poor virtual-real synchronizability, lacks quantitative mathematical support, and cannot realize process closed-loop optimization, a technical scheme of deep fusion of deep learning algorithm and core mathematical formula is proposed. The method constructs a double-branch lightweight reconstruction network embedded with a channel attention mechanism, combines a composite loss function, distortion correction, space-time registration and other quantitative formulas, and sequentially completes network training, high-temperature image preprocessing, end-to-end three-dimensional reconstruction, multi-source data fusion, dynamic twin modeling and virtual-real deviation regulation, realizes single-frame reconstruction delay of less than or equal to 80 ms, reconstruction accuracy of less than or equal to 0.08 mm, and virtual-real synchronous delay of less than or equal to 150 ms, effectively shields high-temperature interference, completes real-time dynamic mapping of the forging forming process and process parameter self-adaptive closed-loop optimization, greatly improves the forging size qualification rate, does not need additional auxiliary hardware, and is suitable for high-temperature forging online precision monitoring and intelligent control scenes of large-scale free forging and die forging production lines.

Description

Technical Field

[0001] This invention belongs to the fields of precision forging intelligent manufacturing, deep learning 3D vision, non-contact precision measurement and digital twin technology. Specifically, it relates to a method for accurately reconstructing the anti-interference 3D morphology of hot metal forgings at 800℃-1250℃ by relying on deep learning algorithms, and completing the construction of dynamic digital twins and process closed-loop optimization of the entire process by combining mathematical models and algorithm formulas. It is particularly suitable for online morphology monitoring, dimensional accuracy control and dynamic simulation of forming process of high-temperature forgings in large free forging and die forging production lines. Background Technology

[0002] High-temperature forging is a core processing technology in aerospace, energy and power, and high-end equipment manufacturing. The geometry, dimensional accuracy, and surface integrity of hot forgings directly determine the mechanical properties and service life of the products. Traditional offline contact measurement suffers from strong hysteresis and equipment fragility. Existing traditional 3D reconstruction technology relies on manually designed features and has weak resistance to interference from high-temperature thermal radiation and strong light overexposure, making it difficult to balance reconstruction accuracy and real-time performance.

[0003] Existing 3D reconstruction technologies face significant technical bottlenecks when applied to high-temperature forging scenarios: conventional methods such as structured light scanning and laser 3D measurement rely on manually designed image features and fixed visual algorithms, which have extremely poor resistance to interference from the high-temperature forgings themselves, such as thermal radiation, strong light overexposure, and surface grayscale distortion. After image features are lost, problems such as a sharp increase in point cloud noise, loss of local morphology, and a sharp drop in reconstruction accuracy will occur. Even when combined with basic optical filtering components, it is impossible to adapt to the complex working conditions of high-temperature transient deformation and irregular surface conditions at the algorithm level. At the same time, the reconstruction process is cumbersome and has high latency, making it difficult to meet the real-time monitoring needs of the forging site.

[0004] At the same time, most existing digital twin technologies for high-temperature forgings rely on traditional 3D reconstruction data and do not utilize the advantages of deep learning in strong feature extraction, noise adaptive suppression, and end-to-end inference. This results in poor synchronization between the virtual and real twin models, distortion of deformation mapping, and an inability to achieve real-time deviation warning and adaptive control of process parameters. Geometric morphology data and forging process data are disconnected, making it difficult to support intelligent closed-loop management of the forging process.

[0005] To address this issue, researchers have proposed various high dynamic range 3D measurement and digital twin construction methods, but these methods all have certain limitations. Existing multi-exposure imaging and polarization filtering methods require additional hardware assistance and have poor adaptability; traditional deep learning reconstruction methods do not incorporate quantitative mathematical formulas, resulting in poor algorithm interpretability, difficulty in quantifying and guaranteeing reconstruction accuracy, and inability to adapt to the strong interference characteristics of high-temperature and harsh working conditions, thus failing to achieve virtual-real synchronization and process closed-loop optimization. Therefore, this invention designs a real-time dynamic digital twin technology for high-temperature forgings based on the fusion of deep learning 3D reconstruction and quantitative formulas, achieving high-precision reconstruction and twin modeling without additional hardware or auxiliary images.

[0006] In summary, existing methods for three-dimensional measurement and digital twinning of high-temperature forgings cannot adequately meet the high-precision, high-efficiency, and interference-resistant online monitoring requirements of the precision forging industry. Developing a novel reconstruction and twinning method adapted to high-temperature working conditions can effectively overcome the interference problems caused by high-temperature thermal radiation and strong light overexposure, improve measurement accuracy and virtual-real synchronization, and has significant practical implications for promoting the intelligent development of the high-end forging industry. Summary of the Invention

[0007] To overcome the shortcomings of the existing technologies, this invention provides a real-time dynamic digital twin method for high-temperature forgings based on deep learning 3D reconstruction. This method is used for high-precision, real-time online 3D morphology monitoring and dynamic digital twin construction of hot metal forgings at 800℃-1250℃. It can efficiently and accurately reconstruct the 3D geometry of high-temperature forgings, achieve dynamic virtual-real synchronization and process closed-loop optimization, without the need for additional auxiliary images or hardware adjustments, and is suitable for online intelligent monitoring scenarios of high-temperature forgings.

[0008] To achieve the above objectives, the technical solution adopted by this invention is: a real-time dynamic digital twin method for high-temperature forgings based on deep learning 3D reconstruction, comprising the following steps:

[0009] S1. Construction and transfer training of deep learning 3D reconstruction network: A dual-branch lightweight network is built for the characteristics of high-temperature forging images. Training optimization and lightweight processing are completed by combining core mathematical formulas to adapt to high-temperature working conditions.

[0010] S2. Deep learning preprocessing and feature enhancement of high temperature images: geometric distortion correction and overexposure area repair of the original acquired images are performed, and feature weighting enhancement is completed through attention mechanism.

[0011] S3, Deep Learning End-to-End Real-Time 3D Reconstruction, inputs feature-enhanced images into the trained network and directly outputs a high-precision 3D mesh model, and evaluates the reconstruction effect through a quantization formula;

[0012] S4. Multi-source data temporal fusion and spatiotemporal alignment: accurately correlate and fuse 3D reconstruction data, temperature field data, and forging process parameters based on timestamps.

[0013] S5. Dynamic digital twin modeling and real-time virtual-real synchronization: Based on the design CAD model, the virtual-real model is aligned by improving the registration algorithm, realizing the construction of a time-series dynamic twin and millisecond-level synchronization.

[0014] S6. Intelligent analysis of virtual and real deviations and closed-loop optimization of processes: quantifies the dimensional deviation between the twin model and the design model, completes hierarchical early warning, and combines optimization formulas to achieve adaptive adjustment of process parameters or shutdown protection.

[0015] Furthermore, the deep learning 3D reconstruction network in S1 is constructed and transferred for training using the following methods:

[0016] S11. Construct an improved dual-branch lightweight deep learning 3D reconstruction network. The feature extraction branch adopts a lightweight ResNet network with an embedded channel attention mechanism (SE module). The core attention weight calculation formula is as follows:

[0017]

[0018] In the formula, For channel attention weights, It is the Sigmoid activation function. This is a global average pooling operation. This is an intermediate feature map. , These are the weight matrices of the fully connected layer; the attention-weighted feature map is as follows:

[0019]

[0020] In the formula, This is an element-wise multiplication operation. This is the effective feature map after filtering out high-temperature interference.

[0021] S12, the 3D shape inference branch adopts a lightweight encoder-decoder structure, outputting a set of vertex coordinates of the 3D mesh model. The vertex reasoning mapping relationship is as follows:

[0022]

[0023] In the formula, For encoder feature compression operation, For the decoder's 3D coordinate reconstruction operation, This represents the total number of grid vertices.

[0024] S13. A multi-dimensional composite loss function is used to complete network training. The formula for the loss function is:

[0025]

[0026] In the formula, , , To determine the loss weight, this invention takes... , , ;

[0027] Geometric loss (mean squared error loss) formula:

[0028] In the formula, To accurately annotate vertex coordinates in 3D, It is an L2 norm;

[0029] Model smoothing loss formula:

[0030] In the formula, As vertex The Laplace operator;

[0031] Temperature adaptability loss formula:

[0032] In the formula, This represents the number of feature points in the high-temperature region. To predict the characteristics of high-temperature regions for network purposes, High-temperature area features are marked on the real scene.

[0033] S14. Perform pruning, quantization, and lightweight optimization on the network. The formula for parameter compression ratio is:

[0034]

[0035] In the formula, For parameter compression ratio, this invention takes , These are the parameters after pruning. Original parameter values; optimized single-frame inference latency. .

[0036] Furthermore, the following methods are used for deep learning preprocessing and feature enhancement of high-temperature images in S2:

[0037] S21. Based on the camera intrinsic parameter matrix, perform geometric distortion correction on the original image. The pixel coordinate mapping formula is:

[0038]

[0039] In the formula, These are the original distorted pixel coordinates. For the corrected pixel coordinates, For the camera intrinsic parameter matrix, This is the inverse distortion correction function. This represents the distance from the pixel to the center of the image.

[0040] S22. Generative adversarial networks are used to repair overexposed and missing areas in the image. The formula for generating the repaired image is:

[0041]

[0042] In the formula, For generator networks, For the missing region mask, The corrected image; the inpainting loss formula is:

[0043]

[0044] In the formula, For L1 loss, To be a true, complete image. To counteract the loss of weight, To generate adversarial losses, For the discriminator network.

[0045] S23. Enhance images through attention-weighted output features. This completes the effective feature enhancement.

[0046] Furthermore, the deep learning-based end-to-end real-time 3D reconstruction in S3 employs the following method:

[0047] The feature-enhanced image is input into the trained network, which outputs a 3D mesh model end-to-end. The mapping relationship is as follows:

[0048]

[0049] In the formula, To reconstruct the complete two-branch network, For mesh patch generation operations, A collection of mesh faces;

[0050] Reconstruction accuracy quantification formula:

[0051] Formula for restoring integrity:

[0052] In the formula, The number of valid vertices. This represents the total number of vertices.

[0053] Furthermore, the following methods are used for time-series fusion and spatiotemporal alignment of multi-source data in S4:

[0054] S41. Data synchronization is completed using a unified high-precision timestamp. Data acquisition cycle formula:

[0055]

[0056] In the formula, For the collection period, , The start and end times for data collection. This represents the total number of data collections.

[0057] S42. Formula for mapping temperature data to 3D vertex space:

[0058] In the formula, This is a projection function from a 3D vertex to a 2D pixel. The temperature value corresponding to the vertex;

[0059] Multi-source data fusion feature vector:

[0060] In the formula, These are the process parameters at the corresponding time points.

[0061] Furthermore, the dynamic digital twin modeling and real-time virtual-real synchronization in S5 adopt the following methods:

[0062] S51, Designing CAD Models Based on Forgings Based on this, an initial twin model is constructed. ;

[0063] S52. Improve the ICP registration algorithm using feature constraints. The objective function is:

[0064]

[0065] In the formula, The number of key feature points. For feature point weights, Let be a rotation matrix. It is a translation vector. For real-time model feature points, For the corresponding feature points of the twin model; the rotation matrix satisfies, ;

[0066] S53, Timing Deformation Synchronization Formula:

[0067] In the formula, For Long Short-Term Memory (LSTM) networks, virtual-real synchronization is delayed. .

[0068] Furthermore, the intelligent analysis of virtual and real deviations and the optimization of process closed loop in S6 adopt the following methods:

[0069] S61. Formula for single vertex size deviation:

[0070] S62. Deviation Level Judgment Rules:

[0071] Normal, Warning, Exceeding Standard

[0072] In the formula, The warning threshold is set with a tolerance of 80%. For exceeding the threshold, a tolerance of 100% is used;

[0073] S63. Closed-loop optimization formula for process parameters:

[0074] Adjusted process parameters:

[0075] In the formula, For parameter adjustment amount, To optimize the gain matrix, This represents the average deviation; when it exceeds the standard... Execute shutdown protection.

[0076] Support System Basic Description

[0077] The method of this invention relies on a lightweight, high-temperature-adaptive hardware acquisition system, which serves only as data input and computing power support, without any hardware structural innovation. The system includes: a high-temperature anti-interference image acquisition unit (equipped with a narrowband filter component and a passive cooling shell to acquire monocular / binocular raw images and temperature data), a multi-source process synchronous acquisition unit (interfacing with a PLC to acquire parameters such as forging pressure and hammer stroke), and an edge computing unit (with a built-in GPU and deep learning inference engine to deploy algorithms and execute formula calculations).

[0078] Beneficial effects of the invention

[0079] Rigorous theoretical support and strong interpretability: Deep integration of deep learning algorithms with core mathematical formulas, from network training and image inpainting to virtual-real registration and closed-loop control, each step is quantitatively expressed, solving the "black box" problem of traditional deep learning algorithms and improving the rigor and scalability of the technical solution.

[0080] Outstanding resistance to high temperature interference: Through the attention mechanism formula and temperature adaptability loss formula, the network can accurately shield high temperature heat radiation and strong light overexposure interference, stably adapt to the entire temperature range of 800℃-1250℃, and achieve reconstruction integrity ≥95%.

[0081] Accuracy and real-time performance are quantifiable and controllable: By using the reconstruction accuracy formula, inference delay formula and synchronization delay formula, the single-frame reconstruction delay is controlled to ≤80ms, the reconstruction accuracy is ≤±0.08mm, and the virtual-real synchronization delay is ≤150ms, which fully meets the needs of industrial sites.

[0082] Closed-loop optimization is precise and efficient: Based on the deviation quantification formula and the process optimization formula, the adaptive fine-tuning of process parameters is realized, and the forging size qualification rate is increased from 92% to over 99.6%, significantly reducing the scrap rate and realizing pre-quality control. Attached Figure Description

[0083] Figure 1 This is a flowchart illustrating the overall implementation process of the method of the present invention.

[0084] Figure 2 This is a diagram illustrating the deep learning 3D reconstruction network architecture and core formulas of this invention. Detailed Implementation

[0085] The present invention will be further described in detail below with reference to specific embodiments. This embodiment is implemented based on the technical solution of the present invention and provides a complete operation process and formula calculation examples. However, the protection scope of the present invention is not limited to the following embodiments.

[0086] Example: Deep learning reconstruction and dynamic twin monitoring of large aerospace high-temperature alloy forgings

[0087] Application: High-temperature alloy die forgings for a certain aero-engine, forging temperature 1050℃, workpiece size Φ500mm×150mm, produced with a 3000T die forging press, design dimensional tolerance ±0.3mm.

[0088] Method Implementation Process

[0089] 1. Network Construction and Training

[0090] Constructing a dual-branch reconstruction network, in the channel attention mechanism It is a 1×1 convolution kernel. The convolution kernel is 1×1, and the mixed dataset contains 50,000 samples. The composite loss function is... , , After 50 rounds of pre-training and 10 rounds of real-world fine-tuning, the pruning quantification rate was achieved. Single-frame inference latency This meets the real-time requirements.

[0091] 2. Image Preprocessing and Feature Enhancement

[0092] Camera intrinsic matrix for After distortion correction of the original image, the GAN inpainting algorithm is used to repair overexposed areas and restore the lost image. Output feature-enhanced image There are no obvious defects or overexposure.

[0093] 3. End-to-end 3D reconstruction

[0094] Will Input network, output 3D mesh model ( (Number of vertices). The calculated reconstruction accuracy is... Reconstructing integrity This meets the accuracy requirements.

[0095] 4. Multi-source data fusion

[0096] Collection cycle Temperature detection point Number of process samplings Temperature data is mapped to vertices using a spatiotemporal registration formula, generating a full-feature vector. This enables precise correlation between geometric, temperature, and process data.

[0097] 5. Dynamic Twin Modeling and Synchronization

[0098] Design Model Based on this, key feature points are selected. One, optimize the rotation matrix using the feature-constrained ICP registration formula. With translation vector After registration, the objective function value converges to Timing deformation synchronization uses an LSTM network, with virtual-real synchronization delay. Real-time mapping of forging deformation of forgings.

[0099] 6. Deviation Analysis and Closed-Loop Optimization

[0100] Preset warning threshold Exceeding the threshold The average deviation is calculated in real time. This triggers an alert. The gain matrix needs to be optimized. Adjustment amount of process parameters Adjusted process parameters Subsequent reconstruction deviation It has returned to the normal range.

[0101] Implementation effect

[0102] In this embodiment, the method integrates deep learning algorithms and mathematical formulas to stably adapt to high-temperature conditions of 1050℃. The reconstruction accuracy and real-time performance meet industrial requirements. The dimensional qualification rate of forgings is increased from 92% to 99.6%, and the scrap rate is reduced by 80%. This enables contactless, intelligent, and quantifiable monitoring and control of high-temperature forgings throughout the entire process.

Claims

1. A method for real-time dynamic digital twin of high-temperature forgings based on deep learning 3D reconstruction, characterized in that, Includes the following steps: S1. Construction and transfer training of deep learning 3D reconstruction network: A dual-branch lightweight network is built for the characteristics of high-temperature forging images. Training optimization and lightweight processing are completed by combining core mathematical formulas to adapt to high-temperature working conditions. S2. Deep learning preprocessing and feature enhancement of high temperature images: geometric distortion correction and overexposure area repair of the original acquired images are performed, and feature weighting enhancement is completed through attention mechanism. S3, Deep Learning End-to-End Real-Time 3D Reconstruction, inputs feature-enhanced images into the trained network and directly outputs a high-precision 3D mesh model, and evaluates the reconstruction effect through a quantization formula; S4. Multi-source data temporal fusion and spatiotemporal alignment: accurately correlate and fuse 3D reconstruction data, temperature field data, and forging process parameters based on timestamps. S5. Dynamic digital twin modeling and real-time virtual-real synchronization: Based on the design CAD model, the virtual-real model is aligned by improving the registration algorithm, realizing the construction of a time-series dynamic twin and millisecond-level synchronization. S6. Intelligent analysis of virtual and real deviations and closed-loop optimization of processes: quantifies the dimensional deviation between the twin model and the design model, completes hierarchical early warning, and combines optimization formulas to achieve adaptive adjustment of process parameters or shutdown protection.

2. The method for real-time dynamic digital twin of high-temperature forgings based on deep learning 3D reconstruction according to claim 1, characterized in that, The construction and transfer training of the deep learning 3D reconstruction network in S1 are as follows: S11. Construct an improved dual-branch lightweight deep learning 3D reconstruction network. The feature extraction branch adopts a lightweight ResNet network with an embedded channel attention mechanism (SE module). The core attention weight calculation formula is as follows: ; In the formula, For channel attention weights, It is the Sigmoid activation function. This is a global average pooling operation. This is an intermediate feature map. , These are the weight matrices of the fully connected layer; the attention-weighted feature map is as follows: ; In the formula, This is an element-wise multiplication operation. The effective feature map after filtering out high-temperature interference; S12, the 3D shape inference branch adopts a lightweight encoder-decoder structure, outputting a set of vertex coordinates of the 3D mesh model. The vertex reasoning mapping relationship is as follows: ; In the formula, For encoder feature compression operation, For the decoder's 3D coordinate reconstruction operation, This represents the total number of grid vertices. S13. A multi-dimensional composite loss function is used to complete network training. The formula for the loss function is: ; In the formula, , , To determine the loss weight, this invention takes... , , ; Geometric loss (mean squared error loss) formula: ; In the formula, To accurately annotate vertex coordinates in 3D, It is an L2 norm; Model smoothing loss formula: ; In the formula, As vertices The Laplace operator; Temperature adaptability loss formula: ; In the formula, This represents the number of feature points in the high-temperature region. To predict the characteristics of high-temperature regions for network purposes, Mark the characteristics of high-temperature areas in the real scene; S14. Perform pruning, quantization, and lightweight optimization on the network. The formula for parameter compression ratio is: ; In the formula, For parameter compression ratio, this invention takes , These are the parameters after pruning. Original parameter values; optimized single-frame inference latency. .

3. The method for real-time dynamic digital twin of high-temperature forgings based on deep learning 3D reconstruction according to claim 1, characterized in that, The deep learning preprocessing and feature enhancement of high-temperature images in S2 are as follows: S21. Based on the camera intrinsic parameter matrix, perform geometric distortion correction on the original image. The pixel coordinate mapping formula is: ; In the formula, These are the original distorted pixel coordinates. For the corrected pixel coordinates, For the camera intrinsic parameter matrix, This is the inverse distortion correction function. The distance from a pixel to the center of the image; S22. Generative adversarial networks are used to repair overexposed and missing areas in the image. The formula for generating the repaired image is: ; In the formula, For generator networks, For the missing region mask, The corrected image; the inpainting loss formula is: ; In the formula, For L1 loss, To be a true, complete image. To counteract the loss of weight, To generate adversarial losses, For discriminator networks; S23. Enhance images through attention-weighted output features. This completes the effective feature enhancement.

4. The method for real-time dynamic digital twin of high-temperature forgings based on deep learning 3D reconstruction according to claim 1, characterized in that, The specific implementation of end-to-end real-time 3D reconstruction using deep learning in S3 is as follows: The feature-enhanced image is input into the trained network, which outputs a 3D mesh model end-to-end. The mapping relationship is as follows: ; In the formula, To reconstruct the complete two-branch network, For mesh patch generation operations, A collection of mesh faces; Reconstruction accuracy quantification formula: ; Formula for restoring integrity: ; In the formula, The number of valid vertices. This represents the total number of vertices.

5. The method for real-time dynamic digital twin of high-temperature forgings based on deep learning 3D reconstruction according to claim 1, characterized in that, The specific steps of multi-source data temporal fusion and spatiotemporal alignment in S4 are as follows: S41. Data synchronization is completed using a unified high-precision timestamp. Data acquisition cycle formula: ; In the formula, For the collection period, , The start and end times for data collection. This represents the total number of data collections. S42. Formula for mapping temperature data to 3D vertex space: ; In the formula, This is a projection function from a 3D vertex to a 2D pixel. The temperature value corresponding to the vertex; Multi-source data fusion feature vector: ; In the formula, These are the process parameters at the corresponding time points.

6. The method for real-time dynamic digital twin of high-temperature forgings based on deep learning 3D reconstruction according to claim 1, characterized in that, The dynamic digital twin modeling and real-time virtual-real synchronization in S5 are specifically as follows: S51, Designing CAD Models Based on Forgings Based on this, an initial twin model is constructed. ; S52. Improve the ICP registration algorithm using feature constraints. The objective function is: ; In the formula, The number of key feature points. For feature point weights, Let be a rotation matrix. It is a translation vector. For real-time model feature points, For the corresponding feature points of the twin model; the rotation matrix satisfies, ; S53, Timing Deformation Synchronization Formula: ; In the formula, For Long Short-Term Memory (LSTM) networks, virtual-real synchronization is delayed. .

7. The method for real-time dynamic digital twin of high-temperature forgings based on deep learning 3D reconstruction according to claim 1, characterized in that, The intelligent analysis of virtual and real deviation and the closed-loop optimization of the process in S6 are as follows: S61. Formula for single vertex size deviation: ; S62. Deviation Level Judgment Rules: Normal, Warning, Exceeding Standard ; In the formula, The warning threshold is set with a tolerance of 80%. For exceeding the threshold, a tolerance of 100% is used; S63. Closed-loop optimization formula for process parameters: ; Adjusted process parameters: ; In the formula, For parameter adjustment amount, To optimize the gain matrix, This represents the average deviation; when it exceeds the standard... Execute shutdown protection.

8. A real-time dynamic digital twin system for high-temperature forgings based on deep learning 3D reconstruction, characterized in that, The method described in any one of claims 1-7 includes a data acquisition unit, a data processing unit, and an output unit; The data acquisition unit is used to acquire images of high-temperature forgings, temperature field data, and forging process parameters. Data processing unit: Completes deep learning 3D reconstruction network training, image preprocessing, 3D reconstruction, multi-source data fusion, dynamic twin modeling, deviation analysis and process optimization calculations; Output unit: Visualizes and transmits 3D models, dynamic twins, deviation results, early warning information and optimization instructions.

Images