Processing process spatio-temporal feature fusion prediction visualization method

Abstract

The invention discloses a processing process spatial-temporal feature fusion prediction visualization method. According to the method, mechanism modeling is carried out on a real physical world through a digital twin system, and spatial dimension feature extraction in the machining process is achieved. On the basis of interpretability of a digital twinning system in a processing process, dimension reduction processing of twinning data is realized. The real-time spatial expression of the shape and position precision of the workpiece is obtained by reconstructing the three-dimensional characteristics of the workpiece in the machining process in the spatial dimension. On the basis, real-time twin data are input into an LSTM neural network model, and processing feature extraction of the time dimension is achieved. And advanced prediction of the machining precision of the manufacturing system is realized on a time sequence. Through the fusion mode of combining the time dimension and the spatial features driven by the DT-LSTM, the prediction precision and the generalization ability of the digital twin model are improved, the calculation cost of the neural network model is greatly reduced, and real-time monitoring and advanced prediction of the processing process are realized.

Description

Technical Field

[0001] The present invention belongs to the field of real-time monitoring and accuracy prediction of precision machining processes, and relates to a method for predicting and visualizing the spatiotemporal features of machining processes that is collaboratively driven by digital twins and deep learning. Background Art

[0002] Smart manufacturing will shift toward highly dynamic, small-batch, high-precision, and flexible customized production. Traditional mass production lacks the flexibility required to meet increasingly personalized customer demands. This trend requires the coordinated operation of various moving parts of machine tools and the dynamic scheduling of production processes. This also brings the dual challenges of spatial and temporal transfer, influenced by cross-process and cross-machine characteristics. In flexible manufacturing systems, the increasing complexity and scale of equipment make machine tool operation and machining processes more susceptible to machining errors. Machining errors often arise from dynamic randomness, multi-source uncertainty, high coupling, and strong interference factors. Therefore, it is extremely difficult to perform multi-source error data fusion and achieve rapid error diagnosis in various machining scenarios. Traditional machining process detection technologies face significant challenges in handling data imbalance, dynamic spatial relationships, and temporal dependencies.

[0003] In the spatial dimension, real-world manufacturing systems exhibit significant spatial correlation, and product processing accuracy is often evaluated using spatial data such as form and position accuracy. Existing methods, such as "Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks" (Cui et al., 2019: 2090-2096), utilize convolutional neural networks (CNNs) to understand spatial relationships in scenes and have achieved significant success in extracting Euclidean spatial features. However, existing models often struggle to fully capture intrinsic spatial features. Because data from a single or a few sensors is limited, it cannot fully reflect the operating state of the equipment, and it ignores the spatial correlation between the operations of different actuators, making it impossible to effectively capture the significant features between different states of a complex system.

[0004] In the time dimension, the previous process of a product during processing often has a direct impact on the subsequent processes. For example, residual stress, surface hardening degree, subsurface damage, etc. are closely related to the previous process. In industrial production, a large number of interconnected sensors are usually used to generate a large amount of time series data. With the increase in the complexity and spatiotemporal dimensions of sensor data, the use of effective machine learning methods for anomaly detection modeling has become a hot topic. Early researchers used methods such as recurrent neural networks (RNNs) and temporal convolutional networks (TCNs) to predict the time dimension, but these methods did not integrate the time features of different dimensions. To alleviate this problem, the existing "A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder", Park et al., 2018, 3(3):1544-1551, replaced the feedforward neural network in the Variational Autoencoder with a long short-term memory (LSTM) network, effectively integrating the two models. The existing "Robust anomaly detection for multivariate time series through temporal GCNs and attention-based VAE" (Shi et al., 2023, 275) proposes an attention-based reconstruction model that uses LSTM to emphasize the importance of variables changing over time and capture temporal dependencies. However, traditional anomaly detection methods do not fully consider the temporal dimension, resulting in poor performance on time series data. Summary of the Invention

[0005] This invention aims to overcome the challenges of shifting dual spatiotemporal feature domains during machining processes, comprehensively exploring and integrating multidimensional spatiotemporal features within trajectory sequences. To address the current challenges of poor representation and computational efficiency in collaborative modeling of spatiotemporal features, a method and system for fusion prediction and visualization of machining process spatiotemporal features driven by digital twins and deep learning is developed. This method enables real-time collaborative fusion modeling of spatiotemporal features within machining processes. This method uses a digital twin system to model the physical world, extracting spatial features of the machining process. Building on the interpretability of the digital twin system, it achieves dimensionality reduction of machining data. By reconstructing the three-dimensional features of the workpiece machining process in the spatial dimension, a real-time spatial representation of the workpiece's geometric and positional accuracy is obtained. Based on this, the real-time machining data is input into an LSTM neural network model to extract machining features in the temporal dimension. This method enables advanced prediction of machining accuracy for manufacturing systems based on time series. This DT-LSTM-driven fusion of temporal and spatial features improves the prediction accuracy and generalization capability of the digital twin model, significantly reduces the computational cost of the neural network model, and enables real-time monitoring and advanced prediction of the machining process, laying the foundation for precise control of manufacturing systems.

[0006] The technical solution of the present invention:

[0007] A method for predicting and visualizing the spatiotemporal features of machining processes, driven by digital twins and deep learning, addresses the significant challenges faced by traditional machining process detection technologies in dealing with data imbalance, dynamic spatial relationships, and time dependencies. This method uses a digital twin system to create a real-time mapping between physical and digital spaces, reducing the dimensionality of real-time data. It also uses a long short-term memory (LSTM) model to extract the temporal features of machining data, ultimately enabling real-time visual monitoring and advanced prediction of the entire machining process. The specific steps are as follows:

[0008] Step 1: Establish a DT-LSTM system for the processing process;

[0009] This paper is based on the five-dimensional model of the digital twin system of the machining process. On this basis, in order to achieve real-time fusion of the spatiotemporal characteristics of the workpiece machining process, the machine tool is used as the smallest independent system and the workpiece machining process is used as the research object, and the digital twin system is extended to the workpiece machining process.

[0010] Step 1: Establish a digital twin system (DT) for CNC machine tools. The main moving parts of CNC machine tools are divided into linear motion parts and rotary motion parts. The linear motion parts include the X-axis feed system and the Z-axis feed system, and the rotary motion parts include the C-axis rotation system and the G-axis rotation system. The main moving parts of the CNC machine tool are dynamically modeled to form a digital twin system DT = {X, Z, C, G} for the machining process. Based on the established digital twin system, starting from the initial machining moment of the workpiece on the physical platform of the machine tool, real-time machining data is synchronously collected and stored to establish a historical database of the workpiece machining process, from time 0 to time t-1.

[0011] Step 2: Use the digital twin system to collect historical processing data from time 0 to t-1 and real-time processing data at the current time t, and synchronously map and construct a digital twin system with physical meaning. Classify, merge, and simplify the real-time processing data according to the corresponding physical structure of the digital twin system. The main moving parts of the CNC machine tool are divided into linear motion parts and rotary motion parts. The processing data with the same motion form are merged: for the X-axis feed system and Z-axis feed system with linear motion form, they are merged and simplified to processing data containing length scalars. For the C-axis rotation system and G-axis rotation system with rotational motion form, they are merged and simplified to processing data containing angle scalars, realizing physical dimensionality reduction of the real-time processing data of the workpiece.

[0012] Step 3: Extract the spatial features of the real-time machining data at the current time t in the digital twin system after physical dimensionality reduction, obtain the spatial expression of the workpiece quality evaluation index, and realize the real-time visualization of the workpiece machining accuracy; the workpiece quality evaluation index includes surface waviness, surface roughness, shape accuracy and position accuracy;

[0013] Step 4: Use the long short-term memory (LSTM) model to reconstruct the time series of historical processing data from time 0 to t-1 and the real-time processing data at the current time t, extract the time dimension features of the real-time processing data at the current time t, and predict the processing status. The predicted processing data of the workpiece after the prediction time step t+1 is obtained and synchronously transmitted back to the digital twin system to obtain the distribution form of the processing data points of the workpiece within the prediction time step in the three-dimensional space coordinate system. The workpiece quality is evaluated based on the distribution form of the processing data points in the three-dimensional space coordinate system and the corresponding advanced control parameters are generated.

[0014] Step 5: In step 1, a digital twin system was established under the premise of ensuring data interpretability, which can extract spatial dimension features of processing process information and perform physical dimensionality reduction of data scale; in step 4, the historical processing data and real-time processing data of the digital twin system are input into the long short-term memory LSTM model to realize the time dimension feature extraction of historical processing data and real-time processing data, and the long short-term memory LSTM model is trained, predicted and verified based on the historical processing data and real-time processing data, ensuring the prediction accuracy of the long short-term memory LSTM on the basis of real-time performance; the prediction parameters of the long short-term memory LSTM model are input into the digital twin system, and the corresponding control parameters are sent to the CNC system; based on the real-time monitoring of the digital twin system and the interpretability of historical processing data and real-time processing data, a DT-LSTM system is established, and the spatial physical information and time series data of the processing process are fused and collaboratively processed in real time through DT and LSTM, so as to seamlessly integrate physical knowledge with data;

[0015] The established DT-LSTM system sends prediction and control parameters to the machine tool, directly controlling the workpiece machining process to ensure stable workpiece machining quality and simultaneously improve machining accuracy and efficiency.

[0016] A modeling method for a predictive visualization system based on the fusion of spatiotemporal features of machining processes, driven by the collaboration of digital twins and deep learning, has been developed. At the DT-LSTM application layer, it was discovered that workpiece machining accuracy is closely related to the motion of the machine tool's primary moving components. The workpiece machining process is essentially the material removal that occurs through the interaction between the workpiece and the cutting tool. The workpiece is connected to the machine bed via the "workpiece-C-axis-Z-axis" connection, while the cutting tool is connected to the bed via the "tool-grinding-axis-X-axis" connection. Therefore, in theory, closely monitoring the motion of the machine tool's primary moving components can achieve real-time monitoring of the workpiece machining process.

[0017] Therefore, a digital twin system of the CNC grinding machine's main moving parts, including the X, Z, C, and G axes, was established through entity mapping. Real-time machining data enables a one-to-one correspondence between the digital twin system and the physical entities in the real world, achieving a virtual-to-physical control and a symbiotic effect.

[0018] Step 2: Establish a long short-term memory (LSTM) model for the digital twin system;

[0019] Although the processing data is directly derived from the physical modeling of the machine tool's mechanical structure, and each processing data has direct and significant physical meaning, in multi-source heterogeneous real-time processing data, due to the independence of each physical quantity feature, the processing data feature dimensions of each moving part are dispersed and discrete in time and space, so the typical features of the sensor signal cannot be directly recognized and extracted by the digital twin system. The actual processing process has a strong time correlation. The previous process often affects the subsequent processing, and in the same process, the workpiece surface is formed by the envelope of the processing trajectory. Therefore, under the conditions of a small range of process parameter changes and relatively stable processing, the corresponding processing data often has a certain time series correlation. Based on this, LSTM can be used to process time series data to capture the time dependency within the sequence. By identifying and retaining long-term patterns in the sequence, the LSTM prediction layer significantly enhances the prediction accuracy of the DT-LSTM system. The construction steps based on the DT-LSTM prediction layer are as follows:

[0020] Step 1: LSTM unit basic framework;

[0021] During the manufacturing process, products are ultimately obtained from raw material through a series of processes and steps. Historical processing data during the manufacturing process influences the current processing status of the workpiece. This historical processing data facilitates real-time monitoring and high-precision prediction. Therefore, a method for fusing spatiotemporal features of the machining process, driven by digital twins and deep learning, is proposed to achieve real-time monitoring and predictive control of manufacturing systems. Based on a real-time interactive simplified physical model established by the digital twin system, an LSTM network is used to implement machining process diagnosis based on historical processing data and current processing status.

[0022] A method for fusing spatiotemporal features of machining processes driven by digital twins and deep learning is proposed to achieve real-time monitoring and predictive control of manufacturing systems. A long short-term memory (LSTM) model is used to enable machining process diagnosis based on real-time machining data and current machining status in a historical database.

[0023] The core components of the LSTM model are the sequence input layer and the LSTM layer. Real-time processing data is the time series data that is fed into the LSTM model. The sequence input layer feeds the corresponding time series data into the LSTM layer, which then learns the long-term correlations between the time steps of the time series data. The LSTM layer contains multiple LSTM units, which are constructed for the real-time processing data of the X-axis feed system, Z-axis feed system, C-axis rotation system, and G-axis rotation system. An LSTM unit is composed of multiple storage cells, each of which has three gating mechanisms: a forget gate, an input gate, and an output gate.

[0024] The learnable weights of the LSTM layer include the input weight W, the recurrent weight R, and the bias b; the LSTM layer constructs the concatenation matrix according to the following equation:

[0025] (1)

[0026] Among them, i, f, o represent the input gate, forget gate, and output gate respectively, g represents the candidate memory unit; the memory unit state c at time t t and the hidden unit state h t They are given by the following formulas:

[0027] (2)

[0028] Where ⊙ represents the Hadamard product, σ c represents the state activation function; the hidden unit state h t The output of the LSTM layer containing this time step; the corresponding input gate, forget gate, candidate memory unit and output gate calculation are given by the following formula:

[0029] (3)

[0030] Among them, x t is the input at time t, that is, the real-time processing data of the digital twin system at time t; the historical input data x0~x0 are processed by the memory unit states and hidden unit states corresponding to i, f, o, g at time 0~t. t The operation of the corresponding time series data is obtained by the output data h0~h t , h t That is the predicted input data for the next moment;

[0031] Use the tanh function as the state activation function:

[0032] (4)

[0033] In formula (3), σ x represents the gate activation function;

[0034] Use the Sigmoid function as the gate activation function:

[0035] (5)

[0036] Step 2: Adaptation of DT and LSTM units;

[0037] The main moving parts of a CNC machine tool include the X-axis feed system, the Z-axis feed system, the C-axis rotation system, and the G-axis rotation system; the corresponding five-dimensional model of the digital twin system is shown as follows:

[0038] (6)

[0039] Among them, X represents the X-axis feed system, Z represents the Z-axis feed system, C represents the C-axis or spindle rotation system, and G represents the G-axis rotation system; the physical entity PE, virtual entity VM, service SS, processing data DD, and connection CN are the corresponding expressions of the main moving parts of the CNC machine tool in the digital twin system;

[0040] Motion components of the same type have structural and dynamic similarities. Therefore, the digital twin system simplifies the data representation structure of CNC machine tools in the real physical world based on physical similarities. That is, the data of the X-axis feed system and the Z-axis feed system are uniformly processed by one LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of the X-axis feed system and the Z-axis feed system; the data of the C-axis rotation system and the G-axis rotation system are uniformly processed by another LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of the C-axis rotation system and the G-axis rotation system:

[0041] (7)

[0042] Among them, LSTM-L represents the LSTM layer of the linear motion component in the digital twin system, LSTM-R represents the LSTM layer of the rotational motion component in the digital twin system, and X d (t), Z d (t), C d (t) are real-time processing data corresponding to the X-axis, Z-axis, C-axis or spindle, including position signal, power signal and feedback signal; the output is the predicted data within the prediction time step corresponding to the X-axis, Z-axis, C-axis or spindle;

[0043] First, the initial state of the LSTM unit and the first time step of the time series data are used to calculate the first output and the updated unit state (c1, h1). Specifically, at time t, the current unit state of the LSTM unit (c t−1 , h t−1 ) and the next time step of the time series data to calculate the output and the updated unit state (c t , h t ); the unit state of the current layer is determined by the hidden unit state h t and the memory cell state c t Composition; hidden unit state h at time t t Contains the output of the LSTM layer at that moment; the memory cell state c t Contains information obtained from the previous moment; at each Czech, the current layer will be in the memory cell state c t Add or remove information in the layer; the current layer uses different gates to control these updates;

[0044] Based on this, the operation process of real-time processing data in LSTM in the digital twin system is established, and the real-time connection of processing data in different processes, steps, and work strokes based on historical processing information is obtained, which can realize real-time monitoring and processing prediction of the processing process.

[0045] Step 3: Training the prediction layer in the DT-LSTM architecture;

[0046] On this basis, the DT-LSTM system is trained online. The digital twin system includes the X-axis feed system, Z-axis feed system, and C-axis rotation system of the CNC machine tool. Based on the data structure of the digital twin system, a three-channel long short-term memory (LSTM) model is set. The number of hidden units in the LSTM layer determines how much information the layer learns. Using more hidden units produces more accurate results, but also increases the risk of overfitting the training data. Therefore, the LSTM layer is set to have 128 hidden units. A higher dropout probability helps improve the model's generalization ability, but at the cost of information loss and slowing the learning process. Therefore, the dropout layer helps prevent overfitting by randomly setting the layer's input to zero, effectively changing the network architecture between training iterations. The DT-LSTM model is trained, validated, and tested. During monitoring, real-time machining data is compared with LSTM-predicted data. When training data is out of tolerance, retraining is performed, and the digital twin system performs corresponding training operations based on the returned data. The workpiece machining accuracy is predicted within an acceptable training accuracy range, and process control parameters are adjusted according to the product's process accuracy requirements.

[0047] Training was performed using Adam optimization with a learning rate of 0.001. To mitigate overfitting and underfitting, the number of training iterations was set to 1000. After iterative training, the performance of the best validation and training models on the test set was compared, and the model with the highest test accuracy and lowest test loss was selected as the final model. This approach effectively reduced the impact of the training iteration setting on model performance. Using this strategy, the optimal number of training iterations for the DT-LSTM model was determined.

[0048] Step 4: Transmission of real-time processed data in the DT-LSTM system;

[0049] During workpiece machining, the main moving parts of a CNC machine tool are controlled using a closed-loop servo feedback system based on a grating scale. This involves sending CNC instructions to a programmable controller (PLC) via an industrial computer. The PLC then transmits the corresponding control signals to the servo drivers of the main moving parts of the CNC machine tool. The servo drivers then transmit the power signals to the corresponding linear and rotary motion components, driving them to produce corresponding movements. The grating scale compares the real-time positions of the linear and rotary motion components with their ideal positions, and transmits the error as a feedback signal to the motion controller of the PLC. The corresponding control algorithm is then executed based on the corresponding feedback error to reduce the real-time error.

[0050] By extracting real-time processing data and connecting it to the LSTM layer, advanced prediction of processing quality can be achieved. The real-time processing data transmitted back by the DT-LSTM system includes power signals, position signals, and feedback signals, which can be expressed as:

[0051] (8)

[0052] Among them, X Pow 、X Fee 、X Pos Represent the real-time power signal, feedback signal and position signal of the X axis respectively; Z Pow , Z Fee , Z Pos Represent the real-time power signal, feedback signal and position signal of the Z axis respectively; C Pow 、C Fee 、C Pos Represent the real-time power signal, feedback signal and position signal of the C-axis respectively;

[0053] Real-time power signals enable accurate identification of machining status, while position and feedback signals reflect the shape and position accuracy of the workpiece. The DT-LSTM system's real-time machining data from time 0 to time t-1 is transmitted as historical data to the LSTM layer. The LSTM layer outputs a predicted value based on the corresponding forget gate and update gate, and compares it with the real-time machining data at time t to determine the current prediction accuracy. If the prediction accuracy meets the algorithm requirements, the prediction value after time t+1 is further calculated and transmitted to the industrial computer. The industrial computer adjusts the machining parameters based on the predicted value.

[0054] Step 3: Dimensionality reduction of real-time processed data within the DT-LSTM system;

[0055] The prediction and visualization method of spatiotemporal feature fusion of machining processes driven by digital twins and deep learning can improve data processing efficiency. The improvement in processing efficiency is shown in the following formula:

[0056] (9)

[0057] Where T is the current process, M is the number of moving parts corresponding to the workpiece processing process, N is the real-time data category of a single moving part, and t is the processing time step; m is the number of moving parts after dimensionality reduction, n is the real-time data category of a single moving part after dimensionality reduction, and s is the processing time;

[0058] Step 4: Real-time processing data spatiotemporal feature fusion visualization method based on DT-LSTM system;

[0059] In the machining of cylindrical surfaces, the machining path with time as the variable presents a spiral-up real-time machining trajectory. In cylindrical surface machining, the X-axis, Z-axis, C-axis, and G-axis each have their own physical meanings in the digital twin system. The X-axis represents the cutting depth, corresponding to the controlled diameter of the cylindrical surface, and is stationary during a single cylindrical surface machining. The Z-axis corresponds to the machining length of the cylindrical surface, and the machining trajectory is uniform linear motion. The C-axis movement directly corresponds to the roundness of the cylindrical surface, and is a uniform rotational motion during cylindrical surface machining. The G-axis corresponds to the machining parameters and is a uniform rotational motion.

[0060] Construct the machine tool space coordinate system and the digital twin system space coordinate system respectively in the machine tool mechanical coordinate system, where the coordinate origin and X, Y, and Z axes of the machine tool space coordinate system and the digital twin system space coordinate system correspond one-to-one to the machine tool mechanical coordinate system respectively;

[0061] On the cylindrical surface, the collected real-time processing data points of the workpiece are subjected to dimensionality reduction processing, and the cylindrical surface is projected into the xOz, xOy and yOz planes of the machine tool space coordinate system in the form of three views. The xOz and yOz planes are the composite motion forms of the x-axis, y-axis and z-axis respectively, showing a typical sine curve shape. The xOy plane is the composite motion form of the x-axis and y-axis, which is an ideal circular trajectory. The two-dimensional spatial data is reduced to a three-axis motion form represented by x, y, and z. Since x, y, and z are actually virtual space axes generated by the X, Z, and C motion axes, they are represented by axes 1, 2, and 3 respectively. The motion forms of axis 1 and axis 2 are both sinusoidal motion, and the phase difference between the two is 90 degrees. Axis 1 and axis 2 correspond to the motion of the processing point on the cylindrical surface, while axis 3 is a linear motion, representing the motion of the processing point on the cylindrical surface along the generatrix direction. Based on this, the spatiotemporal correspondence between the physical motion axes X, Z, and C and the virtual space axes x, y, and z in the machine tool digital twin system is established:

[0062] (10);

[0063] Step 5: Processing data prediction method based on DT-LSTM system;

[0064] Based on the dimensionality reduction method for real-time processing data within the DT-LSTM system established in the third step, the one-dimensional processing data is mapped to a three-dimensional space, and subsequent processing data is predicted. The processing data prediction process based on the DT-LSTM system is divided into three steps.

[0065] Step 1 generates the subsequent prediction data h1p,… of step 1 based on the current experimental data 0-h1t. As the processing progresses, the subsequent prediction data of step 1 is gradually replaced by the subsequent experimental data h1t+1,… of step 1. On this basis, the LSTM unit parameters in formula 1 are verified based on the subsequent experimental data h1t+1,… of step 1 and the prediction data of step 1. When the error does not exceed the prediction setting threshold δ t When predicting step 2 based on the subsequent experimental data h1 t+1, ... of step 1, the predicted data h2 p, ... of step 2 are generated;

[0066] As the processing progresses, the predicted data in step 2 is replaced by the experimental data h2 t+1, ..., and the LSTM unit parameters in Equation 1 are verified accordingly; if the error exceeds the prediction set threshold δ t Execute the corresponding control process, combine the experimental data h2 t+1, ... of step 2 to re-predict step 2, iterate the parameter weight of formula 1, and regenerate the predicted data h2 p, ... of step 2. Finally, combine the current experimental data of step 2 to generate the predicted data h3 p, ... of step 3.

[0067] Repeat the above history-prediction-verification steps, and finally generate the workpiece surface processing data in real time along with the entire workpiece processing process;

[0068] The trained DT-LSTM system is used to predict real-time processing data. By reconstructing the predicted data in space, the predicted workpiece surface processing trajectory is obtained. On this basis, the historical processing data and real-time processing data of the entire processing process are trained, predicted, and verified. The above steps are repeated to achieve training, prediction, and verification of the processing data.

[0069] Beneficial effects of the present invention: In response to the current problems of poor representation capabilities and low computational efficiency of collaborative modeling of spatiotemporal features of machining processes, the present invention invents a method and system for fusion prediction and visualization of spatiotemporal features of machining processes driven by digital twins and deep learning, which realizes real-time collaborative fusion modeling of spatiotemporal features of machining processes. Through DTLSM, real-time fusion and collaborative processing of physical information and neural networks are realized, and physical knowledge and data can be seamlessly integrated. Interpretability is reflected in the unified architecture modeling of digital twins of machine tool moving parts through physical models, which improves the interpretability of data on the basis of ensuring a one-to-one correspondence between data and physical models. Deep learning neural networks, on the basis of simplifying the model and data structure, realize accurate prediction of the machining process through the processing of historical data and key real-time data, providing new ideas for advanced control of the machining process. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a system architecture for predicting and visualizing the spatiotemporal features of the processing process, driven by the collaboration of digital twins and deep learning.

[0071] Figure 2 This is a structural diagram of the LSTM neural network based on the digital twin architecture.

[0072] Figure 3 This is the principle behind the DT-LSTM collaboratively driven spatiotemporal feature fusion method for CNC grinding machine machining. (a) is the closed-loop servo feedback system of the machine tool used in ultra-precision machining; (b) is the transmission pattern of the machine tool's internal control signals within the LSTM layer.

[0073] Figure 4 It is a digital twin-driven spatiotemporal feature fusion method for machining processes. (a) represents the representation of machining data in the temporal and spatial domains; (b) represents the physical meaning of the machining data.

[0074] Figure 5 Figure 1 shows the root mean square error (RMS) and loss curves of the DT-LSTM architecture during iterative training. (a) shows the RMS error curve during DT-LSTM architecture training, and (b) shows the loss curve during DT-LSTM architecture training.

[0075] Figure 6 This is how the DT-LSTM architecture predicts and applies spatiotemporal features within a single time step. On the left: Twin prediction data after DT-LSTM training; on the right: Spatial representation of the DT-LSTM-based advanced twin prediction data.

[0076] Figure 7This is the fusion and application of spatiotemporal features of the DT-LSTM architecture in the machining process. (a) shows the first step of the machining process driven by the DT-LSTM architecture: training, prediction, and verification of processing data; (b) shows the second step of the machining process driven by the DT-LSTM architecture: training, prediction, and verification of processing data; and (c) shows the third step of the machining process driven by the DT-LSTM architecture: training, prediction, and verification of processing data.

[0077] Figure 8 The DT-LSTM predicts and reconstructs the spatial dimensions of workpiece processing data at different time steps. (a) shows the prediction and reconstruction of the spatial dimensions of the workpiece processing data in the first two time steps; (b) shows the prediction and reconstruction of the spatial dimensions of the workpiece processing data in the first three time steps; and (c) shows the prediction and reconstruction of the spatial dimensions of the workpiece processing data by the DT-LSTM during the workpiece processing process.

[0078] Figure 9 This is a comparative verification of the DT-LSTM prediction results for workpiece shape and position accuracy across all processes. (a) is the offline planar measurement of the workpiece's roundness; (b) is the real-time prediction of the workpiece's roundness by DT-LSTM; (c) is the spatial measurement of the workpiece's cylindricity; (d) is the real-time prediction of the workpiece's cylindricity by DT-LSTM; and (e) is the real-time prediction of the workpiece's shape and position accuracy by DT-LSTM. DETAILED DESCRIPTION

[0079] The specific implementation of the present invention is described in detail below in conjunction with the technical solutions and drawings.

[0080] Figure 1 This is a processing process spatiotemporal feature fusion prediction and visualization system driven by digital twins and deep learning. The specific steps of system modeling are as follows:

[0081] Step 1: Establish a DT-LSTM system for the processing process.

[0082] The DT-LSTM architecture established by the present invention is as follows Figure 1 As shown in the figure, based on the five-dimensional digital twin model of the machining process, on this basis, in order to realize the real-time fusion of the spatiotemporal characteristics of the machining process, the machine tool is taken as an independent minimum system and the complete machining process of the workpiece is taken as the research object, and the digital twin system is further expanded to the entire process of the workpiece machining process. The specific steps are as follows:

[0083] Step 1: Establish a digital twin system (DT) for CNC machine tools. The main moving parts of CNC machine tools are divided into linear motion parts and rotary motion parts. The linear motion parts include the X-axis feed system and the Z-axis feed system, and the rotary motion parts include the C-axis rotation system and the G-axis rotation system. The main moving parts of the CNC machine tool are dynamically modeled to form a digital twin system DT = {X, Z, C, G} for the machining process. Based on the established digital twin system, starting from the initial machining moment of the workpiece on the physical platform of the machine tool, real-time machining data is synchronously collected and stored to establish a historical database of the workpiece machining process, from time 0 to time t-1.

[0084] Step 2: Use the digital twin system to collect historical processing data from time 0 to t-1 and real-time processing data at the current time t, and synchronously map and construct a digital twin system with physical meaning. Classify, merge, and simplify the real-time processing data according to the corresponding physical structure of the digital twin system. The main moving parts of the CNC machine tool are divided into linear motion parts and rotary motion parts. The processing data with the same motion form are merged: for the X-axis feed system and Z-axis feed system with linear motion form, they are merged and simplified to processing data containing length scalars. For the C-axis rotation system and G-axis rotation system with rotational motion form, they are merged and simplified to processing data containing angle scalars, realizing physical dimensionality reduction of the real-time processing data of the workpiece.

[0085] Step 3: Extract the spatial features of the real-time machining data at the current time t in the digital twin system after physical dimensionality reduction, obtain the spatial expression of the workpiece quality evaluation index, and realize the real-time visualization of the workpiece machining accuracy; the workpiece quality evaluation index includes surface waviness, surface roughness, shape accuracy and position accuracy;

[0086] Step 4: Use the long short-term memory (LSTM) model to reconstruct the time series of historical processing data from time 0 to t-1 and the real-time processing data at the current time t, extract the time dimension characteristics of the real-time processing data at the current time t, and predict the processing status. Obtain the predicted processing data of the workpiece after the prediction time step t+1 and synchronously transmit it back to the digital twin system to obtain the distribution form of the processing data points of the workpiece within the prediction time step in the three-dimensional space coordinate system. Evaluate the workpiece quality based on the distribution form of the processing data points in the three-dimensional space coordinate system and generate corresponding advanced control parameters.

[0087] Step 5: In step 1, a digital twin system was established under the premise of ensuring data interpretability, which can extract spatial dimension features of processing process information and perform physical dimensionality reduction of data scale; in step 4, the historical processing data and real-time processing data of the digital twin system are input into the long short-term memory LSTM model to realize the time dimension feature extraction of historical processing data and real-time processing data, and the long short-term memory LSTM model is trained, predicted and verified based on the historical processing data and real-time processing data, ensuring the prediction accuracy of the long short-term memory LSTM on the basis of real-time performance; the prediction parameters of the long short-term memory LSTM model are input into the digital twin system, and the corresponding control parameters are sent to the CNC system; based on the real-time monitoring of the digital twin system and the interpretability of historical processing data and real-time processing data, a DT-LSTM system is established, and the spatial physical information and time series data of the processing process are fused and collaboratively processed in real time through DT and LSTM, so as to seamlessly integrate physical knowledge with data;

[0088] Based on this, the present invention takes CNC grinding machine as an example and proposes a three-layer modeling structure of the DT-LSTM architecture, namely, the application layer, the modeling layer, and the prediction layer. Figure 1 As shown, the flow and control of processing data are realized, and the workpiece processing process is directly controlled to ensure the stability of workpiece processing quality and simultaneously improve processing accuracy and processing efficiency.

[0089] A modeling method for a machining process prediction and visualization system based on the fusion of spatiotemporal features, driven by the collaboration of digital twins and deep learning. At the application layer of the DT-LSTM architecture, it was found that workpiece machining accuracy is closely related to the motion of the machine tool's primary moving components. The workpiece machining process is essentially the material removal that occurs through the interaction between the workpiece and the cutting tool. The workpiece is connected to the machine bed via the "workpiece-C-axis-Z-axis" connection, while the cutting tool is connected to the bed via the "tool-grinding-axis-X-axis" connection. Therefore, in theory, closely monitoring the motion of the machine tool's primary moving components can achieve real-time monitoring of the workpiece machining process.

[0090] Therefore, at the modeling layer, a digital twin system of the CNC grinder's main moving components, including the X, Z, and C axes and the grinding axis, was established through entity mapping. Real-time technician data enabled a one-to-one correspondence between the digital twin system and the physical entities in the real world, achieving a virtual-to-real world symbiosis. The modeling and application layers, combined with price data, connectivity, and corresponding services, form the universal five-dimensional digital twin model used in conventional digital twins.

[0091] Step 2: Establish a long short-term memory (LSTM) neural network for the digital twin system.

[0092] Although the processing data is directly derived from the physical modeling of the machine tool's mechanical structure, and each processing data has direct and significant physical meaning, in multi-source heterogeneous real-time processing data, due to the independence of each physical quantity feature, the processing data feature dimensions of each moving part are dispersed and discrete in time and space, so the typical features of the sensor signal cannot be directly recognized and extracted by the digital twin system. The actual processing process has a strong time correlation. The previous process often affects the subsequent processing, and in the same process, the workpiece surface is formed by the envelope of the processing trajectory. Therefore, under the conditions of a small range of process parameter changes and relatively stable processing, the corresponding processing data often has a certain time series correlation. Based on this, LSTM can be used to process time series data to capture the time dependency within the sequence. By identifying and retaining long-term patterns in the sequence, the LSTM prediction layer significantly enhances the prediction accuracy of the DT-LSTM system. The construction steps based on the DT-LSTM prediction layer are as follows:

[0093] Step 1: LSTM unit basic framework;

[0094] During the manufacturing process, products are ultimately obtained from raw material through a series of processes and steps. Historical data from the machining process influences the current state of the workpiece. This historical processing information facilitates real-time monitoring and high-precision prediction. Therefore, a method for fusing spatiotemporal features of machining processes, driven by digital twins and deep learning, is proposed to achieve real-time monitoring and predictive control of manufacturing systems. Based on a real-time interactive simplified physical model established in the digital twin system, an LSTM network is used to implement machining process diagnosis based on historical processing data and current machining status.

[0095] A method for fusing spatiotemporal features of the machining process driven collaboratively by digital twins and deep learning is proposed to realize real-time monitoring and predictive control of the manufacturing system; the long short-term memory (LSTM) model is used to realize machining process diagnosis based on real-time machining data and current machining status in the historical database.

[0096] LSTM is a special type of recurrent neural network (RNN). Its core components are the sequence input layer and the LSTM layer. The sequence input layer inputs time series data into the neural network, and the LSTM layer learns the long-term correlation between the time steps of the sequence data. Figure 2 As shown in Figure 1, an LSTM unit consists of multiple memory cells, each of which contains three gating mechanisms: a forget gate, an input gate, and an output gate, as well as a cell state for maintaining long-term information. LSTM effectively controls the flow of information by allowing information to be selectively retained, forgotten, or output. This allows for the long-term retention of important information and prevents interference from irrelevant information. The basic components, functions, and calculation formulas of an LSTM unit are shown in Table 1:

[0097] Table 1 Basic components and functions of LSTM unit

[0098] Components Function Calculation formula Input gate (i) Controls the level of cell status updates <![CDATA[i t =σ g (W i xt+R i h t−1 +b i )]]> Forget gate (f) Controls the level of cell state reset (forgetting) <![CDATA[f t =σ g (W f xt+R f h t−1 +b f )]]> Candidate unit (g) Adding information to the cell state <![CDATA[g t =σ c (W g xt+R g h t−1 +b g )]]> Output gate (o) Controls the level of cell state added to the hidden state <![CDATA[o t =σ g (W o xt+R o h t−1 +b o )]]>

[0099] The learnable weights of the LSTM layer include the input weights W (InputWeights), the recurrent weights R (RecurrentWeights), and the bias b (Bias). The matrices W, R, and b represent the input weights, recurrent weights, and bias of each component, respectively. This layer constructs the concatenated matrix according to the following equation:

[0100] (11)

[0101] Among them, i, f, o represent the input gate, forget gate, and output gate respectively, g represents the candidate memory unit; the memory unit state c at time t t and the hidden unit state h t They are given by the following formulas:

[0102] (12)

[0103] Where ⊙ represents the Hadamard product, σ c represents the state activation function; the hidden unit state h t The output of the LSTM layer containing this time step; the corresponding input gate, forget gate, candidate memory unit and output gate calculation are given by the following formula:

[0104] (13)

[0105] Among them, x t is the input at time t, that is, the real-time processing data of the digital twin system at time t; the historical input data x0~x0 are processed by the memory unit states and hidden unit states corresponding to i, f, o, g at time 0~t. t The operation of the corresponding time series data is obtained by the output data h0~h t , h t That is the predicted input data for the next moment;

[0106] The hyperbolic tangent function (tanh) is used as the state activation function. The output range of the hyperbolic tangent function is (−1, 1) and it has the characteristic of zero centering, so the tanh function is used to calculate the state activation function:

[0107] (14)

[0108] σ xRepresents the gate activation function. The Sigmoid function maps the real number domain to the (0,1) interval through nonlinear transformation. Its output value has probabilistic meaning. Here, the Sigmoid function is used to calculate the gate activation function:

[0109] (15)

[0110] Step 2: Adaptation of DT and LSTM units;

[0111] In order to make LSTM better suitable for DT system, it is necessary to adapt LSTM according to the digital twin system. The adaptation process is as follows Figure 3 As shown, since a grinder has an additional grinding axis compared to a lathe, this paper uses an ultra-precision grinder as an example to illustrate the digital twin system structure of the machining process established in this invention. The main moving parts of an ultra-precision grinder are two linear motion axes (X and Z axes) and two rotational motion axes (spindle / C axis and grinding axis). The corresponding five-dimensional model of the digital twin system is shown below:

[0112] (16)

[0113] Here, X represents the X-axis linear feed system, Z the Z-axis linear feed system, C the C-axis rotary motion system, and G the grinding axis rotary motion system. The physical entity (PE), virtual model (VM), service system (SS), digital twin data (DD), and connection (CN) represent the corresponding representations of the machine tool's main moving components within the digital twin system.

[0114] Motion components of the same type have structural and dynamic similarities. Therefore, the digital twin system simplifies the data representation structure of CNC machine tools in the real physical world based on physical similarities. That is, the data of the X-axis feed system and the Z-axis feed system are uniformly processed by one LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of the X-axis feed system and the Z-axis feed system; the data of the C-axis rotation system and the G-axis rotation system are uniformly processed by another LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of the C-axis rotation system and the G-axis rotation system:

[0115] (17)

[0116] Among them, LSTM-L represents the LSTM layer of the linear motion component in the digital twin system, LSTM-R represents the LSTM layer of the rotational motion component in the digital twin system, and X d (t), Z d (t), Cd (t) are real-time processing data corresponding to the X-axis, Z-axis, C-axis or spindle, including position signal, power signal and feedback signal; the output is the predicted data within the prediction time step corresponding to the X-axis, Z-axis, C-axis or spindle;

[0117] First, the initial state of the LSTM unit and the first time step of the time series data are used to calculate the first output and the updated unit state (c1, h1). Specifically, at time t, the current unit state of the LSTM unit (c t−1 , h t−1 ) and the next time step of the time series data to calculate the output and the updated unit state (c t , h t ); the unit state of the current layer is determined by the hidden unit state h t and the memory cell state c t Composition; hidden unit state h at time t t Contains the output of the LSTM layer at that moment; the memory cell state c t Contains information obtained from the previous moment; at each Czech, the current layer will be in the memory cell state c t Add or remove information in the layer; the current layer uses different gates to control these updates;

[0118] Based on this, the operation process of real-time processing data in LSTM in the digital twin system is established, and the real-time connection of processing data in different processes, steps, and work strokes based on historical processing information is obtained, which can realize real-time monitoring and processing prediction of the processing process.

[0119] Step 3: Training the prediction layer in the DT-LSTM architecture;

[0120] On this basis, the DT-LSTM model is trained online. The corresponding LSTM structure is shown in Table 2. It consists of four layers: sequence input layer, LSTM layer, dropout layer, and fully connected layer, with a total of 67.9k parameters.

[0121] Table 2 Prediction layer structure in DT-LSTM architecture

[0122]

[0123] As shown in Table 2, the LSTM is set to three channels based on the established DT data structure. The number of hidden units in the LSTM layer determines how much information the layer has learned. Using more hidden units produces more accurate results, but it is also more likely to cause overfitting of the training data. Therefore, the LSTM layer is set to have 128 hidden units. A higher dropout probability helps improve the generalization ability of the model, but at the cost of losing information and slowing down the learning process. Therefore, the dropout layer helps avoid overfitting by randomly setting the input of this layer to zero and effectively changing the network architecture between training iterations. Based on the structure detailed in Table 2, the DT-LSTM model was trained, verified, and tested. During the monitoring process, the real-time processing data is compared with the LSTM predicted data. When the training data is out of tolerance, retraining is performed. The digital twin system performs corresponding training operations based on the returned data, such as Figure 3 As shown in (b) in the figure, the workpiece machining accuracy is predicted within the appropriate range of training accuracy, and the process control parameters are adjusted according to the product process accuracy requirements.

[0124] Adam optimization is used for training, and the learning rate is set to 0.001. In order to mitigate overfitting or underfitting, this example sets the training iteration to 1000 and adopts a specific strategy. Validation is performed every fifty iterations to monitor model performance, and the model with the best validation accuracy and training accuracy is retained. After iterative training, the performance of the best validation and training models on the test set is compared, and the model with the highest test accuracy and lowest test loss is selected as the final model. This method effectively reduces the impact of the training iteration setting on model performance. The accuracy and loss curves of the training iterations are shown as follows: Figure 5 The results show that during the first 300 training iterations, both training and validation loss decreased rapidly, while training and validation accuracy increased rapidly. Subsequently, the training loss continued to slowly decrease to 0.033, while the validation loss stabilized at around 0.028. The training root mean square error (RMSE) continued to slowly decrease to 0.169, while the validation RMSE stabilized at around 0.183. The mean root mean square error was 0.178. Using this strategy, the optimal training iteration number for the DT-LSTM model was determined to be 400.

[0125] Step 4: Transmission of processed data in the DT-LSTM system;

[0126] The control method of the main moving parts of the machine tool in ultra-precision machining mostly adopts a closed-loop servo feedback system based on the grating scale, such as Figure 3As shown in the figure, an industrial personal computer (IPC) sends CNC instructions to a programmable multi-axis controller (PMAC). The PMAC transmits the corresponding control signals to the servo drivers of the corresponding moving parts. The servo drivers then transmit the power signals to the corresponding motion mechanisms, driving them to produce the desired motion. A linear encoder compares the real-time position of the motion mechanism with its ideal position and transmits the error as feedback to the PMAC motion controller. The PMAC motion controller then executes the corresponding control algorithm based on the feedback error to reduce the real-time error and improve control accuracy.

[0127] The established DT-LSTM architecture can monitor the machining process conveniently and accurately. By extracting the machining data and connecting it to the LSTM layer, the machining quality can be predicted in advance, such as Figure 3 By extracting real-time processing data and connecting it to the LSTM layer, the processing quality can be predicted in advance. The real-time processing data sent back by the DT-LSTM system includes power signals, position signals, and feedback signals, which can be expressed as:

[0128] (18)

[0129] Among them, X Pow 、X Fee 、X Pos Represent the real-time power signal, feedback signal and position signal of the X axis respectively; Z Pow , Z Fee , Z Pos Represent the real-time power signal, feedback signal and position signal of the Z axis respectively; C Pow 、C Fee 、C Pos Represent the real-time power signal, feedback signal and position signal of the C-axis respectively;

[0130] Real-time power signals enable accurate identification of machining status, while position and feedback signals reflect the shape and position accuracy of the workpiece. The DT-LSTM system's real-time machining data from time 0 to time t-1 is transmitted as historical data to the LSTM layer. The LSTM layer outputs a predicted value based on the corresponding forget gate and update gate, and compares it with the real-time machining data at time t to determine the current prediction accuracy. If the prediction accuracy meets the algorithm requirements, the prediction value after time t+1 is further calculated and transmitted to the industrial computer. The industrial computer adjusts the machining parameters based on the predicted value.

[0131] If the prediction accuracy meets the algorithm's requirements, the predicted value (time t+1 to...) is further calculated and transmitted to the IPC. The IPC then adjusts the processing parameters appropriately based on the predicted value. Therefore, the LSTM network established in this invention includes a phase in which real-time processing data verifies the LSTM's current prediction based on historical data, a phase in which the LSTM network predicts future processing data in advance, and a phase in which the current processing state is pre-adjusted based on this advance prediction data.

[0132] Step 3: Dimensionality reduction of processed data within the DT-LSTM system;

[0133] The DT-LSTM real-time architecture established in this invention simplifies the physical structure. By physically isolating the data stream, it can reduce the dimensionality of the real-time data stream structure, thereby improving processing efficiency and monitoring accuracy. The processing process fault diagnosis method driven by the digital twin and deep learning in this invention improves processing efficiency. The processing efficiency improvement is shown in the following formula:

[0134] (19)

[0135] Where T represents the current process, M represents the number of moving parts corresponding to the workpiece machining process, N represents the real-time data category for a single moving part, and t represents the machining time step. m represents the number of moving parts after dimensionality reduction, n represents the real-time data for a single moving part after dimensionality reduction, and t represents the machining moment. Taking the ultra-precision CNC grinding machine targeted by this invention as an example, the corresponding initial machining data includes: M(X, Z, C, M) = 4, N(pow, fee, pos, con) = 4. After the physical structure is simplified by the DT-LSTM system, the real-time machining data with temporal and spatial dimensions includes: m(X, Z, C) = 3, N(pow, fee, pos) = 2. By simplifying the M×N-dimensional real-time machining data to m×n-dimensional machining data, the data size is reduced by 62%. This makes the processing of machining data within the same time step more efficient and concise.

[0136] The established real-time digital twin architecture for the machining process significantly reduces modeling and computational costs, improves real-time data transmission during the machining process, and establishes a real-time connection between machining data and electrical data at different moments through physical models. At the machine tool level, the real-time digital twin architecture for the main moving components enables real-time mapping of machine tool motion characteristics, such as current loops, voltage loops, velocity loops, and acceleration loops. The proposed dimensionality reduction of machining data based on the DT-LSTM architecture goes beyond simple data screening and extraction, but instead relies on the virtual-real correspondence between the digital twin and the physical structure. By mapping real-time machining data with temporal and spatial dimensions into machining data within the DTS, the autocorrelation and cross-correlation of the machining data are improved, establishing a complete symmetric relationship between the temporal and spatial mapping. Furthermore, while historical machining data with only a single dimension, such as time or space, such as vibration and temperature, does not participate in subsequent computations in the LSTM prediction layer, it is still synchronously computed within the DT system through the physical model, enabling real-time monitoring of the current machining process.

[0137] Step 4: Real-time processing data spatiotemporal feature fusion visualization method based on DT-LSTM system;

[0138] The present invention describes a method and system for predicting and visualizing the spatiotemporal features of a machining process that is collaboratively driven by digital twins and deep learning. Its interpretability lies in the unified architecture modeling of the digital twins of machine tool moving parts and physical models through the DT-LSTM system, which improves the interpretability of the data while ensuring a one-to-one correspondence between the data and the physical model.

[0139] The third step of dimensionality reduction and simplification of processing data not only greatly reduces the modeling and computing costs of DT-LSTM and improves the real-time data transmission of the processing process, but also establishes a real-time connection between processing data and electrical data at different times through the physical model. At the machine tool level, the real-time digital twin architecture of the main moving parts realizes the real-time mapping of machine tool motion characteristics such as current loop, voltage loop, velocity loop and acceleration loop, thereby improving the interpretability of real-time processing data.

[0140] The DT-LSTM architecture established by the present invention can conveniently and intuitively realize the prediction of spatiotemporal feature processing data of the workpiece processing process. In the processing of cylindrical surfaces, the processing path with time as the variable presents a spiral rising real-time processing trajectory; in the processing of cylindrical surfaces, the X-axis, Z-axis, C-axis, and G-axis each have their own physical meanings in the digital twin system. The X-axis is the cutting depth, corresponding to the controlled diameter of the cylindrical surface, and is stationary in the processing of a single cylindrical surface; the Z-axis corresponds to the processing length of the cylindrical surface, and the processing trajectory is uniform linear motion; the C-axis motion directly corresponds to the roundness of the cylindrical surface, and is a uniform rotational motion in cylindrical surface processing; the G-axis corresponds to the processing parameters and is a uniform rotational motion;

[0141] Construct the machine tool space coordinate system and the digital twin system space coordinate system respectively in the machine tool mechanical coordinate system, where the coordinate origin and X, Y, and Z axes of the machine tool space coordinate system and the digital twin system space coordinate system correspond one-to-one to the machine tool mechanical coordinate system respectively;

[0142] Cylindrical surface machining is a relatively simple surface machining case, which can be regarded as a single single-axis feed motion. Here, the digital twin-driven machining process spatiotemporal feature machining data prediction method established by the present invention is briefly described using cylindrical surface machining as an example. In the machining of cylindrical surfaces, the machining path with time as the variable presents a spiral rising real-time machining trajectory, such as Figure 4 (a) in the figure. The spiral represents the corresponding machining trajectory, and the circles represent the PMAC motion control points. In cylindrical surface machining, the X, Z, C, and M axes each have their own physical meaning in DT. The X axis represents the depth of cut, corresponding to the controlled diameter of the cylindrical surface, and remains stationary during a single cylindrical surface machining pass. The Z axis corresponds to the machining length of the cylindrical surface, and the ideal machining trajectory is uniform linear motion. The C axis motion directly corresponds to the roundness of the cylindrical surface and is ideally a uniform rotational motion in cylindrical surface machining. The grinding axis corresponds to the machining parameters and is a uniform rotational motion.

[0143] The above-mentioned moving parts of the machine tool are the X, Z, C axis systems inside the machine tool, and do not represent the x, y, z spatial coordinates of the workpiece. Even for simple cylindrical surface processing, it is not possible to accurately and intuitively distinguish the direct relationship between the movement of each axis system of the machine tool and the corresponding processing errors of each axis. The DT-LSTM system established by the present invention can solve this problem. Taking the cylindrical surface as an example, the three-dimensional space data points are processed for dimensionality reduction, and the cylindrical surface is projected into the xOz, xOy and yOz planes in the form of three views, as shown in FIG. Figure 4 As shown in (a) in the figure. It is found that the xOz and yOz planes are similar, and are the composite motion forms of the x-axis and y-axis with the z-axis, showing a typical sine curve shape. The xOy plane is the composite motion form of the x-axis and y-axis, which is an ideal circular trajectory. Further reduce the two-dimensional spatial data to a three-axis motion form represented by x, y, and z, as shown in Figure 4 As shown in (b) in the figure. Since x, y, z are actually virtual space axes generated by the X, Z, C motion axes, they are represented by axes 1, 2, and 3 respectively. It is found that the motion forms of the x-axis (axis 1) and the y-axis (axis 2) are similar, both are sinusoidal motions, and the phase difference between the two is 90 degrees. Axis 1 and axis 2 correspond to Figure 4 In (a), the processing point moves on the cylindrical surface, while the z axis (3 axis) is a linear motion, representing Figure 4 The motion of the machining point on the cylindrical surface along the generatrix in (a) is shown. Based on this, the spatiotemporal correspondence between the physical motion axes X, Z, C and the virtual space axes x, y, z in the machine tool digital twin system is established:

[0144] (20)

[0145] Use the trained DT-LSTM architecture to predict the processed data. The model training process is as follows: Figure 5 As shown, the prediction results are Figure 6 As shown. The solid line before time t1 is the actual historical processing data, the dotted line after time t1 is the processing data predicted by the DT-LSTM architecture, and the solid line after time t1 is the real-time processing data. It can be seen that the processing data predicted by the DT-LSTM architecture is highly consistent with the measured results, and the prediction of processing data is well achieved on the time scale. In addition, the DT-LSTM established based on the present invention can generate the spatial data structure of the workpiece in real time, such as Figure 6 As shown in the image on the right, the predicted machining data is spatially reconstructed to obtain the predicted workpiece surface machining trajectory. The solid line before time t1 represents actual historical machining data, the asterisks after time t1 represent the machining data predicted by the DT-LSTM architecture, and the solid line after time t1 represents real-time machining data. This demonstrates that the DT-LSTM architecture established in this paper can effectively predict and reconstruct the spatiotemporal characteristics of machining data.

[0146] Figure 6 It is to reconstruct the spatiotemporal features of the processing data within a single time step, and on this basis, train, predict and verify the processing data of the entire processing process. The principle is as follows Figure 7 shown. Figure 7 (a) in the figure shows exactly Figure 6 The process of predicting the spatiotemporal features of machining data within a single time step. Time t1 marks the beginning of workpiece machining. After time t2, when the workpiece machining state gradually stabilizes, the corresponding twin machining data is collected from times t2 to t4 and used as historical data for LSTM network training. The length of historical data collection is determined by the machining conditions and the twin model architecture. Starting at time t4, the machining process for the next time step can be predicted, for example, from times t4 to t6.

[0147] During the processing of the second time step, the predicted data of time t4-t6 are realized and verified with the predicted data of the first time step, such as Figure 7 As shown in (b) in the figure, based on the historical data collection length, the historical data from time t6 to time t3 is selected as the second time step for LSTM network prediction, and the predicted processed data for the third time step from time t6 to time t7 is generated. The historical data from time t2 to time t3 is stored in the processed data as long-term historical data.

[0148] Similarly, during the processing of the third time step, the predicted data at time t6-t7 are realized and verified with the predicted data in the second time step, as shown in Figure 7 As shown in (c) in the figure, based on the historical data collection length, the historical data from time t7 to time t5 is selected as the third time step for LSTM network prediction. This generates the predicted processed data for the fourth time step, from time t7 to time t8. The historical data from time t2 to time t5 is stored in the processed data as long-term historical data. Repeat the above steps to complete the training, prediction, and validation of the processed data.

[0149] Step 5: Processing data prediction method based on DT-LSTM system;

[0150] Based on the dimensionality reduction method for real-time processing data within the DT-LSTM system established in the third step, the one-dimensional processing data is mapped to a three-dimensional space, and subsequent processing data is predicted. The processing data prediction process based on the DT-LSTM system is divided into three steps.

[0151] like Figure 8 As shown in (a), the subsequent prediction data of step 1 (h1 p, ...) are generated based on the experimental data of step 1 (0-h1 t), which are shown as solid lines and asterisks in three-dimensional space. As the processing progresses, the subsequent prediction data of step 1 are gradually replaced by the subsequent experimental data of step 1 (h1 t+1, ...). On this basis, the LSTM algorithm is verified based on the subsequent experimental data of step 1 (h1 t+1, ...) and the prediction data of step 1. When the error does not exceed the prediction set threshold δ t When the subsequent experimental data of step 1 (h1 t+1, ...) are used to predict step 2, and the prediction data of step 2 (h2 p, ...) are generated, such as Figure 8 As shown in (a) in .

[0152] As the processing progresses, the predicted data of step 2 is replaced by the experimental data (h2 t+1,…), as shown by the solid line in 8 (a), and the LSTM algorithm is verified accordingly. If the error exceeds the prediction threshold δ t , and re-predict step 2 based on the experimental data of step 2 (h2 t+1,…), and generate the predicted data of step 2 (h2 p,…). Finally, the predicted data of step 3 (h3 p,…) is generated based on the current experimental data of step 2. Figure 8 The above history-prediction-verification steps are repeated, and finally the processing data of the workpiece surface is generated in real time during the entire processing process of the workpiece, as shown in (b). Figure 8 As shown in (c) in the figure, the asterisks are the predicted values ​​of the LSTM network, and the solid lines are the measured processing data traces.

[0153] Repeat the above history-prediction-verification steps, and finally generate the workpiece surface processing data in real time along with the entire workpiece processing process;

[0154] The trained DT-LSTM system is used to predict real-time processing data. By reconstructing the predicted data in space, the predicted workpiece surface processing trajectory is obtained. On this basis, the historical processing data and real-time processing data of the entire processing process are trained, predicted, and verified. The above steps are repeated to achieve training, prediction, and verification of the processing data.

[0155] In order to verify the accuracy of the DT-LSTM driven machining process prediction method, the present invention compares the error data of the entire workpiece with the offline measurement data, such as Figure 9 shown. Figure 9 (a) in the figure shows the offline measurement results of the workpiece. The PP values ​​of the five measurements are 1.04, 1.27, 1.19, 1.27 and 1.08 μm, respectively, with an average value of 1.17 μm. Figure 9 (b) in the figure is the machining error distribution predicted by the present invention based on DT-LSTM. The DT-LSTM predicted values ​​are completely distributed in a circle with a radius of 1.17 μm, which is consistent with the measured data.

[0156] Figure 9 (c) in the figure shows the distribution of five offline measurement data along the workpiece surface. Due to the offline measurement procedures and installation process, the measurement cost is high, so the offline measurement data is relatively limited. Figure 9 (d) in the figure shows the full data distribution of workpiece surface machining errors predicted by DT-LSTM. The data points are continuously recorded as the machining process progresses, with a high degree of accuracy and completeness. The machining data and the predicted quantity are significantly higher than those of offline measurements in terms of data breadth, abundance, and spatial distribution density. The accuracy is consistent with offline measurement data, and it has the real-time performance that offline measurement lacks. This allows for real-time monitoring of the machining process in both time and space dimensions. Figure 9 The above results fully verify the accuracy and convenience of the DT-LSTM proposed in this invention, and it is relatively convenient to realize the online monitoring and real-time prediction of workpiece processing quality.

[0157] The present invention provides a method and system for predicting and visualizing the fusion of spatiotemporal features of a machining process driven collaboratively by digital twins and deep learning. To address the current problems of poor collaborative modeling and representation capabilities and low computational efficiency of spatiotemporal features of machining processes, the present invention invents a method and system for predicting and visualizing the fusion of spatiotemporal features of a machining process driven collaboratively by digital twins and deep learning. The digital twin model simplifies the real-time data cost and improves the interpretability of machining data. The deep learning neural network realizes self-learning and self-processing of real-time data, realizes high-precision prediction of the machining process, improves the prediction accuracy and generalization ability of the digital twin model, greatly reduces the computational cost of the neural network model, realizes real-time monitoring and advanced prediction of the machining process, and is of great significance to the accurate prediction and regulation of the machining quality of precision manufacturing systems.

Claims

1. A method for predicting and visualizing the fusion of spatiotemporal features of a machining process, characterized by: Here are the steps: Step 1: Establish a DT-LSTM system for the processing process; In order to achieve real-time fusion of the spatiotemporal characteristics of the workpiece machining process, the machine tool is taken as the smallest independent system and the workpiece machining process is taken as the research object, and the digital twin system is extended to the workpiece machining process; Step 2: Establish a long short-term memory (LSTM) model for the digital twin system; Step 3: Dimensionality reduction of real-time processed data within the DT-LSTM system; Step 4: Real-time processing data spatiotemporal feature fusion visualization method based on DT-LSTM system; Step 5: Processing data prediction method based on DT-LSTM system.

2. The method for predicting and visualizing the spatiotemporal feature fusion of machining processes according to claim 1 is characterized in that: Step 1: Establish a DT-LSTM system for the processing process. The details are as follows: Step 1: Establish a digital twin system for CNC machine tools: The main moving parts of CNC machine tools are divided into linear motion parts and rotary motion parts. The linear motion parts include the X-axis feed system and the Z-axis feed system, and the rotary motion parts include the C-axis rotation system and the G-axis rotation system. The main moving parts of the CNC machine tool are dynamically modeled to form a digital twin system DT = {X, Z, C, G} for the processing process. Based on the established digital twin system, starting from the initial processing moment of the workpiece on the physical platform of the machine tool, real-time processing data is synchronously collected and stored to establish a historical database of the workpiece processing process, from time 0 to time t-1. Step 2: Use the digital twin system to collect historical processing data from time 0 to t-1 and real-time processing data at the current time t, and synchronously map and construct a digital twin system with physical meaning. Classify, merge, and simplify the real-time processing data according to the corresponding physical structure of the digital twin system. The main moving parts of the CNC machine tool are divided into linear motion parts and rotary motion parts. The processing data with the same motion form are merged: for the X-axis feed system and Z-axis feed system with linear motion form, they are merged and simplified to processing data containing length scalars. For the C-axis rotation system and G-axis rotation system with rotational motion form, they are merged and simplified to processing data containing angle scalars, realizing physical dimensionality reduction of the real-time processing data of the workpiece. Step 3: Extract the spatial features of the real-time machining data at the current time t in the digital twin system after physical dimensionality reduction, obtain the spatial expression of the workpiece quality evaluation index, and realize the real-time visualization of the workpiece machining accuracy; the workpiece quality evaluation index includes surface waviness, surface roughness, shape accuracy and position accuracy; Step 4: Use the long short-term memory (LSTM) model to reconstruct the time series of historical processing data from time 0 to t-1 and the real-time processing data at the current time t, extract the time dimension features of the real-time processing data at the current time t, and predict the processing status. The predicted processing data of the workpiece after the prediction time step t+1 is obtained and synchronously transmitted back to the digital twin system to obtain the distribution form of the processing data points of the workpiece within the prediction time step in the three-dimensional space coordinate system. The workpiece quality is evaluated based on the distribution form of the processing data points in the three-dimensional space coordinate system and the corresponding advanced control parameters are generated. Step 5: In step 1, a digital twin system was established under the premise of ensuring data interpretability, which can extract spatial dimension features of processing process information and perform physical dimensionality reduction of data scale; in step 4, the historical processing data and real-time processing data of the digital twin system are input into the long short-term memory LSTM model to realize the time dimension feature extraction of historical processing data and real-time processing data, and the long short-term memory LSTM model is trained, predicted and verified based on the historical processing data and real-time processing data, ensuring the prediction accuracy of the long short-term memory LSTM on the basis of real-time performance; the prediction parameters of the long short-term memory LSTM model are input into the digital twin system, and the corresponding control parameters are sent to the CNC system; based on the real-time monitoring of the digital twin system and the interpretability of historical processing data and real-time processing data, a DT-LSTM system is established, and the spatial physical information and time series data of the processing process are fused and collaboratively processed in real time through DT and LSTM, so as to seamlessly integrate physical knowledge with data; The established DT-LSTM system sends prediction parameters and control parameters to the machine tool, directly controlling the workpiece machining process to ensure the stability of the workpiece machining quality and simultaneously improve the machining accuracy and efficiency.

3. The method for predicting and visualizing the spatiotemporal feature fusion of machining processes according to claim 1, characterized in that: Step 2: Establish a long short-term memory (LSTM) model for the digital twin system. The details are as follows: Step 1: Basic framework of LSTM unit; A method for fusing spatiotemporal features of machining processes driven by digital twins and deep learning is proposed to achieve real-time monitoring and predictive control of manufacturing systems. A long short-term memory (LSTM) model is used to enable machining process diagnosis based on real-time machining data and current machining status in a historical database. The core components of the LSTM model are the sequence input layer and the LSTM layer. Real-time processing data is the time series data that is fed into the LSTM model. The sequence input layer feeds the corresponding time series data into the LSTM layer, which then learns the long-term correlations between the time steps of the time series data. The LSTM layer contains multiple LSTM units, which are constructed for the real-time processing data of the X-axis feed system, Z-axis feed system, C-axis rotation system, and G-axis rotation system. An LSTM unit is composed of multiple storage cells, each of which has three gating mechanisms: a forget gate, an input gate, and an output gate. The learnable weights of the LSTM layer include the input weight W, the recurrent weight R, and the bias b; the LSTM layer constructs the concatenation matrix according to the following equation: (1)； in, Represent the input gate, forget gate, and output gate respectively. Represents the candidate memory unit; the state of the memory unit at time t and hidden unit states They are given by the following formulas: (2)； in, represents the Hadamard product, represents the state activation function; hidden unit state The output of the LSTM layer containing this time step; the corresponding input gate, forget gate, candidate memory unit and output gate calculation are given by the following formula: (3)； in, is the input at time t, that is, the real-time processing data of the digital twin system at time t; through time 0~t The corresponding memory unit state and hidden unit state are proportional to the historical input data The operation is to get the output data under the corresponding time series data , That is the predicted input data for the next moment; Use the tanh function as the state activation function: (4)； In formula (3), represents the gate activation function; Use the Sigmoid function as the gate activation function: (5)； Step 2: Adaptation of DT and LSTM units; The main moving parts of a CNC machine tool include the X-axis feed system, the Z-axis feed system, the C-axis rotation system, and the G-axis rotation system; the corresponding five-dimensional model of the digital twin system is shown as follows: (6)； Among them, X represents the X-axis feed system, Z represents the Z-axis feed system, C represents the C-axis or spindle rotation system, and G represents the G-axis rotation system; the physical entity PE, virtual entity VM, service SS, twin data DD, and connection CN are the corresponding expressions of the main moving parts of the CNC machine tool in the digital twin system; Motion components of the same type have structural and dynamic similarities. Therefore, the digital twin system simplifies the data representation structure of CNC machine tools in the real physical world based on physical similarities. That is, the data of the X-axis feed system and the Z-axis feed system are uniformly processed by one LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of the X-axis feed system and the Z-axis feed system; the data of the C-axis rotation system and the G-axis rotation system are uniformly processed by another LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of the C-axis rotation system and the G-axis rotation system: (7)； Among them, LSTM-L represents the LSTM layer of the linear motion component in the digital twin system, and LSTM-R represents the LSTM layer of the rotational motion component in the digital twin system. 、 、 They are real-time processing data corresponding to the X-axis, Z-axis, C-axis or spindle, including position signal, power signal and feedback signal; the output is the predicted data within the prediction time step corresponding to the X-axis, Z-axis, C-axis or spindle; First, use the initial state of the LSTM unit and the first time step of the time series data to calculate the first output and the updated unit state , specifically: at time t, use the current cell state of the LSTM cell and the next time step of the time series data to calculate the output and updated unit state ; The unit state of the current layer is determined by the hidden unit state and memory cell status Composition; the hidden unit state at time t Contains the output of the LSTM layer at that moment; memory cell state Contains information obtained from the previous moment; at each moment, the current layer will be in the memory cell state Add or remove information in the layer; the current layer uses different gates to control these updates; Step 3: Training of LSTM in DT-LSTM system; The digital twin system includes the X-axis feed system, Z-axis feed system, and C-axis rotation system of the CNC machine tool. Based on the data structure of the digital twin system, the long short-term memory (LSTM) model is set to three channels. Adam optimization is used for training, with a learning rate of 0.001 and 1000 training iterations. After iterative training, the model with the highest test accuracy and lowest test loss is selected as the final long short-term memory (LSTM) model for the digital twin system. Step 4: Transmission of real-time processed data in the DT-LSTM system; During workpiece machining, the main moving parts of a CNC machine tool are controlled using a closed-loop servo feedback system based on a grating scale. This involves sending CNC instructions to a programmable controller (PLC) via an industrial computer. The PLC then transmits the corresponding control signals to the servo drivers of the main moving parts of the CNC machine tool. The servo drivers then transmit the power signals to the corresponding linear and rotary motion components, driving them to produce corresponding movements. The grating scale compares the real-time positions of the linear and rotary motion components with their ideal positions, and transmits the error as a feedback signal to the motion controller of the PLC. The corresponding control algorithm is then executed based on the corresponding feedback error to reduce the real-time error. By extracting real-time processing data and connecting it to the LSTM layer, advanced prediction of processing quality can be achieved. The real-time processing data transmitted back by the DT-LSTM system includes power signals, position signals, and feedback signals, which can be expressed as: (8)； in, Represent the real-time power signal, feedback signal and position signal of the X-axis respectively; Represent the real-time power signal, feedback signal and position signal of the Z axis respectively; Represent the real-time power signal, feedback signal and position signal of the C-axis respectively; Real-time power signals enable accurate judgment of the processing status, and position signals and feedback signals reflect the shape and position accuracy of the workpiece. The real-time processing data of the DT-LSTM system from time 0 to time t-1 is transmitted to the LSTM layer as historical data. The LSTM layer outputs a predicted value based on the corresponding forget gate and update gate, and compares it with the real-time processing data at time t to determine the current prediction accuracy. If the prediction accuracy meets the algorithm requirements, the prediction value after time t+1 is further calculated and transmitted to the industrial computer. The industrial computer adjusts the processing parameters based on the predicted value.

4. The method for predicting and visualizing the spatiotemporal feature fusion of machining processes according to claim 1, characterized in that: Step 3: Dimensionality reduction of real-time data processing within the DT-LSTM system is as follows: The prediction and visualization method of spatiotemporal feature fusion of machining processes driven by digital twins and deep learning can improve data processing efficiency. The improvement in processing efficiency is shown in the following formula: (9)； Among them, T is the current process, M is the number of moving parts corresponding to the workpiece processing process, N is the real-time data category of a single moving part, and t is the processing time step; m is the number of moving parts after dimensionality reduction, n is the real-time data category of a single moving part after dimensionality reduction, and s is the processing time.

5. The method for predicting and visualizing the spatiotemporal feature fusion of machining processes according to claim 1 is characterized in that: Step 4: The specific method for fusion and visualization of spatiotemporal features of real-time processed data based on the DT-LSTM system is as follows: In the machining of cylindrical surfaces, the machining path with time as the variable presents a spiral-up real-time machining trajectory. In cylindrical surface machining, the X-axis, Z-axis, C-axis, and G-axis each have their own physical meanings in the digital twin system. The X-axis represents the cutting depth, corresponding to the controlled diameter of the cylindrical surface, and is stationary during a single cylindrical surface machining. The Z-axis corresponds to the machining length of the cylindrical surface, and the machining trajectory is uniform linear motion. The C-axis movement directly corresponds to the roundness of the cylindrical surface, and is a uniform rotational motion during cylindrical surface machining. The G-axis corresponds to the machining parameters and is a uniform rotational motion. Construct the machine tool space coordinate system and the digital twin system space coordinate system respectively in the machine tool mechanical coordinate system, where the coordinate origin and X, Y, and Z axes of the machine tool space coordinate system and the digital twin system space coordinate system correspond one-to-one to the machine tool mechanical coordinate system respectively; On the cylindrical surface, the collected real-time processing data points of the workpiece are processed by dimensionality reduction, and the cylindrical surface is projected into the machine tool space coordinate system in the form of three views. , and flat; and The planes are axis, Axis and The compound motion of the axis presents a typical sinusoidal shape; The plane is Axis and The compound motion form of the axis is an ideal circular trajectory; the two-dimensional spatial data is reduced to , , The three-axis motion form represented by , , The virtual space axes actually generated by the X, Z, and C motion axes are represented by axes 1, 2, and 3 respectively; the motion forms of axes 1 and 2 are both sinusoidal motions, and the phase difference between the two is 90 degrees; axes 1 and 2 correspond to the motion of the processing point on the cylindrical surface, while axis 3 is a linear motion, representing the motion of the processing point on the cylindrical surface along the generatrix direction; based on this, the physical motion axes X, Z, C and the virtual space axes are established in the digital twin system of the machine tool. , , The space-time correspondence between them: (10)。 6. The method for predicting and visualizing the spatiotemporal feature fusion of machining processes according to claim 1, characterized in that: Step 5: The processing data prediction method based on the DT-LSTM system is as follows: Based on the dimensionality reduction method for real-time processing data within the DT-LSTM system established in the third step, the one-dimensional processing data is mapped to a three-dimensional space, and subsequent processing data is predicted. The processing data prediction process based on the DT-LSTM system is divided into three steps. Step 1: Based on the current experimental data Generate subsequent prediction data for step 1 ,…; As the processing progresses, the subsequent prediction data of step 1 is gradually replaced by the subsequent experimental data of step 1 , ..., based on the subsequent experimental data of step 1 , ... and the predicted data in step 1 are used to check the LSTM unit parameters in formula 1. When the error does not exceed the prediction threshold According to the subsequent experimental data of step 1 , ... make predictions for step 2 and generate prediction data for step 2 ,…; As the machining process progresses, the predicted data in step 2 are replaced by the experimental data. , ..., and check the LSTM unit parameters in Equation 1 accordingly; if the error exceeds the prediction threshold Execute the corresponding control process, combined with the experimental data of step 2 , ...re-predict step 2, iterate the parameter weights of formula 1, and regenerate the prediction data of step 2 , ..., finally combine the current experimental data of step 2 to generate the predicted data of step 3 ,…; Repeat the above history-prediction-verification steps, and finally generate the workpiece surface processing data in real time along with the entire workpiece processing process; The trained DT-LSTM system is used to predict real-time processing data. By reconstructing the predicted data in space, the predicted workpiece surface processing trajectory is obtained. On this basis, the historical processing data and real-time processing data of the entire processing process are trained, predicted, and verified. The above steps are repeated to achieve training, prediction, and verification of the processing data.

Images