A machining process space-time feature fusion prediction visualization method

Abstract

The application discloses a kind of machining process space-time feature fusion prediction visualization method.The method is modeled to real physical world by digital twin system, realize the space dimension feature extraction of machining process.Based on the explainability of machining process digital twin system, realize the dimension reduction processing of twin data.Through the three-dimensional feature of workpiece machining process in space dimension reconstruction, the real-time space expression of workpiece shape and position accuracy is obtained.On this basis, real-time twin data is input into LSTM neural network model, realize the machining feature extraction in time dimension.The lead prediction of manufacturing system machining accuracy is realized in time series.The fusion mode of time dimension and space feature driven by DT-LSTM improves the prediction accuracy and generalization ability of digital twin model, greatly reduces the calculation cost of neural network model, realizes the real-time monitoring and lead prediction of machining process.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of real-time monitoring and precision prediction of precision machining process, and relates to a machining process spatiotemporal feature fusion prediction visualization method driven by digital twinning and deep learning. BACKGROUND

[0002] Intelligent manufacturing will shift to highly dynamic, small batch, high precision and flexible customized production. Traditional mass production lacks the flexibility required to meet customers' increasing individualization needs. This trend requires the coordinated operation of each moving part of the machine tool and the dynamic arrangement of the production process. This also brings the challenge of double spatiotemporal domain transfer affected by cross-process and cross-machine features. In flexible manufacturing systems, the complexity and scale of equipment are constantly upgrading, making it easier for machine tools to run and machining processes to have machining errors. Machining errors often result from dynamic randomness, multi-source uncertainty, high coupling and strong interference factors. Therefore, it is extremely difficult to perform multi-source error data fusion and achieve rapid error diagnosis in various machining scenarios. Traditional machining process detection technology faces great challenges in dealing with data imbalance, dynamic spatial relationships and time dependence.

[0003] In the spatial dimension, real-world manufacturing systems exhibit significant spatial correlation, and the machining precision of products is often evaluated through spatial data such as geometric and dimensional accuracy. Existing "Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks" Cui et al., 2019: 2090-2096 uses convolutional neural networks (CNN) to understand the spatial relationships in the scene and has achieved significant success in extracting Euclidean spatial features. However, existing models often struggle to fully capture the intrinsic spatial features. Due to the limited data from a single or a small number of sensors, the operating state of the equipment cannot be fully reflected, and the spatial correlation between different actuator operations is ignored, making it difficult to effectively obtain significant features between different states of complex systems.

[0004] In the time dimension, the previous process often has a direct impact on the subsequent process during the processing of the product, such as residual stress, surface hardening degree, subsurface damage, etc., which are closely related to the previous process. In industrial production, a large number of interconnected sensors are often used to generate a large amount of time series data. With the increasing complexity and spatio-temporal dimension of sensor data, using effective machine learning methods for anomaly detection modeling has become a hot topic. Early researchers used recurrent neural networks (RNN), temporal convolutional networks (TCN), etc. to make time dimension prediction, but these methods did not integrate different dimensional time features. To alleviate this problem, the existing "A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder", Park et al., 2018, 3(3): 1544-1551, replaces the feedforward neural network in the Variational Autoencoder with a long short-term memory (LSTM) network, effectively integrating the two models. The existing "Robust anomaly detection for multivariate time series through temporal GCNs and attention-based VAE", Shi et al., 2023, 275, proposes an attention-based reconstruction model that uses LSTM to emphasize the importance of variable changes over time and capture temporal dependencies. However, current traditional anomaly detection methods do not fully consider the time dimension, resulting in poor performance on time series data. SUMMARY

[0005] The present application aims to overcome the transfer challenge of the dual space-time feature domain of the processing process, comprehensively explore and integrate the multi-dimensional space-time features in the trajectory sequence, and solve the problems of poor collaborative modeling and low calculation efficiency of space-time features in the processing process. A digital twin and deep learning collaborative driving processing process space-time feature fusion prediction visualization method and system are invented, which realizes real-time collaborative fusion modeling of space-time features in the processing process. The method realizes the extraction of spatial dimension features of the processing process through the mechanism modeling of the real physical world by the digital twin system. On the basis of the interpretability of the digital twin system, the dimension reduction processing of the processing data is realized. By reconstructing the three-dimensional features of the workpiece processing process in the spatial dimension, the real-time spatial expression of the workpiece shape and position accuracy is obtained. On this basis, the real-time processing data is input into the LSTM neural network model, and the processing feature extraction in the time dimension is realized. The advanced prediction of the manufacturing system processing accuracy in the time sequence is realized. The fusion mode of the time dimension and the spatial feature driven by the DT-LSTM improves the prediction accuracy and generalization ability of the digital twin model, greatly reduces the calculation cost of the neural network model, realizes the real-time monitoring and advanced prediction of the processing process, and lays a foundation for the precise regulation and control of the manufacturing system.

[0006] The technical scheme of the present application:

[0007] A digital twin and deep learning collaborative driving processing process space-time feature fusion prediction visualization method, which aims at the great challenge faced by traditional processing process detection technology in dealing with data imbalance, dynamic spatial relationship and time dependence, creates real-time mapping between physical space and digital space through a digital twin system, reduces the dimension of real-time data, extracts the time dimension features of the processing data through a long short-term memory LSTM model, and finally realizes real-time visualization monitoring and advanced prediction of the entire processing process. The specific steps are as follows:

[0008] Step 1: Establish a DT-LSTM system for the processing process;

[0009] Based on the five-dimensional model of the digital twin system of the processing process, the digital twin system is expanded to the workpiece processing process to realize real-time fusion of space-time features in the workpiece processing process, taking the machine tool as the smallest independent system and the workpiece processing process as the research object.

[0010] Step one: Establishing a digital twin (DT) system of a numerical control machine (CNC): The main moving parts of the numerical control machine are divided into linear motion parts and rotary motion parts. The linear motion parts include the X-axis feeding system and the Z-axis feeding system. The rotary motion parts include the C-axis rotating system and the G-axis rotating system. The main moving parts of the numerical control machine are modeled dynamically to form a digital twin system DT = {X, Z, C, G} facing the machining process. Based on the established digital twin system, the initial machining time of the workpiece on the physical platform of the machine tool is taken as the starting point. Real-time machining data is collected and stored synchronously to establish a historical database of the workpiece machining process from 0 to t-1 time;

[0011] Step two: Collecting the historical machining data from 0 to t-1 time and the real-time machining data at the current time t through the digital twin system, and synchronously mapping and constructing a digital twin system with physical meaning. According to the corresponding physical structure of the digital twin system, the real-time machining data is classified, merged, and simplified. The main moving parts of the numerical control machine are divided into linear motion parts and rotary motion parts. The machining data with the same motion form is merged: for the X-axis feeding system and the Z-axis feeding system with linear motion form, the machining data containing length scalar is simplified; for the C-axis rotating system and the G-axis rotating system with rotary motion form, the machining data containing angle scalar is simplified, realizing the physical dimension reduction of the real-time machining data of the workpiece.

[0012] Step three: Extracting the spatial features of the real-time machining data at the current time t in the physically dimension-reduced digital twin system, obtaining the spatial expression of the workpiece quality evaluation index, and realizing the real-time visualization of the workpiece machining precision. The workpiece quality evaluation index includes surface waviness, surface roughness, shape accuracy, and position accuracy.

[0013] Step four: Using the long short-term memory (LSTM) model to reconstruct the time series of the historical machining data from 0 to t-1 time and the real-time machining data at the current time t, extracting the time dimension features of the real-time machining data at the current time t, and predicting the machining state. The predicted machining data of the workpiece at the time step t+1 is obtained and synchronously fed back to the digital twin system to obtain the distribution form of the machining data points of the workpiece in the three-dimensional coordinate system within the predicted time step. Based on the distribution form of the machining data points in the three-dimensional coordinate system, the workpiece quality is evaluated, and the corresponding advance control parameters are generated.

[0014] Step five: Step one establishes a digital twin system under the premise of ensuring data interpretability, which can extract spatial dimension features and physically reduce the data scale of the processing information; Step four inputs the historical processing data and real-time processing data of the digital twin system into the long short-term memory (LSTM) model to realize the time dimension feature extraction of the historical processing data and real-time processing data. Based on the historical processing data and real-time processing data, the long short-term memory (LSTM) model is trained, predicted and verified to ensure the prediction accuracy of the long short-term memory (LSTM) on the basis of real-time performance. The prediction parameters of the long short-term memory (LSTM) model are input into the digital twin system, and the corresponding control parameters are sent to the numerical control system. Based on this, the DT-LSTM system is established on the basis of real-time monitoring of the digital twin system and the interpretability of historical processing data and real-time processing data. The spatial physical information and time series data in the processing process are fused and cooperatively processed in real time through DT and LSTM, and physical knowledge and data are seamlessly integrated.

[0015] The established DT-LSTM system sends the prediction parameters and control parameters to the machine tool to directly control the workpiece processing process, so as to ensure the stability of the workpiece processing quality and simultaneously improve the processing precision and processing efficiency.

[0016] A digital twin and deep learning collaborative driving processing process space-time feature fusion prediction visualization system modeling method is provided. In the DT-LSTM application layer, it is found that the processing precision of the workpiece is closely related to the movement process of the main moving parts of the machine tool. The processing process of the workpiece is essentially the material removal under the interaction of the workpiece and the tool. The workpiece is connected to the bed through "workpiece-C-axis-Z-axis", and the tool is connected to the bed through "tool-grinding axis-X-axis". Therefore, in theory, closely monitoring the movement of the main moving parts of the machine tool can realize real-time monitoring of the workpiece processing process.

[0017] Therefore, the digital twin system of the main moving parts of the numerical control grinding machine is established through entity mapping, including X, Z, C and G axes. Through real-time processing data, the digital twin system and the physical entity can be one-to-one corresponding to the real physical world, achieving the effect of virtual control of real and virtual-real symbiosis.

[0018] Step two: establish a long short-term memory (LSTM) model for the digital twin system;

[0019] Although the processing data is directly derived from the physical modeling of the machine tool mechanical structure, each processing data has a direct and significant physical meaning, but in the multi-source heterogeneous real-time processing data, due to the mutual independence of each physical quantity characteristic, the characteristic dimension of each motion part processing data is scattered and time-space dispersed, so the typical features of the sensing signal cannot be well identified and extracted directly by the digital twin system. The actual processing process has strong time correlation. The previous process often has an impact on the subsequent processing, and in the same process, the workpiece surface is formed by the machining trajectory envelope, so under the condition that the processing parameter change range is not large and the processing is relatively stable, the corresponding processing data often has a certain time series correlation. Accordingly, the LSTM can be used to process time series data, aiming to capture the time dependence within the sequence. By identifying and retaining long-term patterns in the sequence, the LSTM prediction layer significantly enhances the prediction accuracy of the DT-LSTM system. The construction steps of the DT-LSTM prediction layer are as follows:

[0020] Step one: LSTM unit basic framework;

[0021] In the manufacturing process, products are obtained from raw blank materials through a series of processes and steps, and the historical processing data in the processing process will affect the current processing state of the workpiece. Historical processing data is helpful for real-time monitoring and high-precision prediction. Therefore, a digital twin and deep learning collaborative driving processing process space-time feature fusion method is proposed to realize real-time monitoring and prediction control of the manufacturing system. Based on the digital twin system to establish a real-time interactive simplified physical model, the processing process diagnosis based on historical processing data and current processing state is realized through the LSTM network.

[0022] A digital twin and deep learning collaborative driving processing process space-time feature fusion method is proposed to realize real-time monitoring and prediction control of the manufacturing system; through the long short-term memory LSTM model, the processing process diagnosis based on real-time processing data in the historical database and the current processing state is realized;

[0023] The core components of the long short-term memory LSTM model are the sequence input layer and the LSTM layer; the real-time processing data is the time series data input into the long short-term memory LSTM model, the sequence input layer inputs the corresponding time series data into the LSTM layer, and the LSTM layer learns the long-term correlation between the time steps of the time series data; the LSTM layer contains multiple LSTM units, which construct corresponding LSTM units for the real-time processing data of the X-axis feeding system, the Z-axis feeding system, the C-axis rotating system and the G-axis rotating system, respectively. An LSTM unit is composed of multiple storage units, and each storage unit contains three gating mechanisms: forget gate, input gate and output gate;

[0024] The learnable weights of the LSTM layer include input weights W, recurrent weights R, and bias b; the LSTM layer constructs a concatenated matrix according to the following equation:

[0025] (1)

[0026] Where i, f, o represent input gate, forget gate, and output gate respectively, and g represents candidate memory cell; the memory cell state c t and the hidden cell state h t at time t are given by the following equations:

[0027] (2)

[0028] Where ⊙ represents Hadamard product, and σ c represents state activation function; the hidden cell state h t is the output of the LSTM layer at this time step; the calculation of the corresponding input gate, forget gate, candidate memory cell, and output gate is given by the following equation:

[0029] (3)

[0030] Where x t is the input at time t, i.e., the real-time processing data of the digital twin system at time t; the output data h0~h t under the corresponding time series data is obtained by operating the historical input data x0~x t through the memory cell state and hidden cell state of i, f, o, and g at time 0~t; h t is the predicted input data at the next moment;

[0031] The tanh function is used as the state activation function:

[0032] (4)

[0033] In equation (3), σ x represents gate activation function;

[0034] The Sigmoid function is used as the gate activation function:

[0035] (5)

[0036] Step two: adaptation of DT and LSTM unit;

[0037] The main moving parts of the numerical control machine tool include X-axis feeding system, Z-axis feeding system, C-axis rotating system, and G-axis rotating system; the five-dimensional model of the corresponding digital twin system is shown in the following equation:

[0038] (6)

[0039] Wherein, X represents the X-axis feed system, Z represents the Z-axis feed system, C represents the C-axis or spindle rotation system, G represents the G-axis rotation system; the physical entity PE, the virtual entity VM, the service SS, the processing data DD, the connection CN are respectively the corresponding expressions of the main motion components of the numerical control machine tool in the digital twin system;

[0040] The same type of motion components have structural similarity and dynamic similarity, so the digital twin system simplifies the data representation structure of the numerical control machine tool in the real physical world according to the physical similarity, that is, the data of the X-axis feed system and the Z-axis feed system are uniformly processed by one LSTM layer, and the input data corresponding to the LSTM layer is the real-time processing data of the X-axis feed system and the Z-axis feed system; the data of the C-axis rotation system and the G-axis rotation system are uniformly processed by another LSTM layer, and the input data corresponding to the LSTM layer is the real-time processing data of the C-axis rotation system and the G-axis rotation system:

[0041] (7)

[0042] Wherein, LSTM-L represents the LSTM layer of the linear motion component in the digital twin system, LSTM-R represents the LSTM layer of the rotary motion component in the digital twin system, X d (t), Z d (t), C d (t) are the corresponding real-time processing data of the X-axis, Z-axis, C-axis or spindle, including position signal, power signal, feedback signal; the output is the predicted data of the X-axis, Z-axis, C-axis or spindle corresponding to the prediction time step;

[0043] First, use the initial state of the LSTM unit and the first time step of the time series data to calculate the first output and the updated unit state (c1, h1), which is: at time t, use the current unit state (c t−1 , h t−1 ) of the LSTM unit and the next time step of the time series data to calculate the output and the updated unit state (c t , h t ); the unit state of the current layer is composed of the hidden unit state h t and the memory unit state c t ; the hidden unit state h t at time t contains the output of the LSTM layer at that time; the memory unit state c t contains the information obtained from the previous time; at each time, the current layer adds or deletes information in the memory unit state c t ; the current layer uses different gates to control these updates;

[0044] According to the above, the operation process of real-time machining data in the LSTM in the digital twin system is established, the real-time connection of machining data in different processes, steps and working strokes based on historical machining information is obtained, and real-time monitoring and machining prediction of the machining process can be realized.

[0045] Step three: training of the prediction layer in the DT-LSTM architecture;

[0046] On this basis, the DT-LSTM system is trained online. The digital twin system includes the X-axis feeding system, the Z-axis feeding system and the C-axis rotating system of the numerical control machine tool. According to the data structure of the digital twin system, the long short-term memory (LSTM) model is set to three channels. The number of hidden units of the LSTM layer determines how much information the layer has learned. Using more hidden units can produce more accurate results, but it is also more likely to cause overfitting of the training data. Therefore, the LSTM layer is set to have 128 hidden units. A higher dropout rate helps to improve the generalization ability of the model, but at the cost of losing information and slowing down the learning process. Therefore, the dropout layer helps to avoid overfitting by randomly setting the input of this layer to zero and effectively changing the network architecture between training iterations. The DT-LSTM model is trained, verified and tested. In the monitoring process, the real-time machining data is compared with the LSTM predicted data. When the training data is out of tolerance, retraining is performed. The digital twin system performs corresponding training operations according to the feedback data. The machining precision of the workpiece is predicted within the appropriate range of training precision, and the process control parameters are adjusted according to the product process precision requirements.

[0047] Adam optimization is used for training, and the learning rate is set to 0.001. In order to reduce overfitting or insufficient fitting, the training iteration is set to 1000. After iterative training, the performance of the best validation and training model on the test set is compared, and the model with the highest test precision and the lowest test loss is selected as the final model. This method effectively reduces the influence of training iteration setting on model performance. Using this strategy, the best training iteration round of the DT-LSTM model is determined.

[0048] Step four: transmission of real-time machining data in the DT-LSTM system;

[0049] The control mode of the main moving parts of the numerical control machine tool in the workpiece machining process adopts a closed-loop servo feedback system with a grating ruler as a reference, that is, the numerical control machine sends the machining numerical control instructions to the programmable controller through the industrial computer, the programmable controller transmits the corresponding control signals to the servo drivers of the main moving parts of the numerical control machine tool, the servo drivers transmit the power signals to the corresponding linear motion parts and rotary motion parts, and drive the linear motion parts and rotary motion parts to generate corresponding actions; the grating ruler compares the real-time position of the linear motion parts and rotary motion parts with the ideal position, and transmits the error as a feedback signal to the motion controller of the programmable controller, and executes the corresponding control algorithm according to the corresponding feedback error to reduce the real-time error;

[0050] By extracting real-time machining data and accessing the LSTM layer, the machining quality can be predicted in advance; the real-time machining data returned by the DT-LSTM system includes power signals, position signals and feedback signals, which are represented as:

[0051] (8)

[0052] Wherein, X Pow , X Fee , X Pos represent the real-time power signal, feedback signal and position signal of the X-axis respectively; Z Pow , Z Fee , Z Pos represent the real-time power signal, feedback signal and position signal of the Z-axis respectively; C Pow , C Fee , C Pos represent the real-time power signal, feedback signal and position signal of the C-axis respectively;

[0053] The real-time power signal can accurately distinguish the machining state, and the position signal and the feedback signal can reflect the shape and position accuracy of the workpiece; the real-time machining data of the DT-LSTM system at 0~t-1 time is transmitted to the LSTM layer as historical data, the LSTM layer outputs the predicted value according to the corresponding forget gate and update gate, and compares it with the real-time machining data at t time to determine the current prediction accuracy; if the prediction accuracy meets the algorithm requirement, the prediction value after t+1 time is further calculated, and the prediction value is transmitted to the industrial computer; the industrial computer adjusts the machining parameters according to the prediction value;

[0054] Step 3: Dimensionality reduction processing of real-time machining data in the DT-LSTM system;

[0055] The machining process space-time feature fusion prediction visualization method driven by digital twin and deep learning can improve the data processing efficiency, and the processing efficiency is improved as follows:

[0056] (9)

[0057] wherein, T is the current process, M is the number of motion components corresponding to the workpiece machining process, N is the real-time data category of a single motion component, t is the machining time step; m is the number of motion components after dimension reduction, n is the real-time data category of a single motion component after dimension reduction, s is the machining time;

[0058] Fourth step: real-time machining data spatio-temporal feature fusion visualization method based on DT-LSTM system;

[0059] In the machining of a cylindrical surface, the machining path with time as the variable presents a real-time machining trajectory that spirals upward; in the machining of a cylindrical surface, the X-axis, Z-axis, C-axis, and G-axis each have their own physical meaning in the digital twin system. The X-axis is the depth of cut, corresponding to the control diameter of the cylindrical surface, which is stationary during the machining of a single cylindrical surface; the Z-axis corresponds to the machining length of the cylindrical surface, and the machining trajectory is a uniform linear motion; the C-axis motion directly corresponds to the roundness of the cylindrical surface, which is a uniform rotational motion in the machining of a cylindrical surface; the G-axis corresponds to the machining parameters, which is a uniform rotational motion;

[0060] The machine tool space coordinate system and the digital twin system space coordinate system are respectively constructed under the machine tool mechanical coordinate system, wherein the coordinate origin and the X, Y, Z axes of the machine tool space coordinate system and the digital twin system space coordinate system are one-to-one corresponding to the machine tool mechanical coordinate system;

[0061] In the cylindrical surface, the collected workpiece real-time machining data points are processed by dimension reduction, and the cylindrical surface is projected onto the xOz, xOy, and yOz planes of the machine tool space coordinate system in three views. The xOz and yOz planes are respectively the composite motion forms of the x-axis, y-axis, and z-axis, presenting a typical sinusoidal curve shape; the xOy plane is the composite motion form of the x-axis and y-axis, which is an ideal circular trajectory; the two-dimensional space data is reduced to a three-axis motion form represented by x, y, and z; since x, y, and z are actually virtual space axes generated by X, Z, and C motion axes, they are represented by axes 1, 2, and 3 respectively; both 1-axis and 2-axis are sinusoidal motion, and the phase difference between them is 90 degrees; 1-axis and 2-axis correspond to the motion of the machining point on the cylindrical surface respectively, while 3-axis is a straight line motion, representing the motion of the machining point along the generatrix direction on the cylindrical surface; accordingly, the spatio-temporal correspondence between the physical motion axes X, Z, and C and the virtual space axes x, y, and z in the machine tool digital twin system is established:

[0062] (10);

[0063] Fifth step: machining data prediction method based on DT-LSTM system;

[0064] According to the dimension reduction processing method of real-time machining data in the DT-LSTM system established in the third step, one-dimensional machining data is mapped to a three-dimensional space, and subsequent machining data is predicted; the machining data prediction process based on the DT-LSTM system is divided into three steps;

[0065] Step 1 generates subsequent prediction data h1 p, … of step 1 according to current experimental data 0-h1 t; as the machining process proceeds, the subsequent prediction data of step 1 is gradually replaced by subsequent experimental data h1 t+1, … of step 1, and on this basis, the LSTM unit parameters in formula 1 are verified according to the subsequent experimental data h1 t+1, … of step 1 and the prediction data of step 1; when the error does not exceed the prediction threshold δ t Step 2 is predicted according to the subsequent experimental data h1 t+1, … of step 1, and prediction data h2 p, … of step 2 is generated;

[0066] As the machining process proceeds, the prediction data of step 2 is replaced by experimental data h2 t+1, …, and the LSTM unit parameters in formula 1 are verified accordingly; if the error exceeds the prediction threshold δ t , the corresponding control process is executed, step 2 is predicted again in combination with the experimental data h2 t+1, … of step 2, the parameter weights of formula 1 are iterated, and the prediction data h2 p, … of step 2 is regenerated, and finally the prediction data h3 p, … of step 3 is generated in combination with the current experimental data of step 2;

[0067] The above history-prediction-verification steps are repeatedly executed, and finally the machining data of the workpiece surface is generated in real time during the entire machining process of the workpiece;

[0068] The trained DT-LSTM system is used to predict real-time machining data, and the predicted machining trajectory of the workpiece surface is obtained by reconstructing the prediction data in space; on this basis, the historical machining data and real-time machining data of the entire machining process are trained, predicted and verified; the above steps are repeatedly executed to realize the training, prediction and verification of the machining data.

[0069] The beneficial effects of the present application: the present application aims at the problem of poor spatiotemporal feature modeling and representation ability and low calculation efficiency in the current processing process, and invents a digital twin and deep learning collaborative driving spatiotemporal feature fusion prediction visualization method and system for processing process, which realizes real-time collaborative fusion modeling of spatiotemporal features in processing process. Through DTLSM, real-time fusion and collaborative processing of physical information and neural network are realized, and physical knowledge and data are seamlessly integrated. The explainability is reflected in the unified architecture modeling of the machine tool moving parts through digital twinning of physical models, which improves the data explainability on the basis of ensuring one-to-one correspondence between data and physical models. The deep learning neural network simplifies the model and data structure, and realizes the precision prediction of the processing process through the processing of historical data and key real-time data, which provides a new idea for the advanced control of the processing process. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 is a digital twin and deep learning collaborative driving spatiotemporal feature fusion prediction visualization system architecture.

[0071] Figure 2 is an LSTM neural network structure diagram based on a digital twin architecture.

[0072] Figure 3 is a DT-LSTM collaborative driving spatiotemporal feature fusion method for CNC grinding machine processing process. Wherein (a) is the closed-loop servo feedback system of the machine tool in ultra-precision machining; (b) is the transmission rule of the machine tool internal control signal in the LSTM layer.

[0073] Figure 4 is a digital twin driven spatiotemporal feature fusion method for processing process. Wherein (a) is the representation state of the processing data in the time domain and the space domain; (b) is the physical meaning corresponding to the processing data.

[0074] Figure 5 is the root mean square error and loss curve of the DT-LSTM architecture in the iteration training process. Wherein (a) is the root mean square error curve in the training of the DT-LSTM architecture; (b) is the loss curve in the training of the DT-LSTM architecture.

[0075] Figure 6 is the prediction and application of spatiotemporal features in a single time step by the DT-LSTM architecture. Wherein the left side: the twin prediction data after training of the DT-LSTM; the right side: the spatial expression based on the advanced twin prediction data of the DT-LSTM.

[0076] Figure 7is the spatio-temporal feature fusion and application of the DT-LSTM architecture in the machining process. Wherein (a) is the first step machining process data training, prediction and verification driven by the DT-LSTM architecture; (b) is the second step machining process data training, prediction and verification driven by the DT-LSTM architecture; (c) is the third step machining process data training, prediction and verification driven by the DT-LSTM architecture.

[0077] Figure 8 is the prediction and reconstruction of the workpiece machining data in the spatial dimension in different time steps of the DT-LSTM. Wherein (a) is the prediction and reconstruction of the workpiece machining data in the spatial dimension in the first two time steps; (b) is the prediction and reconstruction of the workpiece machining data in the spatial dimension in the first three time steps; (c) is the prediction and reconstruction of the workpiece machining data in the spatial dimension by the DT-LSTM in the workpiece machining process.

[0078] Figure 9 is the comparison and verification of the workpiece shape and position accuracy prediction results by the DT-LSTM. Wherein (a) is the offline plane measurement value of the workpiece roundness; (b) is the real-time prediction value of the workpiece roundness by the DT-LSTM; (c) is the spatial measurement value of the workpiece cylindricity; (d) is the real-time prediction value of the workpiece cylindricity by the DT-LSTM; (e) is the real-time prediction value of the workpiece shape and position accuracy by the DT-LSTM. DETAILED DESCRIPTION

[0079] The specific implementation of the present application is described in detail below in combination with the technical solutions and the drawings.

[0080] Figure 1 is a digital twin and deep learning collaborative driving machining process spatio-temporal feature fusion prediction visualization system, the specific steps of system modeling are as follows:

[0081] Step 1: Establish a DT-LSTM system for the machining process.

[0082] The DT-LSTM architecture established by the present application is shown in Figure 1 Based on the digital twin five-dimensional model of the machining process, in order to realize the real-time fusion of the spatio-temporal features of the machining process, taking the machine tool as an independent minimum system and taking the complete machining process of the workpiece as the research object, the digital twin system is further expanded to the whole process of the workpiece machining process. The specific steps are as follows:

[0083] Step one: Establishing the digital twin (DT) of the numerical control machine (CNC): The main moving parts of the numerical control machine are divided into linear motion parts and rotary motion parts. The linear motion parts include the X-axis feeding system and the Z-axis feeding system. The rotary motion parts include the C-axis rotating system and the G-axis rotating system. The main moving parts of the numerical control machine are modeled dynamically to form a digital twin system DT = {X, Z, C, G} facing the machining process. Based on the established digital twin system, the initial machining time of the workpiece on the physical platform of the machine tool is taken as the starting point, and real-time machining data is collected and stored synchronously to establish a historical database of the workpiece machining process from 0 to t-1 time;

[0084] Step two: Collecting the historical machining data from 0 to t-1 time and the real-time machining data at the current time t through the digital twin system, and synchronously mapping and constructing the digital twin system with physical meaning. According to the corresponding physical structure of the digital twin system, the real-time machining data is classified, merged and simplified. The main moving parts of the numerical control machine are divided into linear motion parts and rotary motion parts. The machining data with the same motion form is merged: for the X-axis feeding system and the Z-axis feeding system with linear motion form, the machining data containing length scalar is simplified; for the C-axis rotating system and the G-axis rotating system with rotary motion form, the machining data containing angle scalar is simplified, realizing the physical dimension reduction of the real-time machining data of the workpiece.

[0085] Step three: Extracting the spatial features of the real-time machining data at the current time t in the physically dimension-reduced digital twin system, obtaining the spatial expression of the workpiece quality evaluation index, and realizing the real-time visualization of the workpiece machining precision. The workpiece quality evaluation index includes surface waviness, surface roughness, shape accuracy and position accuracy.

[0086] Step four: Using the long short-term memory (LSTM) model to reconstruct the time series of the historical machining data from 0 to t-1 time and the real-time machining data at the current time t, extracting the time dimension features of the real-time machining data at the current time t, and predicting the machining state, obtaining the predicted machining data of the workpiece at the predicted time step t+1 and synchronously returning to the digital twin system to obtain the distribution form of the machining data points of the workpiece in the three-dimensional coordinate system within the predicted time step, and evaluating the workpiece quality based on the distribution form of the machining data points in the three-dimensional coordinate system, generating the corresponding advance control parameters.

[0087] Step five: step one establishes a digital twin system under the premise of ensuring data interpretability, which can extract spatial dimension features and physically reduce data scale for processing information; step four inputs the historical processing data and real-time processing data of the digital twin system into the long short-term memory (LSTM) model to realize the time dimension feature extraction of the historical processing data and real-time processing data; based on the historical processing data and real-time processing data, the long short-term memory (LSTM) model is trained, predicted and verified to ensure the prediction accuracy of the long short-term memory (LSTM) on the basis of real-time; the prediction parameters of the long short-term memory (LSTM) model are input into the digital twin system, and the corresponding control parameters are sent to the numerical control system; accordingly, on the basis of real-time monitoring of the digital twin system and the interpretability of historical processing data and real-time processing data, a DT-LSTM system is established to realize real-time fusion and collaborative processing of spatial physical information and time series data in the processing process, and seamlessly integrate physical knowledge and data;

[0088] Accordingly, the application takes a numerical control grinding machine as an example, and proposes a three-layer modeling structure of the application layer, the modeling layer and the prediction layer of the DT-LSTM architecture, as shown in Figure 1 The processing data flow and control are realized, and the workpiece processing process is directly controlled to ensure the stability of the workpiece processing quality and simultaneously improve the processing precision and efficiency.

[0089] A digital twin and deep learning collaborative driving processing process space-time feature fusion prediction visualization system modeling method can be found in the application layer of the DT-LSTM architecture that the processing precision of the workpiece is closely related to the movement process of the main moving parts of the machine tool. The processing process of the workpiece is essentially the material removal under the interaction of the workpiece and the tool, and the workpiece is connected with the bed through "workpiece-C shaft-Z shaft", and the tool is connected with the bed through "tool-grinding shaft-X shaft". Therefore, in theory, closely monitoring the movement of the main moving parts of the machine tool can realize real-time monitoring of the workpiece processing process.

[0090] Therefore, in the modeling layer, the digital twin system of the main moving parts of the numerical control grinding machine is established through entity mapping, including X, Z, C shaft and grinding shaft. Through real-time technical data, the digital twin system and the physical entity can be one-to-one corresponding with the real physical world, achieving the effect of virtual control of real and virtual-real symbiosis. And the modeling layer and the application layer, together with the price data, connection and corresponding service, constitute the general five-dimensional digital twin model in the conventional digital twin.

[0091] Second step: establish a long short-term memory (LSTM) neural network for a digital twin system.

[0092] Although the machining data is directly derived from the physical modeling of the machine tool mechanical structure, each machining data has a direct and significant physical meaning, but in the multi-source heterogeneous real-time machining data, due to the mutual independence of each physical quantity characteristic, the machining data characteristic dimension of each moving part is scattered and time-space dispersed, so the typical characteristics of the sensing signal cannot be well identified and extracted directly by the digital twin system. The actual machining process has strong time correlation. The previous process often has an impact on the subsequent machining, and in the same process, the workpiece surface is formed by the machining trajectory envelope, so under the condition that the machining data corresponding to the process parameter change range is relatively stable, the machining data often has a certain time sequence correlation. Accordingly, the LSTM can be used to process time sequence data, aiming to capture the time dependence within the sequence. By identifying and retaining long-term patterns in the sequence, the LSTM prediction layer significantly enhances the prediction accuracy of the DT-LSTM system. The construction steps of the DT-LSTM prediction layer are as follows:

[0093] Step one: LSTM unit basic framework;

[0094] In the manufacturing process, products are ultimately obtained from raw blank materials through a series of processes and steps, and the historical data in the machining process will affect the current machining state of the workpiece. Machining history information is helpful to realize real-time monitoring and high-precision prediction. Therefore, a digital twin and deep learning collaborative driving machining process space-time feature fusion method is proposed to realize real-time monitoring and prediction control of the manufacturing system. Based on the digital twin system to establish a real-time interactive simplified physical model, the LSTM network is used to realize the machining process diagnosis based on the historical machining data and the current machining state.

[0095] A digital twin and deep learning collaborative driving machining process space-time feature fusion method is proposed to realize real-time monitoring and prediction control of the manufacturing system; through the long short-term memory LSTM model, the machining process diagnosis based on the real-time machining data in the historical database and the current machining state is realized.

[0096] LSTM is a special type of recurrent neural network (RNN), and its core components are sequence input layer and LSTM layer. The sequence input layer inputs time series data into the neural network, and the LSTM layer learns the long-term correlation between time steps of sequence data. As shown in Figure 2 An LSTM unit is composed of multiple storage units, each unit contains three gating mechanisms: forget gate, input gate and output gate, and a unit state for keeping long-term information. LSTM effectively controls information flow by allowing information to be selectively retained, forgotten or output. This can long-term retain important information and prevent interference from irrelevant information. The basic components, functions and calculation formulas of the LSTM unit are shown in Table 1:

[0097] Table 1 Basic components of LSTM unit and their functions

[0098] Component Function Calculation formula Input gate (i) Level of control unit state update i t =σ g (W i xt+R i h t−1 +b i )]]> Forget gate (f) Level of control unit state reset (forget) f t =σ g (W f xt+R f h t−1 +b f )]]> Candidate unit (g) Add information to unit state g t =σ c (W g xt+R g h t−1 +b g )]]> Output gate (o) Level of control unit state added to hidden state o t =σ g (W o xt+R o h t−1 +b o )]]>

[0099] The learnable weights of an LSTM layer include the input weights W, the recurrent weights R, and the bias b. The matrices W, R, and b represent the input weights, the recurrent weights, and the bias of each component, respectively. The layer constructs a concatenated matrix according to the following equation:

[0100] (11)

[0101] where i, f, o, and g denote the input gate, the forget gate, the output gate, and the candidate memory cell, respectively; the memory cell state c t and the hidden state h t at time step t are given by

[0102] (12)

[0103] where denotes the Hadamard product, and σ c denotes the state activation function; the hidden state h t contains the output of the LSTM layer at this time step; and the computations for the input gate, the forget gate, the candidate memory cell, and the output gate are given by

[0104] (13)

[0105] where x t is the input at time step t, that is, the real-time processing data of the digital twin system at time t; the output data h0:t t at the corresponding time series data is obtained by operating the historical input data x0:t t on the memory cell state and the hidden state of i, f, o, and g at time steps 0:t; and h t is the predicted input data at the next time step.

[0106] The hyperbolic tangent function (tanh) is used as the state activation function. The hyperbolic tangent function outputs a range of (−1, 1) and has the feature of zero centralization, so the tanh function is used to calculate the state activation function:

[0107] (14)

[0108] σ xThe gate activation function is represented. The sigmoid function maps the real number domain to the interval (0, 1) through a nonlinear transformation, and the output value has a probability significance. The sigmoid function is used to calculate the gate activation function:

[0109] (15)

[0110] Step two: adaptation of DT and LSTM unit;

[0111] In order to make LSTM better adapt to the DT system, it is necessary to adapt LSTM according to the digital twin system. The adaptation process is as shown in Figure 3 Since the grinding machine has one more grinding shaft than the lathe, the present application takes the ultra-precision grinding machine as an example to illustrate the structure of the machining process digital twin system established by the present application. The main moving parts of the ultra-precision grinding machine are two linear motion axes (X and Z axes) and two rotary motion axes (main shaft / C axis and grinding shaft), and the corresponding five-dimensional model of the digital twin system is as follows:

[0112] (16)

[0113] Among them, X represents the X-axis linear feed system, Z represents the Z-axis linear feed system, C represents the C-axis rotary motion system, and G represents the grinding shaft rotary motion system. Physical entity (Physical entity, PE), virtual entity (Virtual model, VM), service (Service system, SS), machining data (Digital twin data, DD) and connection (Connection, CN) are respectively corresponding expressions of the main moving parts of the machine tool in the digital twin system.

[0114] The same type of moving parts have structural similarity and dynamic similarity, so the digital twin system simplifies the data representation structure of the numerical control machine tool in the real physical world according to the physical similarity, that is, the data of the X-axis feed system and the Z-axis feed system are uniformly processed by one LSTM layer, and the input data corresponding to the LSTM layer is the real-time machining data of the X-axis feed system and the Z-axis feed system; the data of the C-axis rotary system and the G-axis rotary system are uniformly processed by another LSTM layer, and the input data corresponding to the LSTM layer is the real-time machining data of the C-axis rotary system and the G-axis rotary system:

[0115] (17)

[0116] Among them, LSTM-L represents the LSTM layer of the linear motion part in the digital twin system, LSTM-R represents the LSTM layer of the rotary motion part in the digital twin system, X d (t), Z d (t), Cd (t) Real-time machining data corresponding to X-axis, Z-axis, C-axis or main shaft respectively, including position signal, power signal, feedback signal; output is the predicted data in the corresponding predicted time step of X-axis, Z-axis, C-axis or main shaft;

[0117] First, the initial state of the LSTM unit and the first time step of the time series data are used to calculate the first output and the updated unit state (c1, h1), which is: at time t, the current unit state (c t−1 , h t−1 ) of the LSTM unit and the next time step of the time series data are used to calculate the output and the updated unit state (c t , h t ); the unit state of the current layer is composed of the hidden unit state h t and the memory unit state c t ; the hidden unit state h t at time t contains the output of the LSTM layer at this time; the memory unit state c t contains the information obtained from the previous time; at each time, the current layer adds or deletes information in the memory unit state c t ; the current layer uses different gates to control these updates;

[0118] According to this, the operation process of real-time machining data in LSTM in the digital twin system is established, the real-time connection of machining data in different processes, steps and working strokes based on historical machining information is obtained, and real-time monitoring and machining prediction of the machining process can be realized.

[0119] Step three: training of the prediction layer in the DT-LSTM architecture;

[0120] On this basis, the DT-LSTM model is trained online. The corresponding LSTM structure is shown in Table 2, which is composed of four layers of sequence input layer, LSTM layer, dropout layer and full connection layer, and a total of 67.9k parameters.

[0121] Table 2 Structure of prediction layer in DT-LSTM architecture

[0122]

[0123] As shown in Table 2, the LSTM is set to three channels according to the established DT data structure. The number of LSTM layer hidden units determines how much information the layer learns, and using more hidden units produces more accurate results, but also more likely to cause training data overfitting. Therefore, the LSTM layer is set to have 128 hidden units. A higher dropout probability helps improve the generalization ability of the model, but at the cost of losing information and slowing down the learning process. Therefore, the dropout layer helps to avoid overfitting by randomly setting the input of this layer to zero and effectively changing the network architecture between training iterations. Based on the structure detailed in Table 2, the DT-LSTM model is trained, validated and tested. In the monitoring process, real-time machining data is compared with LSTM prediction data, and when the training data is out of tolerance, retraining is performed, and the digital twin system performs corresponding training operations according to the feedback data, as shown in (b) of FIG. 1. The workpiece machining precision is predicted within the appropriate range of training accuracy, and the process control parameters are adjusted according to the product process precision requirements. Figure 3

[0124] The training is performed using Adam optimization with a learning rate of 0.001. To mitigate overfitting or insufficient fitting, this example sets the training iteration to 1000 and adopts a specific strategy. Every fifty iterations are validated to monitor the model performance, and the model is retained with the best validation accuracy and training accuracy. After iterative training, the performance of the best validation and training model on the test set is compared, and the model with the highest test accuracy and the lowest test loss is selected as the final model. This method effectively reduces the impact of training iteration settings on model performance. The accuracy and loss curves of the training iteration are shown in FIGS. 3A and 3B, respectively. The results show that in the first 300 training iterations, both the training and validation loss decrease rapidly, while the training and validation accuracy improve quickly. Subsequently, the training loss continues to decrease slowly to 0.033, while the validation loss stabilizes at around 0.028. The training RMSE continues to decrease slowly to 0.169, while the validation RMSE stabilizes at around 0.183. The mean of the root mean square error is 0.178, and using this strategy, the best training iteration number of the DT-LSTM model is determined to be 400. Figure 5

[0125] Step four: transmission of machining data in the DT-LSTM system;

[0126] The control mode of the main machine tool motion parts in ultra-precision machining mainly uses a closed-loop servo feedback system with a grating ruler as the reference, such as Figure 3 ​​As shown. That is, by industrial personal computer (IPC) to send the machining numerical control instructions to programmable multi-axis controller (PMAC), PMAC will be the corresponding control signal to the corresponding motion parts of the servo driver, servo driver will power signal to the corresponding motion mechanism, drive motion mechanism to produce the corresponding action. Grating will motion mechanism real-time position and ideal position comparison, the error as a feedback signal to PMAC motion controller, PMAC motion controller according to the corresponding feedback error to perform the corresponding control algorithm, in order to reduce the real-time error, improve the control precision.

[0127] Based on the established DT-LSTM architecture can be convenient and accurate to monitor the processing. By extracting processing data and access to LSTM layer can time ahead of the processing quality prediction, such as Figure 3 As shown. By extracting real-time processing data and access to LSTM layer to achieve ahead of the processing quality prediction; DT-LSTM system back to the real-time processing data, including power signals, position signals and feedback signals, expressed as:

[0128] (18)

[0129] Where, X Pow , X Fee , X Pos represent the real-time power signal, feedback signal and position signal of X-axis respectively; Z Pow , Z Fee , Z Pos represent the real-time power signal, feedback signal and position signal of Z-axis respectively; C Pow , C Fee , C Pos represent the real-time power signal, feedback signal and position signal of C-axis respectively;

[0130] Real-time power signal to accurately distinguish the processing state, position signal and feedback signal reflect the shape and position accuracy of workpiece; DT-LSTM system 0~t-1 time of real-time processing data as historical data transmission to LSTM layer, LSTM layer according to the corresponding forget gate and update gate output prediction value, and compared with the real-time processing data at t time, to determine the current prediction accuracy; if the prediction accuracy meets the algorithm requirements, further calculate the prediction value after t+1 time, and the prediction value is transmitted to the industrial personal computer; IPC according to the prediction value of the processing parameters are adjusted;

[0131] If the prediction accuracy meets the algorithm requirements, the predicted value (t+1 time..) is further calculated and transmitted to the IPC. The IPC adjusts the processing parameters according to the predicted value. Therefore, the LSTM network established by the present application includes a real-time processing data verification link based on the current prediction of the LSTM based on historical data, a lead prediction link of the LSTM network for future processing data, and a pre-adjustment and control of the current processing state according to the lead prediction data.

[0132] Step 3: Dimension reduction of processing data in the DT-LSTM system

[0133] The DT-LSTM real-time architecture established by the present application simplifies the physical structure, and through physical isolation of data flow, the dimension of the real-time data flow structure can be reduced to improve processing efficiency and monitoring accuracy. The digital twin and deep learning collaborative driving processing process fault diagnosis method improves the processing efficiency, and the processing efficiency is improved as follows:

[0134] (19)

[0135] Where T is the current process, M is the number of moving parts corresponding to the workpiece processing process, N is the real-time data category of a single moving part, t is the processing time step, m is the number of moving parts after dimension reduction, n is the real-time data of a single moving part after dimension reduction, and t is the processing time. Taking the ultra-precision CNC grinding machine as an example, the initial processing data includes: M(X, Z, C, M)=4, N(pow, fee, pos, con)=4, the real-time processing data of the physical structure after the DT-LSTM system simplification has time and space dimensions: m(X, Z, C)=3, N(pow, fee, pos)=2, by simplifying the MxN-dimensional real-time processing data to m x n-dimensional processing data, the data size of the processing data is finally reduced by 62%. This makes the processing of the processing data more efficient and concise in the same time step.

[0136] The established processing process digital twinning real-time architecture greatly reduces the modeling and calculation cost, improves the real-time transmission of the processing process data, and establishes the real-time connection of the processing data and the electrical data at different times through the physical model, and realizes the real-time mapping of the current loop, the voltage loop, the speed loop and the acceleration loop and other machine tool motion characteristics through the digital twinning real-time architecture of the main motion components at the machine tool level.The processing data dimension reduction operation based on the DT-LSTM architecture proposed in the application is not a simple data screening and extraction, but is based on the virtual-real correspondence of the digital twinning and the physical structure.The real-time processing data with time and space dimensions are mapped into the processing data in the DTS, which improves the autocorrelation and cross-correlation of the processing data and establishes a complete space-time mapping symmetry relationship.On the other hand, for historical processing data with only time or space dimension, such as vibration and temperature, although they do not participate in the subsequent operation of the LSTM prediction layer, they are still synchronized in the DT system through the physical model, and the real-time monitoring of the current processing process is realized.

[0137] Fourth step: real-time processing data space-time feature fusion visualization method based on DT-LSTM system

[0138] The processing process space-time feature fusion prediction visualization method and system based on digital twinning and deep learning collaborative driving provided by the application has the following advantages: the machine tool motion components and the physical model are modeled through the unified architecture of digital twinning in the DT-LSTM system, which improves the data explainability on the basis of ensuring one-to-one correspondence between the data and the physical model.

[0139] Through the dimension reduction simplification of the third step processing data, not only the modeling and calculation cost of the DT-LSTM is greatly reduced, the real-time transmission of the processing process data is improved, but also the real-time connection of the processing data and the electrical data at different times is established through the physical model, and the real-time mapping of the current loop, the voltage loop, the speed loop and the acceleration loop and other machine tool motion characteristics is realized through the digital twinning real-time architecture of the main motion components at the machine tool level, which improves the real-time processing data explainability.

[0140] The DT-LSTM architecture established by the application can conveniently and intuitively realize the space-time feature processing data prediction of the workpiece processing process.In the cylindrical surface processing, the processing path with time as the variable presents a spiral rising real-time processing trajectory;in the cylindrical surface processing, the X-axis, the Z-axis, the C-axis and the G-axis have their own physical meanings in the digital twinning system, the X-axis is the cutting depth, corresponding to the control diameter of the cylindrical surface, which is stationary in single cylindrical surface processing;the Z-axis corresponds to the processing length of the cylindrical surface, and the processing trajectory is a uniform linear motion;the C-axis motion directly corresponds to the roundness of the cylindrical surface, which is a uniform rotary motion in cylindrical surface processing;the G-axis corresponds to the processing parameter, which is a uniform rotary motion.

[0141] A machine tool space coordinate system and a digital twin system space coordinate system are respectively constructed under a machine tool mechanical coordinate system, wherein coordinate origins and X, Y, Z axes of the machine tool space coordinate system and the digital twin system space coordinate system respectively correspond to the machine tool mechanical coordinate system one by one;

[0142] Cylindrical surface machining is a relatively simple curved surface machining case, which can be regarded as a single single-axis feeding motion. In this case, the machining process space-time characteristic machining data prediction method established by the digital twin driving is briefly described. In the machining of the cylindrical surface, the machining path with time as the variable presents a spiral rising real-time machining trajectory, as shown in (a) of FIG. 1. The spiral line is the corresponding machining trajectory, and the circle is the motion control point of PMAC. In the cylindrical surface machining, the X, Z, C and M axes have their respective physical meanings in the DT. The X axis is the depth of cut, corresponding to the control diameter of the cylindrical surface, which is stationary in single cylindrical surface machining. The Z axis corresponds to the machining length of the cylindrical surface, and the ideal machining trajectory is a uniform linear motion. The C axis motion directly corresponds to the roundness of the cylindrical surface, which is an ideal uniform rotation in the cylindrical surface machining. The grinding axis corresponds to the machining parameter, which is a uniform rotation. Figure 4

[0143] The above machine tool motion components are the X, Z and C axis systems inside the machine tool, and do not represent the x, y and z space coordinates of the workpiece. Even for simple cylindrical surface machining, it is not very accurate and intuitive to distinguish the direct relationship between the motion of each axis system of the machine tool and the machining error of each axis. The DT-LSTM system established by the present application can solve this problem. Taking the cylindrical surface as an example, the three-dimensional space data points are processed by dimension reduction, and the cylindrical surface is projected onto the xOz, xOy and yOz planes in three views, as shown in (a) of FIG. 2. It is found that the xOz and yOz planes are similar, which are the composite motion forms of the x and y axes and the z axis, showing a typical sinusoidal curve shape. The xOy plane is the composite motion form of the x and y axes, which is an ideal circular trajectory. Further dimension reduction of the two-dimensional space data to the three-axis motion form represented by x, y and z is shown in (b) of FIG. 2. Since x, y and z are virtual space axes generated by X, Z and C motion axes, they are represented by axes 1, 2 and 3 respectively. It is found that the x axis (1 axis) and the y axis (2 axis) are similar, both of which are sinusoidal motion, and the phase difference between them is 90 degrees. The 1 axis and the 2 axis respectively correspond to the motion of the machining point on the cylindrical surface in (a) of FIG. 1, and the z axis (3 axis) is a straight line motion, representing the motion of the machining point on the cylindrical surface along the generatrix direction in (a) of FIG. 1. Accordingly, the space-time correspondence between the physical motion axes X, Z and C and the virtual space axes x, y and z in the machine tool digital twin system is established as follows: Figure 4 Figure 4 Figure 4 Figure 4

[0144] ​​​​​ (20)

[0145] The trained DT-LSTM architecture is used to predict processed data. The model training process is as follows: Figure 5 As shown, the prediction results are as follows Figure 6 As shown in the diagram. The solid lines before time t1 represent actual historical processing data, the dashed lines after time t1 represent processing data predicted by the DT-LSTM architecture, and the solid lines after time t1 represent real-time processing data. It can be seen that the processing data predicted by the DT-LSTM architecture has a high degree of consistency with the measured results, achieving good prediction of processing data on a time scale. Furthermore, the DT-LSTM established based on this invention can generate the spatial data structure of the workpiece in real time, such as... Figure 6 As shown in the image on the right, the predicted machining trajectory of the workpiece surface is obtained by reconstructing the predicted machining data in space. The solid lines before time t1 represent actual historical machining data, the asterisks after time t1 represent machining data predicted by the DT-LSTM architecture, and the solid lines after time t1 represent real-time machining data. This verifies that the DT-LSTM architecture established in this invention can effectively predict and reconstruct the spatiotemporal characteristics of machining data.

[0146] Figure 6 This involves fusing and reconstructing the spatiotemporal features of processing data within a single time step. Based on this, training, prediction, and validation are performed on the processing data throughout the entire processing process. The principle is as follows: Figure 7 As shown. Figure 7 (a) in the text shows exactly what is shown. Figure 6 The process of predicting the spatiotemporal characteristics of machining data within a single time step. Time t1 is the initial stage of workpiece machining. After time t2, the workpiece machining state gradually stabilizes, and corresponding twin machining data can be collected from time t2 to t4 as historical data for training the LSTM network. The length of historical data collection is determined based on the machining conditions and the twin model architecture. Prediction of the machining process in the next time step can begin at time t4, such as time t4-t6.

[0147] During the processing of the second time step, the predicted data for times t4-t6 is realized and verified with the predicted data in the first time step, such as... Figure 7 As shown in (b) above. Based on this, historical data from time t6 to time t3 is selected as the second time step for prediction by the LSTM network, and the prediction processing data for the third time step from time t6 to time t7 is generated. The historical data from time t2 to time t3 is stored as long-term historical data in the processing data.

[0148] Similarly, during the processing at the third time step, the predicted data for times t6-t7 is realized and verified against the predicted data from the second time step, such as... Figure 7 As shown in (c), the historical data from time t7 to time t5 is selected again based on the historical data collection length as the third time step for prediction by the LSTM network, and the prediction processing data for the fourth time step from time t7 to time t8 is generated. The historical data from time t2 to time t5 is stored as long-term historical data in the processing data. The above steps are repeated to achieve training, prediction, and validation of the processing data.

[0149] Step 5: A method for predicting processing data based on the DT-LSTM system;

[0150] Based on the dimensionality reduction processing method for real-time processing data within the DT-LSTM system established in the third step, one-dimensional processing data is mapped to three-dimensional space, and subsequent processing data is predicted; the processing data prediction process based on the DT-LSTM system is divided into three steps.

[0151] like Figure 8 As shown in (a), subsequent prediction data (h1p, ...) for step 1 are generated based on the experimental data (0-h1t) from step 1, represented by solid lines and asterisks in three-dimensional space, respectively. As the processing progresses, the subsequent prediction data for step 1 is gradually replaced by the subsequent experimental data (h1t+1, ...) from step 1. Based on this, the LSTM algorithm is validated using the subsequent experimental data (h1t+1, ...) and the prediction data from step 1. The validation process is completed when the error does not exceed the prediction threshold δ. t Based on the subsequent experimental data (h1 t+1,…) from step 1, predictions are made for step 2, and prediction data (h2 p,…) for step 2 are generated, such as… Figure 8 As shown in (a) of the document.

[0152] As the processing continues, the predicted data from step 2 is replaced by experimental data (h2 t+1,…), as shown by the solid line in 8(a), and the LSTM algorithm is validated accordingly. If the error exceeds the prediction threshold δ... t The corresponding control flow is executed, and the experimental data from step 2 (h2 t+1,…) is used to re-predict step 2, generating predicted data for step 2 (h2 p,…). Finally, the current experimental data from step 2 is used to generate predicted data for step 3 (h3 p,…). Figure 8 As shown by the solid line in (b) above. Repeating the above history-prediction-verification steps ultimately generates real-time machining data for the workpiece surface throughout the entire machining process, as shown in... Figure 8 As shown in (c) in the figure. The asterisks represent the predicted values ​​of the LSTM network, and the solid lines represent the actual processing data trajectories.

[0153] Repeat the above history-prediction-verification steps to ultimately generate real-time processing data of the workpiece surface throughout the entire processing of the workpiece.

[0154] The trained DT-LSTM system is used to predict real-time machining data. By reconstructing the predicted data in space, the predicted machining trajectory of the workpiece surface is obtained. Based on this, the historical machining data and real-time machining data of the entire machining process are used for training, prediction and verification. The above steps are repeated to achieve training, prediction and verification of machining data.

[0155] To verify the accuracy of the DT-LSTM-driven machining process prediction method, this invention compares the error data of the entire workpiece with offline measurement data, such as... Figure 9 As shown. Figure 9 (a) shows the offline measurement results of the workpiece. The PP values ​​for the five measurements were 1.04, 1.27, 1.19, 1.27 and 1.08 μm, with an average value of 1.17 μm. Figure 9 (b) in the figure shows the processing error distribution based on DT-LSTM prediction in this invention. The DT-LSTM prediction values ​​are completely distributed in a ring with a radius of 1.17 μm, which is in good agreement with the measured data.

[0156] Figure 9 (c) shows the distribution of five offline measurement data along the workpiece surface. Due to the high cost of offline measurement procedures and installation processes, the offline measurement data is relatively limited. Figure 9 (d) in the figure represents the full data distribution of workpiece surface machining errors predicted by DT-LSTM. The data points are continuously recorded as the machining process progresses, exhibiting high accuracy and completeness. The machining data and prediction quantity are significantly higher than offline measurements in terms of data breadth, abundance, and spatial distribution density, while maintaining the same accuracy. Furthermore, it possesses real-time capabilities not found in offline measurements, enabling real-time monitoring of the machining process simultaneously in both time and spatial dimensions. Figure 9 As shown in (e) above, the results fully verify the accuracy and convenience of the DT-LSTM proposed in this invention, enabling convenient online monitoring and real-time prediction of workpiece machining quality.

[0157] The application discloses a machining process space-time feature fusion prediction visualization method and system driven by digital twinning and deep learning, and aims at solving the problems of poor modeling and representation ability and low calculation efficiency of the machining process space-time feature cooperation at present. The application discloses a machining process space-time feature fusion prediction visualization method and system driven by digital twinning and deep learning. The real-time data cost is simplified through the digital twinning model, and the explainability of the machining data is improved. The self-learning and self-processing of the real-time data are realized through the deep learning neural network, the high-precision prediction of the machining process is realized, the prediction precision and the generalization ability of the digital twinning model are improved, the calculation cost of the neural network model is greatly reduced, the real-time monitoring and the advanced prediction of the machining process are realized, and the accurate prediction and regulation and control of the machining quality of the precision manufacturing system are of great significance.

Claims

1. A method for spatiotemporal feature fusion and prediction visualization of a manufacturing process, characterized in that, The steps are as follows: Step 1: Establish a DT-LSTM system for the machining process; To realize the real-time fusion of the space-time characteristics of the workpiece machining process, take the machine tool as the smallest independent system, take the workpiece machining process as the research object, and expand the digital twin system to the workpiece machining process; Step 1: Establish the digital twin system of the numerical control machine tool: The moving parts of the numerical control machine tool are divided into linear motion parts and rotary motion parts, the linear motion parts include X-axis feeding system and Z-axis feeding system, the rotary motion parts include C-axis rotating system and G-axis rotating system; The moving parts of the numerical control machine tool are modeled, forming a digital twin system DT={X,Z,C,G} for the machining process; Based on the established digital twin system, the initial machining time of the workpiece on the machine tool physical platform is taken as the starting point, the real-time machining data is collected and stored synchronously, and the historical database of the workpiece machining process is established from 0 to t-1 time; Step 2: Collect the historical machining data from 0 to t-1 time and the real-time machining data at the current time t through the digital twin system, and synchronously map and construct the digital twin system with physical meaning, and classify, merge and simplify the real-time machining data according to the corresponding physical structure of the digital twin system; The moving parts of the numerical control machine tool are divided into linear motion parts and rotary motion parts, and the machining data with the same motion form is merged: for the X-axis feeding system and Z-axis feeding system with linear motion form, the simplified representation is machining data containing length scalar, for the C-axis rotating system and G-axis rotating system with rotary motion form, the simplified representation is machining data containing angle scalar, realizing the physical dimension reduction of the workpiece real-time machining data; Step 3: Extract the real-time machining data space characteristics of the current time t in the physical dimension reduced digital twin system, obtain the spatial expression of the workpiece quality evaluation index, and realize the real-time visualization of the workpiece machining precision; The workpiece quality evaluation index includes surface waviness, surface roughness, shape accuracy and position accuracy; Step 4: Use the long short-term memory (LSTM) model to reconstruct the time series of the historical machining data from 0 to t-1 time and the real-time machining data at the current time t, extract the time dimension characteristics of the real-time machining data at the current time t, and predict the machining state, obtain the predicted machining data of the workpiece at the predicted time step t+1 and synchronously return to the digital twin system, to obtain the distribution form of the machining data points of the workpiece in the three-dimensional space coordinate system within the predicted time step, and evaluate the workpiece quality based on the distribution form of the machining data points in the three-dimensional space coordinate system, and generate the corresponding lead control parameters; Step five: Step one establishes a digital twin system under the premise of ensuring data interpretability, which can extract spatial dimension features and physically reduce the scale of data in the machining process information; Step four inputs the historical machining data and real-time machining data of the digital twin system into the long short-term memory (LSTM) model to realize the time dimension feature extraction of the historical machining data and real-time machining data. The long short-term memory (LSTM) model is trained and verified based on the historical machining data and real-time machining data, ensuring the prediction accuracy of the long short-term memory (LSTM) on the basis of real-time performance. The prediction parameters of the long short-term memory (LSTM) model are input into the digital twin system, and the corresponding control parameters are sent to the numerical control system. Based on this, the DT-LSTM system is established on the basis of real-time monitoring of the digital twin system and the interpretability of historical machining data and real-time machining data. The spatial physical information and time series data in the machining process are fused and cooperatively processed in real time through DT and LSTM, and physical knowledge and data are seamlessly integrated; The DT-LSTM system sends prediction parameters and control parameters to the machine tool to directly control the workpiece machining process, ensuring the stability of the workpiece machining quality and simultaneously improving the machining accuracy and efficiency; Second step: Establishing a long short-term memory (LSTM) model for digital twin system; Third step: Dimensionality reduction processing of real-time machining data in DT-LSTM system; Fourth step: Real-time machining data spatio-temporal feature fusion visualization method based on DT-LSTM system; Fifth step: Machining data prediction method based on DT-LSTM system.

2. The spatio-temporal feature fusion and prediction visualization method for machining process according to claim 1, characterized in that, Second step: Establishing a long short-term memory (LSTM) model for digital twin system as follows: Step one: Basic framework of LSTM unit; A spatio-temporal feature fusion method for machining process driven by digital twin and deep learning is proposed to realize real-time monitoring and prediction control of manufacturing system. The long short-term memory (LSTM) model is used to diagnose the machining process based on real-time machining data and current machining state in the historical database. The core components of the long short-term memory (LSTM) model are the sequence input layer and the LSTM layer. Real-time machining data is the time series data input into the long short-term memory (LSTM) model. The sequence input layer inputs corresponding time series data into the LSTM layer, which learns the long-term correlation between time steps of time series data. The LSTM layer contains multiple LSTM units, which construct corresponding LSTM units for real-time machining data of X-axis feeding system, Z-axis feeding system, C-axis rotating system, and G-axis rotating system. Each LSTM unit is composed of multiple memory cells, each containing three gating mechanisms: forget gate, input gate, and output gate. The learnable weights of the LSTM layer include input weight W, recurrent weight R, and bias b. The LSTM layer constructs a concatenated matrix according to the following equation: ； where i, f, o represent input gate, forget gate, and output gate respectively, g represents a candidate memory cell; memory cell state c at time t t and hidden cell state h t are given by the following equations, respectively: ； where denotes the Hadamard product, σ c denotes the state activation function; the hidden unit state h t the output of the LSTM layer containing this time step; the computation of the input gate, forget gate, candidate memory unit, and output gate are given by ； wherein x t is the input at time t, that is, the real-time processing data of the digital twin system at time t; the operation of the historical input data x0~x t through the memory cell state and the hidden cell state of i, f, o, and g corresponding to time 0~t, that is, the output data h0~h t under the corresponding time series data is obtained. t h is the predicted input data at the next moment. The tanh function is used as the state activation function: ； In formula (3), σ x denotes a gate activation function; The Sigmoid function is used as the gate activation function: ； Step two: adaptation of DT to LSTM unit; The motion components of the numerical control machine tool include X-axis feeding system, Z-axis feeding system, C-axis rotating system and G-axis rotating system; the five-dimensional model of the corresponding digital twin system is shown in the following formula: ； Wherein, X represents the X-axis feeding system, Z represents the Z-axis feeding system, C represents the C-axis or main shaft rotating system, and G represents the G-axis rotating system; the physical entity PE, virtual entity VM, service SS, twin data DD and connection CN are the corresponding expressions of the motion components of the numerical control machine tool in the digital twin system; The same type of motion components have structural similarity and dynamic similarity, so the digital twin system simplifies the data representation structure of the numerical control machine tool in the real physical world according to the physical similarity, that is, the data of X-axis feeding system and Z-axis feeding system are processed by one LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of X-axis feeding system and Z-axis feeding system; the data of C-axis rotating system and G-axis rotating system are processed by another LSTM layer, and the input data of the corresponding LSTM layer is the real-time processing data of C-axis rotating system and G-axis rotating system: ； Wherein, LSTM-L represents the LSTM layer of the linear motion component in the digital twin system, LSTM-R represents the LSTM layer of the rotating motion component in the digital twin system, X d (t), Z d (t), C d (t) are the real-time machining data corresponding to the X-axis, Z-axis, C-axis or main shaft respectively, including position signal, power signal and feedback signal; the output is the predicted data within the predicted time step corresponding to the X-axis, Z-axis, C-axis or main shaft. The first output and updated cell state (c1, h1) is computed using the initial state of the LSTM unit and the first time step of the time series data, specifically: at time t, the current cell state (c t-1 , h t-1 ) of the LSTM unit and the next time step of the time series data are used to compute the output and updated cell state (c t , h t ); the cell state of the current layer is composed of the hidden cell state h t and the memory cell state c t ; the hidden cell state h t at time t contains the output of the LSTM layer at that time; the memory cell state c t contains information obtained from previous times; at each time step, the current layer adds or removes information in the memory cell state c t ; the current layer uses different gates to control these updates; Step three: training of LSTM in DT-LSTM system; The digital twin system includes the X-axis feeding system, Z-axis feeding system and C-axis rotating system of the numerical control machine tool, and the long short-term memory LSTM model is set to three channels according to the data structure of the digital twin system; The training is carried out using Adam optimization, the learning rate is set to 0.001, and the training iteration is set to 1000; after iterative training, the model with the highest test accuracy and the lowest test loss is selected as the final long short-term memory LSTM model for digital twin system; Step four: transmission of real-time processing data in DT-LSTM system; The control mode of the motion components of the numerical control machine tool during workpiece processing adopts a closed-loop servo feedback system based on a grating ruler, that is, the numerical control instructions for processing are sent to the programmable controller through the industrial computer, the programmable controller transmits the corresponding control signals to the servo drivers of the motion components of the numerical control machine tool, the servo drivers transmit power signals to the corresponding linear motion components and rotating motion components, and drive the linear motion components and rotating motion components to generate corresponding actions; the grating ruler compares the real-time position of the linear motion components and rotating motion components with the ideal position, and transmits the error as a feedback signal to the motion controller of the programmable controller, and executes the corresponding control algorithm according to the corresponding feedback error to reduce the real-time error; By extracting real-time processing data and connecting the LSTM layer, the advance prediction of processing quality is realized; the real-time processing data returned by the DT-LSTM system includes power signal, position signal and feedback signal, which is represented as: (8) ； wherein X Pow , X Fee , X Pos respectively represent the real-time dynamic signal, feedback signal and position signal of the X-axis; Z Pow , Z Fee , Z Pos respectively represent the real-time dynamic signal, feedback signal and position signal of the Z-axis; C Pow , C Fee , C Pos respectively represent the real-time dynamic signal, feedback signal and position signal of the C-axis; Real-time dynamic signals realize accurate identification of machining state, and position signals and feedback signals reflect the shape and position accuracy of the workpiece; the real-time machining data of the DT-LSTM system at 0~t-1 time is transmitted to the LSTM layer as historical data, the LSTM layer outputs the predicted value according to the corresponding forget gate and update gate, and compares it with the real-time machining data at t time to determine the current prediction accuracy; if the prediction accuracy meets the algorithm requirements, the predicted value after t+1 time is further calculated, and the predicted value is transmitted to the industrial computer; the industrial computer adjusts the machining parameters according to the predicted value.

3. The machining process space-time feature fusion prediction visualization method according to claim 1, characterized in that, Step 3: Dimension reduction processing of real-time machining data in the DT-LSTM system is as follows: The machining process space-time feature fusion prediction visualization method driven by digital twinning and deep learning can improve the data processing efficiency, and the improvement of the processing efficiency is as follows: (9) ； Where T is the current process, M is the number of corresponding motion components in the workpiece machining process, N is the real-time data category of a single motion component, t is the machining time step; m is the number of motion components after dimension reduction, n is the real-time data category of a single motion component after dimension reduction, and s is the machining time.

4. The machining process space-time feature fusion prediction visualization method according to claim 1, characterized in that, Step 4: The real-time machining data space-time feature fusion visualization method based on the DT-LSTM system is as follows: In the machining of a cylindrical surface, the machining path with time as the variable presents a spiral rising real-time machining trajectory; in the machining of a cylindrical surface, the X-axis, Z-axis, C-axis and G-axis have their own physical meanings in the digital twinning system, the X-axis is the depth of cut, corresponding to the control diameter of the cylindrical surface, and is stationary in the machining of a single cylindrical surface; the Z-axis corresponds to the machining length of the cylindrical surface, and the machining trajectory is a uniform linear motion; the C-axis motion directly corresponds to the roundness of the cylindrical surface, and is a uniform rotary motion in the machining of a cylindrical surface; the G-axis corresponds to the machining parameters, and is a uniform rotary motion; The machine tool space coordinate system and the digital twinning system space coordinate system are respectively constructed under the machine tool mechanical coordinate system, and the coordinate origin and the X, Y and Z axes of the machine tool space coordinate system and the digital twinning system space coordinate system are one-to-one corresponding with the machine tool mechanical coordinate system. In the cylindrical surface, the collected workpiece real-time machining data points are processed by dimension reduction, and the cylindrical surface is projected to the xOz, xOy and yOz planes of the machine tool coordinate system in three views; the xOz and yOz planes are respectively the composite motion forms of the x-axis, y-axis and z-axis, and present a typical sinusoidal curve shape; the xOy plane is the composite motion form of the x-axis and y-axis, and is an ideal circular trajectory; the two-dimensional space data is reduced to three-axis motion forms represented by x, y and z; since x, y and z are actually virtual space axes generated by X, Z and C motion axes, they are represented by axes 1, 2 and 3 respectively; the motion forms of axes 1 and 2 are both sinusoidal motion, and the phases of the two are 90 degrees apart; axes 1 and 2 correspond to the motion of the machining point on the cylindrical surface respectively, while axis 3 is linear motion, representing the motion of the machining point on the cylindrical surface along the generatrix direction; accordingly, the space-time correspondence between the physical motion axes X, Z and C and the virtual space axes x, y and z in the machine tool digital twin system is established: (10)。 5. The machining process space-time feature fusion prediction and visualization method according to claim 1, characterized in that, Step 5: The machining data prediction method based on the DT-LSTM system is as follows: According to the dimension reduction processing method of real-time machining data in the DT-LSTM system established in the third step, one-dimensional machining data is mapped to three-dimensional space, and subsequent machining data is predicted; the machining data prediction process based on the DT-LSTM system is divided into three steps; Step 1 generates subsequent prediction data h1 p,... of step 1 according to current experimental data 0-h1 t; as the processing proceeds, the subsequent prediction data of step 1 is gradually replaced by subsequent experimental data h1 t+1,... of step 1, and on this basis, the LSTM unit parameters in formula 1 are verified according to the subsequent experimental data h1 t+1,... of step 1 and the prediction data of step 1, and when the error does not exceed the prediction set threshold δ t Step 2 is predicted according to the subsequent experimental data h1 t+1,... of step 1, and the prediction data h2 p,... of step 2 is generated; As the process proceeds, the prediction data of step 2 is replaced by the experimental data h2 t+1, …, and the LSTM unit parameters in formula 1 are verified accordingly; if the error exceeds the prediction set threshold δ t When the error exceeds the prediction set threshold δ, the corresponding control flow is executed, and step 2 is predicted again in combination with the experimental data h2 t+1, … of step 2, the parameter weight of formula 1 is iterated, and the prediction data h2 p, … of step 2 is regenerated, and finally the prediction data h3 p, … of step 3 is generated in combination with the current experimental data of step 2. Repeat the above history-prediction-verification steps to finally generate real-time machining data of the workpiece surface along with the entire machining process of the workpiece; Use the trained DT-LSTM system to predict real-time machining data, and obtain the predicted machining trajectory of the workpiece surface by reconstructing the predicted data in space; on this basis, the historical machining data and real-time machining data of the entire machining process are trained, predicted and verified; repeat the above steps to realize the training, prediction and verification of the machining data.

Images