Motor digital twin model construction method based on knowledge distillation
Through the construction method of motor digital twin model based on knowledge distillation, the motor characteristics are captured using 1D-CNN and LSTM networks, combined with dynamic temperature distillation training, the problem of taking into account both accuracy and real-time performance of the motor digital twin model is solved, and the lightweight and high-precision model migration is achieved, which is suitable for scenarios such as intelligent manufacturing and urban rail transit.
Patent Information
- Application Number
- CN202510501405.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-07-25
AI Technical Summary
The existing motor digital twin models are difficult to balance the accuracy and real-time, the dynamic feature modeling is insufficient, and the edge deployment adaptability is poor, which cannot meet the needs of real-time sensitive scenarios such as intelligent manufacturing and urban rail transit.
Using a knowledge distillation-based method, a motor digital twin model including teacher and student modules is constructed, and motor features are captured through 1D-CNN and LSTM networks, combining dynamic temperature distillation training and feature adaptation layers to achieve lightweight and high-precision model migration.
It realizes high-precision prediction with an error of less than 0.3mH under <20ms delay, reduces memory usage, adapts to edge device deployment, and meets the real-time and accuracy requirements of on-board equipment.
Smart Images
Figure CN120372959A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a digital twin construction system for motors, and particularly to a method for constructing a digital twin model of a motor based on knowledge distillation. Background Art
[0002] There are three major technical bottlenecks in traditional motor digital twin modeling: In the field of physical modeling, although the finite element method can achieve an accuracy of 0.1 mH level, the single simulation takes more than 15 minutes, and it is only applicable to steady-state conditions; although the lumped method compresses the calculation time to 1 ms, it ignores the leakage magnetic effect, resulting in 、 the predicted value being systematically 12% higher, and the temperature sensitivity being significant. In the field of data-driven, although the deep LSTM model reaches an average error of 0.18 mH, it occupies a large amount of memory; although the lightweight TCN solution reduces the number of parameters to 0.9 M, due to the loss of high-frequency features, the error increases when the rotational speed changes suddenly. In the field of model compression, the temperature (T = 5) of traditional knowledge distillation technology causes the soft label to be overly smoothed, and the prediction standard deviation of the student model reaches 0.25 mH, and the difference in the feature map size between the teacher-student models increases the KL divergence loss by 41%.
[0003] The core contradictions of the above methods are reflected in: 1) It is difficult to balance accuracy and real-time performance, and existing solutions cannot achieve an error of <0.3 mH with a delay of <20 ms; 2) The dynamic characteristic modeling is insufficient, and the prediction error of the motor transient conditions (rapid acceleration, regenerative braking) is relatively large; 3) The adaptability for edge deployment is poor, and large models are difficult to run on embedded devices with a memory of <1 GB. The technological evolution shows that the static knowledge distillation solutions before 2022 can only achieve an error of ±0.35 mH with a delay of 25 ms, and cannot meet the dual requirements of on-vehicle devices for real-time performance and accuracy. The root cause is that there are three defects in existing distillation methods: The setting of the temperature parameter does not match the time series characteristics, resulting in low efficiency of knowledge transfer of temporal correlation; there is an essential conflict between the complex architecture of the teacher model (such as multi-level cascaded LSTM) and the lightweight requirement of the student model; the mismatch of feature dimensions causes irreversible loss of high-frequency features during the knowledge distillation process. These defects seriously restrict the application of digital twin models in real-time sensitive scenarios such as intelligent manufacturing and urban rail transit. Summary of the Invention
[0004] In view of the above status quo and technical problems, the present invention provides a method for constructing a digital twin model of a motor based on knowledge distillation, which has many advantages such as simple structure and reliable function.
[0005] In view of the deficiencies of the prior art, the present invention proposes the following innovative technical solutions: A method for constructing a digital twin model of a motor based on knowledge distillation, comprising the following steps:
[0006] Step (1): Construct a multi - physical - field dataset for the permanent - magnet synchronous motor, including the d - axis inductance under different operating conditions , q - axis inductance ;
[0007] Step (2): Construct a knowledge - distillation model, including a teacher module and a student module; among them, the teacher model includes a data - information extraction module and a time - series feature extraction module based on the long - short - term memory network (LSTM) and 1D - CNN network; the student model is trained by the distilled teacher model
[0008] Step (3): Implement a feature - adaptation layer, which is used to adjust the feature representations of different data. Insert a learnable parameter matrix between the one - dimensional convolutional neural network (1D - CNN) and the long - short - term memory neural network (LSTM) in the time - series feature extraction module Implement this feature - adaptation layer
[0009] Step (4): Dynamic - temperature distillation training. Set a dynamic - temperature parameter decay function through distillation in different stages, and adopt different learning rates in different stages to meet the requirements of dynamic changes. Calculate the temperature - weighted KL - divergence loss , and weight it proportionally with the task loss . Transmit the error gradient to the distillation model through loss feedback, driving the student model to imitate the knowledge expression of the teacher model; if the standard is met, the training is completed
[0010] Compared with the prior art, the present invention has the following advantages
[0011] The present invention uses a simulation dataset to construct a digital - twin model, which can make the error value within 0.2 mH, effectively solving the problem of model accuracy
[0012] Adopt the student model for lightweight and miniaturized result migration, greatly reducing the memory occupation
[0013] Capture the local features of the motor inductance signal through 1D - CNN, and establish the long - term time - varying characteristics of the inductance parameters through the LSTM module Brief Description of the Drawings
[0014] In order to more clearly explain the technical features and characteristics of the present invention, the present invention will be further described in detail below according to the specific embodiments of the present invention and in conjunction with the drawings, where
[0015] Figure 1 is a schematic diagram of the knowledge - distillation model of the present invention
[0016] Figure 2 is a schematic diagram of the double - layer 1D - CNN convolutional - neural - network structure of the present invention
[0017] Figure 3 It is a schematic diagram of the three - layer LSTM structure of the present invention;
[0018] Figure 4 It is a schematic diagram of the process of the knowledge distillation model based on CNN + LSTM of the present invention;
[0019] Figure 5 It is a schematic diagram of the finite - element model of the built - in permanent - magnet synchronous motor for simulation;
[0020] Figure 6 It is the data training set;
[0021] Figure 7 It is a schematic diagram of the training result of the traditional teacher model;
[0022] Figure 8 It is a schematic diagram of the training result of the traditional student model;
[0023] Figure 9 It is a schematic diagram of the training result of the teacher model of the present invention;
[0024] Figure 10 It is a schematic diagram of the training result of the student model of the present invention;
[0025] Figure 11 It is the error heat map of the training result of the teacher model of the present invention;
[0026] Figure 12 It is the error heat map of the training result of the student model of the present invention. Detailed implementation manners
[0027] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above - mentioned objectives, the present invention adopts the following technical solutions.
[0028] The concept of the present invention is: using a 1D convolutional neural network to capture the local features of the signal, using an LSTM network to obtain the temporal features of the signal, and using the training result of the teacher model as an example for training the student model, so as to construct a lightweight and miniaturized digital model to complete the construction of the digital twin system.
[0029] More specifically, the present invention provides a method for constructing a motor digital twin model based on knowledge distillation, including the following steps:
[0030] Step (1), construct a multi - physical - field dataset of the permanent - magnet synchronous motor, and use Ansys Maxwell simulation software to analyze the d - axis inductance under different working conditions 、q-axis inductance ;
[0031] Step (2), construct a knowledge distillation model, including a teacher module and a student module; among them, the teacher model includes a data information extraction module based on 1D-CNN and a temporal feature extraction module based on the long short-term memory network LSTM network and 1D-CNN network; the student model is trained by the distilled teacher model;
[0032] The 1D-CNN data information extraction module is used to process the temporal signal, and the convolutional kernel slides along the time axis to capture microsecond-level transient features.
[0033] The temporal feature extraction module is used to capture the cross-time-step dependencies of the sequence data, jointly extract temporal information in the digital layer, and facilitate the training of an accurate large-data teacher model. The teacher model is used to train a miniaturized and lightweight student model; the student model is used for real-time data interaction, provides data reference for the entire digital twin structure, and realizes the lightweight deployment requirements. Among them, the one-dimensional convolutional neural network 1D-CNN contains two layers of convolution (number of channels 16→8, convolutional kernel width 3), and the LSTM network is a three-layer stacked structure (hidden state 64, dropout rate 0.2);
[0034] Step (3), implement a feature adaptation layer, which is used to adjust the feature representations of different data, and insert a learnable parameter matrix between the one-dimensional convolutional neural network 1D-CNN and the long short-term memory network LSTM neural network in the temporal feature extraction module to make the distribution of the input features closer to the mapping of the target feature distribution. The initialization method of the learnable parameter matrix is as follows: Weight initialization; The specific weight initialization is: by adjusting the initial range of the weights according to the dimensions of the input and output, the variance of the activation function values of each layer is kept consistent, so as to accelerate the convergence of the model. The formula is: ;
[0035] Among them, are the input and output dimensions respectively, is the above-mentioned learnable parameter matrix.
[0036] When the feature adaptation layer realizes dimension expansion, bilinear interpolation is used to compensate the feature values, which is beneficial to maintaining the continuity of the entire space:
[0037] ;
[0038] is the output feature vector, is the learnable weight matrix, and k and n are the original index and the target index respectively.
[0039] Step (4), dynamic temperature distillation training, set a dynamic temperature parameter decay function through distillation in different stages , and adopt different learning rates in different stages to meet the requirements of dynamic changes, calculate the temperature-weighted KL divergence loss , and combine it with the task loss Weighted at a ratio of 7:3, and transmit the error gradient to the distillation model through loss feedback to drive the student model to imitate the knowledge expression of the teacher model; if the standard is met, the training is completed.
[0040] In step (2), the specific structure of the teacher model is as follows:
[0041] (1) Temporal feature extraction module
[0042] Input layer: Input 2D temporal data ( , ), and the normalization formula is:
[0043] , ;
[0044] The first 1D-CNN layer: 16 convolution kernels with a width of 3, ReLU activation, and zero padding to maintain the sequence length;
[0045] Max pooling layer: Pooling window 2, stride 2;
[0046] The second 1D-CNN layer: 8 convolution kernels with a width of 3, and the output feature dimension is 16;
[0047] Feature adaptation layer: Perform a linear transformation in the feature adaptation layer, and the formula is: ; Among them, is the output feature vector, is the learnable parameter matrix, is the input feature vector, is the learnable bias vector; LSTM network: Hidden state dimension 64, sequence-to-sequence output mode; Output layer: The fully connected layer outputs , predicted values, and the activation function is Sigmoid.
[0048] (2) Data information extraction module
[0049] Extract high-fidelity data from the input original data information through a CNN neural network, and complete the extraction of the input data through local perception and weight sharing.
[0050] In step (4), the dynamic temperature distillation training is specifically set as follows:
[0051] Temperature-weighted KL divergence loss formula:
[0052] ;
[0053] Where is the output of the i-th class of the Softmax function of the teacher model at temperature T, is the output of the i-th class of the Softmax function of the student model at temperature T, is the input of the teacher model, is the input of the student model, is the weighting coefficient.
[0054] In the teacher model architecture, a dual-channel structure of 1D-CNN and LSTM is adopted, where:
[0055] The 1D-CNN neural network configures a two-layer convolutional network (16→8 channels, convolution kernel width 3) to extract local time-domain features (time window <10ms) of the current / speed signal;
[0056] A three-layer stacked LSTM neural network (hidden state 64, Dropout rate 0.2) captures the long-term time-varying characteristics of the inductance parameters (time constant >100ms);
[0057] The feature adaptation layer realizes the mapping from a 16-dimensional to a 64-dimensional feature space through the learnable matrix to solve the problem of dimension mismatch;
[0058] In the dynamic temperature distillation strategy, an exponentially decaying temperature function is designed: ; realizing three-stage knowledge transfer:
[0059] In the initial stage of training (t = 0 - 500 epochs): T = 5 - 3, enhancing soft transfer;
[0060] In the middle stage of training (t = 501 - 800 epochs): T = 3 - 1.8, balancing hard and soft label supervision;
[0061] In the late stage of training (t = 801 - 1000 epochs): T = 1.8 - 1.0, focusing on key feature learning.
[0062] Figures 1 - 4 FIG. is a schematic structural diagram of the motor digital twin technology based on knowledge distillation provided by the present invention, which includes: a knowledge distillation model, a two-layer 1D convolutional neural network structure, and a three-layer LSTM neural network structure.
[0063] Figure 1Schematic diagram of the knowledge distillation model provided by the present invention Figure 4 Flowchart of the knowledge distillation model training provided by the present invention, which includes a teacher model and a student model. The training of the teacher model on the left starts with data input. After the original data is preprocessed by normalization, feature mining is performed through a dual-branch parallel network: the CNN+LSTM combined network focuses on extracting dynamic patterns in the time dimension, while the independent CNN branch captures static information in the data space dimension. After the dual-branch features are fused in the parameter calculation module, the model weights and biases are updated, and based on the preset iteration conditions, it is determined whether to terminate the training. The training of the student model on the right takes the training data generated by the teacher model as input, and synchronously obtains the real data and the "soft labels" output by the teacher model. After the differences between the two are quantified by the KL divergence loss function, the system determines the training status according to the convergence: if not up to the standard, the error gradient is transmitted to the distillation module through the loss feedback mechanism to drive the student model to imitate the knowledge expression of the teacher model; if up to the standard, the training is completed.
[0064] Figure 2 Schematic diagram of the structure of the double-layer 1D convolutional neural network provided by the present invention, which includes a double-layer convolutional pooling module, a fully connected layer, and an output layer. Through this structure, one-dimensional sequence data can be effectively processed.
[0065] This structure contains at least 2 convolutional structures, which include multiple convolutional kernels for extracting the feature information of the data. Compared with the traditional neural network structure, the CNN convolutional layer structure can achieve parameter sharing, that is, the relationship between two layers of neurons is not one-to-one, but one-to-many, which avoids different neurons from learning the same information, reduces the amount of information calculation, and improves the training efficiency of the model.
[0066] Figure 3 Schematic diagram of the LSTM structure provided by the present invention. Taking the first-layer structure diagram as an example, in the figure 、 are input signals, is the long-term memory input signal, 、 are output signals, is the output of the forgetting gate, are the information update values of the intermediate structure respectively, is the activation function, represents the vector bias.
[0067] ;
[0068] ;
[0069] ;
[0070] ;
[0071] ;
[0072] 。
[0073] This structure is improved from the RNN neural network. The RNN neural network relies on a large amount of data and does not have the function of storing data, which will cause data loss. LSTM has a memory function by adding a cell state unit, so that information can be continuously transmitted for a long time. The structure of LSTM is mainly a gate structure, namely, forget gate, memory gate, and output gate. Such a structure enables selective memory of information during the information transmission process, avoiding the problem of gradient disappearance in the case of a large amount of data to a certain extent, and can continuously update information at each moment, making the entire network output reasonable information.
[0074] In this model, first, the data is divided. In this paper, 50% is taken as the training set and 50% as the prediction set as the input of CNN. The CNN convolutional layer adopts a double-layer convolutional structure. The output channels of the first layer are 16 channels, and the second layer is 8 channels. The width of the convolutional kernel is set to 3. The pooling layer adopts the maximum pooling method, and the pooling stride is 2, which is related to the sequence length. The output feature dimension is 16. A feature adaptation layer is added to adapt to the 64-dimensional feature space in the teacher LSTM model to solve the problem of dimension mismatch. The hidden state dimension of the LSTM model is 64, and 3 layers of LSTM are stacked to enhance the nonlinear modeling ability of the model. The student model is used to imitate the trained teacher model to achieve model compression and knowledge transfer.
[0075] Figure 5 is the finite element simulation model of the motor. The three-phase excitation of the finite element simulation model is set as the rated excitation , the simulation time is 300 ms, and the rated speed is 900 rpm.
[0076] Figure 6 is the data training set. The training data set uses the finite element model , simulation data.
[0077] From Figure 7 , Figure 8 's simulation results, it can be seen that the training accuracy of the traditional model without using the present invention is significantly poor, which is not conducive to the accuracy requirements of the digital twin model data.
[0078] From Figure 9 , Figure 10From the training results, it can be seen that the training results of the model construction method provided by the present invention are good. The relative deviation of the teacher model does not exceed 0.5%, and the error is within 0.2 mH. The relative deviation of the student model does not exceed 1%, and the error is within 0.5 mH.
[0079] From Figure 11 , Figure 12 the training error heat map, it can be seen that the overall error range of the training results of the model of the present invention is very small, meeting the accuracy requirements for the construction of the motor digital twin model and achieving the accuracy of the twin data
[0080] Obviously, the above-described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.
Claims
1. A method for constructing a digital twin model of an electric motor based on knowledge distillation, characterized in that, It includes the following steps: Step (1): Construct a multi-physical field dataset for a permanent magnet synchronous motor, including the d-axis inductance under different working conditions , the q-axis inductance ; Step (2), construct a knowledge distillation model, including a teacher module and a student module; among them, the teacher model includes a data information extraction module and a temporal feature extraction module based on a long short-term memory network (LSTM network) and a 1D-CNN network; the student model is trained by the distilled teacher model; Step (3): Implement the feature adaptation layer, which is used to adjust the feature representations of different data, and insert a learnable parameter matrix between the one-dimensional convolutional neural network 1D-CNN and the long short-term memory network LSTM neural network in the time series feature extraction module Implement this feature adaptation layer; Step (4), dynamic temperature distillation training, set a dynamic temperature parameter decay function through distillation in different stages, and adopt different learning rates in different stages to meet the requirements of dynamic changes, calculate the temperature-weighted KL divergence loss , and weighted with the task loss proportionally, transfer the error gradient to the distillation model through loss feedback, and drive the student model to imitate the knowledge expression of the teacher model; if the standard is met, the training is completed.
2. The method for constructing a motor digital twin model based on knowledge distillation according to claim 1, characterized in that, The data information extraction module is used to process temporal signals, and the convolutional kernel slides along the time axis to capture microsecond-level transient features.
3. A method for constructing a motor digital twin model based on knowledge distillation according to claim 1, characterized in that, The temporal feature extraction module is used to capture the cross-time-step dependencies of sequence data and jointly extract temporal information in the digital layer to facilitate the training of the teacher model.
4. A method for constructing a motor digital twin model based on knowledge distillation according to claim 1, characterized in that, The feature adaptation layer makes the distribution of the input features closer to the mapping of the target feature distribution. Initialize the learnable parameter matrix of the weights ; The initialization is specifically as follows: By adjusting the initial range of the weights according to the dimensions of the input and output, the variances of the activation function values of each layer are kept consistent, thereby accelerating the convergence of the model. The formula is: ; Among them, are the input and output dimensions respectively, is a learnable parameter matrix.
5. A method for constructing a motor digital twin model based on knowledge distillation according to claim 1, characterized in that, The temporal feature extraction module includes: Input layer: Input 2D time-series data , , and the normalization formula is: ， ； The first 1D-CNN layer: 16 convolutional kernels with a width of 3, ReLU activation, and zero padding to maintain the sequence length; The max pooling layer: a pooling window of 2 and a stride of 2; The second 1D-CNN layer: 8 convolutional kernels with a width of 3, and the output feature dimension is 16; The feature adaptation layer: a linear transformation is performed in the feature adaptation layer, and the formula is: ； Among them, is the output feature vector, is the learnable parameter matrix, is the input feature vector, is the learnable bias vector; The LSTM network: the hidden state dimension is 64, and the sequence-to-sequence output mode; Output layer: Output of the fully connected layer , Predicted value, with the activation function being Sigmoid.
6. The method for constructing a motor digital twin model based on knowledge distillation according to claim 5, wherein When the feature adaptation layer realizes dimension expansion, bilinear interpolation is used to compensate for feature values: ； is the output feature vector, is a learnable weight matrix, is the input feature vector, and k and n are the original index and the target index respectively.
7. A method for constructing a digital twin model of a motor based on knowledge distillation according to claim 1, characterized in that In the data information extraction module, high-fidelity data extraction is performed on the input original data information through a CNN neural network, and the input data is extracted through local perception and weight sharing.
8. A method for constructing a motor digital twin model based on knowledge distillation according to claim 1, characterized in that, In step (4), the dynamic temperature distillation training is specifically set as follows: Design an exponential decay temperature function: ; Realize three-stage knowledge transfer: In the initial stage of training, t = 0 - 500 rounds: T = 5 - 3, enhance soft transfer; In the middle stage of training, t = 501 - 800 rounds: T = 3 - 1.8, balance hard and soft label supervision; In the later stage of training, t = 801 - 1000 rounds: T = 1.8 - 1.0, focus on key feature learning.
9. A method for constructing a motor digital twin model based on knowledge distillation according to claim 1, characterized in that, In step (4), the temperature-weighted KL divergence loss formula: ； wherein, is the output of the i-th class of the Softmax function of the teacher model at temperature T, is the output of the i-th class of the Softmax function of the student model at temperature T, is the input of the teacher model, is the input of the student model, is the weighting coefficient.
10. A method for constructing a motor digital twin model based on knowledge distillation according to claim 5, characterized in that, In the three-layer stacked LSTM neural network, the specific expression formula for each layer is: ； ； ； ； ； ； Among them, and are input signals, is the long-term memory input signal, and are output signals, is the forget gate output, are respectively the information update values of the intermediate structure, is the activation function, represents the vector bias.