Comprehensive transmission device anomaly detection method and system based on depth feature mining
Through deep learning methods and improved CNN-LSTM model, combined with principal component analysis and segmentation processing, the detection problems of complex features and time dependencies in the transmission device are solved, and the accuracy and robustness of abnormal detection are improved.
Patent Information
- Application Number
- CN202410954629.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art is difficult to effectively capture the complex characteristics and time dependencies inside the integrated transmission device, resulting in insufficient accuracy and robustness of abnormal detection.
Using a deep learning-based method, dimensionality reduction data is processed through principal component analysis and segmented processing, combined with the improved CNN-LSTM model, the adaptive weighting method is used to perform feature fusion to extract key features in multidimensional time series data.
It improves the accuracy and robustness of abnormal detection, significantly enhances the detection performance of the integrated transmission device, and reduces the computational complexity and model training speed.
Smart Images

Figure CN119004306A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of armored vehicle transmission device anomaly detection, and in particular to a comprehensive transmission device anomaly detection method and system based on deep feature mining. Background Art
[0002] The structure of the integrated transmission device is complex and contains many parts. Its operating status is affected by many factors, such as speed, temperature, pressure, etc. These data are high-dimensional and interrelated. The challenge of using deep learning methods for anomaly detection is how to effectively extract and fuse the key features in these high-dimensional time series data, especially under complex working conditions. Therefore, how to capture the complex features and time dependencies within the integrated transmission device and improve the accuracy and robustness of anomaly detection has become a technical problem that needs to be solved urgently. Summary of the invention
[0003] In view of the above problems, the purpose of the present invention is to provide a comprehensive transmission device anomaly detection method and system based on deep feature mining, which can effectively utilize deep learning methods to process complex and changeable high-dimensional time series data to improve the accuracy and efficiency of transmission device anomaly detection.
[0004] To achieve the above-mentioned purpose, in the first aspect, the technical solution adopted by the present invention is: a method for detecting anomalies of an integrated transmission device based on deep feature mining, which comprises: recording the original data of key parts of the integrated transmission device monitored by multiple sensors into multi-dimensional time series data, and performing preliminary dimensionality reduction through the principal component analysis method to obtain the reduced-dimensional data; segmenting the reduced-dimensional data to obtain multiple groups of sequence fragments, and further reducing the dimensionality of each subsequence fragment in each group of sequence fragments through PAA, and dividing the further reduced-dimensional sequence fragments into a training set and a test set; training the improved CNN-LSTM model through the training set data to obtain the trained optimal model, and using the trained optimal model to detect the test set data to determine whether the data has anomalies.
[0005] Furthermore, the principal component analysis method is used to perform preliminary dimensionality reduction, and the reduced dimensionality data is obtained, including:
[0006] Perform Min-Max normalization on the multidimensional time series data to obtain standardized data, and calculate the covariance matrix of the standardized data;
[0007] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues and corresponding eigenvectors;
[0008] According to the proportion of a single eigenvalue of the covariance matrix in all eigenvalues, several component data whose proportions meet the set conditions are selected as principal components;
[0009] The selected eigenvectors are combined into a projection matrix to map the original data into a new principal component space for preliminary dimensionality reduction, and the reduced dimensionality data are weightedly fused as the final reduced dimensionality data.
[0010] Furthermore, the reduced-dimensional data is weighted and fused, including:
[0011] The multi-layer perceptron model is used to learn the weight of each feature to perform adaptive weight setting and dynamically adjust the weight of each feature;
[0012] The generated weights are applied to the features after PCA and weighted fusion is performed.
[0013] Furthermore, the data after dimensionality reduction is segmented to obtain multiple groups of sequence segments, and each subsequence segment in each group of sequence segments is further reduced in dimensionality by PAA, including:
[0014] Determine the PAA dimension reduction window size, set the window size of the undivided window from small to large, perform PAA data dimension reduction on each window size, calculate the statistical indicators of each subsequence after dimension reduction, and select the largest window with no qualitative change in the trend of the data statistical indicators as the selected window size; wherein the data statistical indicators have comprehensive characteristics of data shape and variation;
[0015] According to the selected window size, the sliding window method is used to divide the data into multiple overlapping subsequences;
[0016] Through PAA, the average value of each subsequence is used to replace the original subsequence, further reducing the dimension of the subsequence.
[0017] Furthermore, the statistical indicators are:
[0018]
[0019] Among them, K is the data statistical index; x i represents the i-th value of the time series; N is the length of the time series; sqrt is the root mean square; μ is the mean.
[0020] Furthermore, the improved CNN-LSTM model includes: parallel CNN and LSTM;
[0021] The data is respectively extracted from signal features by CNN and LSTM, and the signal features are converted into one-dimensional vectors using adaptive weighting method;
[0022] The two one-dimensional vectors are fused according to the weighted concatenation method, and the anomaly detection category is output after passing through the fully connected layer;
[0023] Among them, the two one-dimensional vectors are feature fused according to the weighted splicing method, including:
[0024] Initialize the weights of the features extracted by the CNN module and the LSTM module to obtain the initialized feature fusion weight λ;
[0025] In the forward propagation stage, in the fusion layer, the input feature vectors are combined by weight: V F =λV CNN +(1-λ)V LSTM , where V CNN 、V LSTM The data are respectively extracted through CNN and LSTM to extract signal features, V F is the fused feature;
[0026] Compare the output of the improved CNN-LSTM model with the output target and calculate the cross entropy loss function;
[0027] In the back-propagation phase, the chain rule is used to calculate the gradient of the loss function about the fusion feature weight λ through each layer of the model;
[0028] According to the gradient of the feature fusion weight, the stochastic gradient descent Adam algorithm is applied to update the weight: Among them, λ t+1 is the current weight, η is the learning rate, is the gradient of the loss function with respect to the weights;
[0029] The above process is repeated until the model converges or reaches the predetermined training rounds; as the training progresses, the feature fusion weights will be continuously adjusted to minimize the loss function, thereby finding the optimal feature fusion ratio.
[0030] Furthermore, CNN adopts a 1D CNN structure, which consists of 1 convolutional layer with 4 convolution kernels, 1 pooling layer and 2 fully connected layers;
[0031] Extract local spatiotemporal features from the input sequence through convolutional and pooling layers;
[0032] A residual connection branch is introduced in the convolution layer. The branch consists of one convolution kernel and the other three convolution kernels are connected in series. After the branch is connected in parallel with the output of the first convolution kernel, it is added to the output of the last convolution kernel in series as the output of the convolution layer.
[0033] The convolution operation moves in the time dimension to achieve feature extraction of sequence changes. The length of the convolution kernel is smaller than the length of the input sequence, and weights are shared in the convolution operation.
[0034] In the second aspect, the technical solution adopted by the present invention is: a comprehensive transmission device anomaly detection system based on deep feature mining, which includes: a preliminary dimensionality reduction module, which records the original data of the key parts of the comprehensive transmission device monitored by various sensors into multi-dimensional time series data, and performs preliminary dimensionality reduction through the principal component analysis method to obtain the reduced dimensionality data; a secondary dimensionality reduction module, which segments the reduced dimensionality data to obtain multiple groups of sequence fragments, and further reduces the dimensionality of each subsequence fragment in each group of sequence fragments through PAA, and divides the further reduced dimensionality sequence fragments into a training set and a test set; an anomaly detection module, which trains the improved CNN-LSTM model through the training set data to obtain the trained optimal model, and uses the trained optimal model to detect the test set data to determine whether the data has an anomaly.
[0035] In a third aspect, the technical solution adopted by the present invention is: a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes any one of the above methods.
[0036] In a fourth aspect, the technical solution adopted by the present invention is: a computing device, comprising: one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the above methods.
[0037] The present invention adopts the above technical solution, which has the following advantages:
[0038] 1. The present invention reduces the dimension through PCA, reduces the redundancy of data, speeds up model training, and reduces computational complexity, thereby improving the efficiency and accuracy of anomaly detection.
[0039] 2. The present invention uses the PAA method to make the data processing process more efficient and significantly improve the calculation speed. The reduced-dimensional data can be more easily modeled and analyzed in subsequent machine learning and signal processing tasks, thereby improving the performance and accuracy of the overall algorithm.
[0040] 3. The present invention can detect abnormal situations more accurately, improve the detection performance of the model, and significantly enhance the accuracy and robustness of abnormality detection of the integrated transmission device.
[0041] 4. Through the residual connection structure, the model of the present invention can be trained more stably and optimized more easily, further improving the effect of anomaly detection and the convergence speed of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1is a flow chart of a comprehensive transmission device abnormality detection method based on deep feature mining in an embodiment of the present invention;
[0043] Figure 2 Schematic diagram of an improved CNN-LSTM model in an embodiment of the present invention;
[0044] Figure 3 Schematic diagram of the contribution of different components in the embodiment of the present invention;
[0045] Figure 4 is the confusion matrix of the detection result in the embodiment of the present invention
[0046] Figure 5a This is a comparison chart of the effects of different model training sets in the embodiments of the present invention;
[0047] Figure 5b It is a comparison chart of the test set effects of different models in the embodiments of the present invention. DETAILED DESCRIPTION
[0048] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.
[0049] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0050] The integrated transmission is an important part of tracked armored vehicles, which is used to transmit engine power to the vehicle's movement mechanism and realize the vehicle's gear shifting and steering control. The reliability of the integrated transmission is crucial to ensure the normal operation of the characteristic vehicle. Abnormal detection of the integrated transmission can detect potential problems of the transmission early, take preventive measures, reduce unnecessary downtime and reduce maintenance costs. Therefore, abnormal detection is one of the key technologies to ensure the safe and reliable operation of the integrated transmission. The integrated transmission contains a variety of sensors. How to judge the health status of the vehicle from the monitoring data of these sensors is an important issue at present.
[0051] Most of the existing condition monitoring technologies are based on multidimensional time series data, such as speed, temperature, pressure, etc., which can reflect the working status of each component of the transmission system in real time. Any changes and anomalies in the internal components will be manifested as characteristic changes in the multidimensional time series data. By processing and decomposing these data, the working status information of the components can be directly obtained. However, when dealing with massive high-dimensional data under complex working conditions, traditional machine learning methods often find it difficult to capture the deep features in the data, and the effect is not satisfactory. In contrast, deep learning methods have gradually become the main means of anomaly detection due to their strong adaptability and ability to process large-scale data. Among them, convolutional neural networks (CNNs) perform well in feature extraction and can automatically extract spatial features in data; long short-term memory networks (LSTMs) have significant advantages in processing time series data and can capture the temporal dependencies in data. Combining the advantages of these two models, more accurate anomaly detection can be achieved.
[0052] Therefore, the present invention proposes a comprehensive transmission device anomaly detection method and system based on deep feature mining, and introduces an improved CNN-LSTM fusion model with a residual connection structure. The model uses CNN and LSTM to extract the features of multidimensional time series data respectively, and obtains more comprehensive feature information through fusion; the residual connection structure is used to optimize the CNN layer to accelerate the convergence speed and improve the detection effect. The present invention was experimentally verified by building a comprehensive transmission device test bench, showing the superior performance of the present invention in comprehensive transmission device anomaly detection.
[0053] In one embodiment of the present invention, a method for detecting anomalies of an integrated transmission device based on deep feature mining is provided, which can capture the deep learning model of the complex features and time dependencies within the integrated transmission device to improve the accuracy and robustness of anomaly detection. In this embodiment, Figure 1 As shown, the method comprises the following steps:
[0054] 1) The original data of the key parts of the integrated transmission device monitored by various sensors are recorded as multi-dimensional time series data, and the principal component analysis (PCA) method is used to perform preliminary dimensionality reduction to obtain the reduced-dimensional data;
[0055] 2) Segment the reduced-dimensional data to obtain multiple groups of sequence segments, and further reduce the dimension of each subsequence segment in each group of sequence segments through PAA (segmented aggregation approximation), and divide the sequence segments after further dimension reduction into training set and test set;
[0056] 3) The improved CNN-LSTM model is trained with the training set data to obtain the trained optimal model, and the trained optimal model is used to detect the test set data to determine whether there are any abnormalities in the data.
[0057] In the above step 1), various sensors are used to collect various sensor monitoring data such as the rotation speed, temperature, pressure, etc. of the integrated transmission device to obtain sample multi-dimensional time series data X.
[0058] Specifically, various types of sensors are installed at key parts of the transmission device, including speed sensors, temperature sensors, and pressure sensors. When the transmission device is running, the monitoring data of each sensor is collected in real time through the data acquisition system and recorded as multi-dimensional time series data X. The data includes the monitoring values of each sensor under different operating conditions to ensure data diversity.
[0059] In the above step 1), the principal component analysis method is used to perform preliminary dimensionality reduction to obtain the reduced dimensionality data, which includes the following steps:
[0060] 1.1) Perform Min-Max normalization on the multidimensional time series data to obtain the normalized data x*, and calculate the covariance matrix S of the normalized data;
[0061] Specifically, the Min-Max standardization process is used to ensure that each variable is on the same scale to prevent certain variables from having a disproportionate impact on the results of the principal component analysis. The standardized data x* is:
[0062]
[0063] In the formula, x represents the original data, x min Represents the minimum value of the original data, x max Indicates the maximum value of the original data.
[0064] The covariance matrix reveals the linear correlation between the variables in the data set and provides a basis for PCA analysis. The covariance matrix S is:
[0065]
[0066] In the formula, m represents the number of original variables, X represents the standardized data, represents the mean of the standardized data and T is the transpose.
[0067] 1.2) Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalue λ and the corresponding eigenvector V;
[0068] Among them, the eigenvector represents the main direction in the original data, and the eigenvalue represents the degree of change of the data along these directions. Perform eigenvalue decomposition on the covariance matrix:
[0069] SV=λV
[0070] 1.3) According to the proportion of a single eigenvalue of the covariance matrix in all eigenvalues (referred to as variance contribution rate), select several component data whose proportions meet the set conditions as principal components, and obtain the number of principal components p;
[0071] Among them, you can choose one of the following two conditions to set the condition:
[0072] The first method: selection based on the inflection point principle: plot the variance contribution rates of all components in a line graph from large to small, take the inflection points of the line as nodes, and select all the eigenvectors of the previous part as the principal components.
[0073] The second method is to select the principal components according to a fixed threshold, and arrange the variance contribution rates of all components from large to small. If the cumulative variance contribution rate of the first n principal components reaches the threshold (usually 85% to 95%), then select this part of the eigenvectors as the principal components.
[0074] The variance contribution rate is:
[0075]
[0076] In the formula, α i Expressed as the variance contribution rate of the i-th principal component, λ i It is represented as the i-th eigenvalue, and p represents the number of all eigenvalues (i.e. the number of principal components).
[0077] 1.4) Combine the selected eigenvectors into a projection matrix W = V T , the original data is mapped into the new principal component space for preliminary dimensionality reduction, and the reduced dimensionality data is weightedly fused as the final reduced dimensionality data.
[0078] In this embodiment, the data is mapped to the principal component space by multiplying the original data by the projection matrix, thereby completing the dimensionality reduction representation of the data. The calculation method is:
[0079]
[0080] In this embodiment, weighted fusion is performed on the data after dimensionality reduction, including the following steps:
[0081] 1.4.1) Use the multi-layer perceptron (MLP) model to learn the weight of each feature to perform adaptive weight setting and dynamically adjust the weight of each feature to improve the effect of feature fusion;
[0082] By training the MLP model, it can adaptively generate weights based on input features;
[0083] 1.4.2) Apply the generated weights to the features after PCA and perform weighted fusion.
[0084] In this embodiment, when used, the multidimensional time series data is reduced in dimension by the principal component analysis (PCA) method, the main information in the data is retained, and the data after dimension reduction is weighted and fused. PCA converts high-dimensional data into low-dimensional data, effectively reduces the dimension of the data, improves computational efficiency, and reduces the consumption of computing resources. At the same time, the data after dimension reduction retains the main features in the original data, ensuring that the subsequent model can accurately identify abnormal situations in the data. By using a multi-layer perceptron (MLP) after PCA dimension reduction to obtain adaptive weights for weighted fusion, important features in the data can be dynamically highlighted, thereby generating more expressive features. Adaptive weighted fusion allows the influence of features to change flexibly, captures more complex patterns and relationships, and further enhances the predictive and generalization capabilities of the model.
[0085] In the above step 2), the data after dimensionality reduction is segmented to obtain multiple groups of sequence segments, and each subsequence segment in each group of sequence segments is further reduced in dimensionality by PAA, including the following steps:
[0086] 2.1) Determine the PAA dimension reduction window size, set the window size of the undivided window from small to large, perform PAA data dimension reduction on each window size, calculate the statistical indicators of each subsequence after dimension reduction, and select the largest window with no qualitative change in the trend of the data statistical indicators as the selected window size to retain key information;
[0087] In this embodiment, a data statistical index that comprehensively characterizes the data shape and variation characteristics is constructed. The calculation method of the index K is:
[0088]
[0089] Among them, x i represents the i-th value of the time series; N is the length of the time series; sqrt is the root mean square; μ is the mean:
[0090]
[0091] 2.2) Based on the selected window size, the sliding window method is used to divide the data into multiple overlapping subsequences;
[0092] In this embodiment, preferably, the window stride is 100 and the window width is 300.
[0093] 2.3) The average value of each subsequence is used to replace the original subsequence through PAA to further reduce the dimension of the subsequence.
[0094] The basic principle of PAA is to segment the original time series into subsequences of uniform length and then replace the original data with the average value of each subsequence. This can effectively reduce the amount of data while retaining the trend of the original data as much as possible.
[0095] Simplify a time series of length N into a new series Y = y 1 , y 2 ,..., y M of length M, where M < N. The subsequence segmentation process includes the following steps:
[0096] 2.3.1) Divide the original sequence X into M equal-length segments, each segment having a length. If N is not divisible by M, the length of the last segment may be slightly different.
[0097] 2.3.2) Perform aggregation, calculate the mean value of each segment, and use this mean value to represent the data of that segment. Each y i is the mean value of all data points in its corresponding segment:
[0098]
[0099] where i = 1, 2,..., M. If N is not divisible by M, the last segment y i needs to adjust the calculation method according to the actual number of data points included.
[0100] In this embodiment, when in use, PAA represents the entire sequence by segmenting the time series and calculating the average value of each segment, thereby achieving data dimensionality reduction. Specifically, PAA divides the original time series into several equal-length segments, and the average value of each segment represents the characteristics of that segment. Through PAA, the dimension of the time series can be effectively reduced while retaining the main characteristics of the data, enabling the data after dimensionality reduction to greatly reduce the computational complexity and storage space requirements while maintaining the original information.
[0101] In step 3) above, in this embodiment, the preferred ratio of the training set to the test set is 4:1.
[0102] In step 3) above, as Figure 2 shown, the improved CNN-LSTM model includes: a parallel CNN and LSTM; the parallel structure can utilize the advantages of both CNN and LSTM at the same time. The CNN module is good at capturing local spatio-temporal features, and the LSTM module is good at capturing long-term and short-term dependencies in the sequence. By fusing the two types of features, the model can more comprehensively understand the signal characteristics of the transmission device, reduce feature loss, and improve the detection accuracy.
[0103] Specifically, the data extracts signal features through CNN and LSTM respectively, and uses the adaptive weighting method to convert the signal features into one-dimensional vectors;
[0104] The two one-dimensional vectors are fused according to the weighted splicing method, and the anomaly detection category is output after passing through the fully connected layer. The optimal weight value of the feature splicing is found through the back propagation algorithm and gradient descent optimization in the model training process. The two one-dimensional vectors are fused according to the weighted splicing method, which specifically includes the following steps:
[0105] 3.1) Initialize weights:
[0106] In the model initialization stage, the weights are usually randomly initialized, and the weights of the features extracted by the CNN module and the LSTM module are initialized to obtain the initialized feature fusion weight λ;
[0107] Preferably, in this embodiment, the weights of feature extraction by the CNN module and the LSTM module are both initialized to 0.5.
[0108] 3.2) Forward propagation:
[0109] In the forward propagation stage, in the fusion layer, the input feature vectors are combined by weighting:
[0110] V F =λV CNN +(1-λ)V LSTM
[0111] Where V CNN 、V LSTM The data are respectively extracted through CNN and LSTM to extract signal features, V F The fused features.
[0112] 3.3) Calculate the loss:
[0113] The output of the improved CNN-LSTM model is compared with the output target and the cross entropy loss function is calculated.
[0114] 3.4) Back Propagation:
[0115] In the back-propagation phase, the chain rule is used to calculate the gradient of the loss function with respect to the fusion feature weight λ through each layer of the model.
[0116] 3.5) Update weights:
[0117] According to the gradient of the feature fusion weight λ, the stochastic gradient descent Adam algorithm is applied to update the weight to reduce the value of the loss function. The formula for updating the weight is as follows:
[0118]
[0119] Among them, λ t+1 is the current weight, η is the learning rate, is the gradient of the loss function with respect to the weights.
[0120] 3.6) Iterative training:
[0121] Repeat the above steps on the training samples until the model converges or reaches the predetermined training round. As the training progresses, the feature fusion weights will be continuously adjusted to minimize the loss function and find the optimal feature fusion ratio.
[0122] In this embodiment, CNN adopts a 1D CNN structure, which consists of 1 convolution layer with 4 convolution kernels, 1 pooling layer and 2 fully connected layers:
[0123] Extract local spatiotemporal features from the input sequence through convolutional and pooling layers;
[0124] A residual connection branch is introduced in the convolution layer, which consists of one convolution kernel, and the other three convolution kernels are connected in series; and after the branch is connected in parallel with the output of the first convolution kernel, it is added to the output of the last convolution kernel in series as the output of the convolution layer; in this embodiment, a residual connection structure is introduced in the multi-layer CNN, and a CNN module is used in the branch to adjust the data size to ensure that the two data sizes of the residual connection are the same. The residual connection structure can alleviate the gradient disappearance and gradient explosion problems in deep neural networks, so that the network can more easily learn the difference between the input and the target.
[0125] The convolution operation moves in the time dimension to achieve feature extraction of sequence changes. The length of the convolution kernel is smaller than the length of the input sequence, and weights are shared in the convolution operation.
[0126] Specifically, in the 1D convolutional layer, the forward propagation process of data is
[0127]
[0128] Among them, represents the i-th element of the output sequence of the 1D convolution layer, S[i·s+j] represents the input sequence element for the convolution operation, s represents the stride of the convolution kernel, W[j] represents the j-th element of the convolution kernel, k represents the convolution kernel length, and b represents the bias term.
[0129] In the activation function layer, the forward propagation process of data is as follows:
[0130] Y=f(X)
[0131] Among them, Y represents the output of the activation function layer, f represents the activation function, and X represents the input of the activation function layer. Common activation functions include Sigmoid, Tanh, ReLU, Leaky ReLU, etc.
[0132] Among them, in the 1D pooling layer, the forward propagation process of data is as follows:
[0133] O=Y、 ss
[0134] Among them, O represents the output of the pooling layer, ↓ss represents the downsampling operation of the pooling layer, and the commonly used downsampling methods are average pooling and maximum pooling.
[0135] Among them, the 1D CNN method realizes the function of extracting abstract features from the input sequence through multiple convolutional layers and pooling layers. The convolution operation of 1D CNN moves in the time dimension to realize the feature extraction function of sequence changes. The length of the convolution kernel is much smaller than the length of the input sequence, and the weights are shared in the convolution operation, which can detect the same features at different positions of the sequence and has translation invariance. The pooling operation of 1D CNN can reduce the length of the sequence, which helps to reduce the computational complexity and extract the key features of the input sequence.
[0136] In this embodiment, LSTM includes a forget gate, an input gate, and an output gate, denoted as f t ,i t , o t The forget gate determines the memory unit C of the previous time step. t-1 What information should be forgotten. It uses the Sigmoid activation function to obtain the output of the forget gate. The output of the forget gate f t Is a vector ranging from 0 to 1, used to control C t-1 The degree of forgetfulness. t The calculation formula is as follows:
[0137] f t =σ(w f [H t-1 , X t ]+b f )
[0138] The input gate controls the input information of the current time step. It has two parts. One part uses a tanh function to filter valid information from the input of the current time step as a candidate memory cell unit. The other part uses the Sigmoid activation function to obtain the output i of the input gate t , used to control the effectiveness of candidate memory cells. t , and C t The calculation formula is as follows
[0139] i t =σ(w i [H t -1,X t ]+b i )
[0140]
[0141] Among them, the output gate determines the output information of the current time step. It uses the Sigmoid activation function to obtain the output of the output gate. The output of the output gate is o t The memory cell C at the current time step t Together they determine the output data H of the neuron at the current time step t . o t and H t The calculation formula is as follows:
[0142] o t =σ(w o [H t-1 , X t ]+b o )
[0143] H t =o t ⊙ tanh(C t )
[0144] Among them, the LSTM method effectively obtains the long-term and short-term dependencies of comprehensive transmission monitoring data by introducing "memory units" and gating mechanisms.
[0145] In this embodiment, the improved CNN-LSTM model used adopts a fusion model composed of multiple CNN modules (each CNN module includes a one-dimensional convolution layer, an activation function layer and a one-dimensional pooling layer) and LSTM as a classification model. The original data extracts signal features through CNN and LSTM respectively, and the signal features are converted into a one-dimensional vector form using two linear layers, and then feature fusion is performed according to the splicing method. The signal features extracted by this method are not easy to lose, which is conducive to achieving higher accuracy anomaly detection. Adding a residual connection structure to the multi-layer CNN makes it easier for the network to learn the difference between the target and the input. In order to ensure that the two data sizes of the residual connection are the same, a CNN module is used in the branch to adjust the data size. In addition, Leaky ReLU is used as an activation function to prevent the gradient vanishing and gradient explosion problems in deep neural networks. The fusion feature finally outputs the category of the sequence through the linear layer and the Softmax function to realize the comprehensive transmission anomaly detection, and significantly enhance the accuracy and robustness of the comprehensive transmission device anomaly detection.
[0146] Embodiment, this embodiment provides a comprehensive transmission device anomaly detection method based on PCA-CNN-LSTM, which has great practical reference significance for the product service status monitoring of the comprehensive transmission device. Among various intelligent anomaly detection methods, how to process multi-dimensional sensor data to achieve real-time and effective anomaly detection is a key challenge. By combining principal component analysis (PCA), convolutional neural network (CNN) and long short-term memory network (LSTM), this method provides a better solution. Principal component analysis (PCA) reduces data redundancy through dimensionality reduction processing, retains key features, and improves the efficiency of subsequent model processing. The parallel structure CNN-LSTM model combines the CNN's ability to capture local features and the LSTM's ability to model time series dependencies, making anomaly detection more accurate. Through dimensionality reduction and feature extraction of multi-dimensional time series data, this embodiment realizes accurate monitoring of the operating status of complex transmission devices. Using the PCA-CNN-LSTM fusion model, rich feature information can be extracted from the original data, and the detection accuracy can be further improved through feature fusion. Especially in the processing of high-dimensional data and the capture of abnormal features, it performs well.
[0147] 1) Obtain comprehensive transmission monitoring data, including speed 1, speed 2, speed 3, speed 4, speed 5, speed 6, speed 7, pressure 1, pressure 2, temperature 1, and temperature 2. Some of the data were measured under abnormal operating conditions caused by a large leakage of return oil flow, and have been marked as abnormal data.
[0148] 2) The comprehensive transmission data containing 13 variables were normalized and then PCA was performed. The independent variance contribution rate and cumulative variance contribution rate of each component after PCA were as follows: Figure 3 As shown in Figure 2, the variance contribution rates of the first three components are 83.1%, 12.0% and 4.1% respectively, and their cumulative variance contribution rate reaches 99%, which shows that the three principal components contain most of the information of the original variables and can replace the original variables for anomaly detection.
[0149] 3) Set the sliding window stride to 100 and the window length to 300, and obtain sequence samples of corresponding length. Each sequence sample contains three subsequences, corresponding to three principal component data. The label calibration principle of sequence samples is as follows: if the original sample contained in the sequence contains more than 10 abnormal time steps, then the sequence sample is an abnormal sequence, otherwise it is a normal sequence. After completing the calibration of the sequence data label, it is divided into a training set and a test set in a ratio of 4:1 for model training and testing.
[0150] 4) Train the CNN-LSTM model and adjust the model parameters. The CNN has 3 layers, the convolution kernel size is 3, the number of neurons is 8, 16, and 32, the LSTM stack has 3 layers, the number of neurons is 128, the loss function is cross entropy loss, and the number of iterations is 500.
[0151] 5) Use the trained model to test the comprehensive transmission monitoring data. The test results are as follows: Figure 4 As shown in Figure 2, the anomaly detection rate in the test set is 92.7%. The effect comparison of different models is as follows: Figure 5a , Figure 5b shown.
[0152] In summary, the present invention significantly improves the accuracy and efficiency of anomaly detection of integrated transmission devices through the combination of PCA dimensionality reduction and parallel CNN-LSTM model, which has important technical significance and practical application value.
[0153] In one embodiment of the present invention, a comprehensive transmission device abnormality detection system based on deep feature mining is provided, which includes:
[0154] The initial dimension reduction module records the original data of the key parts of the integrated transmission device monitored by various sensors into multi-dimensional time series data, and performs initial dimension reduction through the principal component analysis method to obtain the reduced dimension data;
[0155] The second dimension reduction module segments the reduced-dimensional data to obtain multiple groups of sequence segments, and further reduces the dimension of each subsequence segment in each group of sequence segments through PAA, and divides the sequence segments after further dimension reduction into a training set and a test set;
[0156] The anomaly detection module trains the improved CNN-LSTM model through the training set data to obtain the trained optimal model, and uses the trained optimal model to detect the test set data to determine whether there are anomalies in the data.
[0157] In the above embodiment, the principal component analysis method is used to perform preliminary dimensionality reduction to obtain the reduced dimensionality data, including:
[0158] Perform Min-Max normalization on the multidimensional time series data to obtain standardized data, and calculate the covariance matrix of the standardized data;
[0159] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues and corresponding eigenvectors;
[0160] According to the proportion of a single eigenvalue of the covariance matrix in all eigenvalues, several component data whose proportions meet the set conditions are selected as principal components;
[0161] The selected eigenvectors are combined into a projection matrix to map the original data into a new principal component space for preliminary dimensionality reduction, and the reduced dimensionality data are weightedly fused as the final reduced dimensionality data.
[0162] In this embodiment, weighted fusion is performed on the data after dimensionality reduction, including:
[0163] The multi-layer perceptron model is used to learn the weight of each feature to perform adaptive weight setting and dynamically adjust the weight of each feature;
[0164] The generated weights are applied to the features after PCA and weighted fusion is performed.
[0165] In the above embodiment, the data after dimensionality reduction is segmented to obtain multiple groups of sequence segments, and each subsequence segment in each group of sequence segments is further reduced in dimensionality by PAA, including:
[0166] Determine the PAA dimension reduction window size, set the window size of the undivided window from small to large, perform PAA data dimension reduction on each window size, calculate the statistical indicators of each subsequence after dimension reduction, and select the largest window with no qualitative change in the trend of the data statistical indicators as the selected window size; wherein the data statistical indicators have comprehensive characteristics of data shape and variation;
[0167] According to the selected window size, the sliding window method is used to divide the data into multiple overlapping subsequences;
[0168] Through PAA, the average value of each subsequence is used to replace the original subsequence, further reducing the dimension of the subsequence.
[0169] The statistical indicators are:
[0170]
[0171] Among them, K is the data statistical index; x i represents the i-th value of the time series; N is the length of the time series; sqrt is the root mean square; μ is the mean.
[0172] In the above embodiment, the improved CNN-LSTM model includes: a CNN and a LSTM in parallel;
[0173] The data is respectively extracted through CNN and LSTM to extract signal features, and the signal features are converted into one-dimensional vectors using the adaptive weighting method:
[0174] The two one-dimensional vectors are fused according to the weighted concatenation method, and the anomaly detection category is output after passing through the fully connected layer;
[0175] Among them, the two one-dimensional vectors are feature fused according to the weighted splicing method, including:
[0176] Initialize the weights of the features extracted by the CNN module and the LSTM module to obtain the initialized feature fusion weight λ;
[0177] In the forward propagation stage, in the fusion layer, the input feature vectors are combined by weight: V F =λV CNN +(1-λ)V LSTM , where V CNN 、V LSTM The data are respectively extracted through CNN and LSTM to extract signal features, V F The fused features are:
[0178] Compare the output of the improved CNN-LSTM model with the output target and calculate the cross entropy loss function;
[0179] In the back propagation phase, the chain rule is used to calculate the gradient of the loss function of each layer of the model with respect to the fusion feature weight λ:
[0180] According to the gradient of the feature fusion weight, the stochastic gradient descent Adam algorithm is applied to update the weight: Among them, λ t+1 is the current weight, η is the learning rate, is the gradient of the loss function with respect to the weights;
[0181] The above process is repeated until the model converges or reaches the predetermined training rounds; as the training progresses, the feature fusion weights will be continuously adjusted to minimize the loss function, thereby finding the optimal feature fusion ratio.
[0182] In this embodiment, CNN adopts a 1D CNN structure, which consists of 1 convolution layer with 4 convolution kernels, 1 pooling layer and 2 fully connected layers;
[0183] Extract local spatiotemporal features from the input sequence through convolutional and pooling layers;
[0184] A residual connection branch is introduced in the convolution layer. The branch consists of one convolution kernel and the other three convolution kernels are connected in series. After the branch is connected in parallel with the output of the first convolution kernel, it is added to the output of the last convolution kernel in series as the output of the convolution layer.
[0185] The convolution operation moves in the time dimension to achieve feature extraction of sequence changes. The length of the convolution kernel is smaller than the length of the input sequence, and weights are shared in the convolution operation.
[0186] In this embodiment, the LSTM includes a forget gate, an input gate, and an output gate;
[0187] The forget gate determines the memory unit C of the previous time step t-1 Which information should be forgotten? The Sigmoid activation function is used to obtain the output of the forget gate.
[0188] The input gate controls the input information of the current time step. It has two parts. One part uses a tanh function to filter valid information from the input of the current time step as a candidate memory cell unit. The other part uses the Sigmoid activation function to obtain the output i of the input gate t , which is used to control the effectiveness of candidate memory cells;
[0189] The output gate determines the output information of the current time step. The Sigmoid activation function is used to obtain the output of the output gate. The output of the output gate is o t The memory cell C at the current time step t The output data of the neurons at the current time step are jointly determined to obtain the long- and short-time dependencies of the comprehensive transmission monitoring data.
[0190] The system provided in this embodiment is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for specific processes and detailed contents, which will not be repeated here.
[0191] A computing device provided in one embodiment of the present invention may be a terminal, which may include: a processor, a communications interface, a memory, a display screen, and an input device. The processor, the communications interface, and the memory communicate with each other through a communication bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory, wherein the non-volatile storage medium stores an operating system and a computer program, and the computer program is executed by the processor to implement the methods in the above embodiments; the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a management network, NFC (near field communication) or other technologies. The display screen may be a liquid crystal display screen or an electronic ink display screen, and the input device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computing device, or an external keyboard, touchpad or mouse, etc. The processor may call the logic instructions in the memory.
[0192] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0193] In one embodiment of the present invention, a computer program product is provided, wherein the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments.
[0194] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores server instructions, and the computer instructions enable a computer to execute the methods provided in the above embodiments.
[0195] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.
[0196] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0197] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A comprehensive transmission device anomaly detection method based on deep feature mining, characterized in that: include: The original data of the key parts of the integrated transmission device monitored by various sensors are recorded as multi-dimensional time series data, and the principal component analysis method is used to perform preliminary dimension reduction to obtain the reduced dimension data; The data after dimensionality reduction is segmented to obtain multiple groups of sequence segments, and each subsequence segment in each group of sequence segments is further reduced in dimension by PAA, and the sequence segments after further dimensionality reduction are divided into a training set and a test set; The improved CNN-LSTM model is trained with the training set data to obtain the trained optimal model, and the trained optimal model is used to detect the test set data to determine whether there are any abnormalities in the data.
2. The comprehensive transmission device abnormality detection method based on deep feature mining as claimed in claim 1, characterized in that: The principal component analysis method is used to perform preliminary dimensionality reduction, and the reduced dimensionality data is obtained, including: Perform Min-Max normalization on the multidimensional time series data to obtain standardized data, and calculate the covariance matrix of the standardized data; Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues and corresponding eigenvectors; According to the proportion of a single eigenvalue of the covariance matrix in all eigenvalues, several component data whose proportions meet the set conditions are selected as principal components; The selected eigenvectors are combined into a projection matrix to map the original data into a new principal component space for preliminary dimensionality reduction, and the reduced dimensionality data are weightedly fused as the final reduced dimensionality data.
3. The comprehensive transmission device abnormality detection method based on deep feature mining as claimed in claim 2 is characterized in that: The reduced-dimensional data is weighted and fused, including: The multi-layer perceptron model is used to learn the weight of each feature to perform adaptive weight setting and dynamically adjust the weight of each feature; The generated weights are applied to the features after PCA and weighted fusion is performed.
4. The comprehensive transmission device abnormality detection method based on deep feature mining as claimed in claim 1, characterized in that: The data after dimensionality reduction is segmented to obtain multiple groups of sequence fragments, and each subsequence fragment in each group of sequence fragments is further reduced in dimensionality through PAA, including: Determine the PAA dimension reduction window size, set the window size of the undivided window from small to large, perform PAA data dimension reduction on each window size, calculate the statistical indicators of each subsequence after dimension reduction, and select the largest window with no qualitative change in the trend of the data statistical indicators as the selected window size; wherein the data statistical indicators have comprehensive characteristics of data shape and variation; According to the selected window size, the sliding window method is used to divide the data into multiple overlapping subsequences; Through PAA, the average value of each subsequence is used to replace the original subsequence, further reducing the dimension of the subsequence.
5. The comprehensive transmission device abnormality detection method based on deep feature mining as claimed in claim 4 is characterized in that: The statistical indicators are: Among them, K is the data statistical index; x i represents the i-th value of the time series; N is the length of the time series; sqrt is the root mean square; μ is the mean.
6. The comprehensive transmission device abnormality detection method based on deep feature mining as claimed in claim 1, characterized in that: Improved CNN-LSTM model, including: parallel CNN and LSTM; The data is respectively extracted from signal features by CNN and LSTM, and the signal features are converted into one-dimensional vectors using adaptive weighting method; The two one-dimensional vectors are fused according to the weighted concatenation method, and the anomaly detection category is output after passing through the fully connected layer; Among them, the two one-dimensional vectors are feature fused according to the weighted splicing method, including: Initialize the weights of the features extracted by the CNN module and the LSTM module to obtain the initialized feature fusion weight λ; In the forward propagation stage, in the fusion layer, the input feature vectors are combined by weight: V F =λV CNN +(1-λ)V LSTM , where V CNN 、V LSTM The data are respectively extracted through CNN and LSTM to extract signal features, V F is the fused feature; Compare the output of the improved CNN-LSTM model with the output target and calculate the cross entropy loss function; In the back-propagation phase, the chain rule is used to calculate the gradient of the loss function of each layer of the model with respect to the fusion feature weight λ; According to the gradient of the feature fusion weight, the stochastic gradient descent Adam algorithm is applied to update the weight: Among them, λ t+1 is the current weight, η is the learning rate, is the gradient of the loss function with respect to the weights; The above process is repeated until the model converges or reaches the predetermined training rounds; as the training progresses, the feature fusion weights will be continuously adjusted to minimize the loss function, thereby finding the optimal feature fusion ratio.
7. The comprehensive transmission device abnormality detection method based on deep feature mining as claimed in claim 6, characterized in that: CNN adopts a 1D CNN structure, which consists of 1 convolutional layer with 4 convolution kernels, 1 pooling layer and 2 fully connected layers; Extract local spatiotemporal features from the input sequence through convolutional and pooling layers; A residual connection branch is introduced in the convolution layer. The branch consists of one convolution kernel and the other three convolution kernels are connected in series. After the branch is connected in parallel with the output of the first convolution kernel, it is added to the output of the last convolution kernel in series as the output of the convolution layer. The convolution operation moves in the time dimension to achieve feature extraction of sequence changes. The length of the convolution kernel is smaller than the length of the input sequence, and weights are shared in the convolution operation.
8. A comprehensive transmission device anomaly detection system based on deep feature mining, characterized in that: include: The initial dimension reduction module records the original data of the key parts of the integrated transmission device monitored by various sensors into multi-dimensional time series data, and performs initial dimension reduction through the principal component analysis method to obtain the reduced dimension data; The second dimension reduction module segments the reduced-dimensional data to obtain multiple groups of sequence segments, and further reduces the dimension of each subsequence segment in each group of sequence segments through PAA, and divides the sequence segments after further dimension reduction into a training set and a test set; The anomaly detection module trains the improved CNN-LSTM model through the training set data to obtain the trained optimal model, and uses the trained optimal model to detect the test set data to determine whether there are anomalies in the data.
9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 7.
10. A computing device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods described in claims 1 to 7.
Citation Information
Cited By
Power ciphertext data anomaly detection method and system based on function encryption
CN119249460A
A method and system for detecting anomaly in power ciphertext data based on function encryption
CN119249460B
Safety management method for comprehensive security and protection integrated management system
CN120070135A
Magnetic suspension track full-automatic assembly line motor abnormity detection system and method
CN120468648A
