Stacker fault diagnosis method based on lightweight MobileViT

Abstract

The invention discloses a stacker fault diagnosis method based on lightweight MobileViT, and relates to the field of stacker fault diagnosis, and the method comprises the steps: S1, deploying a sensor group based on a stacker work site, and obtaining original sensing data of eight modes; s2, preprocessing the obtained original sensing data based on a multi-sensor data fusion preprocessing process of noise reduction reconstruction-fusion-two-dimensional coding to obtain corresponding two-dimensional image data; s3, dividing the obtained two-dimensional image data, and training a MobileViT model based on the training set and the verification set; s4, inputting the test set into the trained MobileViT model to perform fault diagnosis, and performing visual analysis on the result, the lightweight MobileViT model designed by the application can greatly reduce the parameter quantity of a traditional deep learning model; the method has good classification performance, and especially has good practicability in the field of fault diagnosis of the stacker.

Description

TECHNICAL FIELD

[0001] The application relates to the field of stacker fault diagnosis, in particular to a stacker fault diagnosis method based on a light MobileViT. BACKGROUND

[0002] With the rapid development of industrial automation and intelligence, automated warehousing systems have become a key link in modern logistics and production systems. As the core equipment of automated warehousing systems, the running stability and reliability of stackers are crucial to the overall system. However, due to the complex structure of stackers and the long-term operation under high-speed heavy load, frequent start-stop and variable speed conditions, the key components (such as gear racks and guide rails) are prone to wear or sudden failure, which not only threatens the safe operation of the system, but also causes serious economic losses. Therefore, it is of great theoretical significance and engineering application value to realize real-time, accurate and intelligent fault diagnosis of stackers.

[0003] In recent years, research on stacker fault diagnosis has gradually developed. Traditional methods mainly rely on fault tree analysis (FTA) and its extended models. Although related research has made certain achievements in remote monitoring and probabilistic reasoning, such methods essentially rely on static logic models constructed based on prior knowledge, making it difficult to effectively mine dynamic nonlinear feature information generated during stacker operation and lack self-adaptive ability. With the development of deep learning technology, data-driven methods have been gradually introduced into fault diagnosis. Researchers have explored the application of convolutional neural networks (CNN), multi-branch multi-scale CNN (MBSCNN), Transformer and its improved structures in mechanical fault diagnosis, and these methods have achieved good performance in single signal analysis such as vibration and sound. However, when existing deep learning models are directly applied to stacker scenarios, the following problems still exist: first, the fusion and feature representation of multi-sensor heterogeneous information are insufficient, leading to the diagnosis results being easily disturbed by noise and working condition fluctuations; second, the feature extraction capability of single models is limited, making it difficult to simultaneously consider local impact features and long-term sequential dependencies; third, complex deep models often have large parameter quantities and high computational overhead, making it difficult to meet the real-time and lightweight deployment requirements of industrial sites.

[0004] To solve the above problems, this paper proposes a stacker fault diagnosis method based on multi-data fusion and light MobileViT. This method designs a multi-stage signal preprocessing process of "noise reduction reconstruction-fusion-two-dimensional coding" to optimize the decomposition, feature fusion and two-dimensional coding of sound and vibration signals collected by multiple sensors, effectively improving the representability of original signals. On this basis, a light MobileViT model optimized for stacker scenarios is constructed to balance global dependency features and local pattern recognition while reducing model computational complexity, thereby achieving efficient diagnosis of different fault modes of stackers. SUMMARY

[0005] The purpose of the present application is to provide a stacking machine fault diagnosis method based on lightweight MobileViT, which can provide a stacking machine fault diagnosis method based on multi-data fusion and lightweight MobileViT, aiming to solve the problems of insufficient multi-source data fusion, limited feature extraction capability, high model complexity, and limited field deployment of existing stacking machine fault diagnosis methods. By proposing a multi-stage signal preprocessing and optimized lightweight MobileViT network structure, it can fully fuse multi-sensor information, consider local and global feature extraction, and significantly reduce computational overhead while ensuring diagnostic accuracy, thereby realizing efficient, real-time, and low-cost stacking machine intelligent fault diagnosis.

[0006] To achieve the above purpose, the present application provides the following solutions: In a first aspect, the present application provides a stacking machine fault diagnosis method based on lightweight MobileViT, comprising: S1, deploying a sensor group based on a stacking machine work site to obtain original sensor data of eight modalities; S2, preprocessing the obtained original sensor data based on a multi-sensor data fusion preprocessing process of noise reduction reconstruction-fusion-two-dimensional coding to obtain corresponding two-dimensional image data; S3, dividing the obtained two-dimensional image data to obtain a training set, a validation set, and a test set; training a MobileViT model based on the training set and the validation set to obtain a trained MobileViT model; S4, inputting the test set into the trained MobileViT model for fault diagnosis, and visualizing and analyzing the results.

[0007] According to the specific embodiments provided by the present application, the present application has the following technical effects: Through the multi-stage preprocessing of "noise reduction reconstruction-fusion-two-dimensional coding", the effective fusion of multi-source heterogeneous data such as sound and vibration is realized, the robustness of fault diagnosis is significantly improved, and the interference of noise and working condition fluctuations on the diagnosis results is reduced.

[0008] Based on the lightweight MobileViT network structure, the efficient extraction of local impact features and long-term global dependence is considered, which breaks through the technical limitations of traditional CNN, RNN / LSTM, and conventional Transformer, and significantly improves the completeness and accuracy of feature representation.

[0009] Through lightweight network design, the model computational overhead is reduced, the deployment difficulty of advanced deep learning models on edge devices is solved, and the dual satisfaction of high-precision diagnosis and real-time response of stacking machine faults is realized.

[0010] Finally, the accuracy, robustness and real-time performance of the stacker fault diagnosis are significantly improved, reliable technical support is provided for intelligent operation and maintenance in industrial field, and efficient operation and maintenance requirements in actual application are met. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0012] Figure 1 A flowchart of a fault diagnosis method based on multi-data fusion and lightweight MobileViT provided in an embodiment of the present application; Figure 2 An engraving processing production line provided in an embodiment of the present application; Figure 3 Data acquisition point provided in an embodiment of the present application; Figure 4 A data preprocessing flowchart provided in an embodiment of the present application; Figure 5 A lightweight MobileViT model structure provided in an embodiment of the present application; Figure 6 A lightweight attention mechanism provided in an embodiment of the present application; Figure 7 A lightweight MobileViT block provided in an embodiment of the present application; Figure 8 A multi-modal signal comparison chart provided in an embodiment of the present application; Fig. 9(a) is a confusion matrix of ConvNeXt; Fig. 9(b) is a confusion matrix of EfficientNetV2; Fig. 9(c) is a confusion matrix of MobileNetV2; Fig. 9(d) is a confusion matrix of MobileViTXXS; Fig. 9(e) is a confusion matrix of Resnet34; Fig. 9(f) is a confusion matrix of ShuffleNetV2; Fig. 9(g) is a confusion matrix of SwinTransformerTiny; Fig. 9(h) is a confusion matrix of lightweight MobileViT; Figure 10 Accuracy comparison of different model training processes provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] like Figure 1 The embodiment shown in this application provides a stacker crane fault diagnosis method based on lightweight MobileViT, the method comprising: S1. Deploy a sensor group based on the stacker crane's working site to obtain raw sensing data for eight modes; S2. The multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and two-dimensional coding preprocesses the acquired raw sensor data to obtain the corresponding two-dimensional image data. S3. Divide the acquired two-dimensional image data into training set, validation set and test set; train the MobileViT model based on the training set and validation set to obtain the trained MobileViT model. S4. Input the test set into the trained MobileViT model for fault diagnosis, and perform visualization analysis on the results.

[0016] Optionally, step S1, which involves deploying a sensor group at the stacker crane's working site to obtain raw sensing data for eight modes, further includes: Three data collection points are deployed at the stacker crane's working site: the first data collection point, the second data collection point, and the third data collection point. The first data acquisition point is located on the bearing guide rail at the top of the stacker crane, used to monitor its smooth operation or abnormal vibration. The second data acquisition point is set at the gear and rack transmission mechanism adjacent to the ground track, and is used to collect signal characteristics during the meshing transmission process; The third data acquisition point is set on the side of the pallet of the stacker crane to collect signals of the gear and rack transmission during operation; Each data acquisition point is equipped with two triaxial accelerometers and two sound sensors. The accelerometers are used to capture vibration signals in the x, y, and z directions, and the sound sensors are used to capture sound signals. The raw sensing data for eight modes were obtained based on the vibration signals in the x, y, and z directions and the sound signals.

[0017] Specifically, the experimental data of the self-collected data set comes from a warehouse stacker in a carving processing production line, aiming to collect its vibration and sound signals under different health states, and its general structure is as shown in Figure 2 .

[0018] The complete signal collection system is composed of the following parts: an iPotest-1006 data collection instrument, two miniature three-axis IEPE acceleration sensors, and two N2250 free-field measurement microphones. Considering that the fault signals generated by the transmission components of the stacker are mainly concentrated in the low-frequency band, the sampling frequency (fs) of all channels is uniformly set to 10 kHz. In order to accurately capture the running state of different key components, the sensors are strategically placed in three core positions (see Figure 3 (a)(b)(c)): Position one: a group of sensors are installed on the bearing guide rail part at the top of the stacker, used to monitor its smooth running or abnormal vibration; Position two: a group of sensors are installed next to the gear rack transmission mechanism of the ground track, used to collect signal characteristics during meshing transmission; Position three: a group of sensors are installed on the side of the tray of the stacker, used to collect signals of the gear rack transmission during operation; Each position is provided with two three-axis acceleration sensors and two sound sensors, wherein each acceleration sensor can capture vibration signals in x, y, and z directions, and each sound sensor captures the sound signal at the position. Therefore, each time of collection can synchronously obtain 8-channel signal data, including 6 vibration signals (x1, y1, z1, x2, y2, z2) and 2 sound signals (s1, s2). In this way, more abundant information is obtained. The duration of the three fault states is not long, and the fault interval is only about 10-20s, so 200,000 data points under each state are collected, and each data point is separated by 0.0001s.

[0019] In order to comprehensively evaluate the diagnostic performance of the proposed algorithm, the running conditions of the stacker under six typical health and fault states are collected. For each state, long-time continuous signals are collected. The specific composition of the data set is shown in Table 1.

[0020] Table 1. Composition of self-collected data set

[0021] First, a particle swarm optimization-based variational mode decomposition (PSO-VMD) technique is used to denoise and reconstruct the 8-channel signal to enhance the saliency of fault features. Then, a multimodal feature fusion network is used to fuse information from all eight modes. Next, a non-overlapping sliding window method is employed to segment the reconstructed signal into samples, with each sample having a length of 600 data points. Finally, an improved Gram angular field transform is used to convert the one-dimensional time-series signal samples into two-dimensional images, constructing an image dataset that can be directly used for model training and validation.

[0022] Optionally, in step S2, the acquired raw sensor data is preprocessed using a multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and 2D encoding to obtain the corresponding 2D image data, such as... Figure 4 As shown, firstly, adaptive signal decomposition and noise reduction reconstruction are performed using optimized variational mode decomposition (VMD) to suppress background noise. Then, the designed data fusion network is used to fuse multi-sensor data. Finally, Gramm-Aufbau (GADF) is used to encode the fused one-dimensional time-series signal into a two-dimensional image that preserves time correlation, improving data quality and achieving effective signal representation. This also includes: S201. Normalize the raw vibration and sound signals collected from the stacker crane. Assume the raw data is represented as follows: By scaling, its value range is placed between [-1, 1]. ; in, The original data after normalization. For the i-th original data sample value, The minimum value of the original data sample. The maximum value of the original data sample; S202. Convert the scaled time series points into points in a polar coordinate system, where time points correspond to polar angles and signal values ​​correspond to radii. The resulting time series in polar coordinates is expressed by the following formula: ; in, yes The corresponding timestamp, Normalization factor; S203. Reconstruct the time series in polar coordinates using the Gramian Angular Field (GAF). Let the polar coordinate angle corresponding to the normalized time series be... θ i and θ j By calculating angles and θ i + θj cos( θ i + θ j sin( θ i - θ j sin( θ i - θ j ), the Gramian Angular Summation Field (GASF) matrix and the Gramian Angular Difference Field (GADF) matrix are constructed, and the calculation formulas of the two are as follows: ; ; Based on the generated matrix, the corresponding two-dimensional image data is obtained.

[0023] Optionally, the MobileViT model trained in the S3 further comprises: The overall structure of the MobileViT network is constructed, including a standard convolutional layer, a lightweight feature extraction backbone network composed of an improved MV2 module and an SE module, a MobileViT block, a global pooling layer, and a classification head; Firstly, the input channel number is expanded through 1x1 point convolution, then the feature is extracted through 3x3 depth separable convolution, and finally the channel number is compressed back to the original number through 1x1 point convolution again, and the cross-layer transmission of the feature is realized through residual connection; Specifically, Figure 5 is the overall structure of the lightweight MobileViT network proposed, which is obtained by stacking standard convolution, improved MV2 module, and lightweight MobileViT block, wherein L represents the number of repeated stacking of the lightweight Transformer module.

[0024] In the proposed lightweight MobileViT network architecture, a lightweight and efficient reverse residual block is used as the basic building unit. This module aims to maintain a low computational overhead while effectively capturing feature information and facilitating the flow of information and the propagation of gradients through residual connections when certain conditions are met. First, the input channel number is expanded through a 1x1 point convolution, then a 3x3 depth separable convolution is used for feature extraction, and finally the channel number is compressed back to the original number through a 1x1 point convolution. To prevent the loss of low-dimensional feature information during the process of first expanding and then reducing the dimension, ReLU6 is selected as the activation function. ReLU6 is a restricted linear unit (Rectified Linear Unit) activation function that limits the output of ReLU to between 0 and 6. Its expression is: .

[0025] The SE module is used to adjust the adaptive weight of the obtained input feature map, including global average pooling of each channel, obtaining channel weight coefficients through two layers of 1x1 convolution and nonlinear activation function, and weighting the original feature map channel by channel, thereby obtaining an optimized feature map; The global feature acquisition module is used to map the optimized feature map to a feature sequence and input it into a multi-head self-attention mechanism to model the long-range dependencies between different spatial positions in the feature map and obtain a feature representation containing global context information. The MobileViT block is the main component of the MobileViT model. To improve the accuracy of the model for stacker fault diagnosis without increasing the computational load, a lightweight MobileViT block is proposed for improvement, as shown in Figure 7 The key modifications include: (1) modifying the transformer module: using lightweight attention instead of self-attention, and changing the linear projection in FFN to convolution projection; (2) modifying the convolution in the Fusion module to a depth separable convolution, and introducing an efficient channel attention after the final convolution; To adapt to the limited computing capacity of on-site equipment, the MobileViT-XXS model is selected as the basis for improvement.

[0026] Optionally, the SE module is also used for: First, use a pointwise convolution to expand the channel number, then use a depth separable convolution to perform spatial filtering on the expanded feature map, and then use a global adaptive average pooling to globally compress the features of each channel, use a 1x1 convolution layer to compress the channels, and increase the non-linear expression ability through an activation function; Reduce the channel dimension through another 1x1 convolution, and use an activation function to generate a weighting coefficient for each channel; The weight coefficient of each channel is used to adjust each channel of the input feature map, to amplify important features and suppress unimportant features.

[0027] Specifically, in order to enhance the representation ability of the network, a Squeeze-and-Excitation (SE) module is introduced after the depth separable convolution. The improved module first uses point-wise convolution to expand the channel number, then uses a depth separable convolution to perform spatial filtering on the expanded feature map, then uses global adaptive average pooling to globally compress the features of each channel, uses a 1x1 convolution layer to compress the channels, and uses an activation function (ReLU) to increase the nonlinear expression ability. Then, another 1x1 convolution is used to restore the channel dimension, and an activation function is used to generate a weight coefficient for each channel. Finally, these weights are used to adjust each channel of the input feature map, amplify important features, and suppress unimportant features.

[0028] Optionally, in order to obtain global feature information from the signal, the multi-head self-attention mechanism uses complex matrix multiplication and multi-dimensional exponential operation, and the computational complexity is mainly in linear transformation, query matrix and key matrix point multiplication calculation, weighted summation and multi-head splicing, and the specific operation amount is: (1) wherein, represents the height of the input image, represents the width of the input image, represents the number of channels.

[0029] The calculation complexity is mainly dominated by In order to reduce the calculation amount, a lightweight attention mechanism is designed, which replaces the complex matrix multiplication by broadcast calculation, and the structure is as shown in Figure 6 Also includes: (1) using convolution layer to perform convolution projection on the input with a shape of , which independently linearly transforms the channel dimension at each spatial position; Specifically, a convolution layer with is used to perform convolution projection on an input with a shape of , which is equivalent to independently linearly transforming the channel dimension at each spatial position. Here, the weight matrix corresponding to Q, K, and V is not defined separately, but is included in the convolution layer. After passing through this convolution layer, an output with a shape of is obtained; Q, K, and V matrices are obtained through a convolution layer with , and the calculation is as follows: wherein the input has a dimension of , is included in the convolution layer, The shape is , All shapes are .

[0030] (2) Information flow is achieved through context aggregation of channels. First, broadcast operation is performed on the Q matrix, then element-wise multiplication is performed on the Q matrix and the V matrix, and then summation operation is performed on the time dimension to obtain the global context vector of each channel. Specifically, information flow is achieved through context aggregation of channels. Since the Q matrix and the V matrix are uniquely different, when calculating their product, Q needs to be multiplied by each dimension c of K. This means that the Q matrix needs to be broadcast, and then the two matrices need to be multiplied element-wise. Finally, the time dimension is summed to obtain the global context vector for each channel. This process can be represented as: (2) in, Indicates a broadcast operation; (3) The obtained global context vector is transmitted to the value matrix through element-wise multiplication via broadcasting to obtain key fine-grained features from the global scope of the signal. ; Specifically, The contextual information contained therein is transmitted to the value matrix through element-wise multiplication broadcast, thereby obtaining key fine-grained features from the global scope of the signal. The process is represented as: (3) (4) Convolutional projection and regularization are performed on the obtained key fine-grained features using a convolutional layer and a dropout layer respectively, to obtain the output feature map enhanced by global context attention, which is used as the output of the global feature acquisition module. Among them, the global context vector obtained in step (2) can be regarded as the attention weight for modeling the importance of each channel, and is used to weight the value matrix in step (3); Lightweight self-attention can be defined as: (4) The core of this attention mechanism lies in designing a single-channel, position-based attention score. This score is used to weight the Key and aggregate the global context for each channel. Then, this global context for each channel is used to scale the corresponding channel features of the Value across all positions. Based on this process, the computational cost of lightweight attention can be expressed as: (5) The output form of the module is given by formula (9), and the final output is formed by weighting the features by the global context vector and then passing through Dropout. To further analyze the lightweight characteristics of this module, the computational complexity (FLOPs) is derived in formula (10), which shows that this module only contains 2HWC 2 + 3HWC + HW operation amount, meeting the lightweight design requirements. This design cleverly uses the broadcast mechanism to avoid position attention calculation based on spatial position pairs, reducing the complexity to , thereby significantly reducing the computational complexity compared to multi-head self-attention when the input image size is large.

[0031] Optionally, S4 test diagnosis and result visualization analysis. Input the test set into the trained model for fault diagnosis, and visualize the results.

[0032] First, through the ablation experiment, the influence of the proposed data preprocessing process on the classification result is analyzed, thereby proving the necessity of each step.

[0033] Second, the proposed deep learning model is compared with other existing models on the self-collected data set to evaluate the advancement of the proposed method.

[0034] Finally, through comparison on public data sets, the generalization of the proposed method in different scenarios and the advantages compared with existing methods are verified.

[0035] To ensure the fairness of the experiment, all experiments in this paper are carried out on a high-performance server equipped with two NVIDIA GeForce RTX4090 GPUs, each with 24 GB of independent video memory. In terms of software environment, PyTorch is used as the deep learning framework, and GPU acceleration support is provided by CUDA 12.2. The operating system is Linux, and the version of the NVIDIA driver is 535.230.02.

[0036] First, set the parameter configuration of the proposed model as shown in Table 2, select the "gear and rack debris" category fault in Table 1, and verify the effectiveness of the proposed model for data fusion, where a comparison chart of a complete fault waveform is selected as shown in Figure 8 Figure 8 ​As can be seen, only the x1, y1 and z1 vibration signals in the original signal exhibit relatively obvious periodic fault impacts, while the fault characteristics of other modalities (such as s1 and s2 sound signals) are submerged in strong background noise. After data fusion processing, the final generated signal not only successfully inherits and amplifies the key fault characteristics in x1, y1 and z1, but also effectively suppresses noise and irrelevant details from other modalities. This significantly improves the signal-to-noise ratio of the fused signal, making the fault impact characteristics more prominent, thereby providing high-quality data input for subsequent precise fault diagnosis.

[0037] Table 2. Lightweight MobileVit network parameter settings

[0038] To systematically verify the effectiveness and contribution of each core module in the fault diagnosis framework proposed in this study, a series of ablation experiments are designed. The innovations of this study mainly include two aspects: (1) a data preprocessing process based on PSO-VMD and wavelet fusion network, which is used to convert multi-source one-dimensional signals into high-quality two-dimensional feature maps; (2) a lightweight MobileViT classification model for the feature maps. The ablation experiment design is as follows: Experiment 1 (baseline model): The vibration and sound signals collected from the stacker are one-dimensional data, but the base model selected in this study requires two-dimensional input. Therefore, the baseline scheme uses the recognized GADF technique to convert the original signal into a two-dimensional image, and uses the standard MobileViTXXS model for classification. This experiment aims to measure the performance level that can be achieved by a standard and mature technical process; Experiment 2 (verify improved GADF method): In this experiment, the GADF module in the baseline scheme is replaced with the improved GADF method designed in this paper, while the classifier still uses the standard MobileViTXXS. By comparing the results of Experiment 2 and Experiment 1, the effectiveness of the one-dimensional to two-dimensional image conversion method proposed can be evaluated; Experiment 3 (verify data denoising method): Based on Experiment 2, the PSO-VMD signal decomposition and reconstruction module is introduced, i.e., using PSO algorithm optimized VMD to denoise the original signal, and then using the improved GADF to convert it into a two-dimensional image input fault diagnosis model. By comparing the results of Experiment 3 and Experiment 2, the effectiveness of the data denoising method proposed can be evaluated; Experiment 4 (verify fusion strategy): Based on Experiment 3, a fusion network is further introduced, and the integration of multi-sensor data fusion can form a more comprehensive and discriminative feature representation by comparison and verification; Experiment 5 (full scheme in this paper): This experiment contains the proposed complete method. On the basis of using the advanced data processing flow (PSO-VMD + fusion network), the lightweight MobileViT model is used as the classifier. By comparing the results of Experiment 4 and Experiment 5, the effectiveness of the improvement of the model architecture itself can be determined. The experimental results are shown in Table 3.

[0039] Through the above series of ablation experiments, the results clearly show that each module of the improved GADF, PSO-VMD signal decomposition and denoising, feature fusion strategy, and lightweight MobileViT model in this study has made a positive and significant contribution to the final fault diagnosis performance. From the baseline accuracy of 82.0% to the final accuracy of 98.0%, the overall effectiveness and superiority of the multi-module collaborative model proposed in this paper are fully verified.

[0040] Table 3. Ablation experiment results comparison

[0041] To verify the effectiveness and superiority of the method proposed in this paper in the stacker fault diagnosis task, a series of comparative experiments were conducted on the self-collected stacker dataset. The self-collected stacker data was preprocessed by the data preprocessing flow proposed in this paper to obtain the picture dataset, which was input into the lightweight MobileViT model proposed in this paper and several classical deep learning models, including the baseline model MobileViTXXS, ResNet34, MobileNet V2, Vgg, SwinTransformer2, and ConvNeXt. To ensure fairness, all models were trained and evaluated on a unified hardware platform (NVIDIA GeForce RTX 4090 GPU) and software framework (Python 3.10 was used for programming language, PyTorch 2.1.1 was used for deep learning framework, and CUDA 12.1 was used for GPU acceleration). This paper will analyze the experimental results from the dimensions of model complexity and model classification performance using multiple evaluation indicators.

[0042] As can be seen from Table 4, the accuracy of the lightweight MobileViT model proposed in this paper reaches 97%, which is significantly better than other models, and the parameter amount of the lightweight MobileViT model is only 900 and 399, which is much lower than the ResNet34, MobileNetV2, ShuffleNetV2 and other models, and also has certain reduction compared with the baseline model MobileViTXXS. As classic lightweight CNN models, MobileNetV2 and ShuffleNetV2 indeed have significant reduction in parameter amount compared with ordinary CNN networks (such as ResNet34), but the accuracy is also greatly reduced, which shows that the hybrid model proposed in this paper further lightens the model while ensuring the accuracy.

[0043] Table 4. Comparison of parameter amount and accuracy of different networks

[0044] Figure 9 shows the confusion matrix of the model proposed in this paper and other comparison methods on the self-collected data set. Figure 9(a) is the confusion matrix of ConvNeXt; Figure 9(b) is the confusion matrix of EfficientNetV2; Figure 9(c) is the confusion matrix of MobileNetV2; Figure 9(d) is the confusion matrix of MobileViTXXS; Figure 9(e) is the confusion matrix of Resnet34; Figure 9(f) is the confusion matrix of ShuffleNetV2; Figure 9(g) is the confusion matrix of SwinTransformerTiny; Figure 9(h) is the confusion matrix of lightweight MobileViT; it can be seen that: except for the baseline model MobileViTXXS and the model proposed in this paper, the recognition accuracy of other models for the "tray into sundries" category is less than 90%, which proves that the misjudgment rate of other models for this type of fault is high, but in actual working conditions, the materials on the tray are complex and varied, and "tray into sundries" is a common fault, and the recognition accuracy of the model proposed in this paper for tray into sundries is as high as 97.0%, and the recognition rate for all faults is higher than that of all comparison methods.

[0045] To verify the effectiveness of the method proposed in this paper, further comparison tests are carried out on the bearing data set in Paderborn, Germany. This data set is widely used in the field of fault diagnosis and contains vibration signal data of 32 bearings in three categories of health, artificial damage and real wear, with a sampling frequency of up to 64 kHz. To ensure consistency, data of six categories are selected from the data set, and the data preprocessing method proposed in this paper is used to convert them into picture format, and then input into multiple network models including the lightweight MobileViT designed in this paper for training, verification and testing.

[0046] Figure 10The accuracy curves of multiple network models during the training process are shown. It can be observed from the figure that most models show rapid growth in accuracy at the beginning of training, but the model proposed in this paper shows better learning trajectory and final performance. After experiencing a short fluctuation of 20 rounds, the model proposed in this paper enters a sustained and stable performance climbing stage, and surpasses most models at around the 40th round. In the later stage of training, the highest validation accuracy is achieved, and excellent stability is maintained.

[0047] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0048] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In the embodiments provided by the present application, any reference to memory, database or other medium can include at least one of non-volatile and volatile memory. Non-volatile memory can include read only memory (Read Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0049] ​The database involved in each of the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, without being limited thereto. The processor involved in each of the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0050] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as there is no contradiction.

[0051] The principles and implementation manners of the present application are described by using specific examples herein, and the above embodiments are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A stacker crane fault diagnosis method based on lightweight MobileViT, characterized in that, The method includes: S1. Deploy a sensor group based on the stacker crane's working site to obtain raw sensing data for eight modes; S2. The multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and two-dimensional coding preprocesses the acquired raw sensor data to obtain the corresponding two-dimensional image data. S3. Divide the acquired two-dimensional image data into training set, validation set and test set; train the MobileViT model based on the training set and validation set to obtain the trained MobileViT model. S4. Input the test set into the trained MobileViT model for fault diagnosis, and perform visualization analysis on the results.

2. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 1, characterized in that, The S1 method, which deploys a sensor group at the stacker crane's working site to obtain raw sensing data in eight modes, also includes: Three data collection points are deployed at the stacker crane's working site: the first data collection point, the second data collection point, and the third data collection point. The first data acquisition point is located on the bearing guide rail at the top of the stacker crane, used to monitor its smooth operation or abnormal vibration. The second data acquisition point is set at the gear and rack transmission mechanism adjacent to the ground track, and is used to collect signal characteristics during the meshing transmission process; The third data acquisition point is set on the side of the pallet of the stacker crane to collect signals of the gear and rack transmission during operation; Each data acquisition point is equipped with two triaxial accelerometers and two sound sensors. The accelerometers are used to capture vibration signals in the x, y, and z directions, and the sound sensors are used to capture sound signals. The raw sensing data for eight modes were obtained based on the vibration signals in the x, y, and z directions and the sound signals.

3. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 1, characterized in that, The S2 process, which involves a multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and 2D encoding, preprocesses the acquired raw sensor data to obtain the corresponding 2D image data. It also includes: S201. Normalize the raw vibration and sound signals collected from the stacker crane. Assume the raw data is represented as follows: By scaling, its value range is placed between [-1, 1]. ； in, The original data after normalization. For the i-th original data sample value, The minimum value of the original data sample. The maximum value of the original data sample; S202. Convert the scaled time series points into points in a polar coordinate system, where time points correspond to polar angles and signal values ​​correspond to radii. The resulting time series in polar coordinates is expressed by the following formula: ； in, yes The corresponding timestamp, Normalization factor; S203. Reconstruct the time series in polar coordinates using the Gram angle field. Let the polar coordinate angle corresponding to the normalized time series be... θ i and θ j By calculating the cosine of the sum of two polar coordinate angles, cos( θ i + θ j ) and the sine of the difference between the two polar coordinate angles sin( θ i - θ j Construct the sum-type Gram angle field matrix and the difference-type Gram angle field matrix, and the calculation formulas for both are as follows: ； ； Based on generation The matrix is ​​encoded to obtain the corresponding two-dimensional image data.

4. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 1, characterized in that, The trained MobileViT model obtained in S3 also includes: The overall structure of the MobileViT network is constructed, including standard convolutional layers, a lightweight feature extraction backbone network composed of improved MV2 modules and SE modules, MobileViT blocks, global pooling layers, and a classification head; The MV2 module is improved by first expanding the number of input channels through 1×1 point convolution, then extracting features through 3×3 depthwise separable convolution, and finally compressing the number of channels back to the original number through 1×1 point convolution again, and realizing cross-layer feature transfer through residual connections. The SE module is used to adaptively adjust the weights of the obtained input feature map. This includes performing global average pooling on each channel, obtaining channel weight coefficients through two layers of 1×1 convolution and non-linear activation functions, and weighting the original feature map channel by channel to obtain the optimized feature map. The global feature acquisition module is used to map the optimized feature map to the feature sequence and input it into the multi-head self-attention mechanism to model the long-range dependencies between different spatial locations in the feature map and obtain feature representations containing global context information. The MobileViT block is used to fuse local detail features extracted by the convolutional network with global features obtained through a multi-head self-attention mechanism to output a fused feature map that combines local texture information and global semantic information. A global pooling layer is used to perform global average pooling or global weighted pooling on the fused feature map in the spatial dimension to obtain a fixed-length global feature vector. The classification head, including one or more fully connected layers and a Softmax activation function, is used to map the global feature vector to the predicted probability of each fault category, thereby outputting the fault diagnosis results of the stacker crane.

5. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 4, characterized in that, The SE module is also used for: First, pointwise convolution is used to expand the number of channels. Then, a depthwise separable convolution is used to perform spatial filtering on the expanded feature map. Next, global adaptive average pooling is used to globally compress the features of each channel. A 1×1 convolutional layer is used to compress the channels, and an activation function is used to increase the non-linear expressive power. The channel dimension is restored by another 1×1 convolution, and the weighting coefficients for each channel are generated using an activation function; Each channel of the input feature map is adjusted based on the weighting coefficient of each channel to obtain the optimized feature map.

6. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 4, characterized in that, The global feature acquisition module further includes: (1) The convolutional layer used has the shape of The input is convolved and projected, and the channel dimension is linearly transformed independently at each spatial location; (2) Information flow is achieved through context aggregation of channels. First, broadcast operation is performed on the Q matrix, then element-wise multiplication is performed on the Q matrix and the V matrix, and then summation operation is performed on the time dimension to obtain the global context vector of each channel. (3) The obtained global context vector is transmitted to the value matrix through element-wise multiplication via broadcasting to obtain key fine-grained features from the global scope of the signal. ; (4) Convolutional projection and regularization are performed on the obtained key fine-grained features using a convolutional layer and a dropout layer respectively, to obtain the output feature map enhanced by global context attention, which is used as the output of the global feature acquisition module.

Images