Stacker fault diagnosis method based on lightweight MobileViT

Abstract

The application discloses a stacking machine fault diagnosis method based on a lightweight MobileViT, relates to the field of stacking machine fault diagnosis, and comprises the following steps: S1, deploying a sensor group based on a stacking machine work site to obtain original sensing data of eight modes; S2, preprocessing the obtained original sensing data based on a denoising reconstruction-fusion-two-dimensional coding multi-sensor data fusion preprocessing procedure to obtain corresponding two-dimensional image data; S3, dividing the obtained two-dimensional image data, and training a MobileViT model based on a training set and a verification set; and S4, inputting a test set into the trained MobileViT model for fault diagnosis, and performing visual analysis on the result. The lightweight MobileViT model designed in the application can greatly reduce the parameter quantity of a traditional deep learning model, has good classification performance, and has good practicability in the field of stacking machine fault diagnosis.

Description

Technical Field

[0001] This application relates to the field of stacker crane fault diagnosis, and in particular to a stacker crane fault diagnosis method based on lightweight MobileViT. Background Technology

[0002] With the rapid development of industrial automation and intelligence, automated warehousing systems have become a crucial link in modern logistics and production systems. As the core equipment of automated warehousing systems, the operational stability and reliability of stacker cranes are vital to the overall system. However, due to their complex structure and long-term operation under conditions of high speed, heavy load, frequent start-stop, and variable speed, key components (such as gears, racks, and guide rails) are prone to wear or sudden failures, threatening not only the safe operation of the system but also causing serious economic losses. Therefore, achieving real-time, accurate, and intelligent fault diagnosis for stacker cranes has significant theoretical and engineering application value.

[0003] In recent years, research on fault diagnosis for stacker cranes has been gradually carried out, with traditional methods mainly relying on fault tree analysis (FTA) and its extended models. Although related research has achieved certain results in remote monitoring and probabilistic reasoning, these methods essentially rely on static logical models constructed from prior knowledge, making it difficult to effectively mine the dynamic nonlinear characteristic information generated during the operation of stacker cranes, and lacking adaptive capabilities. With the development of deep learning technology, data-driven methods have gradually been introduced into fault diagnosis. Researchers have explored the application of convolutional neural networks (CNN), multi-branch multi-scale CNN (MBSCNN), Transformer and its improved structures in mechanical fault diagnosis. These methods have achieved good performance in the analysis of single signals such as vibration and sound. However, when directly applying existing deep learning models to stacker crane scenarios, the following shortcomings still exist: First, the fusion and feature representation of heterogeneous information from multiple sensors are insufficient, making the diagnostic results susceptible to noise and operating condition fluctuations; second, the feature extraction capability of a single model is limited, making it difficult to simultaneously take into account local impact characteristics and long-term temporal dependencies; third, complex deep models often have a large number of parameters and high computational cost, making it difficult to meet the requirements of real-time and lightweight deployment in industrial settings.

[0004] To address the aforementioned issues, this paper proposes a stacker crane fault diagnosis method based on multi-data fusion and lightweight MobileViT. This method employs a multi-stage signal preprocessing workflow of "noise reduction and reconstruction - fusion - two-dimensional encoding" to optimize, decompose, fuse, and encode sound and vibration signals collected from multiple sensors, effectively improving the representativeness of the original signals. Based on this, a lightweight MobileViT model optimized for stacker crane scenarios is constructed to balance global dependency features and local pattern recognition while reducing model computational complexity, thereby achieving efficient diagnosis of different fault modes in stacker cranes. Summary of the Invention

[0005] The purpose of this application is to provide a stacker crane fault diagnosis method based on lightweight MobileViT. This method, which integrates multi-data fusion and lightweight MobileViT, aims to address the shortcomings of existing stacker crane fault diagnosis methods, such as insufficient multi-source data fusion, limited feature extraction capabilities, excessive model complexity, and limited on-site deployment. By proposing a lightweight MobileViT network structure with multi-stage signal preprocessing and optimization, it can fully integrate multi-sensor information, consider both local and global feature extraction, and significantly reduce computational overhead while maintaining diagnostic accuracy. This achieves efficient, real-time, and low-cost intelligent fault diagnosis for stacker cranes.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a stacker crane fault diagnosis method based on lightweight MobileViT, the method comprising: S1. Deploy a sensor group based on the stacker crane's working site to obtain raw sensing data for eight modes; S2. The multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and two-dimensional coding preprocesses the acquired raw sensor data to obtain the corresponding two-dimensional image data. S3. Divide the acquired two-dimensional image data into training set, validation set and test set; train the MobileViT model based on the training set and validation set to obtain the trained MobileViT model. S4. Input the test set into the trained MobileViT model for fault diagnosis, and perform visualization analysis on the results.

[0007] According to the specific embodiments provided in this application, this application has the following technical effects: Through multi-stage preprocessing of "noise reduction and reconstruction - fusion - two-dimensional coding", the effective fusion of heterogeneous data from multiple sources such as sound and vibration is achieved, which significantly improves the robustness of fault diagnosis and reduces the interference of noise and operating condition fluctuations on the diagnostic results.

[0008] Based on the lightweight MobileViT network structure, it can efficiently extract both local impact features and long-term global dependencies, breaking through the technical limitations of traditional CNN, RNN / LSTM and conventional Transformer, and greatly improving the completeness and accuracy of feature representation.

[0009] By reducing model computational overhead through lightweight network design, the deployment challenges of advanced deep learning models on edge devices are solved, achieving both high-precision fault diagnosis and real-time on-site response for stacker cranes.

[0010] Ultimately, this significantly improves the accuracy, robustness, and real-time performance of stacker crane fault diagnosis, providing reliable technical support for intelligent operation and maintenance in industrial sites and meeting the needs of efficient operation and maintenance in practical applications. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a fault diagnosis method based on multi-data fusion and lightweight MobileViT provided in one embodiment of this application; Figure 2 This is an example of an engraving production line provided in one embodiment of this application; Figure 3 This is a data acquisition point selection provided in one embodiment of this application; Figure 4 This is a data preprocessing flowchart provided in one embodiment of this application; Figure 5 This is a lightweight MobileViT model structure provided in one embodiment of this application; Figure 6 This is a lightweight attention mechanism provided in one embodiment of this application; Figure 7 This is a lightweight MobileViT block provided in one embodiment of this application; Figure 8 This is a comparison diagram of multimodal signals provided in one embodiment of this application; Figure 9(a) shows the confusion matrix of ConvNeXt; Figure 9(b) shows the confusion matrix of EfficientNetV2; Figure 9(c) shows the confusion matrix of MobileNetV2; Figure 9(d) shows the confusion matrix of MobileViTXXS; Figure 9(e) shows the confusion matrix of ResNet34; Figure 9(f) shows the confusion matrix of ShuffleNetV2; Figure 9(g) shows the confusion matrix of SwinTransformerTiny; Figure 9(h) shows the confusion matrix of Lightweight MobileViT. Figure 10 This is a comparison of the accuracy of different model training processes provided in one embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] like Figure 1 The embodiment shown in this application provides a stacker crane fault diagnosis method based on lightweight MobileViT, the method comprising: S1. Deploy a sensor group based on the stacker crane's working site to obtain raw sensing data for eight modes; S2. The multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and two-dimensional coding preprocesses the acquired raw sensor data to obtain the corresponding two-dimensional image data. S3. Divide the acquired two-dimensional image data into training set, validation set and test set; train the MobileViT model based on the training set and validation set to obtain the trained MobileViT model. S4. Input the test set into the trained MobileViT model for fault diagnosis, and perform visualization analysis on the results.

[0016] Optionally, step S1, which involves deploying a sensor group at the stacker crane's working site to obtain raw sensing data for eight modes, further includes: Three data collection points are deployed at the stacker crane's working site: the first data collection point, the second data collection point, and the third data collection point. The first data acquisition point is located on the bearing guide rail at the top of the stacker crane, used to monitor its smooth operation or abnormal vibration. The second data acquisition point is set at the gear and rack transmission mechanism adjacent to the ground track, and is used to collect signal characteristics during the meshing transmission process; The third data acquisition point is set on the side of the pallet of the stacker crane to collect signals of the gear and rack transmission during operation; Each data acquisition point is equipped with two triaxial accelerometers and two sound sensors. The accelerometers are used to capture vibration signals in the x, y, and z directions, and the sound sensors are used to capture sound signals. The raw sensing data for eight modes were obtained based on the vibration signals in the x, y, and z directions and the sound signals.

[0017] Specifically, the experimental data for the self-collected dataset comes from a stacker crane in the carving production line. The aim is to collect its vibration and sound signals under different health conditions. Its general structure is as follows: Figure 2 As shown.

[0018] The complete signal acquisition system consists of the following components: one iPotest-1006 data acquisition unit, two miniature triaxial IEPE accelerometers, and two N2250 free-field measurement microphones. Considering that the fault signals generated by the stacker crane's drive components are mainly concentrated in the low-to-mid frequency range, the sampling frequency (fs) of all channels is uniformly set to 10kHz. To accurately capture the operating status of different key components, the sensors are strategically placed in three core locations (see...). Figure 3 (a)(b)(c)): Location 1: A set of sensors is installed on the bearing guide rail at the top of the stacker crane to monitor its smooth operation or abnormal vibration; Location 2: A set of sensors is installed close to the gear and rack transmission mechanism of the ground track to collect signal characteristics during the meshing transmission process; Location 3: A set of sensors is installed on the side of the pallet of the stacker crane to collect the signal of the gear and rack transmission during operation. Two triaxial accelerometers and two sound sensors are set at each location. Each accelerometer can capture vibration signals in the x, y, and z directions, and each sound sensor captures the sound signal at that location. Therefore, each acquisition can simultaneously obtain signal data from 8 channels, including 6 vibration signals (x1, y1, z1, x2, y2, z2) and 2 sound signals (s1, s2) to obtain richer information. The three fault states last for a short time, only about 10-20 seconds in the fault range. Therefore, 200,000 data points were collected under each state, with each data point spaced 0.0001 seconds apart.

[0019] To comprehensively evaluate the diagnostic performance of the proposed algorithm, experiments were conducted to collect data on the operation of the stacker crane under six typical health and fault states. For each state, long-term continuous signals were collected. The specific composition of the dataset is shown in Table 1.

[0020] Table 1. Composition of Self-Collected Datasets

[0021] First, a particle swarm optimization-based variational mode decomposition (PSO-VMD) technique is used to denoise and reconstruct the 8-channel signal to enhance the saliency of fault features. Then, a multimodal feature fusion network is used to fuse information from all eight modes. Next, a non-overlapping sliding window method is employed to segment the reconstructed signal into samples, with each sample having a length of 600 data points. Finally, an improved Gram angular field transform is used to convert the one-dimensional time-series signal samples into two-dimensional images, constructing an image dataset that can be directly used for model training and validation.

[0022] Optionally, in step S2, the acquired raw sensor data is preprocessed using a multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and 2D encoding to obtain the corresponding 2D image data, such as... Figure 4 As shown, firstly, adaptive signal decomposition and noise reduction reconstruction are performed using optimized variational mode decomposition (VMD) to suppress background noise. Then, the designed data fusion network is used to fuse multi-sensor data. Finally, Gramm-Aufbau (GADF) is used to encode the fused one-dimensional time-series signal into a two-dimensional image that preserves time correlation, improving data quality and achieving effective signal representation. This also includes: S201. Normalize the raw vibration and sound signals collected from the stacker crane. Assume the raw data is represented as follows: By scaling, its value range is placed between [-1, 1]. ; in, The original data after normalization. For the i-th original data sample value, The minimum value of the original data sample. The maximum value of the original data sample; S202. Convert the scaled time series points into points in a polar coordinate system, where time points correspond to polar angles and signal values ​​correspond to radii. The resulting time series in polar coordinates is expressed by the following formula: ; in, yes The corresponding timestamp, Normalization factor; S203. Reconstruct the time series in polar coordinates using the Gramian Angular Field (GAF). Let the polar coordinate angle corresponding to the normalized time series be... θ i and θ j By calculating angles and θ i + θj cosine ( θ i + θ j ) and angle difference θ i - θ j sine sin( θ i - θ j Construct the sum-type Gramian Angular Summation Field (GASF) matrix and the difference-type Gramian Angular Difference Field (GADF) matrix. The formulas for calculating both are as follows: ; ; Based on generation The matrix is ​​encoded to obtain the corresponding two-dimensional image data.

[0023] Optionally, the trained MobileViT model obtained in S3 further includes: The overall structure of the MobileViT network is constructed, including standard convolutional layers, a lightweight feature extraction backbone network composed of improved MV2 modules and SE modules, MobileViT blocks, global pooling layers, and a classification head; First, the number of input channels is expanded by 1×1 point convolution, then features are extracted by 3×3 depthwise separable convolution, and finally the number of channels is compressed back to the original number by 1×1 point convolution again, and features are transferred across layers through residual connections. Specifically, Figure 5 The overall structure of the proposed lightweight MobileViT network is obtained by stacking standard convolutions, improved MV2 modules, and lightweight MobileViT blocks, where L represents the number of repeated stacks of lightweight Transformer modules.

[0024] In the proposed lightweight MobileViT network architecture, lightweight and efficient inverse residual blocks are used as the basic building blocks. This module aims to effectively capture feature information while maintaining low computational overhead, and promote information flow and gradient propagation through residual connections under specific conditions. First, the number of input channels is expanded using 1×1 point convolutions, then features are extracted using 3×3 depthwise separable convolutions, and finally, the number of channels is compressed back to its original number using another 1×1 point convolution. To prevent the loss of low-dimensional feature information during the dimensionality increase and subsequent reduction process, ReLU6 is chosen as the activation function. ReLU6 is a Restricted Linear Unit (RLU) activation function that restricts the output of ReLU to between 0 and 6. Its expression is: .

[0025] The SE module is used to adaptively adjust the weights of the obtained input feature map, including performing global average pooling on each channel, obtaining channel weight coefficients through two layers of 1×1 convolution and non-linear activation functions, and weighting the original feature map channel by channel to obtain the optimized feature map. The global feature acquisition module is used to map the optimized feature map to the feature sequence and input it into the multi-head self-attention mechanism to model the long-range dependencies between different spatial locations in the feature map and obtain feature representations containing global context information. The MobileViT block is a major component of the MobileViT model. To improve the accuracy of this model for stacker crane fault diagnosis without increasing computational load, a lightweight MobileViT block was proposed as an improvement. Its structure is as follows: Figure 7 As shown, the key modifications include: (1) Modifying the transformer module: replacing self-attention with lightweight attention, and changing the linear projection in FFN to convolutional projection; (2) Modifying the Fusion module. The convolution was modified to depthwise separable convolution, and efficient channel attention was introduced after the final convolution. To adapt to the limited computing power of the field equipment, the MobileViT-XXS model was selected as the base model for improvement.

[0026] Optionally, the SE module is further configured to: First, pointwise convolution is used to expand the number of channels. Then, a depthwise separable convolution is used to perform spatial filtering on the expanded feature map. Next, global adaptive average pooling is used to globally compress the features of each channel. A 1×1 convolutional layer is used to compress the channels, and an activation function is used to increase the non-linear expressive power. The channel dimension is restored by another 1×1 convolution, and the weighting coefficients for each channel are generated using an activation function; Each channel of the input feature map is adjusted based on the weighting coefficient of each channel to obtain the optimized feature map.

[0027] Specifically, to enhance the network's representational capabilities, this paper introduces a Squeeze-and-Excitation (SE) module after the depthwise separable convolution. The improved module first expands the number of channels using pointwise convolution, then spatially filters the expanded feature map using a depthwise separable convolution, globally compresses the features of each channel using global adaptive average pooling, further compresses the channels using a 1×1 convolutional layer, and increases non-linear expressiveness through an activation function (ReLU). Next, another 1×1 convolution restores the channel dimension, and an activation function generates weighted coefficients for each channel. Finally, these weights are used to adjust each channel of the input feature map, amplifying important features and suppressing unimportant features.

[0028] Optionally, in order to obtain global feature information from the signal, the global feature acquisition module employs complex matrix multiplication and multidimensional exponential operations in its multi-head self-attention mechanism. Its computational complexity mainly lies in linear transformation, calculation of the dot product of the query matrix and the key matrix, weighted summation, and multi-head concatenation. Specifically, the computational complexity is expressed as follows: (1) in, Indicates the height of the input image. Indicates the width of the input image. Indicates the number of channels.

[0029] Its computational complexity is mainly due to To reduce computational complexity, this paper proposes a lightweight attention mechanism that replaces complex matrix multiplication with broadcast computation, as shown in the following structure: Figure 6 As shown. Also includes: (1) Utilization The convolutional layer pairs have the shape of The input is convolved and projected, and the channel dimension is linearly transformed independently at each spatial location; Specifically, using The convolutional layer pairs have the shape of The input is convolved and projected, which is equivalent to performing a linear transformation on the channel dimension independently at each spatial location. Here, the weight matrices corresponding to Q, K, and V are not defined separately but are included in the convolutional layer. The result after this convolutional layer has the shape... The output; through The convolutional layers yield Q, K, and V matrices, calculated as follows: Where input Dimensions , Included in convolutional layers, The shape is , All shapes are .

[0030] (2) Information flow is achieved through context aggregation of channels. First, broadcast operation is performed on the Q matrix, then element-wise multiplication is performed on the Q matrix and the V matrix, and then summation operation is performed on the time dimension to obtain the global context vector of each channel. Specifically, information flow is achieved through context aggregation of channels. Since the Q matrix and the V matrix are uniquely different, when calculating their product, Q needs to be multiplied by each dimension c of K. This means that the Q matrix needs to be broadcast, and then the two matrices need to be multiplied element-wise. Finally, the time dimension is summed to obtain the global context vector for each channel. This process can be represented as: (2) in, Indicates a broadcast operation; (3) The obtained global context vector is transmitted to the value matrix through element-wise multiplication via broadcasting to obtain key fine-grained features from the global scope of the signal. ; Specifically, The contextual information contained therein is transmitted to the value matrix through element-wise multiplication broadcast, thereby obtaining key fine-grained features from the global scope of the signal. The process is represented as: (3) (4) Convolutional projection and regularization are performed on the obtained key fine-grained features using a convolutional layer and a dropout layer respectively, to obtain the output feature map enhanced by global context attention, which is used as the output of the global feature acquisition module. Among them, the global context vector obtained in step (2) can be regarded as the attention weight for modeling the importance of each channel, and is used to weight the value matrix in step (3); Lightweight self-attention can be defined as: (4) The core of this attention mechanism lies in designing a single-channel, position-based attention score. This score is used to weight the Key and aggregate the global context for each channel. Then, this global context for each channel is used to scale the corresponding channel features of the Value across all positions. Based on this process, the computational cost of lightweight attention can be expressed as: (5) Equation (9) gives the output form of this module, which is the final output after weighting the features through the global context vector and then passing through Dropout. To further analyze the lightweight characteristics of this module, Equation (10) derives its computational complexity (FLOPs), showing that this module only contains 2HWC. 2 With a computational complexity of +3HWC + HW, this design meets the requirements of lightweight design. By cleverly utilizing a broadcast mechanism, it avoids positional attention calculations based on spatial location pairs, reducing the complexity to a minimum. This significantly reduces computational complexity compared to multi-head self-attention when the input image size is large.

[0031] Optionally, in S4, test diagnosis and result visualization analysis are performed. The test set is input into the trained model for fault diagnosis, and the results are visualized and analyzed.

[0032] First, through ablation experiments, the impact of the proposed data preprocessing process on the classification results was analyzed, thus proving the necessity of each step.

[0033] Secondly, the proposed deep learning model is compared with other existing models on a self-collected dataset to evaluate the advancement of the proposed method.

[0034] Finally, by comparing the proposed method on public datasets, we verify its generalization ability in different scenarios and its advantages over existing methods.

[0035] To ensure fairness, all experiments were conducted on a high-performance server equipped with two NVIDIA GeForce RTX 4090 GPUs, each with 24 GB of dedicated video memory. For the software environment, PyTorch was used as the deep learning framework, with GPU acceleration support provided by CUDA 12.2. The operating system was Linux, with NVIDIA driver version 535.230.02.

[0036] First, the parameter configuration of the proposed model is set as shown in Table 2. The "Gear and Rack with Foreign Objects" fault category from Table 1 is selected to verify the effectiveness of the proposed model in data fusion. A comparison chart of a complete fault waveform is shown below. Figure 8 As shown. From Figure 8As can be seen, only the x1, y1, and z1 vibration signals in the original signal exhibit relatively obvious periodic fault impacts, while the fault characteristics of other modes (such as s1 and s2 sound signals) are submerged by strong background noise. After data fusion processing, the final generated signal not only successfully inherits and amplifies the key fault characteristics in x1, y1, and z1, but also effectively suppresses noise and irrelevant details from other modes. This significantly improves the signal-to-noise ratio of the fused signal, making the fault impact characteristics more prominent, thus providing high-quality data input for subsequent accurate fault diagnosis.

[0037] Table 2. Lightweight MobileVit Network Parameter Settings

[0038] To systematically verify the effectiveness and contribution of each core module in the fault diagnosis framework proposed in this study, a series of ablation experiments were designed. The innovations of this study mainly include two aspects: (1) a data preprocessing process based on PSO-VMD and wavelet fusion network, used to convert multi-source one-dimensional signals into high-quality two-dimensional feature maps; (2) a lightweight MobileViT classification model for this feature map. The ablation experiment design is as follows: Experiment 1 (Baseline Model): The vibration and sound signals collected from the stacker crane are all one-dimensional data. However, the baseline model selected in this study requires two-dimensional input. Therefore, the baseline scheme uses the recognized Gram angle difference field division technique to convert the original signal into a two-dimensional image and uses the standard MobileViTXXS model for classification. This experiment aims to measure the performance level achievable by a standard and mature technical process. Experiment 2 (Validation of the Improved GADF Method): This experiment replaces the GADF module in the baseline scheme with the improved GADF method designed in this paper, while the classifier still uses the standard MobileViTXXS. By comparing the results of Experiment 2 with those of Experiment 1, the effectiveness of the proposed one-dimensional to two-dimensional image conversion method can be evaluated. Experiment 3 (Verification of Data Denoising Method): Based on Experiment 2, a PSO-VMD signal decomposition and reconstruction module is introduced. That is, the original signal is denoised using VMD optimized by the PSO algorithm, and then converted into a two-dimensional image input into the fault diagnosis model using the improved GADF. By comparing the results of Experiment 3 with those of Experiment 2, the effectiveness of the proposed data denoising method can be evaluated. Experiment 4 (Verification of Fusion Strategy): Based on Experiment 3, a fusion network is further introduced to verify that the fusion of multi-sensor data can form a more comprehensive and discriminative feature representation. Experiment 5 (Complete Scheme of this Paper): This experiment incorporates the proposed complete method. Based on an advanced data processing workflow (PSO-VMD + fusion network), a lightweight MobileViT model is used as the classifier. Comparing the results of Experiments 4 and 5 clearly demonstrates the effectiveness of the improvements to the model architecture itself. Detailed experimental results are shown in Table 3.

[0039] Through a series of ablation experiments, the results clearly demonstrate that each module of the improved GADF, PSO-VMD signal decomposition and denoising, feature fusion strategy, and lightweight MobileViT model proposed in this study makes a positive and significant contribution to the final fault diagnosis performance. The accuracy gradually improved from a baseline of 82.0% to a final accuracy of 98.0%, fully validating the overall effectiveness and superiority of our proposed multi-module collaborative model.

[0040] Table 3. Comparison of ablation test results

[0041] To verify the effectiveness and superiority of the proposed method in stacker crane fault diagnosis, a series of comparative experiments were conducted on a self-collected stacker crane dataset. The self-collected stacker crane data underwent the data preprocessing workflow proposed in this paper to obtain an image dataset, which was then input into the proposed lightweight MobileViT model and several classic deep learning models, including baseline models MobileViTXXS, ResNet34, MobileNet V2, Vgg, SwinTransformer2, and ConvNeXt. To ensure fairness in the comparison, all models were trained and evaluated on a unified hardware platform (NVIDIA GeForce RTX 4090 GPU) and software framework (Python 3.10 programming language, PyTorch 2.1.1 deep learning framework, and CUDA 12.1 GPU acceleration). This paper will comprehensively analyze the experimental results from two dimensions: model complexity and model classification performance, using multiple evaluation metrics.

[0042] As shown in Table 4, the proposed lightweight MobileViT model achieves an accuracy of 97%, significantly outperforming other models. Furthermore, the lightweight MobileViT model has only 900 and 399 parameters, far fewer than models like ResNet34, MobileNetV2, and ShuffleNetV2, and also shows a reduction compared to the baseline model MobileViTXXS. While MobileNetV2 and ShuffleNetV2, as classic lightweight CNN models, do indeed have significantly fewer parameters than ordinary CNN networks (such as ResNet34), their accuracy is also greatly reduced. Therefore, the hybrid model proposed in this paper further reduces the model's weight while maintaining accuracy.

[0043] Table 4. Comparison of accuracy with different network parameter counts

[0044] Figure 9 shows the confusion matrices of the proposed model and other comparative methods on the self-collected dataset. Figure 9(a) is the confusion matrix of ConvNeXt; Figure 9(b) is the confusion matrix of EfficientNetV2; Figure 9(c) is the confusion matrix of MobileNetV2; Figure 9(d) is the confusion matrix of MobileViTXXS; Figure 9(e) is the confusion matrix of ResNet34; Figure 9(f) is the confusion matrix of ShuffleNetV2; Figure 9(g) is the confusion matrix of SwinTransformerTiny; and Figure 9(h) is the confusion matrix of lightweight MobileViT. It can be seen that, except for the baseline model MobileViTXXS and the proposed model, the recognition accuracy of other models for the category of "pallet ingress" is less than 90%, which proves that other models have a high false positive rate for this type of fault. However, in actual working conditions, the materials on the pallet are complex and varied, and "pallet ingress" is a very common fault. The proposed model has a recognition accuracy of up to 97.0% for pallet ingress, and its recognition rate for all faults is higher than that of all comparative methods.

[0045] To verify the effectiveness of the proposed method, a comparative experiment was further conducted on the Paderborn bearing dataset in Germany. This dataset is widely used in the field of fault diagnosis and contains vibration signal data from 32 bearings in three categories: healthy bearings, bearings with artificial damage, and bearings with actual wear, at a sampling frequency as high as 64 kHz. To ensure consistency, data from six categories were selected and converted into image format using the data preprocessing method proposed in this paper. These images were then input into multiple network models, including the lightweight MobileViT model designed in this paper, for training, validation, and testing.

[0046] Figure 10The graph compares the accuracy curves of multiple network models during training. It can be observed that most models show rapid accuracy growth in the early stages of training, but the model proposed in this paper exhibits a superior learning trajectory and final performance. After a brief fluctuation of 20 epochs, the proposed model enters a phase of sustained and stable performance improvement, surpassing most models around the 40th epoch. In the later stages of training, it achieves the highest validation accuracy and maintains excellent stability.

[0047] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0048] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (Read-Only Memory). Memory includes ROM, magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0049] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0050] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0051] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A stacker crane fault diagnosis method based on lightweight MobileViT, characterized in that, The method includes: S1. Deploy a sensor group based on the stacker crane's working site to obtain raw sensing data for eight modes; S2. The multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and two-dimensional coding preprocesses the acquired raw sensor data to obtain the corresponding two-dimensional image data. S3. Divide the acquired two-dimensional image data into training set, validation set and test set; train the MobileViT model based on the training set and validation set to obtain the trained MobileViT model. Obtaining the trained MobileViT model includes the following steps: The overall structure of the MobileViT network is constructed, including standard convolutional layers, a lightweight feature extraction backbone network composed of improved MV2 modules and SE modules, MobileViT blocks, global pooling layers, and a classification head; The MV2 module is improved by first expanding the number of input channels through 1×1 point convolution, then extracting features through 3×3 depthwise separable convolution, and finally compressing the number of channels back to the original number through 1×1 point convolution again, and realizing cross-layer feature transfer through residual connections. The SE module is used to adaptively adjust the weights of the obtained input feature map. This includes performing global average pooling on each channel, obtaining channel weight coefficients through two layers of 1×1 convolution and non-linear activation functions, and weighting the original feature map channel by channel to obtain the optimized feature map. The global feature acquisition module is used to map the optimized feature map to the feature sequence and input it into the multi-head self-attention mechanism to model the long-range dependencies between different spatial locations in the feature map and obtain feature representations containing global context information. The MobileViT block is used to fuse local detail features extracted by the convolutional network with global features obtained through a multi-head self-attention mechanism to output a fused feature map that combines local texture information and global semantic information. A global pooling layer is used to perform global average pooling or global weighted pooling on the fused feature map in the spatial dimension to obtain a fixed-length global feature vector. The classification head, including one or more fully connected layers and a Softmax activation function, is used to map the global feature vector to the predicted probability of each fault category, thereby outputting the fault diagnosis results of the stacker crane. S4. Input the test set into the trained MobileViT model for fault diagnosis, and perform visualization analysis on the results.

2. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 1, characterized in that, The S1 method, which deploys a sensor group at the stacker crane's working site to obtain raw sensing data in eight modes, also includes: Three data collection points are deployed at the stacker crane's working site: the first data collection point, the second data collection point, and the third data collection point. The first data acquisition point is located on the bearing guide rail at the top of the stacker crane, used to monitor its smooth operation or abnormal vibration. The second data acquisition point is set at the gear and rack transmission mechanism adjacent to the ground track, and is used to collect signal characteristics during the meshing transmission process; The third data acquisition point is set on the side of the pallet of the stacker crane to collect signals of the gear and rack transmission during operation; Each data acquisition point is equipped with two triaxial accelerometers and two sound sensors. The accelerometers are used to capture vibration signals in the x, y, and z directions, and the sound sensors are used to capture sound signals. The raw sensing data for eight modes were obtained based on the vibration signals in the x, y, and z directions and the sound signals.

3. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 1, characterized in that, The S2 process, which involves a multi-sensor data fusion preprocessing workflow based on noise reduction, reconstruction, fusion, and 2D encoding, preprocesses the acquired raw sensor data to obtain the corresponding 2D image data. It also includes: S201. Normalize the raw vibration and sound signals collected from the stacker crane. Assume the raw data is represented as follows: By scaling, its value range is placed between [-1, 1]. ； in, The original data after normalization. For the i-th original data sample value, The minimum value of the original data sample. The maximum value of the original data sample; S202. Convert the scaled time series points into points in a polar coordinate system, where time points correspond to polar angles and signal values ​​correspond to radii. The resulting time series in polar coordinates is expressed by the following formula: ； in, yes The corresponding timestamp, Normalization factor; S203. Reconstruct the time series in polar coordinates using the Gram angle field. Let the polar coordinate angle corresponding to the normalized time series be... θ i and θ j By calculating the cosine of the sum of two polar coordinate angles, cos( θ i + θ j ) and the sine of the difference between the two polar coordinate angles sin( θ i - θ j Construct the sum-type Gram angle field matrix and the difference-type Gram angle field matrix, and their calculation formulas are as follows: ； ； Based on generation The matrix is ​​encoded to obtain the corresponding two-dimensional image data.

4. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 1, characterized in that, The SE module is also used for: First, pointwise convolution is used to expand the number of channels. Then, a depthwise separable convolution is used to perform spatial filtering on the expanded feature map. Next, global adaptive average pooling is used to globally compress the features of each channel. A 1×1 convolutional layer is used to compress the channels, and an activation function is used to increase the non-linear expressive power. The channel dimension is restored by another 1×1 convolution, and the weighting coefficients for each channel are generated using an activation function; Each channel of the input feature map is adjusted based on the weighting coefficient of each channel to obtain the optimized feature map.

5. The stacker crane fault diagnosis method based on lightweight MobileViT according to claim 1, characterized in that, The global feature acquisition module further includes: (1) The convolutional layer used has the shape of The input is convolved and projected, and the channel dimension is linearly transformed independently at each spatial location; (2) Information flow is achieved through context aggregation of channels. First, broadcast operation is performed on the Q matrix, then element-wise multiplication is performed on the Q matrix and the V matrix, and then summation is performed on the time dimension to obtain the global context vector of each channel. (3) The obtained global context vector is transmitted to the value matrix through element-wise multiplication via broadcasting to obtain key fine-grained features from the global scope of the signal. ; (4) Convolutional projection and regularization are performed on the obtained key fine-grained features using a convolutional layer and a dropout layer respectively, to obtain the output feature map enhanced by global context attention, which is used as the output of the global feature acquisition module.

Images