Fault traceability and decision-making method and device based on basic model, equipment and medium

Abstract

The invention provides a fault traceability and decision-making method and device based on a basic model, equipment and a medium. The method comprises the steps of collecting historical multi-source fault data of different parts from an industrial system; preprocessing the historical multi-source fault data, and normalizing the preprocessed historical multi-source fault data to obtain historical multi-source one-dimensional data; generating a to-be-trained image according to the historical multi-source one-dimensional data, and obtaining an industrial system component fault type, a fault reason and a solution corresponding to the to-be-trained image; constructing a fault diagnosis data set; constructing a large fault diagnosis model according to the fault diagnosis data set and a pre-trained multi-modal basic model; and inputting multi-source fault data acquired in real time into the fault diagnosis large model to obtain a target fault type, a target fault reason and a target solution corresponding to the multi-source fault data. According to the invention, the accuracy and efficiency of fault detection can be improved, and a fault solution can be directly obtained, so that the labor cost is reduced.

Description

Technical Field

[0001] This application relates to the field of fault detection technology, and in particular to a fault tracing and decision-making method, apparatus, equipment and medium based on a basic model. Background Technology

[0002] Current fault analysis techniques mainly rely on machine learning and deep learning methods. They typically classify faulty parts by component, train deep neural networks through category labels to obtain a classification model, and thus identify the type of fault.

[0003] However, this method has certain limitations. First, after fault classification, experts still need to manually review relevant data for further analysis to determine the specific cause of the fault, possible misdiagnosis, and repair suggestions. This process is not only time-consuming and labor-intensive but also highly dependent on expert experience. Furthermore, due to the complexity of actual industrial environments, various types of faults may exist, making it difficult for traditional classification models to eliminate the possibility of errors, which can easily lead to misjudgments or omissions.

[0004] Therefore, a new fault tracing and decision-making method based on a fundamental model is urgently needed to solve the above problems. Summary of the Invention

[0005] In view of this, this application provides a fault tracing and decision-making method, apparatus, equipment and medium based on a basic model, which can improve the accuracy and efficiency of fault detection and directly obtain fault solutions, thereby reducing labor costs.

[0006] A first aspect of this application provides a fault tracing and decision-making method based on a base model, comprising: collecting historical multi-source fault data of different components from an industrial system; preprocessing the historical multi-source fault data and normalizing the preprocessed historical multi-source fault data to obtain historical multi-source one-dimensional data, wherein the preprocessing includes at least data cleaning, signal enhancement, and data augmentation; generating a training image based on the historical multi-source one-dimensional data, and obtaining the industrial system component fault type, fault cause, and solution corresponding to the training image; constructing a fault diagnosis dataset, wherein the fault diagnosis dataset includes the training image and the corresponding industrial system component fault type, fault cause, and solution; constructing a large-scale fault diagnosis model based on the fault diagnosis dataset and a pre-trained multimodal base model; and inputting real-time collected multi-source fault data into the large-scale fault diagnosis model to obtain the target fault type, target fault cause, and target solution corresponding to the multi-source fault data.

[0007] In one possible implementation, the preprocessing of the historical multi-source fault data includes: data cleaning of the historical multi-source fault data, wherein the data cleaning at least includes: resampling and aligning the sampling time of the historical multi-source fault data; interpolating and repairing short-segment missing values; removing outliers and external noise from long-segment missing values; and signal enhancement of the historical multi-source fault data, wherein the signal enhancement at least includes: applying a high-pass filter or wavelet detrending to the baseline-corrected signal in the historical multi-source fault data; and further processing the data. In historical multi-source fault data, vibration, current, and other signals are bandpass filtered according to the fault frequency band; the one-dimensional time-domain signal of the multi-source fault data is converted into a time-frequency graph as an image encoding input; the historical multi-source fault data is augmented, which includes at least: adding weak Gaussian noise to the historical multi-source fault data; performing slight time-axis stretching, compression, or jittering on the historical multi-source fault data; randomly cutting out small segments of signals from the historical multi-source fault data and replacing them with adjacent data segments; and randomly masking some channel data in the historical multi-source fault data.

[0008] In one possible implementation, the fault diagnosis dataset includes a training dataset and a test dataset; the step of constructing a large fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal base model includes: loading the multimodal base model, training the multimodal base model based on the training dataset to obtain an original large fault diagnosis model; evaluating the original large fault diagnosis model based on the test dataset, and optimizing the original large fault diagnosis model by combining efficient parameter fine-tuning techniques and evaluation results to obtain the large fault diagnosis model.

[0009] In one possible implementation, optimizing the original fault diagnosis model by combining efficient parameter fine-tuning technology and evaluation results includes: when the evaluation result indicates that the model parameters of the original fault diagnosis model do not meet the preset parameter requirements, fine-tuning the model parameters according to the following formula: ;in, ,and .

[0010] In one possible implementation, generating multiple training images based on the historical multi-source one-dimensional data includes: reshaping the historical multi-source one-dimensional data into a two-dimensional matrix, stitching them row by row in channel order using a rasterization layout to form a two-dimensional data structure; normalizing the two-dimensional data structure to the 0–255 range according to the maximum and minimum values ​​and copying it into three channels to generate the training images.

[0011] In one possible implementation, the image to be trained is generated according to the following formula: ;in, The image to be trained, The pixel values ​​of the two-dimensional data structure. The maximum value in the two-dimensional data structure. It is the minimum value in the two-dimensional data structure.

[0012] In one possible implementation, the method further includes: using the fault diagnosis large model as a teacher model to construct a student model corresponding to the teacher model; performing distillation training on the student model based on the teacher model to obtain an initial student model; removing redundant attention heads, channels, or weight parameters from the initial student model and quantizing the initial student model to obtain a lightweight final student model; and inputting the multi-source fault data into the final student model to obtain the target fault type, the target fault cause, and the target solution.

[0013] Secondly, embodiments of this application also provide a fault tracing and decision-making device based on a basic model, comprising: a data acquisition module, a processing module, a generation module, a first construction module, a second construction module, and an input module; the data acquisition module is used to acquire historical multi-source fault data of different components from an industrial system; the processing module is used to preprocess the historical multi-source fault data and normalize the preprocessed historical multi-source fault data to obtain historical multi-source one-dimensional data, wherein the preprocessing includes at least data cleaning, signal enhancement, and data augmentation; the generation module is used to generate a training image based on the historical multi-source one-dimensional data and obtain the industrial system component fault type, fault cause, and solution corresponding to the training image; the first construction module is used to construct a fault diagnosis dataset, wherein the fault diagnosis dataset includes the training image and the corresponding industrial system component fault type, fault cause, and solution; the second construction module is used to construct a large fault diagnosis model based on the fault diagnosis dataset and a pre-trained multimodal basic model; the input module is used to input the real-time acquired multi-source fault data into the large fault diagnosis model to obtain the target fault type, target fault cause, and target solution corresponding to the multi-source fault data.

[0014] Thirdly, embodiments of this application also provide an electronic device, the electronic device including a processor and a memory, the memory being used to store instructions, and the processor being used to call the instructions in the memory, causing the electronic device to execute the fault tracing and decision-making method based on the basic model as described in the first aspect.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the fault tracing and decision-making method based on the basic model as described in the first aspect.

[0016] Compared with related technologies, the embodiments of this application have at least the following advantages: By preprocessing historical multi-source fault data, which includes at least data cleaning, signal enhancement, and data augmentation, the quality of the historical multi-source fault data in the subsequent image representation stage can be improved, and the generalization ability of the subsequently constructed fault diagnosis model can be enhanced. By constructing a fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal base model, not only is the accuracy and efficiency of fault detection improved, but the fault diagnosis model can also better adapt to the actual situation of industrial systems, reducing the probability of false alarms and false negatives. In addition, since the fault diagnosis dataset includes the solutions corresponding to the images to be trained, after inputting the real-time collected multi-source fault data into the fault diagnosis model, the target solution corresponding to the multi-source fault data can be directly output, thereby avoiding subsequent analysis processes and greatly reducing labor costs.

[0017] The technical effects achieved by the second, third, and fourth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the steps of a fault tracing and decision-making method based on a basic model, as provided in an embodiment of this application; Figure 2 Another flowchart of the fault tracing and decision-making method based on a basic model provided in an embodiment of this application; Figure 3 A functional block diagram of a fault tracing and decision-making device based on a basic model provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0020] The following description sets forth many specific details to provide a full understanding of this application. The described embodiments are only some, not all, of the embodiments of this application.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0022] It should be further noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0023] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0024] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0025] For ease of understanding, some concepts related to the embodiments of this application are illustrated and explained by way of example for reference.

[0026] The base model is a large-scale AI neural network pre-trained on massive amounts of data. It can handle various tasks such as text, images, and videos, and forms the foundation for many AI applications. Based on the Transformer architecture, it typically has billions of parameters and can adapt to a wide range of tasks such as natural language processing and computer vision without complex programming. Typical examples include the GPT series and Dale, which are driving the upgrade of robotics, autonomous driving, and other fields from task-specific to open-world applications.

[0027] Distillation training is a technique that allows a "small model" to learn from the knowledge of a "large model," effectively improving the performance of the small model while reducing computational and storage costs.

[0028] Efficient parameter fine-tuning technique: By inserting a low-rank matrix into the original model weights to adapt to specific tasks, the training cost is greatly reduced while maintaining model performance.

[0029] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of an embodiment of the fault tracing and decision-making method based on the basic model of this application. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0030] It should be noted that the fault tracing and decision-making method based on the basic model in this application can be applied to fault detection in industrial systems. The executing entity can be a fault tracing and decision-making device based on the basic model. For example, when detecting faults in different components of an industrial system, the fault tracing and decision-making device based on the basic model can generate the target fault type, target fault cause, and target solution corresponding to the collected multi-source fault data. Of course, the fault tracing and decision-making method based on the basic model can also be applied to other scenarios requiring fault detection, and this application does not specifically limit its application in this regard.

[0031] The specific process of this embodiment is as follows: Figure 1 As shown, it includes the following steps: S101 collects historical multi-source fault data of different components from industrial systems.

[0032] In some embodiments, the types of historical multi-source fault data to be collected are determined, such as vibration signals, temperature signals, and pressure signals. Sensors are installed at various key parts of the industrial system to collect fault data in real time. At the same time, the accuracy and reliability of the sensors, as well as the real-time and continuous nature of data acquisition, are ensured.

[0033] S102, preprocess the historical multi-source fault data, and normalize the preprocessed historical multi-source fault data to obtain historical multi-source one-dimensional data.

[0034] Specifically, preprocessing includes at least data cleaning, signal enhancement, and data augmentation.

[0035] In some embodiments, preprocessing of historical multi-source fault data includes: data cleaning of historical multi-source fault data, wherein data cleaning includes at least: resampling and time alignment of historical multi-source fault data; interpolation repair of short missing values; outlier removal and external noise removal of long missing values; signal enhancement of historical multi-source fault data, wherein signal enhancement includes at least: baseline correction of signals with baseline drift in historical multi-source fault data using high-pass filters or wavelet detrending; bandpass filtering of vibration, current, and other signals in historical multi-source fault data according to the fault frequency band using bandpass filters; conversion of one-dimensional time-domain signals of multi-source fault data into time-frequency graphs as image encoding input; data augmentation of historical multi-source fault data, wherein data augmentation includes at least: adding weak Gaussian noise to historical multi-source fault data; performing minor time-axis stretching, compression, or jittering on historical multi-source fault data; randomly cutting out small segments of signals in historical multi-source fault data and replacing them with adjacent data segments; and randomly masking some channel data in historical multi-source fault data.

[0036] To facilitate understanding, the preprocessing process for historical multi-source fault data in this embodiment will be explained in detail in three types: 1. Data Cleaning: Cleaning historical multi-source fault data to ensure the reliability and consistency of input data. This includes: (1) Sampling alignment and synchronization processing: To address the possible differences in sampling rates or timestamp drift between different sensors, multi-source data is resampled and time-aligned; signal delay is calculated and synchronized using cross-correlation method to ensure timing consistency between channels.

[0037] (2) Missing value repair and bad pixel removal: Short missing values ​​are repaired by linear interpolation or cubic spline interpolation; long missing segments are marked and removed as appropriate. Instantaneous abnormal peak values ​​are detected by the median absolute deviation (MAD) method and repaired by median filtering or truncation method.

[0038] (3) Abnormal interference removal: Notch filters are used to suppress common power frequency interference in industrial systems (such as 50 Hz / 60 Hz) and remove external noise interference to the signal.

[0039] The historical multi-source fault data processed by the above steps can significantly improve the stability of subsequent image processing and model training.

[0040] 2. Signal Enhancement: To enhance the fault-related features in historical multi-source fault data, this embodiment may optionally introduce the following enhancement strategies: (1) Detrending processing: For signals with obvious baseline drift, a high-pass filter or wavelet detrending method is used to perform baseline correction.

[0041] (2) Filtering and noise reduction: For vibration, current and other signals, a bandpass filter is used according to the fault frequency band to retain only the frequency components containing effective information. For non-stationary signals, wavelet thresholding can be used to retain transient characteristics.

[0042] (3) Time-frequency feature transformation: One-dimensional time-domain signals can be converted into time-frequency maps as image encoding inputs, including but not limited to: STFT (Short Time Fourier Transform) to obtain amplitude spectrum; CWT (Continuous Wavelet Transform) to obtain wavelet scale map; and spectrum energy distribution map.

[0043] The aforementioned features can be used as image input alone, or they can be stitched together with historical multi-source fault data row by row as multi-channel two-dimensional image input, further enhancing the expressive power of fault features.

[0044] 3. Data Augmentation: To expand the training sample space, especially in handling situations where samples for certain fault categories are scarce in industrial systems, this embodiment can augment the time-domain signal and image samples, including but not limited to: (1) Signal-level augmentation of historical multi-source one-dimensional data: Additive noise enhancement: Add weak Gaussian noise to the signal to simulate real acquisition fluctuations; Time warp and jitter: Stretch, compress or jitter the signal on a small time axis to simulate sensor sampling instability; Signal clipping and splicing: Randomly cut off small segments of signal and replace them with adjacent data segments to simulate frame loss in industrial environments; Channel discarding: Randomly block some channel data to enhance the robustness of the model to sensor failures; Signal mixing: Linearly combine two types of signal samples in a certain proportion for training to improve the model's ability to distinguish boundary samples.

[0045] (2) Image-level augmentation of the two-dimensional training image: small-scale image enhancement with small translation, scaling, brightness perturbation, etc., while keeping the time direction unchanged; adding slight blur or noise to simulate the quantization error that may occur during the image generation process; Cutout / CutMix augmentation to simulate the situation of missing channels.

[0046] (3) Text-level augmentation of multimodal prompts such as fault causes and solutions: synonym replacement generates multiple semantically consistent but different text prompts; richer question-and-answer formats are generated through back translation (such as Chinese → English → Chinese); non-key prompts are randomly omitted to improve the model's adaptability to prompt noise.

[0047] S103, generate a training image based on historical multi-source one-dimensional data, and obtain the fault type, fault cause and solution of the industrial system component corresponding to the training image.

[0048] In some embodiments, generating multiple training images based on historical multi-source one-dimensional data includes: reshaping the historical multi-source one-dimensional data into a two-dimensional matrix, stitching them row by row in channel order using a rasterization layout to form a two-dimensional data structure; normalizing the two-dimensional data structure to the 0–255 range according to the maximum and minimum values ​​and copying it into three channels to generate training images.

[0049] In some embodiments, the training image is generated according to the following formula: ; in, For the training image, The pixel values ​​are two-dimensional data structures. The maximum value in a two-dimensional data structure. It represents the minimum value in a two-dimensional data structure.

[0050] S104, Construct a fault diagnosis dataset, which includes training images and corresponding industrial system component fault types, fault causes, and solutions.

[0051] In some embodiments, relevant papers and materials are consulted to construct a fault diagnosis dataset based on the fault types of various parts of the industrial system. Specifically, a system has several components, each with different fault types, and each fault type corresponds to some possible causes and corresponding solutions. For different faults of each component, the image path, image size, and prompt words are written into the question section of the fault diagnosis dataset, while the system fault type, system fault cause, and system fault solution corresponding to the image are written into the answer section of the fault diagnosis dataset, forming a complete fault diagnosis dataset.

[0052] S105, Construct a large fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal basic model.

[0053] In some embodiments, the fault diagnosis dataset includes a training dataset and a test dataset; constructing a large fault diagnosis model based on the fault diagnosis dataset and a pre-trained multimodal base model includes: loading the multimodal base model, training the multimodal base model based on the training dataset to obtain the original large fault diagnosis model; evaluating the original large fault diagnosis model based on the test dataset, and optimizing the original large fault diagnosis model by combining efficient parameter fine-tuning techniques and evaluation results to obtain the large fault diagnosis model.

[0054] In some embodiments, the original fault diagnosis model is optimized by combining efficient parameter fine-tuning techniques and evaluation results, including: when the evaluation results indicate that the model parameters of the original fault diagnosis model do not meet the preset parameter requirements, the model parameters are fine-tuned according to the following formula: ;in, ,and .

[0055] To facilitate understanding, the following is a detailed explanation of how this embodiment trains the large-scale fault diagnosis model: 1. Select a suitable multimodal base model, for example, choose the InternVL2.5-8B model.

[0056] 2. Train all linear layers of the multimodal base model and fix the other layers.

[0057] 3. Divide the fault diagnosis dataset into a training set and a test set in a 5:1 ratio. Use the training set to train the multimodal base model and calculate the loss, then update the model parameters.

[0058] Specifically, the training process of the multimodal base model includes two branches: text feature extraction and image feature extraction. Text feature extraction uses a pre-trained text encoder to encode the text cues given in the training set into the latent space as instruction embedding features. Image... First, the image is divided into a series of image blocks according to a specified number of blocks. These image patches are reshaped to a suitable fixed size by the model and then encoded into a latent space by a pre-trained visual encoder with fixed parameters to obtain visual labels. These visual labels are then projected onto a frozen linear layer to form visual embedding features. These instruction embedding features and visual embedding features are trained using LoRA on a pre-trained base model. The output text and instructions from the dataset are used to calculate the loss, and backpropagation is used to optimize the linear layer of the multimodal base model.

[0059] 4. Determine whether to end training based on the loss curve of the model training, and evaluate the multimodal base model on the test set. If the model does not achieve the expected results, adjust the training parameters, such as LoRA rank and LoRA alpha, and repeat the training process until the model converges to the expected level.

[0060] The LoRA formula for model training is: ;in ,and Freeze parameters during training. Training only and The parameters. For The forward propagation process becomes: .

[0061] S106, input the real-time collected multi-source fault data into the fault diagnosis big model to obtain the target fault type, target fault cause and target solution corresponding to the multi-source fault data.

[0062] Compared with related technologies, the embodiments of this application have at least the following advantages: By preprocessing historical multi-source fault data, which includes at least data cleaning, signal enhancement, and data augmentation, the quality of the historical multi-source fault data in the subsequent image representation stage can be improved, and the generalization ability of the subsequently constructed fault diagnosis model can be enhanced. By constructing a fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal base model, not only is the accuracy and efficiency of fault detection improved, but the fault diagnosis model can also better adapt to the actual situation of industrial systems, reducing the probability of false alarms and false negatives. In addition, since the fault diagnosis dataset includes the solutions corresponding to the images to be trained, after inputting the real-time collected multi-source fault data into the fault diagnosis model, the target solution corresponding to the multi-source fault data can be directly output, thereby avoiding subsequent analysis processes and greatly reducing labor costs.

[0063] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the steps of an embodiment of the fault tracing and decision-making method based on a basic model, as described in this application. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted. This fault tracing and decision-making method based on a basic model can be applied to the aforementioned fault tracing and decision-making device based on a basic model, but is not limited thereto, and the embodiments of this application do not limit it in this regard.

[0064] This embodiment is a further improvement on the aforementioned embodiment. The main improvement lies in that, after constructing the large fault diagnosis model, a lightweight student model corresponding to the large fault diagnosis model is also constructed. This approach enables the large fault diagnosis model to be deployed on edge devices, industrial terminal equipment, or embedded systems for real-time inference.

[0065] The specific process of this embodiment is as follows: Figure 2 As shown, it includes the following steps: S201 collects historical multi-source fault data of different components from industrial systems.

[0066] S202, preprocess the historical multi-source fault data, and normalize the preprocessed historical multi-source fault data to obtain historical multi-source one-dimensional data.

[0067] S203, generate a training image based on historical multi-source one-dimensional data, and obtain the fault type, fault cause and solution of the industrial system component corresponding to the training image.

[0068] S204. Construct a fault diagnosis dataset, which includes training images and corresponding industrial system component fault types, fault causes, and solutions.

[0069] S205, Construct a large fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal basic model.

[0070] S206. Input the real-time collected multi-source fault data into the fault diagnosis big model to obtain the target fault type, target fault cause and target solution corresponding to the multi-source fault data.

[0071] S201 to S206 of this embodiment are similar to S101 to S106 of the previous embodiment. To avoid repetition, they will not be described again here.

[0072] S207 uses the fault diagnosis model as the teacher model to construct a student model corresponding to the teacher model.

[0073] In some embodiments, the student model may be built on a lightweight multimodal architecture, such as reducing the number of Transformer layers, reducing the hidden dimension, reducing the visual encoder patch size, or reducing the text model depth.

[0074] S208, Distillation training is performed on the student model based on the teacher model to obtain the initial student model.

[0075] In some embodiments, the distillation training process includes: the student model simultaneously learning: the probability distribution output by the teacher model (soft label distillation); the intermediate feature layer of the teacher model (feature distillation); and the actual labeled data (hard labels). Its distillation loss function can be expressed as: ; in, The supervised loss of the student model on the real labels, Output the KL divergence between the probability distributions of the student model and the teacher model. For intermediate layer feature matching loss, , , These are the weighting coefficients.

[0076] S209: Remove redundant attention heads, channels, or weight parameters from the initial student model and quantize the initial student model to obtain the lightweight final student model.

[0077] In some embodiments, based on distillation, redundant attention heads, channels, or weight parameters in the model are removed using structural pruning techniques, including: sparsification of linear layers that do not affect performance; pruning of less important self-attention heads; and pruning of the embedding channel or patch projection of the visual encoder.

[0078] In some embodiments, this embodiment supports the following quantization methods: INT8 quantization: suitable for GPU / CPU industrial terminals; INT4 or mixed precision quantization: suitable for low-computing-power MCUs or embedded devices; KV Cache quantization: significantly reduces memory usage during inference.

[0079] In some embodiments, after model distillation, pruning, and quantization are completed, the lightweight final student model is exported in an inference format suitable for terminal devices, including but not limited to: ONNX format for cross-platform inference; TensorRTEngine format for GPUs or industrial edge gateways; TFLite or NCNN format for ARM, embedded devices, or mobile devices; C model code or quantization weight table for microcontrollers (MCUs) or FPGAs.

[0080] During the conversion process, the operator type, tensor layout, and dynamic shape of the model are adapted to ensure efficient execution on low-power devices.

[0081] In some embodiments, to meet the real-time and stability requirements of actual industrial sites, this embodiment employs the following inference acceleration technology on the terminal side: Image feature cache: The visual features obtained by the image encoder are cached on the terminal, and the features are reused when analyzing the same channel or similar time slices of data multiple times, which greatly reduces the amount of computation. Edge Runtime Optimization: Use lightweight inference engines, such as ONNX Runtime, TensorRT, and TFLite Micro, to support features such as operator fusion, memory reuse, and low-precision acceleration. Dynamic batch processing and streaming inference: Streaming window inference is used for continuously sampled sensor data, and the inference batch size is dynamically adjusted according to the terminal load to ensure a stable frame rate; KV Cache (for text decoding): By quantizing and reusing the attention cache of the language model, the memory usage and redundant computation during the inference process are reduced.

[0082] In some embodiments, considering the limited computing power and storage of edge devices, this embodiment proposes the following resource management strategy: Layer-by-Layer Loading: The model is split into visual and language modules and loaded on demand, improving execution efficiency under limited storage conditions; On-demand inference strategy: The complete inference process is only triggered when abnormal waveform changes are detected, reducing unnecessary energy consumption; Low-power mode switching: Based on the utilization of the device's CPU / GPU / TPU, adaptive switching between model precision modes (FP16 → INT8 → INT4) is achieved.

[0083] In some embodiments, to adapt to signal distribution shifts caused by changes in operating conditions and equipment aging in industrial systems, this embodiment further provides a collaborative update mechanism between the terminal side and the server side, including: Online sampling data feedback server: Feeds back anonymous sampling data allowed by the user for incremental training or redistillation.

[0084] Delta Update: The terminal only downloads the updated LoRA weights or quantization weights patches, reducing the size of the update package.

[0085] Lightweight Continual Learning: Small-scale self-adaptive learning is performed through lightweight LoRA or Adapter modules without changing the main model structure.

[0086] S210: Input the multi-source fault data into the final student model to obtain the target fault type, target fault cause, and target solution.

[0087] Compared with related technologies, the embodiments of this application have at least the following advantages: By preprocessing historical multi-source fault data, which includes at least data cleaning, signal enhancement, and data augmentation, the quality of the historical multi-source fault data in the subsequent image representation stage can be improved, and the generalization ability of the subsequently constructed fault diagnosis model can be enhanced. By constructing a fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal base model, not only is the accuracy and efficiency of fault detection improved, but the fault diagnosis model can also better adapt to the actual situation of industrial systems, reducing the probability of false alarms and false negatives. In addition, since the fault diagnosis dataset includes the solutions corresponding to the images to be trained, after inputting the real-time collected multi-source fault data into the fault diagnosis model, the target solution corresponding to the multi-source fault data can be directly output, thereby avoiding subsequent analysis processes and greatly reducing labor costs.

[0088] Based on the same idea as the fault tracing and decision-making method based on the basic model in the above embodiments, this application also provides a fault tracing and decision-making device based on the basic model. This device can be used to execute the aforementioned fault tracing and decision-making method based on the basic model. For ease of explanation, the structural diagram of the embodiment of the fault tracing and decision-making device based on the basic model only shows the parts related to the embodiments of this application. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0089] like Figure 3 As shown, the fault tracing and decision-making device 30 based on the basic model includes a data acquisition module 301, a processing module 302, a generation module 303, a first construction module 304, a second construction module 305, and an input module 306. In some embodiments, the above modules can be programmable software instructions stored in memory and executable by a processor. It is understood that in other embodiments, the above modules can also be program instructions or firmware embedded in the processor.

[0090] The acquisition module 301 is used to acquire historical multi-source fault data of different components from the industrial system; The processing module 302 is used to preprocess the historical multi-source fault data and normalize the preprocessed historical multi-source fault data to obtain historical multi-source one-dimensional data. The preprocessing includes at least data cleaning, signal enhancement and data augmentation. The generation module 303 is used to generate a training image based on the historical multi-source one-dimensional data, and to obtain the fault type, fault cause and solution of the industrial system component corresponding to the training image; The first construction module 304 is used to construct a fault diagnosis dataset, wherein the fault diagnosis dataset includes the image to be trained and the corresponding fault type of the industrial system component, the fault cause and the solution; The second construction module 305 is used to construct a large fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal basic model. The input module 306 is used to input the real-time collected multi-source fault data into the fault diagnosis big model to obtain the target fault type, target fault cause and target solution corresponding to the multi-source fault data.

[0091] The fault tracing and decision-making device 30 based on the basic model provided in the above embodiments can realize the technical solutions described in the above embodiments of the fault tracing and decision-making method based on the basic model. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the fault tracing and decision-making method based on the basic model, and will not be repeated here.

[0092] Please refer to point 4. Figure 4 This is a schematic diagram of an embodiment of the electronic device of this application.

[0093] In some embodiments, processor 401 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 402 or process data, such as the fault tracing and decision-making method based on the basic model in this invention.

[0094] In some embodiments, processor 401 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 401 may be local or remote. In some embodiments, processor 401 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0095] In some embodiments, memory 402 may be an internal storage unit of electronic device 400, such as a hard disk or memory of electronic device 400. In other embodiments, memory 402 may also be an external storage device of electronic device 400, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 400.

[0096] Furthermore, the memory 402 may include both internal storage units of the electronic device 400 and external storage devices. The memory 402 is used to store application software and various types of data installed on the electronic device 400.

[0097] In some embodiments, display 403 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 403 is used to display information from electronic device 400 and to display visual user applications. Components 401-403 of electronic device 400 communicate with each other via a system bus.

[0098] In one embodiment, when processor 401 executes the fault tracing and decision-making program based on the basic model in memory 402, the following steps can be implemented: Historical multi-source fault data of different components in industrial systems; The historical multi-source fault data is preprocessed and normalized to obtain historical multi-source one-dimensional data. The preprocessing includes at least data cleaning, signal enhancement and data augmentation. The training image is generated based on the historical multi-source one-dimensional data, and the fault type, fault cause and solution of the industrial system component corresponding to the training image are obtained. Construct a fault diagnosis dataset, wherein the fault diagnosis dataset includes the image to be trained and the corresponding fault type of the industrial system component, the cause of the fault, and the solution; A large-scale fault diagnosis model is constructed based on the fault diagnosis dataset and the pre-trained multimodal basic model. The real-time collected multi-source fault data is input into the fault diagnosis model to obtain the target fault type, target fault cause, and target solution corresponding to the multi-source fault data.

[0099] It should be understood that when the processor 401 executes the fault tracing and decision-making program based on the basic model in the memory 402, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0100] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 400 mentioned. Electronic device 400 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 400 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0101] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the fault tracing and decision-making methods based on the basic model provided in the above-described method embodiments.

[0102] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0103] The above provides a detailed description of the fault tracing and decision-making method, apparatus, equipment, and medium based on the basic model provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A fault tracing and decision-making method based on a basic model, characterized in that, include: Historical multi-source fault data of different components in industrial systems; The historical multi-source fault data is preprocessed and normalized to obtain historical multi-source one-dimensional data. The preprocessing includes at least data cleaning, signal enhancement and data augmentation. The training image is generated based on the historical multi-source one-dimensional data, and the fault type, fault cause and solution of the industrial system component corresponding to the training image are obtained. Construct a fault diagnosis dataset, wherein the fault diagnosis dataset includes the image to be trained and the corresponding fault type of the industrial system component, the cause of the fault, and the solution; A large-scale fault diagnosis model is constructed based on the fault diagnosis dataset and the pre-trained multimodal basic model. The real-time collected multi-source fault data is input into the fault diagnosis model to obtain the target fault type, target fault cause, and target solution corresponding to the multi-source fault data.

2. The fault tracing and decision-making method based on a basic model according to claim 1, characterized in that, The preprocessing of the historical multi-source fault data includes: The historical multi-source fault data is cleaned, wherein the data cleaning includes at least: resampling and aligning the sampling time of the historical multi-source fault data; interpolating and repairing short missing values; removing outliers and external noise from long missing values; The signal enhancement of the historical multi-source fault data includes at least the following: baseline correction of signals with baseline drift in the historical multi-source fault data using a high-pass filter or wavelet detrending; band-pass filtering of vibration, current, and other signals in the historical multi-source fault data according to the fault frequency band using a band-pass filter; and conversion of the one-dimensional time-domain signal of the multi-source fault data into a time-frequency graph as an image encoding input. The historical multi-source fault data is augmented, and the data augmentation includes at least: adding weak Gaussian noise to the historical multi-source fault data; performing slight time-axis stretching, compression, or jittering on the historical multi-source fault data; randomly cutting out small segments of signals from the historical multi-source fault data and replacing them with adjacent data segments; and randomly masking some channel data in the historical multi-source fault data.

3. The fault tracing and decision-making method based on a basic model according to claim 1, characterized in that, The fault diagnosis dataset includes a training dataset and a test dataset; The construction of a large-scale fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal base model includes: Load the multimodal base model, and train the multimodal base model according to the training dataset to obtain the original fault diagnosis large model; The original fault diagnosis model is evaluated based on the test dataset, and optimized by combining efficient parameter fine-tuning technology and evaluation results to obtain the original fault diagnosis model.

4. The fault tracing and decision-making method based on a basic model according to claim 3, characterized in that, The optimization of the original fault diagnosis model by combining efficient parameter fine-tuning technology and evaluation results includes: If the evaluation result indicates that the model parameters of the original fault diagnosis model do not meet the preset parameter requirements, the model parameters are fine-tuned according to the following formula: ;in, ,and .

5. The fault tracing and decision-making method based on a basic model according to claim 1, characterized in that, The step of generating multiple training images based on the historical multi-source one-dimensional data includes: The historical multi-source one-dimensional data is reshaped into a two-dimensional matrix and then stitched together row by row in channel order using a rasterized layout method to form a two-dimensional data structure. The two-dimensional data structure is normalized to the 0–255 range according to the maximum and minimum values ​​and copied into three channels to generate the image to be trained.

6. The fault tracing and decision-making method based on a basic model according to claim 5, characterized in that, The training image is generated according to the following formula: ； in, The image to be trained, The pixel values ​​of the two-dimensional data structure. The maximum value in the two-dimensional data structure. It is the minimum value in the two-dimensional data structure.

7. The fault tracing and decision-making method based on a basic model according to any one of claims 1 to 6, characterized in that, The method further includes: Using the fault diagnosis model as the teacher model, a student model corresponding to the teacher model is constructed. The student model is trained by distillation based on the teacher model to obtain the initial student model; Remove redundant attention heads, channels, or weight parameters from the initial student model, and quantize the initial student model to obtain the lightweight final student model. The multi-source fault data is input into the final student model to obtain the target fault type, the target fault cause, and the target solution.

8. A fault tracing and decision-making device based on a basic model, characterized in that, include: The module includes a data acquisition module, a processing module, a generation module, a first construction module, a second construction module, and an input module. The acquisition module is used to collect historical multi-source fault data of different components from the industrial system; The processing module is used to preprocess the historical multi-source fault data and normalize the preprocessed historical multi-source fault data to obtain historical multi-source one-dimensional data. The preprocessing includes at least data cleaning, signal enhancement and data augmentation. The generation module is used to generate a training image based on the historical multi-source one-dimensional data, and to obtain the industrial system component fault type, fault cause and solution corresponding to the training image; The first construction module is used to construct a fault diagnosis dataset, wherein the fault diagnosis dataset includes the image to be trained and the corresponding fault type of the industrial system component, the cause of the fault, and the solution; The second building module is used to build a large fault diagnosis model based on the fault diagnosis dataset and the pre-trained multimodal base model; The input module is used to input real-time collected multi-source fault data into the fault diagnosis model to obtain the target fault type, target fault cause, and target solution corresponding to the multi-source fault data.

9. An electronic device, the electronic device comprising a processor and a memory, characterized in that, The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the fault tracing and decision-making method based on the basic model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the fault tracing and decision-making method based on the basic model as described in any one of claims 1 to 7.

Images