A bearing fault diagnosis method and system based on knowledge enhancement and multi-path distillation

Abstract

The present application discloses a bearing fault diagnosis method and system based on knowledge enhancement and multi-path distillation, which relates to the fields of intelligent operation and maintenance and industrial equipment health management. The method includes obtaining sensor signals and visual image data of the bearing operation; based on an asynchronous dual-channel architecture, correspondingly extracting signal features of the sensor signals and image features of the visual image data, and performing time synchronization on the signal features and the image features; using a multi-modal bottleneck Transformer module to fuse the synchronized signal features and the synchronized image features; based on a maintenance knowledge graph dynamically constructed from a bearing maintenance manual, combining a text generation model to map the fused features to a semantic space and generate a fault diagnosis report. The present application can improve the recognition accuracy, real-time performance and interpretability of diagnosis results of bearing faults.

Description

Technical Field

[0001] This application relates to the field of intelligent operation and maintenance and industrial equipment health management, and in particular to a bearing fault diagnosis method and system based on knowledge enhancement and multi-path distillation. Background Technology

[0002] Rolling bearings, often referred to as the "joints of industry," are core components in mass-produced customized equipment, and their health directly impacts equipment operational accuracy and production continuity. Mass-produced customized manufacturing, combining assembly line efficiency with market responsiveness, has become a strategic focus for advanced manufacturing. However, the diverse customization demands, heterogeneous data architectures, and knowledge silos inherent in this model mean that the chain reaction triggered by bearing failures can be more destructive, potentially leading to supply chain disruptions and reduced production flexibility.

[0003] Existing bearing health assessment methods are mainly divided into three categories: signal processing methods, model-driven methods, and data-driven intelligent methods. Signal processing methods rely on techniques such as Fast Fourier Transform (FFT) and Empirical Mode Decomposition (EMD) to extract fault features, but their reliability decreases under noise interference and nonlinear operating conditions, and their computational complexity is difficult to meet the real-time requirements of customized manufacturing. Model-driven methods build dynamic models based on physical principles, but their accuracy depends on a complete understanding of the fault mechanism, and their modeling capabilities are insufficient in complex customized scenarios. Data-driven methods, especially deep learning methods, have become mainstream, such as multi-scale CNN and LSTM fusion models and signal-graph neural networks.

[0004] However, the existing methods still have three major defects: (1) the problem of asynchronous time for heterogeneous data: the difference in acquisition frequency between sensor signals and visual images leads to asynchronous data time. Existing interpolation alignment methods will lose key transient degradation features and reduce diagnostic accuracy; (2) the contradiction between feature redundancy and computational efficiency: cross-modal feature coupling generates a large amount of redundant information. The traditional Transformer fusion framework lacks a targeted redundancy suppression mechanism, and the computational overhead is too large, which cannot meet the real-time requirements of agile manufacturing; (3) insufficient integration of maintenance knowledge: existing models are mostly black box structures, which only realize fault classification without integrating domain knowledge. The generated diagnostic results lack semantic interpretability and are difficult to connect with the actual maintenance process. Summary of the Invention

[0005] The purpose of this application is to provide a bearing fault diagnosis method and system based on knowledge enhancement and multi-path distillation, which can improve the accuracy, real-time performance, and interpretability of bearing fault identification results.

[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a bearing fault diagnosis method based on knowledge enhancement and multi-path distillation, the bearing fault diagnosis method based on knowledge enhancement and multi-path distillation comprising: Acquire sensor signals and visual image data of bearing operation; Based on an asynchronous dual-channel architecture, signal features of sensor signals and image features of visual image data are extracted, and time synchronization of signal features and image features is performed. The asynchronous dual-channel architecture includes a signal channel, an image channel, and a time calibration unit. The signal channel is used to extract signal features using finite impulse response filtering based on Hanning windows combined with adaptive feature decomposition. The image channel is used to extract image features using a width ensemble learning layer combining width learning and deep networks. The time calibration unit is used to synchronize the time of signal features and image features using a timestamp alignment algorithm. The multimodal bottleneck Transformer module is used to fuse synchronized signal features and synchronized image features. The multimodal bottleneck Transformer module is used to limit cross-modal attention flow by inserting a fusion bottleneck token, compress feature dimensions, perform cross-modal feature fusion and information distillation, and obtain fused features. A maintenance knowledge graph dynamically constructed based on the bearing maintenance manual is combined with a text generation model to map fused features to the semantic space and generate a fault diagnosis report.

[0007] Optionally, the acquisition of sensor signals and visual image data of bearing operation further includes: Perform initial alignment of sensor signals according to timestamps; The visual image data is processed by grayscale conversion, noise reduction, and size normalization.

[0008] Optionally, the signal feature extraction process specifically includes: Construct a health indicator matrix based on sensor signals; The health index matrix is ​​filtered using a Hanning window finite impulse response filter, and the optimal intrinsic mode function is iteratively solved with the objective function of maximizing the correlation kurtosis. Multiple optimal intrinsic mode function components and residual terms are extracted iteratively to obtain the decomposed signal features.

[0009] Optionally, the image feature extraction process specifically includes: Visual image data is projected onto n parallel feature spaces to obtain n sets of high-dimensional features; The n sets of high-dimensional features are reduced in dimensionality using random singular value decomposition; Based on the dimensionality-reduced features, an enhanced feature is generated using a mapping function, and the enhanced features are concatenated and then simplified again by random singular value decomposition to obtain the image features.

[0010] Optionally, the multimodal bottleneck Transformer module includes: a cross-modal interaction layer and an attention bottleneck layer; The cross-modal interaction layer is used to establish a correlation mapping between signal features and image features through a collaborative attention mechanism; The attention bottleneck layer is used to insert S fusion bottleneck tokens into the synchronized signal features and synchronized image features.

[0011] Optionally, the process of maintaining the knowledge graph construction is as follows: The bearing maintenance manual is preprocessed to extract entities; the entities include: component type entity, failure mode entity, and repair strategy entity. By establishing relationships between entities through lexical processing, semantic analysis, and knowledge integration, an ontology is formed that includes component hierarchical relationships and fault-repair mappings. Integrate multi-source industrial data to update the ontology and dynamically build and maintain the knowledge graph.

[0012] Optionally, the step of establishing inter-entity relationships through lexical processing, semantic analysis, and knowledge integration to form an ontology containing component hierarchical relationships and fault-repair mappings specifically includes: Lexical processing is performed using PDF parsing and text extraction algorithms, word segmentation algorithms based on industrial lexicons, and stop word filtering algorithms. Rule-based entity recognition algorithms, dependency parsing algorithms, and ontology modeling algorithms are used, with the entity recognition algorithm performing semantic analysis. Based on the RDF standard, serialization conversion transforms ontology metadata into RDF triples, which are then stored using the Neo4j graph database.

[0013] Optionally, the multi-source industrial data includes: structured text knowledge in bearing maintenance manuals, actual inspection records and maintenance logs from industrial sites, real-time sensor signals collected by vibration sensors, temperature sensors and current sensors during bearing operation, visual image data of bearings captured by industrial cameras, technical specifications and parameter documents provided by equipment manufacturers, historical fault case databases and corresponding maintenance strategy records, and domain expert experience knowledge bases.

[0014] Optionally, the text generation model is a bidirectional autoregressive Transformer model.

[0015] Secondly, this application provides a bearing fault diagnosis system based on knowledge enhancement and multi-path distillation, the bearing fault diagnosis system based on knowledge enhancement and multi-path distillation comprising: Data acquisition equipment is used to acquire sensor signals and visual image data of bearing operation; An asynchronous feature processing device is used to extract signal features from sensor signals and image features from visual image data based on an asynchronous dual-channel architecture, and to synchronize the signal features and image features in time. The asynchronous dual-channel architecture includes a signal channel, an image channel, and a time calibration unit. The signal channel is used to extract signal features using a finite impulse response filter based on a Hanning window combined with adaptive feature decomposition. The image channel is used to extract image features using a width ensemble learning layer combining width learning and deep networks. The time calibration unit is used to synchronize the signal features and image features in time using a timestamp alignment algorithm. A feature fusion device is used to fuse synchronized signal features and synchronized image features using a multimodal bottleneck Transformer module; the multimodal bottleneck Transformer module is used to limit cross-modal attention flow by inserting a fusion bottleneck token, compress feature dimensions, perform cross-modal feature fusion and information distillation, and obtain fused features; The output device is used to dynamically construct a maintenance knowledge graph based on the bearing maintenance manual. Combined with a text generation model, the fused features are mapped to the semantic space to generate a fault diagnosis report.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a bearing fault diagnosis method and system based on knowledge enhancement and multi-path distillation. Compared with existing single-modal fault diagnosis schemes (such as methods relying solely on signal processing or image processing), it effectively solves the problem of temporal mismatch in multimodal data by using an asynchronous dual-channel architecture (signal processing via Feature Mode Decomposition (FMD) and image processing via a Broad-based Ensemble Learning Layer (BELL)) and a dynamic time calibration mechanism. This allows for accurate capture of minute bearing fault features, significantly improving the ability to identify minute faults. Furthermore, the diagnostic reliability after feature synchronization is far superior to single-modal processing schemes, with stronger accuracy and stability of diagnostic results. Compared with the shortcomings of traditional Transformer and other multimodal fusion schemes, such as "feature redundancy leading to high computational overhead and difficulty in balancing accuracy and efficiency," the Multimodal Bottleneck Transformer (MBT) module in this application achieves this by fusing bottleneck tokens (Fusion Bottleneck). Tokens (FBT) restrict cross-modal attention flow, suppressing redundant and invalid features while retaining key discriminative fault features. This improves fault diagnosis accuracy in complex industrial scenarios while reducing model computational complexity, meeting the real-time diagnostic requirements of industrial sites and achieving a balance between high precision and high efficiency. Compared to existing "black box" fault diagnosis models that only output fault types and lack semantic explanations, this application uses a maintenance knowledge graph dynamically constructed from bearing maintenance manuals, combined with a text generation model, to generate interpretable fault diagnosis reports that conform to industrial standards. The reports comprehensively cover key information required for on-site maintenance, including fault type, fault cause, and repair suggestions. They are highly adaptable to actual maintenance processes, providing engineers with clear operational guidance and avoiding the problem of "knowing the fault but not knowing how to handle it," significantly improving the practical value of the diagnostic results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of a bearing fault diagnosis method based on knowledge enhancement and multi-path distillation in one embodiment of this application. Figure 2 A schematic diagram of the Eigenmode Decomposition (FMD) process provided in this application; Figure 3 A flowchart illustrating the width-integrated learning layer (BELL) provided in this application; Figure 4 This is the confusion matrix of this application on the CWRU dataset. Figure 5 This is a T-SNE visualization of this application on the CWRU dataset. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] In one exemplary embodiment, such as Figure 1 As shown, a bearing fault diagnosis method based on knowledge enhancement and multi-path distillation is provided, comprising the following steps S101 to S104: Wherein: S101, acquire sensor signals and visual image data of bearing operation; As a specific embodiment, the sensor signals are acquired through vibration sensors, temperature sensors, and current sensors, and the visual image data is acquired through an industrial camera. To ensure the accuracy of the acquired data, the sampling parameters can be dynamically adjusted, wherein the sensor signal sampling frequency range is 1kHz-50kHz, and the industrial camera frame rate range is 10fps-60fps. The asynchronous feature processing module achieves synchronization between the signal and image data through a timestamp alignment algorithm, with a synchronization error of less than 10ms.

[0022] Following S101 are: Perform initial alignment of sensor signals according to timestamps; To eliminate image noise (such as Gaussian noise) and improve texture detail recognition, image data was acquired by an industrial high-speed camera, covering surface texture and deformation images of the bearing's outer ring, inner ring, and rolling elements. The images underwent preprocessing including image enhancement, noise reduction, and size normalization.

[0023] S102, based on an asynchronous dual-channel architecture, extracts signal features from sensor signals and image features from visual image data, and synchronizes the signal features and image features in time. The asynchronous dual-channel architecture includes a signal channel, an image channel, and a time calibration unit. The signal channel uses a Hanning window-based finite impulse response (FIR) filter combined with adaptive feature decomposition to extract signal features. The image channel uses a width ensemble learning layer combining width learning and deep networks to extract image features. The time calibration unit uses a timestamp alignment algorithm to synchronize the signal features and image features in time. As a specific implementation, the time calibration unit performs cross-modal synchronization to eliminate signal-image time misalignment, laying the foundation for subsequent fusion. Cross-modal time synchronization is implemented by mapping signal features and image features to a unified time axis using a timestamp alignment algorithm. Specifically, signal features are resampled using a sliding window (window size 300, step size 100) to match the signal feature frame rate with the image frame rate; the time deviation between the two types of features is calculated and corrected through linear interpolation to ensure that each frame corresponds to one set of signal features.

[0024] Specifically, the signal feature extraction process includes: S11, constructing a health indicator matrix based on sensor signals, can merge multi-sensor signals and reduce data dimensionality. Multi-sensor data are initially aligned by timestamps, and a composite health indicator is constructed using an optimized weighted fusion method. ; in, For the first Each sensor (vibration sensor / temperature sensor / current sensor) in Performance monitoring data collected in real time (known quantities, collected by sensors in real time). The fusion coefficient is an unknown quantity, with a value range of [0,1] and ), the monitoring time is , The observation data is ; S12 uses Hanning window finite impulse response (FIR) filtering to filter the health index matrix, and iteratively solves for the optimal intrinsic mode function (IMF) with the objective function of maximizing the correlation kurtosis. Specifically, loading K FIR filter banks are initialized using multiple Hanning windows, and the decomposition mode number n and filter length are set. Using the maximization of correlation kurtosis as the objective function, iteratively update the filter coefficients to solve the constraint problem: The constraints are: ; in, Indicates the input period; M represents the shift order, and N is the signal length. Indicates the first One modal component; Indicate its relevant kurtosis; Indicates the first A length of For an FIR filter with a length of (256, known quantity), the specific process of Feature Mode Decomposition (FMD) is as follows: Figure 2 As shown; S13, iteratively extract multiple optimal intrinsic mode function components and residual terms to obtain the decomposed signal features.

[0025] Iteratively extract R (e.g., 8) IMF components and residual terms to obtain the features of the reconstructed signal after decomposition. ; in, Capturing fault characteristics across different frequency bands, ideally, It should be equal to the original signal. However, due to numerical calculation errors, there may be slight differences between the two.

[0026] Specifically, such as Figure 3 As shown, the image feature extraction process specifically includes: S21, Project the visual image data onto n parallel feature spaces to obtain n sets of high-dimensional features. ; Specifically, visual image data Projected onto n feature spaces, each space outputs: ,in For projection function, , These are the weight matrix and the bias term, respectively. S22, using Randomized Singular Value Decomposition (RSVD) to analyze the n sets of high-dimensional features. Perform dimensionality reduction; ;in, , For left and right singular orthogonal matrices, It is a diagonal matrix arranged in descending order of singular values. Composed of large singular values, Approximately 0 To simplify the mapping features.

[0027] S23, Based on the reduced-dimensional features, a mapping function is used. The generation of enhanced features is specifically implemented as follows: , The weight matrix is ​​a randomized orthogonalized matrix; It is the bias vector; For the generated j-th enhanced feature, , The activation function is used; the enhanced features are concatenated and then simplified again by random singular value decomposition to obtain the simplified matrix. The newly added set of features is designated as the (n+1)th set of features. The resulting enhancement features are: ,in, , , , Randomly generated; the general characteristics of the network are and the enhancement features it generates are All new features are defined as follows: Finally, image features are obtained. .

[0028] S103, the multimodal bottleneck Transformer module is used to fuse the synchronized signal features and the synchronized image features; the multimodal bottleneck Transformer module is used to limit the cross-modal attention flow by inserting a fusion bottleneck token, compress the feature dimension, perform cross-modal feature fusion and information distillation, and obtain fused features; The multimodal bottleneck Transformer module can solve the information redundancy problem caused by the cross-coupling of multimodal features and realize the transformation of "heterogeneous features → homogeneous fusion". The multimodal bottleneck Transformer module includes: a cross-modal interaction layer and an attention bottleneck layer. The cross-modal interaction layer is used to establish a correlation mapping between signal features and image features through a collaborative attention mechanism, mapping signal (512-dimensional) and image (1024-dimensional) features to the same 512-dimensional space; The attention bottleneck layer is used to insert S fusion bottleneck tokens into the synchronized signal features and synchronized image features, and to filter redundant attention weights by dimensional constraints, thereby reducing computational complexity.

[0029] The specific implementation process of the multimodal bottleneck Transformer module includes: Characteristics of the synchronized signal Image features after synchronization The signal features are uniformly mapped to the feature space through a fully connected layer to obtain the mapped signal features. and mapped image features : ; FBT is inserted into the bottleneck unit of the multimodal bottleneck Transformer module to restrict cross-modal attention flow and compress redundant information. S (S=3~5, adaptively adjusted according to data dimension) fusion bottleneck tokens are inserted into the input sequence of the unified signal and image features. , construct standard input The FBT is used to limit cross-modal attention flow, allowing only signal and image features to exchange information through the FBT, thus reducing redundant computation.

[0030] Intermodal feature distillation is achieved through interlayer averaging: ; Where i is the modality index, representing different data modalities; the value range is i=1,2, specifically referring to: i=1: signal feature modality (SIG, from the FMD signal processing channel); i=2: image feature modality (IMG, from the BELL image processing channel); w is the network layer index, representing the depth level of the Transformer network; the value range is w=1,2,...,L (L is the total number of Transformer layers), where w is the feature representation of the current layer; w+1 is the feature representation of the next layer; Let i be the feature representation of the i-th mode in the w-th layer. This represents the characteristic modes of the signal at the w-th layer; This represents the image feature mode at layer w; it is used to preserve mode-specific features and provide a basis for subsequent feature distillation. The feature representation of the Fusion Bottleneck Token (FBT) at layer w; The parameter set for the i-th modality includes parameters from the multi-head attention mechanism, modality-specific projection matrices, and bias terms. It controls the distribution of attention weights for different modalities to ensure the independence and specificity of different modalities during the fusion process. The FBT feature obtained after processing the i-th mode using MBT; The final FBT feature is obtained after interlayer averaging; S104, based on the maintenance knowledge graph dynamically constructed from the bearing maintenance manual, combines a text generation model to map fused features to the semantic space, generating a fault diagnosis report. In other words, it maps fused features to an interpretable diagnosis report, realizing the transformation from "data features to knowledge semantics".

[0031] The fault diagnosis report includes the fault type, severity, and fault mechanism; the fault types include single-point outer ring fault, outer ring crack, outer ring wear, inner ring spalling, inner ring corrosion, inner ring compound fault, rolling element wear, rolling element crack, and rolling element spalling.

[0032] Specifically, the process of constructing the maintenance knowledge graph is as follows: S31, perform text preprocessing on the bearing maintenance manual to extract entities; the entities include: component type entities, fault mode entities, and repair strategy entities; text preprocessing is to remove pictures, tables, and irrelevant characters in the manual, retain texts such as fault descriptions, cause analyses, and repair methods, perform word segmentation using the Jieba word segmentation tool (industrial domain word library), and remove stop words such as "of" and "in". S32, establish the association relationships between entities through lexical processing, semantic analysis, and knowledge integration to form an ontology containing component hierarchical relationships and fault-repair mappings. Entity types: bearing components (inner ring, outer ring, rolling elements, cage), fault types (spalling, wear, crack, corrosion), repair methods (component replacement, lubrication optimization, laser shock peening); relationship types: "component-fault" (such as inner ring-spalling), "fault-cause" (such as spalling-overload), "fault-repair" (such as wear-grease replacement). Among them, the ontology is constructed using the Seven-Step Method to define ontology classes (5 core classes: component class, fault class, cause class, repair class, attribute class), object properties, and data properties (such as fault diameter, repair cycle).

[0033] S321, perform lexical processing using PDF parsing and text extraction algorithms, word segmentation algorithms based on industrial domain word libraries, and stop word filtering algorithms. S322, based on rule-based entity recognition algorithms, dependency syntactic analysis algorithms, and ontology modeling algorithms, the entity recognition algorithm performs semantic analysis. S323, based on the serialization conversion of the RDF standard, convert the ontology metadata into RDF triples, store them using the Neo4j graph database, and construct nodes and edges to support Cypher queries. S33, integrate multi-source industrial data to update the ontology and dynamically construct the maintenance knowledge graph. The multi-source industrial data includes, but is not limited to: structured text knowledge in the bearing maintenance manual, actual maintenance records and maintenance logs in the industrial field, real-time sensor signals collected by vibration sensors, temperature sensors, and current sensors during the operation of the bearing, visual image data of the bearing captured by industrial cameras, technical specifications and parameter documents provided by equipment manufacturers, historical fault case libraries and corresponding maintenance strategy records, and domain expert experience knowledge bases.

[0034] As a specific embodiment, the text generation model is a bidirectional and auto-regressive Transformer (BART) model.

[0035] One implementation of the BART text generation model involves calculating the cosine similarity between the fused features and entity vectors in the knowledge graph. The formula is: ; in, and The components of vectors E and F are represented respectively. Entities that meet the preset similarity threshold (such as "inner circle", "peeling", "overload") are retained to form fault triples. Inference results are generated through multi-hop queries (such as "inner circle-peeling → peeling-overload → overload-check load"). I represents the dimension of these two vectors (i.e. the number of elements contained in the vector).

[0036] The diagnostic text generated by the BART inference unit transforms the model's output feature vectors into natural language that engineers can understand. For interpretable text generation, the BART model is initialized using a pre-trained model "facebook / bart-base". During fine-tuning, the batch size is set to 32, the learning rate to 5e-5, and the epoch to 10. The training data consists of paired "fault triples - expert reports". The corresponding text sequences are randomly masked (masking rate 15%) and rearranged (rearrangement rate 10%) to simulate real-world industrial text noise. The noisy sequence is input to the BART decoder to generate a diagnostic report that conforms to industry standards.

[0037] Table 1 shows the comparison results of the bearing fault diagnosis method of this application with other methods on the CWRU dataset. DC-CMLN is a domain-constrained cascaded multi-receptor learning network, MA 1DCNN is a multi-attention one-dimensional convolutional neural network, MRA-CNN is a multi-scale residual attention convolutional neural network, DCA-BiGRU is a dual-path convolutional network based on attention mechanism and bidirectional gated recurrent units, DRSNs is a deep residual reduction network, and MBS CNN is a multi-branch and multi-scale convolutional neural network. As shown in Table 1, the accuracy and F1 score of this application are superior to advanced multimodal or single-modal methods such as DC-CMLN (98.99% / 98.99%), MA 1DCNN (97.55% / 97.55%), and MRA-CNN (97.71% / 97.70%). Figure 4 The confusion matrix shows that the values ​​on its diagonal are all close to 1, indicating that the classification is almost completely correct; T-SNE visualization ( Figure 5The data shows that the features of different fault states are clearly clustered and the inter-class separation is high, which further proves the powerful feature discrimination ability of this application.

[0038] Table 1

[0039] As a specific implementation, signal decomposition can be replaced with "Variational Mode Decomposition (VMD)", image feature extraction can be replaced with "Lightweight Convolutional Neural Networks (CNN)"; MBT can be replaced with "Graph Attention Networks (GAT) with attention bottleneck"; and the BART text generation model can be replaced with "T5-small model".

[0040] In one exemplary embodiment, a bearing fault diagnosis system with knowledge enhancement and multi-path distillation is provided, comprising: The data acquisition equipment is used to acquire sensor signals and visual image data of bearing operation; the data acquisition equipment integrates vibration sensors, temperature sensors, current sensors and industrial cameras to achieve synchronous acquisition of multi-source data; An asynchronous feature processing device is used to extract signal features from sensor signals and image features from visual image data based on an asynchronous dual-channel architecture, and to synchronize the signal features and image features in time. The asynchronous dual-channel architecture includes a signal channel, an image channel, and a time calibration unit. The signal channel uses a finite impulse response filter based on a Hanning window combined with adaptive feature decomposition to extract signal features. The image channel uses a width ensemble learning layer combining width learning and deep networks to extract image features. The time calibration unit uses a timestamp alignment algorithm to synchronize the signal features and image features in time. The asynchronous feature processing device deploys the algorithmic logic for the signal processing channel and the image processing channel, and supports dynamic adjustment of sampling parameters. The feature fusion device is used to fuse synchronized signal features and synchronized image features using a multimodal bottleneck Transformer module. The multimodal bottleneck Transformer module is used to limit cross-modal attention flow by inserting a fusion bottleneck token, compress feature dimensions, perform cross-modal feature fusion and information distillation, and obtain fused features. The feature fusion device hardware accelerates the inference process of the multimodal bottleneck Transformer (MBT), with an inference latency of <50ms. The output device is used to dynamically construct a maintenance knowledge graph based on the bearing maintenance manual. Combined with a text generation model, it maps fused features to a semantic space to generate a fault diagnosis report. The output device includes a knowledge graph database and a text reasoning engine, supporting real-time updates of the knowledge graph. The output module displays the diagnostic results via an industrial touchscreen and generates a PDF maintenance report. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0041] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0042] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0043] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0044] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0045] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A bearing fault diagnosis method based on knowledge enhancement and multi-path distillation, characterized in that, The knowledge-enhanced and multi-path distillation-based bearing fault diagnosis method includes: Acquire sensor signals and visual image data of bearing operation; Based on an asynchronous dual-channel architecture, signal features of sensor signals and image features of visual image data are extracted, and time synchronization of signal features and image features is performed. The asynchronous dual-channel architecture includes a signal channel, an image channel, and a time calibration unit. The signal channel is used to extract signal features using finite impulse response filtering based on Hanning windows combined with adaptive feature decomposition. The image channel is used to extract image features using a width ensemble learning layer combining width learning and deep networks. The time calibration unit is used to synchronize the time of signal features and image features using a timestamp alignment algorithm. The multimodal bottleneck Transformer module is used to fuse synchronized signal features and synchronized image features. The multimodal bottleneck Transformer module is used to limit cross-modal attention flow by inserting a fusion bottleneck token, compress feature dimensions, perform cross-modal feature fusion and information distillation, and obtain fused features. A maintenance knowledge graph dynamically constructed based on the bearing maintenance manual is combined with a text generation model to map fused features to the semantic space and generate a fault diagnosis report.

2. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 1, characterized in that, The acquisition of sensor signals and visual image data of bearing operation further includes: Perform initial alignment of sensor signals according to timestamps; The visual image data is processed by grayscale conversion, noise reduction, and size normalization.

3. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 1, characterized in that, The signal feature extraction process specifically includes: Construct a health indicator matrix based on sensor signals; The health index matrix is ​​filtered using a Hanning window finite impulse response filter, and the optimal intrinsic mode function is iteratively solved with the objective function of maximizing the correlation kurtosis. Multiple optimal intrinsic mode function components and residual terms are extracted iteratively to obtain the decomposed signal features.

4. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 1, characterized in that, The image feature extraction process specifically includes: Visual image data is projected onto n parallel feature spaces to obtain n sets of high-dimensional features; The n sets of high-dimensional features are reduced in dimensionality using random singular value decomposition; Based on the dimensionality-reduced features, an enhanced feature is generated using a mapping function, and the enhanced features are concatenated and then simplified again by random singular value decomposition to obtain the image features.

5. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 1, characterized in that, The multimodal bottleneck Transformer module includes: a cross-modal interaction layer and an attention bottleneck layer; The cross-modal interaction layer is used to establish a correlation mapping between signal features and image features through a collaborative attention mechanism; The attention bottleneck layer is used to insert S fusion bottleneck tokens into the synchronized signal features and synchronized image features.

6. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 1, characterized in that, The process of maintaining the knowledge graph construction is as follows: The bearing maintenance manual is preprocessed to extract entities; the entities include: component type entity, failure mode entity, and repair strategy entity. By establishing relationships between entities through lexical processing, semantic analysis, and knowledge integration, an ontology is formed that includes component hierarchical relationships and fault-repair mappings. Integrate multi-source industrial data to update the ontology and dynamically build and maintain the knowledge graph.

7. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 6, characterized in that, The process of establishing inter-entity relationships through lexical processing, semantic analysis, and knowledge integration to form an ontology that includes component hierarchical relationships and fault-repair mappings specifically includes: Lexical processing is performed using PDF parsing and text extraction algorithms, word segmentation algorithms based on industrial lexicons, and stop word filtering algorithms. Rule-based entity recognition algorithms, dependency parsing algorithms, and ontology modeling algorithms are used, with the entity recognition algorithm performing semantic analysis. Based on the RDF standard, serialization conversion transforms ontology metadata into RDF triples, which are then stored using the Neo4j graph database.

8. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 6, characterized in that, The multi-source industrial data includes: structured text knowledge in bearing maintenance manuals, actual inspection records and maintenance logs from industrial sites, real-time sensor signals collected by vibration sensors, temperature sensors and current sensors during bearing operation, visual image data of bearings captured by industrial cameras, technical specifications and parameter documents provided by equipment manufacturers, historical fault case databases and corresponding maintenance strategy records, and domain expert experience knowledge bases.

9. The bearing fault diagnosis method based on knowledge enhancement and multi-path distillation according to claim 1, characterized in that, The text generation model is a bidirectional autoregressive Transformer model.

10. A bearing fault diagnosis system based on knowledge enhancement and multi-path distillation, characterized in that, The knowledge-enhanced and multi-path distillation bearing fault diagnosis system includes: Data acquisition equipment is used to acquire sensor signals and visual image data of bearing operation; An asynchronous feature processing device is used to extract signal features from sensor signals and image features from visual image data based on an asynchronous dual-channel architecture, and to synchronize the signal features and image features in time. The asynchronous dual-channel architecture includes a signal channel, an image channel, and a time calibration unit. The signal channel is used to extract signal features using a finite impulse response filter based on a Hanning window combined with adaptive feature decomposition. The image channel is used to extract image features using a width ensemble learning layer combining width learning and deep networks. The time calibration unit is used to synchronize the signal features and image features in time using a timestamp alignment algorithm. A feature fusion device is used to fuse synchronized signal features and synchronized image features using a multimodal bottleneck Transformer module; the multimodal bottleneck Transformer module is used to limit cross-modal attention flow by inserting a fusion bottleneck token, compress feature dimensions, perform cross-modal feature fusion and information distillation, and obtain fused features; The output device is used to dynamically construct a maintenance knowledge graph based on the bearing maintenance manual. Combined with a text generation model, the fused features are mapped to the semantic space to generate a fault diagnosis report.

Images