A lubricating equipment health management method, system, terminal and medium

Abstract

The application relates to the field of lubricating equipment management, and specifically provides a lubricating equipment health management method, system, terminal and medium, which comprises the following steps: collecting multi-modal data of lubricating equipment, including time series structured data such as vibration, unstructured data such as text and oil detection data; standardizing the data and obtaining multi-modal features; aligning the data through timestamp calibration and space mapping, constructing a multi-modal heterogeneous graph containing multi-modal nodes and multi-type edges; generating a unified equipment state representation vector through cross-modal graph attention calculation; inputting a pre-trained neural network model, and outputting fault types and probabilities, remaining service life and health scores; based on the results, receiving queries through a natural language interface, and generating a visual report and / or an operation and maintenance work order. The application realizes accurate identification and early warning of lubricating equipment faults, and improves the accuracy of equipment health management and operation and maintenance efficiency.

Description

Technical Field

[0001] This application relates to the field of lubrication equipment management, specifically to a lubrication equipment health management method, system, terminal, and medium. Background Technology

[0002] In modern industrial production, lubrication equipment (such as lubrication oil circuit systems, hydraulic systems, gearboxes, and bearing lubrication units) is a key subsystem ensuring the normal operation of mechanical equipment. Its health status directly affects the reliability, safety, and operating efficiency of the entire machine. As industrial equipment develops towards larger scale, greater complexity, and continuous operation, the operating conditions faced by lubrication equipment are becoming increasingly demanding. Its failure modes are characterized by diversity, strong early-stage concealment, and multi-factor coupling. Traditional maintenance methods based on single sensor threshold alarms or periodic manual inspections are no longer sufficient to meet the current demands for high reliability, high efficiency, and intelligent operation. Related technologies for the health management of lubrication equipment mainly include the following categories.

[0003] Monitoring and diagnostic methods based on single-modal data. For example, vibration sensors collect equipment vibration signals, and time-domain or frequency-domain analysis (such as FFT and wavelet transform) is used to extract features, combined with expert experience or simple machine learning models for fault identification; or oil analyzers are used to detect indicators such as metal particle concentration and water content in the oil to determine lubrication status and wear degree. These methods rely on only a single type of data, such as time-series signals or laboratory indicators, resulting in limited information dimensions and difficulty in comprehensively reflecting the equipment's operating status. Especially in the early stages of a fault, a single modality often cannot provide sufficient diagnostic criteria, leading to high rates of missed and false alarms, and insufficient ability to identify complex coupled faults.

[0004] Monitoring methods based on simple fusion of multimodal data. Some related technologies attempt to "stitch together" or "parallel input" data from different sources, such as vibration signals, oil detection data, and inspection text descriptions, and then perform joint analysis through multi-input neural networks. However, such methods typically only superficially integrate multimodal data, such as inputting concatenated features into fully connected layers, without deeply addressing core issues such as semantic alignment, temporal synchronization, spatial correlation, and causal consistency between different modalities. This results in the fused feature representation lacking physical meaning and logical coherence, making it difficult for the model to effectively uncover potential correlations between multi-source information, and limiting the accuracy and interpretability of fault diagnosis.

[0005] Health assessment methods based on general artificial intelligence models. Related solutions introduce deep learning models (such as BERT, LSTM, and GNN) to model multimodal data. However, most of these methods are directly transplanted from fields such as natural language processing and computer vision, and suffer from incompatibility issues in lubrication equipment scenarios. On the one hand, general models are not optimized for the failure modes of lubrication equipment (such as bearing pitting, gear wear, and oil circuit blockage) and data characteristics (such as the non-stationarity of time-series signals, the nonlinear degradation of oil indicators, and the technical terminology of inspection texts), resulting in poor generalization ability. On the other hand, existing model training relies on large-scale labeled data, while failure samples of lubrication equipment in industrial settings are scarce, especially early and rare failures, leading to high labeling costs. This causes the models to easily overfit in small-sample scenarios, making it difficult to adapt to the personalized needs of different equipment models and operating conditions. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a method, system, terminal, and medium for lubrication equipment health management, enabling accurate identification and early warning of lubrication equipment faults, thereby improving the accuracy of equipment health management and operational efficiency.

[0007] In a first aspect, the technical solution of the present invention provides a method for health management of lubrication equipment, comprising the following steps: Collect multimodal data from lubrication equipment, including time-series structured data from vibration, temperature, and oil pressure sensors, unstructured data (text, images, and audio) from intelligent inspection terminals, and detection data (oil particles, moisture, and viscosity) from oil detectors. Multimodal features are obtained by standardizing the collected multimodal data. The extracted multimodal features are time-stamped and spatially mapped to align cross-modal data in the time dimension. Then, a multimodal heterogeneous graph is constructed based on the aligned cross-modal data. The nodes in the graph include vibration time series features, oil features, image features, text semantic vectors and audio features, and the edges include temporal adjacency relationships, spatial topological relationships and fault causal relationships. Perform cross-modal graph attention computation on a multimodal heterogeneous graph to learn the semantic association weights between nodes of different modalities and generate a unified device state representation vector that integrates multi-source information; A unified state representation vector is input into a pre-trained neural network model for lubrication equipment fault prediction and health assessment. This model includes a text semantic encoding branch, a temporal signal processing branch, and a multimodal feature fusion branch. After processing, the model outputs the fault type and its probability of occurrence, the predicted value of remaining service life, and a comprehensive health status score. Among them, the text semantic encoding branch adopts a BERT model incrementally pre-trained on the lubrication equipment maintenance manual corpus, and the temporal signal processing branch is used to extract local and global features of vibration signals. During training, the model first undergoes self-supervised pre-training, and then performs transfer learning on a small amount of labeled data. Based on the model output, the system receives device status query commands through a natural language interaction interface, generates visual reports, and / or creates maintenance work orders.

[0008] Secondly, the technical solution of the present invention provides a lubrication equipment health management system, comprising: The multimodal data acquisition module is used to collect multimodal data from lubrication equipment, including time-series structured data from vibration, temperature, and oil pressure sensors, unstructured data such as text, images, and audio from intelligent inspection terminals, and detection data of oil particles, moisture, and viscosity from oil detectors. The multimodal feature acquisition module is used to standardize the collected multimodal data to obtain multimodal features; The multimodal heterogeneous graph construction module is used to perform time stamp calibration and spatial mapping on the extracted multimodal features to achieve alignment of cross-modal data in the time dimension. Then, a multimodal heterogeneous graph is constructed based on the aligned cross-modal data. The nodes in the graph include vibration time series features, oil features, image features, text semantic vectors and audio features, and the edges include temporal adjacency relationships, spatial topological relationships and fault causal relationships. The device state representation vector generation module performs cross-modal graph attention calculation on a multimodal heterogeneous graph to learn the semantic association weights between nodes of different modalities and generate a unified device state representation vector that integrates multi-source information. The fault prediction and health assessment module is used to input the unified state representation vector into a pre-trained neural network model for fault prediction and health assessment of lubrication equipment. This model includes a text semantic encoding branch, a time-series signal processing branch, and a multimodal feature fusion branch. After processing by the model, it outputs the fault type and its probability of occurrence, the predicted value of remaining service life, and the comprehensive health status score. Among them, the text semantic encoding branch adopts a BERT model incrementally pre-trained on the lubrication equipment maintenance manual corpus, and the time-series signal processing branch is used to extract local and global features of vibration signals. The device status query module is used to receive device status query commands through a natural language interaction interface based on the model output results, generate visual reports and / or create maintenance work orders.

[0009] Thirdly, the technical solution of the present invention provides a terminal, comprising: Memory, used to store lubrication equipment health management programs; A processor is configured to implement the steps of the lubrication equipment health management method as described in any of the preceding claims when executing the lubrication equipment health management program.

[0010] Fourthly, the present invention provides a computer-readable storage medium storing a lubrication equipment health management program, wherein the lubrication equipment health management program, when executed by a processor, implements the steps of the lubrication equipment health management method as described in any of the above claims.

[0011] As can be seen from the above technical solutions, this application has the following advantages: 1. This application constructs a heterogeneous graph containing multimodal nodes and multi-type association edges by collecting time-series structured data such as vibration, temperature, and oil pressure, unstructured inspection data such as text, images, and audio, and detection data such as oil particles, moisture, and viscosity. Based on cross-modal graph attention calculation, it dynamically learns the semantic association weights between nodes of different modalities and generates a unified equipment state representation vector that integrates multi-source information. This breaks through the limitations of traditional single-modal or simple splicing and fusion, and can simultaneously capture vibration anomalies, oil deterioration, fault clues in text descriptions and appearance anomalies in images, thus improving the accuracy of early fault identification and root cause analysis. 2. This application, through the collaborative design of text semantic encoding branch, time-series signal processing branch and multimodal feature fusion branch, combined with self-supervised pre-training and few-shot transfer learning, enables the model to accurately identify fault types, predict remaining service life and output a comprehensive health status score even with a small amount of labeled data. This effectively overcomes the problems of scarce fault samples and high labeling costs in industrial fields, and improves the model's generalization ability and adaptability to actual deployment. 3. This application receives device status query commands through a natural language interaction interface, combines the fault type, remaining lifespan and health score output by the model, generates a visual report and automatically creates maintenance work orders, realizing intelligent management of the entire process. Maintenance personnel can directly query the health status of the device through voice or text, obtain explainable fault causes and maintenance suggestions, reduce the professional threshold, and improve the speed of maintenance response and the scientific nature of decision-making. 4. This application constructs a heterogeneous graph through timestamp calibration, spatial mapping and fault causal relationship constraints, and further adjusts the edge weights in the graph structure based on physical mechanisms, so that the multimodal fusion process conforms to the operating mechanism of lubrication equipment, improves the physical meaning of feature representation and the credibility of model decision, and is suitable for industrial scenarios with high reliability requirements. Attached Figure Description

[0012] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of a lubrication equipment health management method provided in an embodiment of the present invention.

[0014] Figure 2 This is a schematic block diagram of a lubrication equipment health management system provided in an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0018] Figure 1 This is a schematic flowchart of a lubrication equipment health management method provided in an embodiment of the present invention. Figure 1 The executing entity can be a lubrication equipment health management system. The lubrication equipment health management method provided in this embodiment of the invention is executed by a computer device, and correspondingly, the lubrication equipment health management system runs on the computer device. Depending on different needs, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0019] like Figure 1 As shown, the method includes the following steps.

[0020] S1 collects multimodal data from lubrication equipment, including time-series structured data from vibration, temperature, and oil pressure sensors, unstructured data (text, images, and audio) from intelligent inspection terminals, and detection data (oil particles, moisture, and viscosity) from oil detectors.

[0021] S2, standardizes the collected multimodal data to obtain multimodal features.

[0022] S3. The extracted multimodal features are time-stamped and spatially mapped to achieve alignment of cross-modal data in the time dimension. Then, a multimodal heterogeneous graph is constructed based on the aligned cross-modal data. The nodes in the graph include vibration time series features, oil features, image features, text semantic vectors and audio features, and the edges include temporal adjacency relationships, spatial topological relationships and fault causal relationships.

[0023] S4 performs cross-modal graph attention computation on the multimodal heterogeneous graph to learn the semantic association weights between nodes of different modalities and generate a unified device state representation vector that integrates multi-source information.

[0024] S5 inputs the unified state representation vector into a pre-trained neural network model for lubrication equipment fault prediction and health assessment. This model includes a text semantic encoding branch, a time-series signal processing branch, and a multi-modal feature fusion branch. After processing by the model, it outputs the fault type and its probability of occurrence, the predicted value of remaining service life, and the comprehensive health status score. Among them, the text semantic encoding branch adopts a BERT model incrementally pre-trained on the lubrication equipment maintenance manual corpus, and the time-series signal processing branch is used to extract local and global features of vibration signals. During training, the model first performs self-supervised pre-training, and then performs transfer learning on a small amount of labeled data.

[0025] S6, based on the model output results, receives device status query commands through a natural language interaction interface, generates visual reports and / or creates maintenance work orders.

[0026] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another lubrication equipment health management method is provided, which includes the following steps.

[0027] S101, train the neural network model for lubrication equipment fault prediction and health assessment.

[0028] First, self-supervised pre-training is performed on publicly available industrial datasets, including vibration signal mask prediction, oil feature contrast learning, and cross-modal reconstruction tasks, to learn general feature representations. Then, a focus loss function is used to perform transfer learning on a small amount of labeled data from the target device.

[0029] (1) Vibration signal mask prediction task.

[0030] The vibration signal masking prediction task randomly masks the temporal feature vector of the vibration signal and predicts the original signal value or local time-frequency features at the masked location through the model in order to learn the local and global temporal dependencies of the vibration signal.

[0031] Specifically, let the time-series feature vector of the original vibration signal be... ,in For time steps, The feature dimension is for a single step; feature vectors of randomly selected time steps (e.g., proportion p∈[0.1,0.3]) are used. Replace with mask marker After obtaining the mask, input The model predicts the original features at the mask location using an encoder (such as a Transformer or a temporal convolutional network). And minimize the following reconstruction loss:

[0032] in, The set of time step indices to be masked. It is the L2 norm. The temporal characteristics of the vibration signal at time t, This is a temporal feature for predicting mask positions using an encoder.

[0033] (2) Oil characteristics comparison learning task.

[0034] The oil feature comparison learning task constructs positive and negative sample pairs from the multidimensional feature vectors (such as metal particle concentration and water content) obtained from oil detection. By maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs, the semantic representation of the oil state is learned.

[0035] Specifically, let the oil feature vector be... Positive sample pairs under the same equipment / operating condition are generated through data augmentation (such as randomly occluding some detection indicators and Gaussian noise perturbation). And negative sample pairs under different equipment / operating conditions Features are mapped to the contrast space via a projection head (such as an MLP). And optimized based on the InfoNCE loss function:

[0036] in, For projection function, For temperature hyperparameters, This is an indicator function; it is 1 when k=i, and 0 otherwise. This represents the number of all candidate samples involved in the summation.

[0037] (3) Cross-modal reconstruction task.

[0038] Cross-modal reconstruction tasks mask some modal inputs from multimodal features and jointly predict key information of the masked modality using the remaining modal features, in order to learn the complementarity and joint representation ability between modalities.

[0039] Specifically, masking oil characteristics Utilizing vibration characteristics and text features Reconstructing oil characteristics using a decoder And minimize the reconstruction error:

[0040] Through joint optimization of the above three tasks during the self-supervised pre-training phase, the total loss function is:

[0041] in, , , These are the task weight hyperparameters. Through joint optimization, the model learns vibration anomaly patterns, oil state semantics, and common feature representations across multiple modes, without relying on a large amount of labeled data.

[0042] The transfer learning phase involves supervised fine-tuning on a small amount of labeled data from the target lubrication equipment, focusing on the ability to identify key fault categories. This includes the following sub-steps.

[0043] 1) Preparation of labeled data.

[0044] Collect a small number of labeled samples of the target equipment under real-world operating conditions (such as normal state, bearing wear, gear pitting, and other fault types; the number of samples is typically tens to hundreds). Each sample contains multimodal feature inputs. and corresponding fault type labels ( (Total number of fault categories).

[0045] 2) Focal Loss optimization. To address the class imbalance problem caused by the high proportion of common categories (such as normal state) and the low proportion of a few fault categories (such as severe wear) in labeled data, a focus loss function is used to replace the traditional cross-entropy loss, thereby enhancing the model's learning weights for difficult samples (a few fault categories).

[0046] The focus loss function is defined as follows:

[0047] in, This represents the number of samples in the current batch. For fault category index, To predict the category to which sample i belongs in the model The probability (output via Softmax). For the true category label of sample i, Set a larger category balance factor (usually for a few fault classes). ), The focusing parameter is used to adjust the weights of easy and difficult samples when the model predicts the probability of a certain class. At lower (difficult samples) The value increases, thus amplifying the loss contribution of that sample.

[0048] By focusing on the loss function, the model prioritizes optimizing the classification boundaries of a few fault categories during the fine-tuning phase, thereby improving the sensitivity to identify early potential faults.

[0049] 3) Model parameter update.

[0050] By combining the general feature extractor (such as encoder weights) obtained during the pre-training phase, freezing some low-level network parameters (such as shallow convolutional kernels in temporal convolutions), and fine-tuning only the parameters of the high-level feature fusion layer and the classification head, the total loss is minimized through backpropagation. Update the model parameters until convergence. The self-supervised pre-training stage employs multi-task learning, including vibration mask prediction, oil state comparison learning, and cross-modal reconstruction. This allows the model to grasp general features of vibration anomaly patterns, oil state semantics, and multimodal correlations without requiring extensive labeled data, thus addressing the data scarcity problem in industrial scenarios. The transfer learning stage focuses on optimizing the loss function to specifically improve the identification ability of a few fault categories. Even with limited labeled data (e.g., 10-50 records per category), it maintains high fault prediction accuracy (≥95%) and adapts to the specific operating conditions of the target equipment. The overall process, through the collaborative design of pre-training and fine-tuning, ensures the model possesses both versatility and specificity, improving the generalization performance and practical deployment effectiveness of the lubrication equipment health management system.

[0051] S102, collects multimodal data of lubrication equipment.

[0052] The collected multimodal data includes time-series structured data from vibration, temperature, and oil pressure sensors; unstructured data (text, images, and audio) from intelligent inspection terminals; and detection data (oil particles, moisture, and viscosity) from oil detectors.

[0053] S103, collect multimodal features.

[0054] The collected multimodal data is standardized to obtain multimodal features, which includes the following steps.

[0055] S103.1 performs denoising, missing value imputation, and normalization on the time-series structured sensor data, and extracts time-domain and frequency-domain features to form a time-series feature vector.

[0056] The time-series structured data from vibration, temperature, and oil pressure sensors are aligned according to timestamps, sampled at a uniform frequency, and encapsulated into standard time-series data formats, such as CSV, InfluxDB, or custom time-series data structures.

[0057] Text descriptions, images, and audio recordings from intelligent inspection terminals are bound to structured forms via an industrial-grade data acquisition app or terminal to record inspection time, location, equipment number, and associated media files.

[0058] The test data such as oil particle size, moisture content, and viscosity from the oil analyzer are processed according to the data format output by the laboratory or online testing instrument. Key parameters are extracted and packaged into test data records with timestamps.

[0059] For time-series structured sensor data such as vibration, temperature, and oil pressure, the following preprocessing operations are performed: Denoising: High-frequency noise and interference signals are removed by using sliding window filtering, wavelet thresholding, or moving average methods. Missing value imputation: For missing data points in a time series, linear interpolation, forward imputation, or interpolation methods based on statistical characteristics are used to process them; Normalization / Standardization: For each type of time-series signal (such as vibration amplitude, oil pressure value), Z-score standardization or Min-Max normalization is used to map it to a uniform numerical range and eliminate dimensional differences. Feature extraction: Extract key time-domain features (such as mean, variance, peak value, kurtosis, peak-to-peak value) and frequency-domain features (such as main frequency, band energy, and spectral entropy) from time-series signals to form a structured time-series feature vector.

[0060] S103.2, for unstructured text, image and audio data, respectively, a domain-optimized OCR model, a convolutional neural network model and a speech recognition model are used to extract structured semantic feature vectors.

[0061] Text data: Domain-optimized OCR recognition models (for handwritten or printed inspection records) and natural language pre-trained models (such as BERT-like models) are used to extract key fault descriptions, abnormal keywords and semantic information, which are then further encoded into fixed-dimensional text semantic vectors.

[0062] Image data: Using convolutional neural networks (such as ResNet, EfficientNet) or domain-adjusted image classification / detection models, analyze equipment appearance images and oil color / impurity images to extract image feature vectors or identify specific fault modes, including oil emulsification, oil leakage, and stain deposition.

[0063] Audio data: Audio recordings of device noises, environmental noises, etc. are converted into text using speech recognition (ASR) technology, and then semantic vectors are extracted through text processing; or audio feature extraction models (such as Mel spectrum, MFCC, WaveNet features) are directly used to extract audio time-frequency feature vectors.

[0064] S103.3 Extract target oil health index parameters from oil detection data and normalize them to form an oil feature vector.

[0065] For the oil particle size, moisture content, viscosity, acid value, and additive content output by the oil analyzer, the following processing is performed: Extract key oil health indicators, such as: concentration of metal wear particles, number of ferromagnetic particles, water percentage, and viscosity index; These indicators are standardized in units and dimensions to form an oil characteristic vector, which is used to reflect the lubrication status and wear degree.

[0066] S103.4 encapsulates all modal data in a unified format, includes timestamps and device metadata, and outputs a multimodal feature set.

[0067] All modal data processed in the above steps, including temporal feature vectors, text semantic vectors, image feature vectors, audio feature vectors, and oil feature vectors, are encapsulated according to a unified data structure. Each data sample corresponds to a multimodal feature set and is accompanied by corresponding metadata information such as timestamp, device ID, and sensor location. The output multimodal features will serve as inputs for cross-modal data alignment, graph structure construction, and joint representation learning in subsequent steps, ensuring that each modal data has the basic conditions for fusion in terms of format, dimension, semantics, and time dimension.

[0068] S104, construct a multimodal heterogeneous graph.

[0069] To address the inconsistencies in temporal reference, spatial location, and logical relationships among different modal data, and to achieve accurate alignment and logical relationship modeling of multimodal data in the temporal dimension, a cross-modal data alignment and heterogeneous graph construction process is further implemented. Extracted multimodal features are timestamped and spatially mapped to achieve temporal alignment of cross-modal data. Then, a multimodal heterogeneous graph is constructed based on the aligned cross-modal data. By constructing the heterogeneous graph and performing graph attention calculation, deep fusion of cross-modal semantics is achieved, uncovering potential fault correlations between different data sources and improving early fault identification capabilities. The nodes in this graph include vibration time-series features, oil features, image features, text semantic vectors, and audio features; the edges include temporal adjacency relationships, spatial topological relationships, and fault causal relationships.

[0070] S104.1, Timestamp calibration of multimodal features.

[0071] Since multimodal data (such as vibration signals, oil analysis data, inspection text / images / audio) collected from different sensors, inspection terminals and testing instruments usually have independent sampling timestamps or recording times, in order to ensure the temporal consistency of subsequent cross-modal correlation analysis, timestamp calibration processing is first performed on all modal feature data.

[0072] S104.11, Extract the original timestamp information of each modality data.

[0073] Extract the original time identifiers corresponding to each modal feature from the original data sources (such as vibration sensor data packet headers, oil testing instrument output logs, and intelligent inspection terminal collection time records), including absolute time, relative timestamps (such as equipment running time, sampling sequence number), or event trigger time.

[0074] S104.12, Determine the unified time base and time alignment strategy.

[0075] Based on the actual needs of lubrication equipment operation monitoring, a unified reference time base should be selected, or a time alignment strategy should be formulated using key events as anchor points. The reference time base can be the equipment startup time, the time of the most recent maintenance, a certain standard time point, or key events such as the time of failure or the time of regular inspection. For time-series data, time axis standardization is performed according to a fixed sampling frequency or time window; for non-time-series data, it is mapped to the nearest time window or time node based on its acquisition time or the time of related events.

[0076] S104.13, perform timestamp matching and data synchronization.

[0077] Based on a unified time benchmark, timestamps of different modal features are matched to align multimodal features belonging to the same time window to the same time node. The same time window can be the same sampling period, the same inspection time, or the same time period before and after a fault. For modalities with different time resolutions, interpolation, sliding window aggregation, or nearest neighbor matching are used to achieve synchronization in the time dimension, ensuring that the features of each node are correlated in the time dimension during subsequent graph construction.

[0078] S104.2, Spatial mapping of multimodal features.

[0079] For the acquisition modes deployed at different physical locations on the lubrication equipment, if their spatial location information has an impact on fault analysis, further spatial mapping processing is performed to map the features of each mode to a unified spatial coordinate system or equipment topology.

[0080] S104.21: Extract the physical location information of each modality data.

[0081] Obtain the physical location identifiers corresponding to each modal data from the equipment CAD drawings, sensor installation manuals, or data acquisition terminal metadata, such as equipment coordinates, area number, and sensor installation point ID.

[0082] S104.22: Construct a spatial topology model of equipment.

[0083] Based on the actual physical structure of the lubrication equipment, including the distribution of gearboxes, pipelines, and bearing housings, a spatial adjacency relationship or topological connection model between the various components of the equipment is established to describe the spatial proximity or logical association between different data collection locations.

[0084] S104.23: Mapping and associating multimodal features based on spatial location.

[0085] Each modal feature is mapped to the corresponding node or region in the equipment spatial topology model based on its physical location information. For modes that are spatially adjacent or logically related (such as vibration sensor and oil detector in the same lubrication circuit), they are marked as having spatial topological association, which is used for the construction of edge relationships in the subsequent heterogeneous graph.

[0086] S104.3, Constructing heterogeneous graph nodes and edges based on multimodal data with time alignment and spatial mapping.

[0087] After completing timestamp calibration and spatial mapping, the modal features are unified to the same time node and spatial location, and then the construction of a multimodal heterogeneous graph is performed.

[0088] S104.31 defines the node types in a heterogeneous graph.

[0089] Each aligned feature of a modality is treated as an independent graph node, and the node types include, but are not limited to: Vibration timing feature nodes correspond to the vibration signal feature vectors within a time window; Oil feature nodes correspond to the detection index vectors of oil particle size, moisture, viscosity, etc. within the time window; Image feature nodes are feature vectors corresponding to equipment appearance images or oil images collected within a time window; Text semantic vector nodes correspond to the semantic embedding vectors of inspection text descriptions or fault reports within the time window; Audio feature nodes correspond to the audio feature vectors of device noise or environmental noise within a time window.

[0090] S104.32: Define the edge type in a heterogeneous graph.

[0091] Based on the time alignment results and device physical structure information, define the relationships between nodes (i.e., graph edges). Edge types include: Temporal adjacency indicates that the data corresponding to two nodes are adjacent in time or belong to the same time window, such as vibration and oil data within the same sampling period; Spatial topology relationship indicates that the sensors or detection points corresponding to two nodes are adjacent in the physical structure of the equipment or belong to the same functional area, such as the vibration sensor and the oil sampling point in the same lubrication circuit. Fault causation indicates that there is a fault propagation or logical causal relationship between two nodes. For example, an increase in the concentration of metal particles in the oil usually precedes abnormal vibration, or a specific text description is related to subsequent abnormal oil indicators.

[0092] S104.33: Construct a set of nodes and edges to generate a multimodal heterogeneous graph.

[0093] The multimodal features within each time window (or critical event moment) that have undergone time alignment and spatial mapping are added to the graph as nodes of the corresponding type. According to the edge type defined in step S104.32, nodes with temporal adjacency, spatial topology, or fault causal relationship are connected to form a multimodal heterogeneous graph G=(V,E) containing a set of nodes V and a set of edges E, which serves as the input basis for subsequent cross-modal graph attention calculation and unified state representation generation.

[0094] S105, Generate a unified device state representation vector.

[0095] Performing cross-modal graph attention computation on a multimodal heterogeneous graph to learn the semantic association weights between nodes of different modalities and generating a unified device state representation vector that integrates multi-source information includes the following steps.

[0096] S105.1, initialize the feature representation vector for each modal node, which is obtained by standardizing and extracting features from the collected multimodal data.

[0097] Suppose there are Q nodes in the graph, each node The initial feature vector is: ,

[0098] in, It is the first The original features of each modal node, such as vibration signal time-frequency features, oil index vectors, and text semantic embedding.

[0099] S105.2 For each target node in a multimodal heterogeneous graph, the semantic association weight between the current target node and each of its neighboring nodes is calculated based on the graph attention mechanism.

[0100] Specifically, for each target node in the graph traverse its set of neighboring nodes (Determined by the edge relationships in the graph, including nodes connected by time, space, or causality, which may originate from different modalities), computation With each neighbor node The semantic association weights (i.e., attention coefficients) between them The semantic association weight is calculated using the following formula:

[0101] in, Let be the semantic association weight between node i and its neighbor node j. , Let be the feature representation vectors of node i and node j, respectively. The weight matrix is ​​a learnable matrix. This is a learnable attention parameter vector. This is a vector concatenation operation. It is a non-linear activation function. Let i be the set of neighboring nodes of node i; It is an exponential function used to amplify the similarity score before normalization.

[0102] The semantic association weight calculation process first measures the feature correlation between node pairs through a learnable weight matrix and attention vector, then introduces non-linearity using LeakyReLU, and finally uses Softmax to transform the correlation score into a probability that sums to 1 (attention coefficient), thereby assigning greater weight to more "important" neighbors when aggregating neighbor information.

[0103] S105.3, Based on the calculated semantic association weights, perform weighted aggregation on the feature representations of neighboring nodes and update the feature representation of the target node.

[0104] Specifically, the attention weights calculated in the previous step are used. For the target node All neighboring nodes The features are weighted and summed to obtain the node at the th node. New feature representation of layers :

[0105] S105.4, through the superposition of single-layer or multi-layer graph attention networks, the feature vector of the target node output by the last layer is taken as the unified device state representation vector that integrates multi-source information. This vector contains comprehensive information from different modalities such as vibration, oil, image, text and audio after semantic alignment and graph attention weighting.

[0106] In practical implementation, multi-layer graph attention networks (GATs) can be stacked, with each layer further aggregating higher-order neighbor information, enabling the feature representation of each node to progressively fuse global multimodal context information. Finally, the feature vector of the target node output from the last layer (such as a virtual node representing the "overall device state," or a fused node) is taken as:

[0107] This refers to a unified device state representation vector that integrates information from multiple sources. This vector contains comprehensive information from different modalities such as vibration, oil, images, text, and audio, which has been semantically aligned and weighted by graph attention. It is used for subsequent fault prediction and health assessment.

[0108] In some optional implementations, after constructing a multimodal heterogeneous graph containing multimodal nodes and various types of edges, to further improve the interpretability, professionalism, and reliability of cross-modal fusion, an edge weight adjustment mechanism based on prior knowledge and physical mechanisms in the lubrication equipment field is introduced. This mechanism dynamically adjusts or optimizes the weights of edges connecting different nodes in the graph using physical guidance, enabling the graph attention mechanism to prioritize modal association paths with practical engineering significance and physical relevance when aggregating multimodal node information. This achieves more accurate and reliable physical-guided cross-modal fusion. Specifically, adjusting the edge weights in the multimodal heterogeneous graph based on physical mechanism constraints includes the following steps.

[0109] Step 1: For each edge in the multimodal heterogeneous graph, label the physical relationship type of the edge according to the modal types of the two nodes and the data source of the edge, including temporal adjacency edges, spatial topological edges, and fault causal edges, and assign an initial physical weight to each edge according to the physical relationship type. .

[0110] Initial physical weights are a quantitative measure of the physical relationships represented by edges. They reflect the importance, strength of association, and other characteristics of the physical relationship in a specific application scenario or analysis task.

[0111] For time-adjacent edges, their initial physical weights The initial physical weight of the adjacency edge is primarily determined by the temporal proximity of the two nodes. For example, if the events represented by the two nodes are closely linked in the time series with a short time interval, the initial physical weight can be set to a relatively large value to highlight this strong temporal correlation; conversely, if the time interval is long, the initial physical weight can be set to a smaller value. The specific numerical calculation can be performed as follows: Assuming the timestamps of node i and node j are ti and tj respectively, a time interval threshold Δt is defined. When |ti - tj| < Δt, = 1-(|ti - tj| / Δt); when |ti - tj|≥Δt, = ε (ε is a small constant, such as 0.1, which represents the weak association weight when the time interval exceeds the threshold).

[0112] The initial physical weight of a spatial topology edge is determined by the spatial location and connectivity of the entities represented by the nodes. If two nodes are directly adjacent in space, such as within the same region or connected by a physical link, then the initial physical weight of the edge can be set to a higher value. It can be quantified based on factors such as spatial distance and connection method. For example, the reciprocal of the spatial distance can be used to represent the weight, i.e. = 1 / ,in Let represent the spatial distance between node i and node j. For some special connection methods, such as connections with higher bandwidth or lower latency in the network topology, the weight value can be appropriately increased based on the above.

[0113] The initial physical weights of causal edges from faults are primarily determined based on the probability and impact of fault propagation. If a fault event (node ​​i) has a high probability of triggering another fault event (node ​​j), then the initial physical weights of the causal edges from node i to node j should be set relatively high. This probability information can be obtained through statistical analysis of historical fault data. For example, by analyzing a large number of fault cases, the probability that fault i leads to fault j can be statistically determined. Then the initial physical weights = At the same time, the severity of the fault's impact can be considered, and the probability can be adjusted accordingly. For example, when the impact of fault j is high, the probability can be adjusted accordingly. Multiply by an amplification factor α (α>1) to appropriately increase the weight.

[0114] Step 2, calculate the semantic association weights based on the graph attention mechanism. With the initial physical weights defined based on physical rules We perform weighted fusion to obtain the adjusted edge weights, denoted as follows:

[0115] in, These are adjustable hyperparameters used to balance the fusion ratio of data-driven and physical-guided approaches.

[0116] After completing the edge weight adjustment, the adjusted edge weights will be... Applied to the graph attention computation process, this approach enables target nodes to prioritize fusing modal node features that are physically correlated in terms of time, space, and fault causality when aggregating neighbor information. This results in more accurate multimodal information fusion, more reliable early fault correlation detection, and more interpretable and credible equipment state characterization. Ultimately, based on physically guided cross-modal fusion, a unified equipment state characterization vector that integrates multi-source information and conforms to the operating mechanism of lubrication equipment is generated for subsequent fault prediction and health assessment.

[0117] S106, Fault Prediction and Health Assessment.

[0118] The unified state representation vector is input into a pre-trained neural network model for lubrication equipment fault prediction and health assessment. This model includes a text semantic encoding branch, a time-series signal processing branch, and a multimodal feature fusion branch. After processing by the model, the fault type and its probability of occurrence, the predicted value of remaining service life, and the comprehensive health status score are output.

[0119] The text semantic encoding branch uses a BERT model that has been incrementally pre-trained on a lubrication equipment maintenance manual corpus.

[0120] The temporal signal processing branch employs a backbone network combining multi-scale temporal convolution and a self-attention mechanism to extract local and global features of vibration signals. This includes multi-scale temporal convolution units that use parallel dilated convolutional structures to extract local vibration features at different time scales; a self-attention mechanism unit for modeling long-range temporal dependencies and key feature regions; and a feature fusion gating unit for dynamically adjusting the contribution weights of each input modality in the final decision.

[0121] The multimodal feature fusion branch uses a gated fusion mechanism to dynamically adjust the contribution of each modality to the final prediction and each output term.

[0122] For fault type probability prediction, the model infers the possible fault types and their probabilities of occurrence of the current lubrication equipment based on a unified state representation vector. The output is a multi-class probability distribution vector, such as [normal state: 92%, rolling bearing inner ring wear: 5%, gear tooth surface pitting: 2%, oil circuit blockage: 1%], which is used for early fault identification, root cause analysis, and risk classification. For remaining service life prediction, the model predicts the remaining time that the current equipment or key components (such as bearings, gears, lubrication systems) can still operate normally. The output is in the form of a scalar value or a prediction distribution, which supports predictive maintenance decisions, arranges maintenance or replacement in advance, and avoids unplanned downtime.

[0123] Based on the health status score or health index, the model outputs a comprehensive health score to intuitively reflect the overall operational health status of the equipment. The output is a single numerical value, for example: Health Index = 85 / 100, indicating that the equipment is in good condition. If it falls below a certain threshold, an alert is triggered, providing maintenance personnel with a quick and intuitive understanding of the equipment's health status and assisting in decision-making.

[0124] In addition to directly providing the predicted category or numerical value, the model also offers interpretable intermediate results or attention weights for interpretability analysis results. For example, it shows which mode (such as vibration signal or oil particles) has the greatest impact on the current prediction, which time-series characteristics are most critical for fault diagnosis, and graph attention weights display which nodes (modes / features) contribute significantly to the final decision. Output formats can include attention maps, feature importance ranking tables, modal contribution analysis, etc., enhancing system transparency and credibility.

[0125] For fault severity assessment, the severity of detected faults is further evaluated (e.g., minor, moderate, severe) to distinguish the urgency level. The output is in the form of classification labels (e.g., severe / warning / normal) or graded values ​​to assist in prioritizing operations and maintenance and optimizing resource allocation.

[0126] S107, Device Status Inquiry.

[0127] Based on the model output, the system receives device status query commands through a natural language interaction interface, generates visual reports, and / or creates maintenance work orders.

[0128] In the lubrication equipment health management method, after multimodal data fusion, graph attention calculation, and neural network model inference, the model outputs key information reflecting the current health status of the equipment, including fault types and their probabilities, predicted remaining service life, and comprehensive health status scores. To transform this data into decision support information that maintenance personnel can intuitively understand and execute, this method further includes an equipment status query step. It receives equipment status query commands from users (such as maintenance engineers and equipment managers) through a natural language interaction interface, automatically generates visual reports and / or automatically creates maintenance work orders based on the model inference results, achieving closed-loop management from data insight to maintenance action, and improving maintenance efficiency and the scientific nature of decision-making.

[0129] S107.1, Natural Language Interaction Interface receives device status query commands.

[0130] It integrates an intelligent natural language interaction module, built on a large language model, which supports users initiating device status query requests to the system through various interaction methods, including text input, voice commands, and input via mobile app or web interface. User query commands can take the following forms: Explicit status queries: such as "What is the current operating status of device XX?", "Are there any abnormal faults?", "How many hours of remaining lifespan are left?"; Implicit maintenance needs: such as "Please check the health status of the lubrication system", "Generate today's equipment health report", "Remind me of the equipment that needs maintenance"; Specific fault tracing: such as "What caused the abnormal vibration yesterday?", "What is the fault type corresponding to the abnormal oil level detection?" The natural language interaction interface uses the intent recognition module to semantically parse the text or voice commands input by the user, extract key query elements such as target device ID, query time range, type of status of interest (fault / life / health score), and output format requirements (report / work order), and then passes the parsed structured query request to the backend model inference result calling module.

[0131] S107.2, Generate a visualization report based on the model output results.

[0132] Based on the query command parsed from the natural language interaction interface, target information related to the query request is extracted from the model inference results (i.e., the fault type and probability, remaining service life, health status score, etc. output by step S106). Through visualization components and report generation engine, a structured, graphical, and easy-to-understand visualization report is automatically generated.

[0133] Health status overview display: In the form of dashboard, status card or rating list, the current comprehensive health score, health level, main risk indicators and key parameter trends of the equipment (such as remaining life countdown, vibration amplitude change curve) are displayed intuitively.

[0134] Fault Details and Root Cause Analysis: If the model detects a potential fault or abnormal state, the report will detail the fault type and its probability of occurrence, associated modal evidence, possible causes of the fault, and visualization charts. Possible causes of the fault are based on knowledge base reasoning or historical case matching, such as "insufficient lubrication due to long-term high-load operation".

[0135] Maintenance Recommendations and Forecast Information: The report also provides predicted remaining service life, recommended maintenance measures, and maintenance priority suggestions. The visual report can be output via a web interface, mobile app, or PDF document for maintenance personnel to view, print, or share in real time.

[0136] S107.3 automatically creates maintenance work orders.

[0137] When a high-risk fault is detected, or when a user's query command explicitly includes the intent to "create a work order" or "notify maintenance," the automatic generation and flow of maintenance work orders will be further triggered.

[0138] Based on the key information output by the model (fault type, probability, location, and suggested measures), and combined with the context such as the target device ID and query time in the query command, a standardized operation and maintenance work order is automatically generated. The content includes basic device information, fault description and classification, recommended handling measures and priorities, the basis for the associated model reasoning, generation time, and responsible person.

[0139] The generated maintenance work orders are automatically pushed to the enterprise maintenance management platform, mobile maintenance APP, or the communication terminal of the designated person in charge through the system interface, triggering the maintenance task process.

[0140] The above text provides a detailed description of an embodiment of a lubrication equipment health management method. Based on the lubrication equipment health management method described in the above embodiment, this invention also provides a lubrication equipment health management system corresponding to the method.

[0141] Figure 2This is a schematic block diagram of a lubrication equipment health management system provided in an embodiment of the present invention. In this embodiment, the lubrication equipment health management system 200 can be divided into multiple functional modules according to its functions, such as... Figure 2 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and which are stored in memory.

[0142] The multimodal data acquisition module 210 is used to acquire multimodal data of the lubrication equipment, including time-series structured data from vibration, temperature and oil pressure sensors, unstructured data of text, images and audio from intelligent inspection terminals, and detection data of oil particles, moisture and viscosity from oil detectors.

[0143] The multimodal feature acquisition module 220 is used to standardize the collected multimodal data to obtain multimodal features.

[0144] The multimodal heterogeneous graph construction module 230 is used to perform time stamp calibration and spatial mapping on the extracted multimodal features to achieve alignment of cross-modal data in the time dimension. Then, a multimodal heterogeneous graph is constructed based on the aligned cross-modal data. The nodes in the graph include vibration time series features, oil features, image features, text semantic vectors and audio features, and the edges include temporal adjacency relationships, spatial topological relationships and fault causal relationships.

[0145] The device state representation vector generation module 240 performs cross-modal graph attention calculation on the multimodal heterogeneous graph to learn the semantic association weights between nodes of different modalities and generate a unified device state representation vector that integrates multi-source information.

[0146] The fault prediction and health assessment module 250 is used to input the unified state representation vector into a pre-trained neural network model for fault prediction and health assessment of lubrication equipment. This model includes a text semantic encoding branch, a time-series signal processing branch, and a multi-modal feature fusion branch. After processing by the model, it outputs the fault type and its probability of occurrence, the predicted value of remaining service life, and the comprehensive health status score. Among them, the text semantic encoding branch adopts a BERT model incrementally pre-trained on the lubrication equipment maintenance manual corpus, and the time-series signal processing branch is used to extract local and global features of vibration signals.

[0147] The device status query module 260 is used to receive device status query commands through a natural language interaction interface based on the model output results, generate visual reports and / or create maintenance work orders.

[0148] The lubrication equipment health management system of this embodiment is used to implement the aforementioned lubrication equipment health management method. Therefore, the specific implementation of this system can be found in the embodiment section of the lubrication equipment health management method above. Thus, the specific implementation can be referred to the description of the corresponding embodiments, and will not be elaborated here.

[0149] Furthermore, since the lubrication equipment health management system of this embodiment is used to implement the aforementioned lubrication equipment health management method, its function corresponds to the function of the above method, and will not be described again here.

[0150] Figure 3 This is a schematic diagram of a terminal 300 provided in an embodiment of the present invention, including: a processor 310, a memory 320, and a communication unit 330. The processor 310 is used to implement the flow steps of the above-described lubrication equipment health management method embodiment when implementing the lubrication equipment health management program stored in the memory 320.

[0151] The terminal 300 includes a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It can be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0152] The memory 320 can be used to store the execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile memory terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 is able to perform some or all of the steps in the above method embodiments.

[0153] The processor 310 serves as the control center of the storage terminal, connecting various parts of the electronic terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0154] The communication unit 330 is used to establish a communication channel, enabling the storage terminal to communicate with other terminals. It can receive user data sent by other terminals or send user data to other terminals.

[0155] The present invention also provides a computer storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0156] The computer storage medium stores a lubrication equipment health management program, which, when executed by the processor, implements the process steps of the above-described lubrication equipment health management method embodiment.

[0157] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or other media capable of storing program code. It includes several instructions to cause a computer terminal (which may be a personal computer, server, or a second terminal, network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0158] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0159] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0161] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for health management of lubrication equipment, characterized in that, Includes the following steps: Collect multimodal data from lubrication equipment, including time-series structured data from vibration, temperature, and oil pressure sensors, unstructured data (text, images, and audio) from intelligent inspection terminals, and detection data (oil particles, moisture, and viscosity) from oil detectors. Multimodal features are obtained by standardizing the collected multimodal data. The extracted multimodal features are time-stamped and spatially mapped to align cross-modal data in the time dimension. Then, a multimodal heterogeneous graph is constructed based on the aligned cross-modal data. The nodes in the graph include vibration time series features, oil features, image features, text semantic vectors and audio features, and the edges include temporal adjacency relationships, spatial topological relationships and fault causal relationships. Perform cross-modal graph attention computation on a multimodal heterogeneous graph to learn the semantic association weights between nodes of different modalities and generate a unified device state representation vector that integrates multi-source information; A unified state representation vector is input into a pre-trained neural network model for lubrication equipment fault prediction and health assessment. This model includes a text semantic encoding branch, a temporal signal processing branch, and a multimodal feature fusion branch. After processing, the model outputs the fault type and its probability of occurrence, the predicted value of remaining service life, and a comprehensive health status score. Among them, the text semantic encoding branch adopts a BERT model incrementally pre-trained on the lubrication equipment maintenance manual corpus, and the temporal signal processing branch is used to extract local and global features of vibration signals. During training, the model first undergoes self-supervised pre-training, and then performs transfer learning on a small amount of labeled data. Based on the model output, the system receives device status query commands through a natural language interaction interface, generates visual reports, and / or creates maintenance work orders.

2. The lubrication equipment health management method according to claim 1, characterized in that, The collected multimodal data is standardized to obtain multimodal features, specifically including: The time-series structured sensor data is denoised, missing values ​​are filled in, and normalized. Time-domain and frequency-domain features are extracted to form a time-series feature vector. For unstructured text, image, and audio data, we use domain-optimized OCR models, convolutional neural network models, and speech recognition models, respectively, to extract structured semantic feature vectors. For oil detection data, extract the target oil health index parameters and normalize them to form an oil feature vector; All modal data are encapsulated in a unified format, with timestamps and device metadata included, and a multimodal feature set is output.

3. The lubrication equipment health management method according to claim 1, characterized in that, Perform cross-modal graph attention computation on a multimodal heterogeneous graph to learn semantic association weights between nodes of different modalities, and generate a unified device state representation vector that integrates multi-source information, specifically including: Each modal node is initialized with its feature representation vector, which is obtained by standardizing and extracting features from the collected multimodal data; For each target node in a multimodal heterogeneous graph, the semantic association weights between the current target node and each of its neighboring nodes are calculated based on a graph attention mechanism. The semantic association weights are calculated using the following formula. in, Let be the semantic association weight between node i and its neighbor node j. , Let be the feature representation vectors of node i and node j, respectively. The weight matrix is ​​a learnable matrix. This is a learnable attention parameter vector. This is a vector concatenation operation. It is a non-linear activation function. Let i be the set of neighboring nodes of node i; Based on the calculated semantic association weights, the feature representations of neighboring nodes are weighted and aggregated to update the feature representation of the target node; The feature vector of the target node output by the last layer is taken as a unified device state representation vector that integrates multi-source information by superimposing single-layer or multi-layer graph attention networks. This vector contains comprehensive information from different modalities such as vibration, oil, image, text and audio after semantic alignment and graph attention weighting.

4. The lubrication equipment health management method according to claim 3, characterized in that, The method also includes a step of adjusting the edge weights in a multimodal heterogeneous graph based on physical mechanism constraints, specifically including: For each edge in a multimodal heterogeneous graph, the physical relationship type of the edge is labeled according to the modal types of the two nodes and the data source of the edge, including temporal adjacency edges, spatial topological edges, and fault causal edges. An initial physical weight is then assigned to each edge based on the physical relationship type. ; The semantic association weights calculated based on the graph attention mechanism With the initial physical weights defined based on physical rules We perform weighted fusion to obtain the adjusted edge weights, denoted as follows: in, It is an adjustable hyperparameter used to balance the fusion ratio of data-driven and physical-guided approaches.

5. The lubrication equipment health management method according to claim 1, characterized in that, Training a neural network model for lubrication equipment failure prediction and health assessment includes the following steps: Perform self-supervised pre-training on publicly available industrial datasets, including vibration signal mask prediction, oil feature contrast learning, and cross-modal reconstruction tasks, to learn general feature representations; Transfer learning is performed on a small amount of labeled data on the target device using a focused loss function.

6. The lubrication equipment health management method according to claim 5, characterized in that, The vibration signal masking prediction task randomly masks the temporal feature vector of the vibration signal and predicts the original signal value or local time-frequency features at the masked location using a model to learn the local and global temporal dependencies of the vibration signal. The loss function for this task is... in, The set of time step indices to be masked. It is the L2 norm. The temporal characteristics of the vibration signal at time t are... To predict the temporal characteristics of the mask position using the encoder; The oil feature comparison learning task constructs positive and negative sample pairs from the multidimensional feature vectors obtained from oil detection. By maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs, it learns the semantic representation of the oil state. The loss function for this task is... Let the oil feature vector be... Data augmentation is used to generate positive sample pairs under the same equipment or operating conditions. And negative sample pairs under different equipment or operating conditions ; in, This indicates that the feature is projected through the projection head. Mapped to contrast space, For projection function, For temperature hyperparameters, This is an indicator function; it is 1 when k=i, and 0 otherwise. This represents the number of all candidate samples involved in the summation. Cross-modal reconstruction tasks mask some modal inputs from multimodal features and jointly predict key information of the masked modality using the remaining modal features to learn the complementarity and joint representation ability between modalities. The loss function for this task is: in, To utilize vibration and text features, a decoder is used to analyze the masked oil feature vector. The process generates a reconstructed oil feature vector. Through joint optimization of the above three tasks during the self-supervised pre-training phase, the total loss function is: in, , , This is the task weight hyperparameter.

7. The lubrication equipment health management method according to claim 5, characterized in that, The focusing loss function used in performing transfer learning is: in, This represents the number of samples in the current batch. For fault category index, To predict the category to which sample i belongs in the model The probability, For the true category label of sample i, This is the category balance coefficient. For focusing parameters.

8. A health management system for lubrication equipment, characterized in that, include: The multimodal data acquisition module is used to collect multimodal data from lubrication equipment, including time-series structured data from vibration, temperature, and oil pressure sensors, unstructured data such as text, images, and audio from intelligent inspection terminals, and detection data of oil particles, moisture, and viscosity from oil detectors. The multimodal feature acquisition module is used to standardize the collected multimodal data to obtain multimodal features; The multimodal heterogeneous graph construction module is used to perform time stamp calibration and spatial mapping on the extracted multimodal features to achieve alignment of cross-modal data in the time dimension. Then, a multimodal heterogeneous graph is constructed based on the aligned cross-modal data. The nodes in the graph include vibration time series features, oil features, image features, text semantic vectors and audio features, and the edges include temporal adjacency relationships, spatial topological relationships and fault causal relationships. The device state representation vector generation module performs cross-modal graph attention calculation on a multimodal heterogeneous graph to learn the semantic association weights between nodes of different modalities and generate a unified device state representation vector that integrates multi-source information. The fault prediction and health assessment module is used to input the unified state representation vector into a pre-trained neural network model for fault prediction and health assessment of lubrication equipment. This model includes a text semantic encoding branch, a time-series signal processing branch, and a multimodal feature fusion branch. After processing by the model, it outputs the fault type and its probability of occurrence, the predicted value of remaining service life, and the comprehensive health status score. Among them, the text semantic encoding branch adopts a BERT model incrementally pre-trained on the lubrication equipment maintenance manual corpus, and the time-series signal processing branch is used to extract local and global features of vibration signals. The device status query module is used to receive device status query commands through a natural language interaction interface based on the model output results, generate visual reports and / or create maintenance work orders.

9. A terminal, characterized in that, include: Memory, used to store lubrication equipment health management programs; A processor is configured to implement the steps of the lubrication equipment health management method as described in any one of claims 1 to 7 when executing the lubrication equipment health management program.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a lubrication equipment health management program, which, when executed by a processor, implements the steps of the lubrication equipment health management method as described in any one of claims 1 to 7.

Images