A multimodal intelligent recommendation system for equipment maintenance knowledge combining digital twins

Abstract

This invention discloses a multimodal intelligent recommendation system for equipment maintenance knowledge that combines digital twins, belonging to the field of equipment maintenance technology. It addresses the technical problems of static recommendation strategies and the difficulty in simultaneously achieving edge-cloud collaborative global optimization in existing solutions. Through synchronous acquisition of multimodal data driven by digital twins, a long short-term memory network model integrating attention mechanisms is used, employing a loss function weighted by decay rate deviation, and strong correlation feature filtering and dynamic weighting based on mutual information entropy. A time-varying feature matrix is ​​used to capture the evolution of equipment status over time. A knowledge graph with both physical entity association accuracy and causal reasoning capabilities is constructed, achieving end-to-end interpretability from data, features, entities, causality, and decision-making. The three-layer architecture of high-frequency response at the edge, global optimization in the cloud, and synchronized parameters through federated learning can resolve the contradiction between the real-time processing requirements of high-frequency data in industrial equipment maintenance and the dependence on a global knowledge graph.

Description

Technical Field

[0001] This invention relates to the field of equipment maintenance and inspection technology, specifically to a multimodal intelligent recommendation system for equipment maintenance and inspection knowledge that combines digital twins. Background Technology

[0002] With the increasing intelligence of industrial equipment, predictive maintenance (PHM) has become a core technology for ensuring the safe operation of equipment. However, existing technical solutions suffer from several drawbacks when implemented, including insufficient accuracy in multimodal feature fusion and Remaining Life (RUL) prediction, poor interpretability of fault tracing due to the lack of causal reasoning capabilities in knowledge graphs, and difficulty in balancing real-time performance with global optimization due to the fragmented data processing between edge and cloud.

[0003] To address the aforementioned shortcomings, this invention proposes a multimodal intelligent recommendation system for equipment maintenance knowledge that incorporates digital twins, thereby resolving the deficiencies of existing technical solutions. Summary of the Invention

[0004] The purpose of this invention is to provide a multimodal intelligent recommendation system for equipment maintenance knowledge that combines digital twins, in order to solve the technical problem that it is difficult to balance the static nature of recommendation strategies and the global optimization of edge-cloud collaboration in existing solutions.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A multimodal intelligent recommendation system for equipment maintenance knowledge combining digital twins includes:

[0007] Multimodal feature processing module: Based on real-time synchronized equipment operation data from digital twins, it predicts the remaining lifespan of key components through a long short-term memory network, and extracts vibration spectrum, temperature gradient, and image texture features that are strongly correlated with lifespan decay to construct a time-varying feature matrix;

[0008] Specifically, an LSTM network with an integrated attention mechanism is constructed, with preprocessed multimodal temporal features as input and remaining lifetime prediction as output.

[0009] The loss function is designed as follows: Where MSE() is the mean square error function; ω1 is the weighting coefficient of the MSE loss term; ω2 is the weighting coefficient of the decay rate deviation term; Δλ=λ pred -λ true , for predicting the attenuation rate λ pred Compared with the true attenuation rate λ true deviation, Indicates the rate of lifespan decay; RUL true This represents the actual remaining lifespan of the equipment.

[0010] Causal relationship enhancement processing module: It integrates predictive features and historical maintenance data to construct a two-layer knowledge graph containing fault causal chains. The bottom layer is the physical entity association layer, and the upper layer is the causal reasoning layer. It quantifies the causal strength between entities through a causal Bayesian network.

[0011] Hybrid Recommendation Strategy Dynamic Analysis Module: Based on the device's current RUL value and the causal path of the knowledge graph, reinforcement learning is used to dynamically adjust the weight ratio of content-based causal reasoning and collaborative filtering. The RUL value is analyzed and the causal reasoning priority mode is dynamically triggered. Based on the calculated recommendation adoption rate, the implementation effect of the dynamically triggered causal reasoning priority mode is actively monitored, evaluated and controlled.

[0012] Edge-cloud collaborative optimization module: Edge nodes process high-frequency sensor data in real time and generate preliminary recommendations, while the cloud performs causal path mining of the knowledge graph and optimization of recommendation model parameters based on global maintenance and inspection big data, and achieves parameter synchronization through federated learning.

[0013] Preferably, based on real-time data mapping between the digital twin and the physical device, time-series sensor data, visual data, and text data are collected;

[0014] The collected multimodal data is preprocessed and feature-enhanced to obtain a multimodal temporal feature set;

[0015] A long short-term memory network with an attention mechanism is constructed, with preprocessed multimodal temporal features as input and remaining lifetime prediction as output; the model is trained using full life cycle failure data generated by digital twin simulation.

[0016] Preferably, when constructing the time-varying feature matrix, the weighted feature vectors are arranged according to the time series, using the time window Δt as the unit, to construct the time-varying feature matrix X. time-var ying :X time-var ying =[F weighted (t0)F weighted (t0+Δt)...F weighted [(t0+nΔt)]; where F weighted t0 is the weighted eigenvector; n is the number of columns in the matrix; t0 is the initial time point.

[0017] Preferably, physical entities and their relationships are identified based on fused data, and an entity-relationship network is constructed through entity extraction, relationship definition, and entity association matrix construction.

[0018] Preferably, a causal inference layer is constructed by mining fault causal chains based on causal Bayesian networks, quantifying the causal strength between entities, and building a causal inference layer.

[0019] Preferably, the average causal effect ACE is calculated using the conditional probability table of a causal Bayesian network, with the formula: ACE(v i-v j )=P(v j =1|do(v i =1))-P(v j =1|do(v i =0); where, do(v i =1) represents the intervention operation, indicating the forced cause variable v i Occurs and is set to 1; do(v i =0), indicating that the forced cause variable v i It occurs, and its value is set to 0; P(v j =1|do(v i =1)) is the intervention v i When = 1, the result variable v j The probability of occurrence; P(v) j =1|do(v i =0) is the intervention v i When = 0, the result variable v j The probability of occurrence; ACE(v) i -v j ) represents the causal strength.

[0020] Preferably, when fusing the two-layer maps, the upper-layer causal variables are associated with the lower-layer physical entities; for each new batch of maintenance data or predicted features, the causal strength is updated using the following formula:

[0021] ACE new =α·ACE old +(1-α)·ACE new-data Where α is the historical weighting coefficient; ACE old Causal strength calculated before the addition of new data; ACE new-data The causal strength is calculated based on the newly added data.

[0022] Preferably, the reward function r is defined. t Quantify the effect of the recommendation strategy under the current weight combination, and guide the agent to learn the optimal weight adjustment strategy: r t =β·Acc pred -χ·C cost +δ·S path ·Π(RUL current ≤RUL threshold ); where Acc pred The accuracy of fault prediction for recommended maintenance strategies; C cost To maintain the normalized cost value; Π() is an indicator function, when RUL current ≤RUL threshold RUL takes 1 if the condition is met, otherwise takes 0. current For the current projected remaining lifetime, RULthreshold The threshold for triggering the causal reasoning priority mode; S path β represents the causal path strength; β, χ, and δ are all weighting coefficients.

[0023] Preferably, the time-varying feature matrix X is extracted based on compressed data. time-var ying Use a lightweight CB model to generate recommendations:

[0024]

[0025] Where, ω CB,edge ω CF,edge Fixed weights at the edge of the causal inference model and the collaborative filtering model, respectively; Sim CB,k , These represent the causal reasoning similarity and collaborative filtering score of the k-th recommendation item, respectively.

[0026] Preferably, when optimizing cloud-based recommendation model parameters and synchronizing with federated learning, the cloud optimizes the recommendation model parameters based on a global knowledge graph, and distributes the optimized parameters to edge nodes through federated learning. The FedProx algorithm is used to achieve cloud-edge parameter synchronization, and the edge nodes update the formula locally.

[0027] in, To update the edge node model parameters; For global model parameters in the cloud; L edge The loss is the local recommendation loss for edge nodes; τ is the gradient of the local loss function at the edge node; τ is the learning rate. For the proximal term coefficient.

[0028] Compared to existing solutions, the beneficial effects achieved by this invention are:

[0029] This invention achieves real-time data consistency between physical devices and virtual models through synchronous acquisition of multimodal data driven by digital twins; the long short-term memory network model integrating attention mechanism can effectively improve the prediction accuracy of the remaining life of key components through the decay rate deviation weighted loss function; the strong correlation feature selection and dynamic weighting based on mutual information entropy can reduce the interference of redundant features; and the use of time-varying feature matrix to capture the evolution law of device state over time can provide time-series structured data support for subsequent knowledge graph construction and reasoning recommendation.

[0030] This invention constructs a knowledge graph that combines the accuracy of physical entity association with the ability of causal reasoning, thereby achieving full-link interpretability from data, features, entities, causality, and decision-making. This can provide reliable knowledge support for fault diagnosis and life prediction in predictive maintenance.

[0031] This invention addresses the shortcomings of traditional hybrid recommendation systems that rely heavily on data association and neglect physical mechanisms by using reinforcement learning to dynamically balance and collaboratively filter weights. This is achieved by ensuring recommendation diversity while forcing a reliance on physical causality when the device is nearing the end of its lifespan.

[0032] This invention resolves the contradiction between the real-time processing requirements of high-frequency data and the dependence on global knowledge graphs in industrial equipment maintenance through a three-layer architecture of high-frequency response at the edge, global optimization in the cloud, and parameter synchronization of federated learning. It achieves multi-dimensional optimization goals of low latency, high accuracy, and privacy protection for recommendation strategies. Attached Figure Description

[0033] The invention will now be further described with reference to the accompanying drawings.

[0034] Figure 1 This is a flowchart illustrating the operation of a multimodal intelligent recommendation system for equipment maintenance knowledge that incorporates digital twins, as described in this invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] like Figure 1 As shown, this invention is a multimodal intelligent recommendation system for equipment maintenance knowledge combining digital twins, comprising:

[0037] Multimodal feature processing module: Based on real-time synchronized equipment operation data from a digital twin, it predicts the remaining lifespan of key components through a long short-term memory network and extracts vibration spectrum, temperature gradient, and image texture features strongly correlated with lifespan decay to construct a time-varying feature matrix; the specific steps include:

[0038] Based on real-time data mapping between digital twins and physical devices, time-series sensor data, visual data, and text data are collected.

[0039] Among them, the time-series sensing data includes vibration signals S acquired through an accelerometer. vib(t), where x′, y′, t is the temperature field data collected by the infrared sensor; where x′ and y′ are two-dimensional spatial coordinates, corresponding to the pixel positions of the temperature field image collected by the infrared sensor, x′∈[0,639], y′∈[0,511], determined by the resolution of the infrared sensor 640×512, representing the temperature distribution at different locations on the surface of the device; t is a time parameter, in seconds, representing the timestamp of temperature data collection, with a synchronization frequency ≥1kHz, consistent with the real-time data mapping frequency of the digital twin, used to capture the dynamic changes of the temperature field as the device runs;

[0040] Visual data: Images of equipment surfaces acquired by industrial cameras (I) vis (t), where P(x,y,z) is the 3D structural data acquired by the lidar; where x, y, and z correspond to the X-axis (horizontal), Y-axis (vertical), and Z-axis (depth) coordinates of the Cartesian coordinate system, respectively, acquired by the lidar, with a point cloud density of 100 points / mm. 2 After voxel mesh downsampling and RANSAC algorithm preprocessing, the structural contour features of the equipment are preserved for analysis of physical morphological changes such as equipment deformation and cracks.

[0041] Text data: Historical maintenance work orders (D) retrieved from the MES system. test This includes unstructured text such as fault descriptions and maintenance measures;

[0042] The collected multimodal data is preprocessed and feature-enhanced to obtain a multimodal temporal feature set containing vibration spectrum, visual feature sequence, and semantic sequence. The preprocessing and feature enhancement include vibration signal denoising, image feature enhancement, and text semantic encoding, which are all existing conventional technical solutions. The specific implementation steps are not described here.

[0043] Construct an LSTM network with an incorporating attention mechanism. LSTM stands for Long Short-Term Memory. The input is preprocessed multimodal temporal features, and the output is the predicted remaining lifetime.

[0044] The loss function is designed as follows: Where MSE() is the mean squared error function; ω1 is the weighting coefficient of the MSE loss term, with a default value of 0.7; ω2 is the weighting coefficient of the decay rate deviation term, with a default value of 0.3; Δλ=λ pred -λ true , for predicting the attenuation rate λ pred Compared with the true attenuation rate λ true deviation, Indicates the rate of lifespan decay; RUL true The tag value represents the actual remaining lifespan of the equipment, expressed in hours, and is obtained through a full lifecycle experiment or digital twin simulation.

[0045] The model is trained using full lifecycle fault data generated by digital twin simulation, and an early stopping method is used to prevent overfitting.

[0046] Among them, the full life cycle fault data includes 10 typical fault modes, which can be determined based on historical fault data.

[0047] When performing strong correlation feature selection and dynamic weighting, the mutual information entropy I(F, RUL) between each modal feature and the relative minimum information value (RUL) is calculated, where F is a single feature variable in the multimodal time-series feature set. Strongly correlated features with I(F, RUL) ≥ 0.8 are selected, specifically including:

[0048] Vibration signals: kurtosis, peak factor, 1 / 3 octave band energy;

[0049] Temperature data: Hotspot temperature gradient

[0050] Image features: percentage of the faulty area, edge gradient magnitude;

[0051] Weights are dynamically assigned based on feature importance: Where M is the total number of features, i is the feature index, used to identify the position of a single specific feature in the multimodal feature set, with a value range of i = 1, 2, ..., M, and j is the summation index, used to traverse all features, with a value range consistent with i, i.e., j = 1, 2, ..., M, to generate a weighted feature vector.

[0052] When constructing the time-varying feature matrix, the weighted feature vectors are arranged according to the time series, with a time window Δt = 5 minutes as the unit, to construct the time-varying feature matrix X. time-var ying :X time-var ying =[F weighted (t0)F weighted (t0+Δt)...F weighted [(t0+nΔt)]; where n is the number of columns in the matrix, RUL pred To predict the remaining lifetime, a dynamic characterization of the feature is achieved as the lifetime decays; t0 is the initial time point.

[0053] In this embodiment of the invention, multimodal data synchronous acquisition driven by digital twins can achieve real-time data consistency between physical devices and virtual models, and provide a precise data foundation for subsequent predictive maintenance; the long short-term memory network model that integrates attention mechanism can effectively improve the prediction accuracy of the remaining lifespan of key components through the decay rate deviation weighted loss function; the strong correlation feature selection and dynamic weighting based on mutual information entropy can reduce redundant feature interference, reduce feature dimensionality while maintaining high correlation with lifespan decay; and the use of time-varying feature matrix to capture the evolution law of device state over time can provide temporal structured data support for subsequent knowledge graph construction and reasoning recommendation.

[0054] Causal Relationship Enhancement Module: This module integrates predictive features with historical maintenance data to construct a two-layer knowledge graph containing fault causal chains. The bottom layer is the physical entity association layer, and the upper layer is the causal reasoning layer. It quantifies the causal strength between entities using a causal Bayesian network. Specific steps include:

[0055] Predictive features: Time-varying feature matrix X time-var ying The dynamic weighted feature F in weighted (t), such as vibration kurtosis, temperature gradient, etc.;

[0056] Historical maintenance data: Unstructured text data such as equipment maintenance records, fault reports, and sensor logs, for example, "2024-03-15, abnormal bearing noise, ball bearing replaced";

[0057] When integrating predictive features with historical maintenance data, numerical predictive features are Z-score standardized: in, The standardized eigenvalues; F i′ (t) represents the original observation value of the i′th feature at time t; μ i′ σ is the historical mean of feature i′; i′ The historical standard deviation of feature i′;

[0058] It should be noted that by unifying the data scale, the bias in the graph weights caused by differences in feature dimensions can be avoided; unstructured text data is preprocessed and aligned with numerical features, thus achieving cross-modal data fusion.

[0059] Based on the identification of physical entities and their relationships using fused data, an entity-relationship network is constructed through entity extraction, relationship definition, and entity association matrix construction.

[0060] When extracting entities, the process includes equipment, components, fault types, maintenance actions, and predictive features. Specifically, these include "spindle motor," "ball bearing," "wear," "replacement," and "kurtosis." Text entities are identified using a BiLSTM-CRF model, and structured feature names are extracted using regular expressions, such as extracting "ball bearing" from sensor logs.

[0061] When defining associations, we include attribute relationships, spatial relationships, and fault associations;

[0062] Attribute relationships: such as "ball bearing-attribute-kurtosis", indicating that kurtosis is a monitoring feature of the bearing;

[0063] Spatial relationships: such as "spindle motor - includes - ball bearing";

[0064] Fault association: such as "wear - occurs in - ball bearing";

[0065] When constructing the entity association matrix, the entity association strength matrix A∈R is defined. E×E E represents the total number of entities, and element A pq The normalized value representing the co-occurrence frequency of entities p and q:

[0066] Where Count(p, q) is the number of times entities p and q co-occur in the same maintenance record or feature window; max(Count(p, ·)) is the maximum number of times entity p co-occurs with all other entities, used to normalize A. pq ∈[0,1];

[0067] It should be noted that constructing the "entity-relationship" topology of the physical world, reflecting the objective correlation between devices, components, features, and faults, can provide the underlying entity network for upper-level causal reasoning; the correlation strength matrix A can be used as a weight prior for subsequent causal relationship mining.

[0068] Based on causal Bayesian networks, fault causal chains are mined, the causal strength between entities is quantified, and a causal inference layer is constructed.

[0069] Select key failure variables V = {v1, v2, ..., v3} from the underlying entities. K For example, "bearing wear", "overheating", "abnormal vibration", the variable status is two values: 1 indicates that it has occurred, and 0 indicates that it has not occurred;

[0070] The PC algorithm is used to learn the causal graph structure G = (V, E) from historical fault data, where edge e ij ∈E, meaning v i It is v j The direct cause; for example, "overheating - bearing wear";

[0071] It should be explained that the Peter-Clark (PC) algorithm is an algorithm used to learn the structure of causal graphs from observed data. The core idea of ​​the PC algorithm is to gradually build the skeleton of the causal network, i.e., the undirected graph, through conditional independence tests, and then use observable asymmetry to orient the edges, thereby obtaining a partially directed acyclic graph or a complete directed acyclic graph. The PC algorithm is usually divided into two main stages: the first stage: building the undirected graph skeleton; the second stage: edge orientation. The implementation of the PC algorithm is a conventional existing technical solution, and the specific implementation steps will not be elaborated here.

[0072] The average causal effect ACE is calculated using the conditional probability table of a causal Bayesian network. The formula is: ACE(v i -v j )=P(v j =1|do(v i =1))-P(v j =1|do(v i =0); where, do(v i =1) represents the intervention operation, indicating the forced cause variable v i It occurs and is set to 1, for example, by experimentally controlling the temperature to be too high; do(v i =0), indicating that the forced cause variable v i It occurs, and its value is set to 0; P(v j =1|do(v i =1)) is the intervention v i When = 1, the result variable v j The probability of occurrence; P(v) j =1|do(v i =0) is the intervention v i When = 0, the result variable v j The probability of occurrence; ACE(v) i -v j The absolute value represents the causal strength, ranging from [-1, 1]. The larger the absolute value, the stronger the causal relationship. A positive sign indicates positive causality, and a negative sign indicates inhibition.

[0073] It should be noted that by using causal Bayesian networks to distinguish between correlation and causation, such as "abnormal vibration" and "bearing wear" being causal relationships rather than just co-occurrence associations, the limitation of "correlation ≠ causation" in traditional knowledge graphs can be overcome.

[0074] Quantifying the causal strength by using the ACE value can provide priority for fault tracing. For example, "overheating-wear" with ACE=0.8 should be investigated first.

[0075] When merging two-layer maps, the upper-layer causal variables are associated with the lower-layer physical entities. For example, "bearing wear" corresponds to the lower-layer entities "wear" and "ball bearing".

[0076] For each new batch of maintenance data or predicted features, the causal strength is updated using the following formula:

[0077] ACE new =α·ACE old +(1-α)·ACE new-data Where α is the historical weighting coefficient, used to balance the stability of old data and the timeliness of new data, and its value ranges from [0.5, 0.8]; ACE old Causal strength calculated before the addition of new data; ACE new-data The causal strength is calculated based on the newly added data;

[0078] It is worth noting that the two-layer structure enables the transition between physical entity association and causal reasoning. It not only preserves the objective attribute relationships of entities, but also enables the "root cause localization" of faults through upper-layer rules, such as tracing from "abnormal equipment noise" to "ball wear". The dynamic update mechanism allows the map to be continuously optimized as equipment operation data accumulates, and can adapt to dynamic scenarios such as equipment aging and changes in operating conditions in a timely and reliable manner.

[0079] In this embodiment of the invention, by constructing a knowledge graph that combines the accuracy of physical entity association with the ability of causal reasoning, the interpretability of the entire chain from data, features, entities, causality, and decision-making is realized, which can provide reliable knowledge support for fault diagnosis and life prediction in predictive maintenance.

[0080] The hybrid recommendation strategy dynamic analysis module: Based on the device's current RUL value and the causal path of the knowledge graph, it uses reinforcement learning to dynamically adjust the weight ratio of content-based causal inference and collaborative filtering, analyzes the RUL value, dynamically triggers the causal inference priority mode, and actively monitors, evaluates, and controls the implementation effect of the dynamically triggered causal inference priority mode based on the calculated recommendation adoption rate; the specific steps include:

[0081] The current RUL value refers to the predicted remaining useful life of the equipment at the current moment.

[0082] We extract input features for the recommendation system from knowledge graphs and real-time device data, and define the state space for reinforcement learning:

[0083] Among them, RUL current For the current projected remaining lifetime; S path D represents the causal path strength. entity For entity association density; Sim CB Cosine similarity for causal reasoning models; For score prediction in the collaborative filtering model, the collaborative filtering score is based on the maintenance records of similar equipment; ω CB，t-1 ω CF，t-1 These are the weights of the causal inference model and the collaborative filtering model at the previous time step, respectively, with an initial value of 0.5 for both.

[0084] It needs to be explained that the causal reasoning model: by analyzing the equipment's own characteristic attributes, such as sensor data, fault types, maintenance records, etc., it calculates the similarity between the current equipment and historical fault cases, and recommends maintenance strategies for similar cases;

[0085] Collaborative filtering model: It does not rely on the device characteristics themselves, but finds a group of devices with similar behaviors by analyzing the historical maintenance behavior of a large number of devices or users, and uses the maintenance records of the group to recommend strategies for the current device;

[0086] The construction and training of causal reasoning models and collaborative filtering models are both existing conventional technical solutions, and the specific implementation steps will not be elaborated here;

[0087] It should be noted that integrating device health status, causal information from the knowledge graph, and the output of the recommendation model into the state space can provide a comprehensive basis for dynamic weight adjustment, avoid recommendation bias caused by a single feature, and effectively improve the robustness of the strategy.

[0088] The action space of reinforcement learning is defined as the continuous adjustment range of causal inference weights and collaborative filtering weights, and a weight update formula based on RUL and causal path is designed:

[0089] The agent's action a t The adjustment amount Δω for the weights of the causal inference model CB ,satisfy:

[0090] a t ∈[-0.1,0.1], ω CB，t =clip(ω CB，t-1 (0, 1); where the collaborative filtering weights automatically satisfy ω CF,t =1-ω CB,t ;Δω CB The step size for each weight adjustment is limited to ±0.1 to avoid drastic fluctuations; clip() is a truncation function to ensure ω... CB,t ∈[0,1];

[0091] The initial weights are adaptively initialized using RUL and causal path strength:

[0092]

[0093] Among them, RUL threshold The threshold for triggering the causal reasoning priority mode; Spath S represents the strength of the strongest causal path in the knowledge graph. path ∈[0,1];

[0094] It should be explained that action space constraints can ensure smooth weight adjustments and avoid abrupt changes in recommendation policies; by designing prior logic with lower RUL, stronger causal paths, and higher weights for causal inference, a reasonable initial point can be provided for reinforcement learning.

[0095] Define reward function r t Quantify the effectiveness of the recommendation strategy under the current weight combination, and guide the agent to learn the optimal weight adjustment strategy:

[0096] r t =β·Acc pred -χ·C cost +δ·S path ·Π(RUL current ≤RUL threshold );

[0097] Among them, Acc pred The accuracy of fault prediction for recommended maintenance strategies; C cost To maintain the normalized cost value; Π() is an indicator function, when RUL current ≤RUL threshold 1 is used when the condition is met, and 0 is used otherwise; β, χ, and δ are all weighting coefficients with initial values ​​of 0.6, 0.3, and 0.1, respectively. Accuracy is prioritized, followed by cost reduction, and finally, causal reasoning is encouraged. The accuracy-cost-causal reasoning approach avoids excessive maintenance due to a single pursuit of accuracy, or ignoring physical causality by relying on collaborative filtering, such as similar devices with different failure mechanisms.

[0098] When the remaining lifespan of the equipment is below a threshold, the weighting formula is dynamically adjusted to forcibly enhance the causal reasoning recommendation weights guided by causal reasoning:

[0099] Among them, real-time monitoring of RUL current If RUL is satisfied current ≤RUL threshold Then, it enters priority mode and updates the weights as follows: in, For the causal reasoning weight in priority mode, the maximum value of the current causal reasoning weight and the causal path strength is taken to ensure that causal reasoning dominates;

[0100] In the priority mode, the causal reasoning recommendation results are further filtered through the causal chain of the knowledge graph:

[0101] Among them, Rec CBThe candidate recommendation list generated for the causal inference model, such as "replace bearing" and "clean lubrication system"; path(rec) is the causal path corresponding to the recommendation item, such as "replace bearing" corresponding to the path "overheating-wear-replacement";

[0102] When actively monitoring, evaluating, and controlling the implementation effect of the dynamic triggering causal reasoning priority mode, the recommendation adoption rate corresponding to all generated candidate recommendation lists is calculated;

[0103] The recommendation adoption rate is calculated by dividing the total number of valid recommendations adopted by the total number of invalid recommendations adopted. Valid recommendations adopted refer to the adoption of recommendations from the candidate recommendation list. Invalid recommendations adopted refer to the failure to adopt recommendations from the candidate recommendation list.

[0104] When analyzing the adoption rate of recommendations to determine the effectiveness of the dynamic triggering causal reasoning priority mode, if the adoption rate is greater than or equal to the adoption threshold, the effectiveness of the dynamic triggering causal reasoning priority mode is considered to be normal, and the application of the existing priority mode is maintained. The specific value of the adoption threshold is not limited; it can be determined based on the parameters required for the model's operation or by professionals in the field based on their work experience and requirements.

[0105] If the recommendation adoption rate is less than the recommendation adoption threshold, the implementation effect of the dynamic triggering causal reasoning priority mode is determined to be abnormal, and the application of the existing priority mode is optimized and adjusted.

[0106] This includes optimizing and adjusting the application of existing priority modes, including but not limited to optimizing and adjusting existing recommendation strategy rules and existing prior logic.

[0107] In this embodiment of the invention, by using reinforcement learning to dynamically balance the weights of collaborative filtering, while ensuring the diversity of recommendations, the device is forced to rely on physical causality when it is nearing the end of its lifespan. This can solve the problem of traditional hybrid recommendations that emphasize data association but neglect physical mechanisms, effectively reducing maintenance costs and improving the accuracy of fault prediction.

[0108] Edge-cloud collaborative optimization module: Edge nodes process high-frequency sensor data in real time and generate preliminary recommendations, while the cloud performs causal path mining of the knowledge graph and optimizes recommendation model parameters based on global maintenance and inspection big data, achieving parameter synchronization through federated learning. Specific steps include:

[0109] Wavelet threshold compression was applied to the raw sensor data to preserve key features: Where W(x(t)) are wavelet transform coefficients, using the db4 wavelet basis, with 3 decomposition levels; λ is the threshold value. σ is the noise standard deviation, and N is the data length;

[0110] In addition, the compression ratio CR = original data size / compressed data size, with a typical value of CR = 10:1; for example, 1GB compressed to 100MB.

[0111] Extracting the time-varying feature matrix X from compressed data time-var ying Use a lightweight CB model to generate recommendations:

[0112]

[0113] Where, ω CB，edge ω CF，edge Fixed weights at the edge of the causal inference model and the collaborative filtering model, respectively; Sim CB，k , These are the causal reasoning similarity and collaborative filtering score for the k-th recommendation item, respectively.

[0114] The cloud platform receives compressed data and preliminary recommendation results uploaded by edge nodes, integrates them with global maintenance and inspection big data, and updates the causal paths of the knowledge graph:

[0115] Cloud-integrated compressed data from N edge nodes Construct the global feature matrix:

[0116]

[0117] By aggregating the local model parameters of edge nodes through federated averages, the direct uploading of raw data is avoided, thus achieving privacy protection.

[0118] The average causal effect (ACE) of causal paths in the knowledge graph is recalculated based on global data. The formula is:

[0119] Among them, ACE global The updated global causal path strength is used to correct the cloud-based knowledge graph; The local causality strength calculated for the i-th edge node; ω i These are the weighting coefficients;

[0120] It should be explained that global data fusion can avoid data silos and improve the statistical significance of causal path mining, such as ACE bias caused by insufficient sample size of a single edge node.

[0121] When optimizing cloud-based recommendation model parameters in sync with federated learning, the cloud optimizes the recommendation model parameters based on a global knowledge graph and distributes the optimized parameters to edge nodes through federated learning.

[0122] Global causal path strength ACEglobal To minimize the recommendation loss function L, we need to constrain the process. cloud :

[0123] in, This is the label for the actual maintenance effect of the i-th edge node, where 1 indicates valid and 0 indicates invalid; ACE pred The strength of the causal path predicted by the model; η is the regularization coefficient, which balances recommendation accuracy and causal consistency.

[0124] The FedProx algorithm is used to achieve cloud-edge parameter synchronization, and the local update formula for edge nodes is as follows:

[0125] in, To update the edge node model parameters; For global model parameters in the cloud, such as ω CB ω CF L edge The loss is the local recommendation loss for edge nodes; The gradient of the local loss function for edge nodes; τ is the learning rate, with a default value of 0.01; This is the coefficient for the proximal term, which controls the deviation between the edge parameters and the cloud parameters. The default value is 0.001.

[0126] It should be noted that regularization terms are used to ensure that the optimization direction of the recommendation model is consistent with the causal path of the knowledge graph, thus avoiding the disconnect between data-driven recommendation and physical mechanisms.

[0127] The FedProx algorithm mitigates parameter divergence caused by data heterogeneity at edge nodes through proximal terms, such as differences in failure modes among different devices, and can effectively improve synchronization stability.

[0128] It should be explained that in the FedProx algorithm, in order to deal with the problems of data heterogeneity and system heterogeneity in federated learning, a concept called proximal is introduced. This proximal term is mainly used to limit the difference between the client model update and the global model, thereby improving the stability and convergence of the training process.

[0129] In this embodiment of the invention, a three-layer architecture of high-frequency response at the edge, global optimization in the cloud, and synchronization of federated learning parameters can resolve the contradiction between the need for real-time processing of high-frequency data and the dependence on global knowledge graphs in industrial equipment maintenance, and achieve the multi-dimensional optimization goals of low latency, high accuracy, and privacy protection for recommendation strategies.

[0130] In the several embodiments provided by this invention, it should be understood that the disclosed methods can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative; for instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation.

[0131] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0133] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the essential characteristics of the present invention.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multimodal equipment maintenance knowledge intelligent recommendation system combining digital twins, characterized in that, include: Multimodal feature processing module: Based on real-time synchronized equipment operation data from digital twins, it predicts the remaining lifespan of key components through a long short-term memory network, and extracts vibration spectrum, temperature gradient, and image texture features that are strongly correlated with lifespan decay to construct a time-varying feature matrix; Specifically, an LSTM network with an integrated attention mechanism is constructed, with preprocessed multimodal temporal features as input and remaining lifetime prediction as output. ; The loss function is designed as follows: Where MSE() is the mean squared error function; These are the weighting coefficients for the MSE loss term; This is the weighting coefficient for the attenuation rate deviation term; To predict the decay rate Compared with the actual decay rate deviation, , representing the rate of lifetime decay; This represents the actual remaining lifespan of the equipment. Additionally, calculate the mutual information entropy between each modal feature and the RUL. F is a single feature variable in the multimodal time series feature set, and the selection... Strong correlation features ≥0.8; Causal relationship enhancement processing module: It integrates predictive features and historical maintenance data to construct a two-layer knowledge graph containing fault causal chains. The bottom layer is the physical entity association layer, and the upper layer is the causal reasoning layer. It quantifies the causal strength between entities through a causal Bayesian network. Hybrid Recommendation Strategy Dynamic Analysis Module: Based on the device's current RUL value and the causal path of the knowledge graph, reinforcement learning is used to dynamically adjust the weight ratio of content-based causal reasoning and collaborative filtering. The RUL value is analyzed and the causal reasoning priority mode is dynamically triggered. Based on the calculated recommendation adoption rate, the implementation effect of the dynamically triggered causal reasoning priority mode is actively monitored, evaluated and controlled. The recommendation adoption rate is calculated by dividing the total number of valid recommendations adopted by the total number of invalid recommendations adopted. Valid recommendations adopted refer to the adoption of recommendations from the candidate recommendation list. Invalid recommendations adopted refer to the failure to adopt recommendations from the candidate recommendation list. When analyzing the recommendation adoption rate to determine the implementation effect of the dynamic triggering causal reasoning priority mode, if the recommendation adoption rate is greater than or equal to the recommendation adoption threshold, the implementation effect of the dynamic triggering causal reasoning priority mode is determined to be normal, and the application of the existing priority mode is maintained. If the recommendation adoption rate is less than the recommendation adoption threshold, the implementation effect of the dynamic triggering causal reasoning priority mode is determined to be abnormal, and the application of the existing priority mode is optimized and adjusted. Edge-cloud collaborative optimization module: Edge nodes process high-frequency sensor data in real time and generate preliminary recommendations, while the cloud performs causal path mining of the knowledge graph and optimization of recommendation model parameters based on global maintenance and inspection big data, and achieves parameter synchronization through federated learning.

2. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins as described in claim 1, characterized in that, Based on real-time data mapping between digital twins and physical devices, time-series sensor data, visual data, and text data are collected. The collected multimodal data is preprocessed and feature-enhanced to obtain a multimodal temporal feature set; A long short-term memory network with an attention mechanism is constructed. The input is preprocessed multimodal temporal features, and the output is the remaining lifetime prediction. The model is trained using full lifecycle fault data generated through digital twin simulation.

3. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins as described in claim 2, characterized in that, When constructing the time-varying feature matrix, a time window is used. Using units as the unit, the weighted eigenvectors are arranged according to the time series to construct a time-varying feature matrix. ;in, These are the weighted eigenvectors; n is the number of columns in the matrix; This is the initial time point.

4. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins according to claim 3, characterized in that, Based on the identification of physical entities and their relationships using fused data, an entity-relationship network is constructed through entity extraction, relationship definition, and entity association matrix construction.

5. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins according to claim 4, characterized in that, Based on causal Bayesian networks, fault causal chains are mined, the causal strength between entities is quantified, and a causal inference layer is constructed.

6. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins according to claim 5, characterized in that, The average causal effect ACE is calculated using the conditional probability table of a causal Bayesian network, using the following formula: ;in, For intervention operations, it represents the forced causal variable. It occurs, and its value is set to 1; , indicating a forced causal variable It occurs, and its value is set to 0; For intervention Time result variable The probability of occurrence; For intervention Time result variable The probability of occurrence; The strength of causality.

7. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins as described in claim 6, characterized in that, When fusing two-layer maps, the upper-layer causal variables are associated with the lower-layer physical entities; for each new batch of maintenance data or predicted features, the causal strength is updated using the following formula: ;in, Historical weighting coefficients; Causal strength calculated before the addition of new data; The causal strength is calculated based on the newly added data.

8. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins according to claim 7, characterized in that, Define reward function Quantify the effectiveness of the recommendation strategy under the current weight combination, and guide the agent to learn the optimal weight adjustment strategy: ; in, The accuracy of fault prediction for recommended maintenance strategies; To maintain the normalized cost value; For indicator functions, when The value is 1 if the condition is met, and 0 otherwise. For the current prediction of remaining lifespan, The threshold for triggering the causal reasoning priority mode; For causal path strength; All are weighting coefficients.

9. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins as described in claim 8, characterized in that, Extracting Time-Varying Feature Matrix from Compressed Data Use a lightweight CB model to generate recommendations: ;in, The edge weights are fixed for the causal inference model and the collaborative filtering model, respectively. These represent the causal reasoning similarity and collaborative filtering score of the k-th recommendation item, respectively.

10. The intelligent recommendation system for multimodal equipment maintenance knowledge combining digital twins according to claim 9, characterized in that, When optimizing cloud-based recommendation model parameters and synchronizing with federated learning, the cloud optimizes the recommendation model parameters based on the global knowledge graph and distributes the optimized parameters to edge nodes through federated learning. The FedProx algorithm is used to achieve cloud-edge parameter synchronization.

Images