Digital twinning-combined multi-modal equipment maintenance and inspection knowledge intelligent recommendation system

Abstract

The invention discloses a multi-mode equipment maintenance and inspection knowledge intelligent recommendation system combined with digital twinning, and belongs to the technical field of equipment maintenance and inspection. The method is used for solving the technical problem that in an existing scheme, recommendation strategy staticizing and edge cloud collaborative global optimization are difficult to consider at the same time. According to the method, multi-modal data synchronous acquisition driven by digital twinning is carried out, a long-short-term memory network model of an attention mechanism is fused, a loss function is weighted through an attenuation rate deviation, strong correlation feature screening and dynamic weighting based on mutual information entropy are carried out, and a time-varying feature matrix is utilized to capture an evolution rule of an equipment state along with time; a knowledge graph with physical entity association precision and causal reasoning ability is constructed, and a three-layer architecture including full-link interpretability of data, features, entities, causals and decisions, edge-end high-frequency response-cloud global optimization-federated learning parameter synchronization is realized. The contradiction between high-frequency data real-time processing requirements and global knowledge graph dependence in industrial equipment maintenance can be solved.

Description

Technical Field

[0001] The present invention relates to the field of equipment maintenance technology, and in particular to a multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twins. Background Art

[0002] As industrial equipment becomes increasingly intelligent, predictive maintenance (PHM) has become a core technology for ensuring safe equipment operation. Existing technical solutions suffer from shortcomings such as insufficient accuracy in multimodal feature fusion and remaining useful life (RUL) prediction, poor interpretability of fault tracing due to a lack of causal reasoning capabilities in knowledge graphs, and the fragmentation of edge and cloud data processing, making it difficult to balance real-time performance and global optimization.

[0003] In response to the above-mentioned defects, the present invention proposes a multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twins to solve the defects of the existing technical solutions. Summary of the Invention

[0004] The purpose of the present invention is to provide a multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twins, which is used to solve the technical problems in existing solutions that are difficult to balance the static nature of recommendation strategies and the global optimization of edge-cloud collaboration.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twins, including:

[0007] Multimodal feature processing module: Based on the real-time synchronized equipment operation data of the digital twin, the module predicts the remaining life of key components through a long short-term memory network, extracts vibration spectrum, temperature gradient, and image texture features that are strongly correlated with life attenuation, and constructs a time-varying feature matrix.

[0008] Among them, an LSTM network integrating attention mechanism is constructed, the input is the preprocessed multimodal time series feature, and the output is the remaining life prediction value

[0009] The loss function is designed as: Where MSE() is the mean square error function; ω1 is the weight coefficient of the MSE loss term; ω2 is the weight coefficient of the attenuation rate deviation term; Δλ=λ pred -λ true , is the predicted decay rate λ pred and the true decay rate λ true The deviation, Indicates the life decay rate; RUL true The actual remaining life of the equipment;

[0010] Causal relationship enhancement processing module: fuse predictive features and historical maintenance data, build a double-layer knowledge graph containing fault causal chain, the bottom layer is a physical entity association layer, and the upper layer is a causal reasoning layer, and the causal strength between entities is quantified through a causal Bayesian network;

[0011] Mixed recommendation strategy dynamic analysis module: based on the current RUL value of the device and the causal path of the knowledge graph, the weight proportion of content-based causal reasoning and collaborative filtering is dynamically adjusted by reinforcement learning, the RUL value is analyzed, and the causal reasoning priority mode is dynamically triggered, and the implementation effect of the dynamically triggered causal reasoning priority mode is actively supervised and evaluated and controlled based on the calculated recommendation adoption degree;

[0012] Edge cloud collaborative optimization module: edge nodes process high-frequency sensor data in real time and generate preliminary recommendations, the cloud performs causal path mining and recommendation model parameter optimization of the knowledge graph based on global maintenance big data, and parameter synchronization is achieved through federated learning.

[0013] Preferably, based on the real-time data mapping of the digital twin and the physical device, time series sensor data, visual data and text data are collected;

[0014] The collected multi-modal data is preprocessed and feature enhanced to obtain a multi-modal time series feature set;

[0015] A long short-term memory network with an attention mechanism is constructed, the input is the preprocessed multi-modal time series feature, and the output is the remaining life prediction value; the model is trained by the full life cycle fault data generated by digital twin simulation.

[0016] Preferably, when constructing the time-varying feature matrix, the weighted feature vector is arranged in time sequence in units of time window Δt, and the time-varying feature matrix X time-var ying is constructed. time-var ying : X weighted = [F weighted (t0)F weighted (t0+Δt)...F weighted (t0+nΔt)]; wherein F is a weighted feature vector; n is the number of matrix rows; t0 is the initial time point.

[0017] Preferably, based on the fusion data, the physical entities and the associated relationships are identified, and an entity-relation network is constructed through entity extraction, associated relationship definition and entity association matrix construction.

[0018] Preferably, based on the causal Bayesian network, the fault causal chain is mined, and the causal strength between entities is quantified to construct a causal reasoning layer.

[0019] Preferably, the average causal effect ACE is calculated through the conditional probability table of the causal Bayesian network, and the formula is: ACE(v i-v j ) = P(v j = 1 | do(v i = 1)) - P(v j = 1 | do(v i = 0)); wherein, do(v i = 1) is an intervention operation, indicating that the cause variable v i occurs and its value is forced to be 1; do(v i = 0) is an intervention operation, indicating that the cause variable v i occurs and its value is forced to be 0; P(v j = 1 | do(v i = 1)) is the probability of the result variable v i occurs when the intervention v j = 1; P(v j = 1 | do(v i = 0)) is the probability of the result variable v i occurs when the intervention v j = 0; ACE(v i -v j ) is the causal strength.

[0020] Preferably, when the double-layer graph is fused, the upper-layer causal variable is associated with the bottom-layer physical entity; and each time a batch of new maintenance data or predicted features is added, the causal strength is updated by the following formula:

[0021] ACE new = a ACE old + (1-a) ACE new-data ; wherein, a is a historical weight coefficient; ACE old is the causal strength calculated before the new data is added; and ACE new-data is the causal strength calculated based on the new data.

[0022] Preferably, a reward function r t is defined to quantify the effect of the recommended strategy under the current weight combination, and to guide the agent to learn the optimal weight adjustment strategy: r t = β Acc pred - χ C cost + δ S path · Π (RUL current ≤ RUL threshold ); wherein, Acc pred is the fault prediction accuracy of the recommended maintenance strategy; C cost is a maintenance cost normalized value; Π() is an indicator function, taking 1 when RUL current ≤ RUL threshold , and 0 otherwise, RUL current is the current predicted remaining useful life, RULthreshold is a threshold value for triggering the causal reasoning priority mode; S path is a causal path strength; β, χ, δ are weight coefficients.

[0023] Preferably, the time-varying feature matrix X is extracted based on compressed data time-var ying , a lightweight CB model is called to generate recommendations:

[0024]

[0025] wherein ω CB,edge , ω CF,edge are edge fixed weights of the causal reasoning model and the collaborative filtering model respectively; Sim CB,k , are the causal reasoning similarity and the collaborative filtering score of the kth recommended item respectively.

[0026] Preferably, when the cloud recommendation model parameter optimization and the federated learning are synchronized, the cloud optimizes the recommendation model parameters based on the global knowledge graph, and the optimized parameters are distributed to the edge nodes through the federated learning, the FedProx algorithm is used to realize the cloud-edge parameter synchronization, and the edge node local update formula is as follows:

[0027] wherein, is the updated edge node model parameter; is the cloud global model parameter; L edge is the edge node local recommendation loss; is the gradient of the edge node local loss function; τ is the learning rate; is the proximal item coefficient.

[0028] Compared with the prior art, the present application has the following beneficial effects:

[0029] The present application can realize the real-time data consistency of physical equipment and virtual models through the digital twin driven multi-modal data synchronous acquisition, the long short-term memory network model fused with the attention mechanism can effectively improve the prediction accuracy of the remaining life of the key components through the attenuation rate bias weighted loss function, the strong correlation feature screening and dynamic weighting based on mutual information entropy can reduce the interference of redundant features, and the time-varying feature matrix can capture the evolution law of the equipment state with time, thereby providing time sequence structured data support for subsequent knowledge graph construction and reasoning recommendation.

[0030] The present application can realize the full-link explainability from data, features, entities, causality and decision by constructing a knowledge graph with physical entity correlation accuracy and causal reasoning capability, and can provide reliable knowledge support for fault diagnosis and life prediction in predictive maintenance.

[0031] By strengthening learning dynamic balance and collaborative filtering weights, the present invention ensures recommendation diversity while forcing reliance on physical causal relationships when the device approaches the end of its life, thus addressing the defects of traditional hybrid recommendation that focuses on data association but neglects physical mechanisms.

[0032] The present invention uses a three-layer architecture of high-frequency response at the edge, global optimization in the cloud, and synchronization of federated learning parameters to resolve the contradiction between the real-time processing requirements of high-frequency data in industrial equipment maintenance and the reliance on the global knowledge graph, and achieve the multi-dimensional optimization goals of low latency, high precision, and privacy protection for recommendation strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings.

[0034] Figure 1 This is a flowchart of the operation of a multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twins in the present invention. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0036] like Figure 1 As shown, the present invention is a multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twins, including:

[0037] Multimodal feature processing module: Based on the real-time synchronized equipment operation data of the digital twin, the module predicts the remaining life of key components through a long short-term memory network, extracts vibration spectrum, temperature gradient, and image texture features that are strongly correlated with life attenuation, and constructs a time-varying feature matrix. The specific steps include:

[0038] Based on real-time data mapping between digital twins and physical devices, collect time-series sensor data, visual data, and text data;

[0039] Among them, time series sensing data: vibration signal S is collected by acceleration sensor vib(t), the infrared sensor collects temperature field data T(x', y', t); wherein x' and y' are two-dimensional space coordinates, corresponding to the pixel position of the temperature field image collected by the infrared sensor, x'∈[0, 639], y'∈[0, 511], determined by the resolution of the infrared sensor 640*512, representing the temperature distribution at different positions on the device surface; t is the time parameter, unit: second, representing the time stamp of temperature data collection, the synchronous frequency is greater than or equal to 1 kHz, consistent with the real-time data mapping frequency of the digital twin, used to capture the dynamic changes of the temperature field with the running time of the device;

[0040] Visual data: industrial camera collects device surface image I vis (t), the laser radar collects three-dimensional structure data P(x, y, z); wherein x, y and z respectively correspond to the X axis (lateral), Y axis (longitudinal) and Z axis (depth) coordinates of the Cartesian coordinate system, collected by the laser radar, the point cloud density is 100 points / mm 2 After down-sampling by voxel grid and pre-processing by RANSAC algorithm, the device structure contour features are retained, which are used to analyze the physical morphological changes such as device deformation and crack;

[0041] Text data: obtain historical maintenance work order D from MES system test , including fault description, maintenance measures and other unstructured text;

[0042] The collected multi-modal data are pre-processed and feature enhanced to obtain a multi-modal time series feature set containing vibration spectrum, visual feature sequence and semantic sequence; wherein the pre-processing and feature enhancement include vibration signal denoising, image feature enhancement and text semantic coding, which are all existing conventional technical solutions, and the specific implementation steps are not described here;

[0043] An LSTM network with attention mechanism is constructed, wherein LSTM is long short-term memory network, the input is the pre-processed multi-modal time series feature, and the output is the remaining useful life prediction value

[0044] Wherein, the loss function is designed as: Wherein, MSE() is the mean square error function; ω1 is the weight coefficient of the MSE loss term, the default value is 0.7; ω2 is the weight coefficient of the decay rate bias term, the default value is 0.3; Δλ = λ pred -λ true , the predicted decay rate λ pred and the deviation of the real decay rate λ true , Indicates the life decay rate; RUL true is the real remaining useful life of the device, unit: hour, the label value is obtained by full life cycle experiment or digital twin simulation;

[0045] The model is trained using full lifecycle fault data generated by digital twin simulation, using early stopping to prevent overfitting.

[0046] Among them, the full life cycle failure data includes 10 typical failure modes, and the specific ones can be determined based on historical failure data;

[0047] When performing strong correlation feature screening and dynamic weighting, the mutual information entropy I(F, RUL) between each modal feature and RUL is calculated. F is a single feature variable in the multimodal time series feature set. Strong correlation features with I(F, RUL) ≥ 0.8 are screened, including:

[0048] Vibration signal: kurtosis, crest factor, 1 / 3 octave energy;

[0049] Temperature data: hotspot temperature gradient

[0050] Image features: fault area ratio, edge gradient amplitude;

[0051] Dynamically assign weights based on feature importance: Among them, M is the total number of features, i is the feature index, which is used to identify the position of a single specific feature in the multimodal feature set, and the value range is i = 1, 2, ..., M. j is the summation index, which is used to traverse all features, and the value range is consistent with i, that is, j = 1, 2, ..., M, to generate a weighted feature vector

[0052] When constructing the time-varying feature matrix, the weighted feature vectors are arranged in time series with the time window Δt = 5 minutes as the unit to construct the time-varying feature matrix X time-var ying :X time-var ying =[F weighted (t0)F weighted (t0+Δt)...F weighted (t0+nΔt)]; where n is the number of matrix columns, RUL pred To predict the remaining life, the dynamic characterization of the characteristics along with the life decay process is realized; t0 is the initial time point.

[0053] In the embodiment of the application, through digital twin driven multi-modal data synchronous acquisition, real-time data consistency of physical equipment and virtual model can be realized, and accurate data basis is provided for subsequent predictive maintenance; through the attenuation rate bias weighted loss function of the long short-term memory network model fused with the attention mechanism, the prediction accuracy of the remaining life of the key components can be effectively improved; based on the strong correlation feature screening and dynamic weighting based on mutual information entropy, the interference of redundant features can be reduced, the feature dimension is reduced while maintaining high correlation with life decay; the time-varying feature matrix is used to capture the evolution law of the equipment state with time, which can provide time sequence structured data support for subsequent knowledge graph construction and reasoning recommendation.

[0054] Causal relationship enhancement processing module: fuse predictive features and historical maintenance data, build a double-layer knowledge graph containing fault causal chain, the bottom layer is a physical entity correlation layer, and the upper layer is a causal reasoning layer, and the causal strength between entities is quantified through a causal Bayesian network; the specific steps include:

[0055] Predictive features: time-varying feature matrix X time-var ying Dynamic weighted features F weighted (t), such as vibration kurtosis, temperature gradient, etc.

[0056] Historical maintenance data: device maintenance records, fault reports, sensor logs, and other unstructured text data, such as "2024-03-15, bearing abnormal noise, replace the ball";

[0057] When fusing predictive features and historical maintenance data, the numerical predictive features are standardized by Z-score: Wherein, is the standardized feature value; F i′ (t) is the original observation value of the i'th feature at time t; μ i′ is the historical mean of feature i'; and σ i′ is the historical standard deviation of feature i';

[0058] It should be noted that by unifying the data scale, the graph weight deviation caused by the dimensional difference of the features can be avoided; the unstructured text data is aligned with the numerical features through preprocessing, realizing cross-modal data fusion.

[0059] Based on the fused data, physical entities and correlation relationships are identified, and an entity-relation network is constructed through entity extraction, correlation relationship definition, and entity correlation matrix construction;

[0060] Entity extraction includes equipment, components, fault types, maintenance actions, and predictive features. Specifically, examples include "spindle motor," "ball bearing," "wear," "replacement," and "kurtosis." The BiLSTM-CRF model is used to identify text entities, and regular expressions are used to extract structured feature names, such as "ball bearing" from sensor logs.

[0061] When defining association relationships, attribute relationships, spatial relationships, and fault relationships are included;

[0062] Attribute relationship: For example, "ball bearing-attribute-kurtosis" indicates that kurtosis is a monitoring feature of the bearing;

[0063] Spatial relationship: such as "spindle motor - including - ball bearing";

[0064] Fault association: such as "wear-occurs-on-ball bearing";

[0065] When constructing the entity association matrix, define the entity association strength matrix A∈R E×E , E is the total number of entities, element A pq Represents the normalized co-occurrence frequency of entities p and q:

[0066] Where Count(p, q) is the number of times entities p and q co-occur in the same maintenance record or feature window; max(Count(p, )) is the maximum number of times entity p co-occurs with all other entities, which is used to normalize A pq ∈[0,1];

[0067] It should be noted that constructing the "entity-relationship" topological structure of the physical world reflects the objective associations between devices, components, features, and faults, and can provide an underlying entity network for upper-level causal reasoning; the association strength matrix A can be used as a weight prior for subsequent causal relationship mining.

[0068] Mining fault causal chains based on causal Bayesian networks, quantifying the causal strength between entities, and building a causal reasoning layer;

[0069] Filter key fault variables V = {v1, v2, ..., v K For example, "bearing wear", "overtemperature", and "abnormal vibration", the variable state is binary: 1 means it occurs, and 0 means it does not occur;

[0070] Use the PC algorithm to learn the causal graph structure G = (V, E) from historical failure data, where the edge e ij ∈E, indicating v i It is v j Direct cause; for example, "excessive temperature - bearing wear";

[0071] It needs to be explained that the PC algorithm (Peter-Clark Algorithm) is an algorithm for learning the structure of a causal graph from observation data; the core idea of the PC algorithm is to gradually build the skeleton of a causal network, that is, an undirected graph, through conditional independence tests, and then use observable asymmetry to orient the edges, so as to obtain a partially directed acyclic graph or a complete directed acyclic graph; the PC algorithm is usually divided into two main stages: the first stage: building the skeleton of an undirected graph; the second stage: edge orientation; the implementation of the PC algorithm is an existing conventional technical solution, and specific implementation steps will not be repeated here;

[0072] The average causal effect ACE is calculated through the conditional probability table of the causal Bayesian network, and the formula is: ACE(v i -v j )=P(v j =1|do(v i =1))-P(v j =1|do(v i =0));Where do(v i =1) is an intervention operation, which means forcing the cause variable v i to occur and taking its value as 1, such as controlling the temperature to be too high through an experiment; do(v i =0) means forcing the cause variable v i to occur and taking its value as 0; P(v j =1|do(v i =1)) is the probability of the result variable v j occurring when the intervention v i =1; P(v j =1|do(v i =0) is the probability of the result variable v j occurring when the intervention v i =0; ACE(v i -v j ) is the causal strength, with a value range of [-1, 1], and the greater the absolute value, the stronger the causal relationship, and a positive sign indicates a positive causal relationship, and a negative sign indicates an inhibitory effect.

[0073] It should be noted that the causal Bayesian network is used to distinguish between correlation and causality, for example, "vibration anomaly" and "bearing wear" are causal relationships, not just co-occurrence associations, which can break through the limitations of traditional knowledge graphs "correlation ≠ causality";

[0074] By quantifying the causal strength through the ACE value, the priority of fault tracing can be provided, for example, "temperature too high-wear" with ACE = 0.8 needs to be prioritized for troubleshooting;

[0075] When the double-layer graph is fused, the upper causal variable is associated with the bottom physical entity, for example, "bearing wear" corresponds to the bottom entity "wear" and "ball bearing";

[0076] With each batch of new maintenance data or predicted features, the causal strength is updated by the following formula:

[0077] ACE new = alpha ACE old + (1-alpha) ACE new-data ; Wherein, alpha is a historical weight coefficient, used to balance the stability of old data and the timeliness of new data, the value range is [0.5, 0.8]; ACE old is the causal strength calculated before the new data; ACE new-data is the causal strength calculated based on the new data;

[0078] It is worth noting that the double-layer structure realizes the leap of physical entity association-causal reasoning, which not only retains the objective attribute relationship of the entity, but also realizes the "root cause positioning" of the fault through the upper layer rules, for example, from "equipment abnormal noise" to "ball wear"; The dynamic updating mechanism makes the graph continuously optimized with the accumulation of equipment operation data, which can adapt to dynamic scenarios such as equipment aging and working condition changes in a timely and reliable manner.

[0079] In the embodiment of the application, by constructing a knowledge graph with physical entity association accuracy and causal reasoning ability, full-link explainability is realized from data, features, entities, causality and decision-making, which can provide reliable knowledge support for fault diagnosis and life prediction in predictive maintenance.

[0080] The mixed recommendation strategy dynamic analysis module: based on the current RUL value of the equipment and the causal path of the knowledge graph, the weight proportion of content-based causal reasoning and collaborative filtering is dynamically adjusted by reinforcement learning, the RUL value is analyzed, and the causal reasoning priority mode is dynamically triggered, and the implementation effect of the dynamically triggered causal reasoning priority mode is actively supervised and evaluated and controlled based on the calculated recommendation adoption degree; The specific steps include:

[0081] Among them, the current RUL value refers to the remaining useful life prediction value of the equipment at the current time;

[0082] The input features of the recommendation system are extracted from the knowledge graph and real-time data of the equipment, and the state space of reinforcement learning is defined:

[0083] Among them, RUL current is the current predicted remaining life; S path is the causal path strength; D entity is the entity association density; Sim CB is the cosine similarity of the causal reasoning model; For the score prediction of the collaborative filtering model, the collaborative filtering score based on the similar equipment maintenance records; ω CB，t-1 、ω CF，t-1 Respectively, the weight of the last time causal inference model, collaborative filtering model, the initial value is 0.5;

[0084] It needs to be explained that the causal inference model: by analyzing the characteristics of the equipment itself, such as sensor data, fault type, maintenance record, etc., the similarity between the current equipment and the historical fault case is calculated, and the maintenance strategy of the similar case is recommended;

[0085] Collaborative filtering model: not dependent on the characteristics of the equipment itself, but by analyzing a large number of historical maintenance behaviors of equipment or users, finding similar equipment groups, using the maintenance records of the group to recommend strategies for the current equipment;

[0086] The construction and training of causal inference model and collaborative filtering model are existing conventional technical solutions, and the specific implementation steps are not described here;

[0087] It needs to be explained that the state space fusion equipment health state-knowledge graph causal information-recommendation model output can provide comprehensive decision basis for dynamic weight adjustment, and can avoid the recommendation deviation caused by single feature, and can effectively improve the strategy robustness.

[0088] The action space of reinforcement learning is defined as the continuous adjustment range of causal inference weight and collaborative filtering weight, and the weight update formula based on RUL and causal path is designed:

[0089] The action a t of the agent is the adjustment amount of the causal inference model weight Δω CB , which satisfies:

[0090] a t ∈[-0.1,0.1], ω CB，t = clip(ω CB，t-1 ,0,1); wherein the collaborative filtering weight automatically satisfies ω CF,t =1-ω CB,t ; Δω CB is the single weight adjustment step, limited to ±0.1, to avoid violent fluctuations; clip() is a clipping function to ensure ω CB,t ∈[0,1];

[0091] The initial value of the weight is adaptively initialized by RUL and causal path intensity:

[0092]

[0093] Wherein, RUL threshold is the threshold for triggering the causal inference priority mode; Spath S path ∈[0,1];

[0094] It needs to be explained that the action space restriction can ensure the weight adjustment smoothness and avoid the recommendation strategy mutation; by designing the lower the RUL, the stronger the causal path, the higher the causal reasoning weight, the prior logic can provide a reasonable initial point for reinforcement learning.

[0095] Define the reward function r t , which quantifies the effect of the recommendation strategy under the current weight combination, and guides the agent to learn the optimal weight adjustment strategy:

[0096] r t =β·Acc pred -χ·C cost +δ·S path ·Π(RUL current ≤RUL threshold );

[0097] Where Acc pred is the fault prediction accuracy of the recommendation maintenance strategy; C cost is the normalized value of maintenance cost; Π() is an indicator function, which takes 1 when RUL current ≤RUL threshold , otherwise 0; β, χ, δ are weight coefficients, the initial values are 0.6, 0.3, 0.1 respectively, which prioritize accuracy, then reduce cost, and finally encourage causal reasoning application; accuracy-cost-causal reasoning application, to avoid over-maintenance caused by single pursuit of accuracy, or neglecting physical causality by relying on collaborative filtering, such as similar devices but different failure mechanisms.

[0098] When the remaining life of the device is lower than the threshold, the weight formula is adjusted dynamically to force the causal reasoning recommendation weight to be enhanced by the causal reasoning guidance:

[0099] Where RUL current is monitored in real time, and if RUL current ≤RUL threshold , the priority mode is entered and the weight is updated as: Where is the causal reasoning weight in the priority mode, which takes the maximum value of the current causal reasoning weight and the causal path strength, to ensure that causal reasoning dominates;

[0100] In the priority mode, the causal reasoning recommendation result is further filtered by the causal chain of the knowledge graph:

[0101] Where Rec CBThe candidate recommendation list generated for the causal inference model, such as "replace bearing", "clean the lubrication system"; path(rec) is the causal path corresponding to the recommended item, such as "replace bearing" corresponding to the path "temperature is too high-wear-replace";

[0102] When actively monitoring and evaluating the implementation effect of the dynamically triggered causal inference priority mode and controlling it, the recommendation adoption degree corresponding to all generated candidate recommendation lists is calculated and obtained;

[0103] Wherein, the recommendation adoption degree is obtained by calculating the ratio of the total number of effective recommendation adoption to the total number of invalid recommendation adoption; effective recommendation adoption refers to the selection of adopting the recommended items in the candidate recommendation list; invalid recommendation adoption refers to the selection of not adopting the recommended items in the candidate recommendation list;

[0104] When data analysis is performed on the recommendation adoption degree to determine the implementation effect of the dynamically triggered causal inference priority mode, if the recommendation adoption degree is greater than or equal to the recommendation adoption threshold, it is determined that the implementation effect of the dynamically triggered causal inference priority mode is normal, and the application of the existing priority mode is maintained; the specific value of the recommendation adoption threshold is not limited, which can be determined according to the running design requirements of the model, or can be determined by the professional technical personnel according to the work experience and work requirements;

[0105] If the recommendation adoption degree is less than the recommendation adoption threshold, it is determined that the implementation effect of the dynamically triggered causal inference priority mode is abnormal, and the application of the existing priority mode is optimized and adjusted;

[0106] Wherein, the optimization and adjustment of the application of the existing priority mode includes but is not limited to the optimization and adjustment of the existing recommendation strategy rules, the optimization and adjustment of the existing priori logic.

[0107] In the embodiment of the application, the weight of reinforcement learning and collaborative filtering is dynamically balanced, which can solve the defects of traditional hybrid recommendation that pays more attention to data association and less attention to physical mechanism, can effectively reduce maintenance cost and improve fault prediction accuracy while ensuring recommendation diversity and relying on physical causal relationship when the equipment approaches the end of life.

[0108] Edge cloud collaborative optimization module: edge nodes process high-frequency sensor data in real time and generate preliminary recommendations, cloud performs causal path mining and recommendation model parameter optimization based on global dimension data, and realizes parameter synchronization through federated learning. The specific steps include:

[0109] Wavelet threshold compression is adopted for the original sensor data to retain key features: Wherein, W(x(t)) is the wavelet transform coefficient, db4 wavelet base is adopted, and the decomposition layer is 3 layers; λ is the threshold value, σ is the noise standard deviation, and N is the data length;

[0110] In addition, the compression ratio CR = original data amount / compressed data amount, and a typical value CR = 10:1; for example, 1GB is compressed to 100MB;

[0111] Extracting the time-varying feature matrix X based on the compressed data time-var ying , and calling a lightweight CB model to generate recommendations:

[0112]

[0113] where ω CB，edge , ω CF，edge are the edge fixed weights of the causal inference model and the collaborative filtering model, respectively; Sim CB，k , are the causal inference similarity and the collaborative filtering score of the kth recommended item, respectively;

[0114] The cloud platform receives the compressed data and preliminary recommendation results uploaded by the edge nodes, fuses the global dimension big data, and updates the knowledge graph causal path:

[0115] The cloud integrates the compressed data of N edge nodes Construct a global feature matrix:

[0116]

[0117] By federated averaging of the local model parameters of the edge nodes, direct uploading of the original data is avoided, and privacy protection is achieved;

[0118] Based on the global data, the average causal effect ACE of the causal path in the knowledge graph is recalculated, and the formula is:

[0119] where ACE global is the updated global causal path strength, which is used to correct the cloud knowledge graph; is the local causal strength calculated by the ith edge node; ω i is the weight coefficient;

[0120] It needs to be explained that through global data fusion, data silos can be avoided, and the statistical significance of causal path mining can be improved, for example, ACE deviation caused by insufficient sample size of a single edge node.

[0121] When performing cloud recommendation model parameter optimization and federated learning synchronization, the cloud optimizes the recommendation model parameters based on the global knowledge graph, and distributes the optimized parameters to the edge nodes through federated learning:

[0122] The global causal path strength ACE globalTo constrain, minimize the recommended loss function L cloud :

[0123] wherein, is the actual maintenance effect label of the i th edge node, 1 represents valid, 0 represents invalid; ACE pred is the causal path strength predicted by the model; η is a regularization coefficient, balancing recommendation accuracy and causal consistency;

[0124] The FedProx algorithm is adopted to realize cloud-edge parameter synchronization, and the local update formula of the edge node is:

[0125] wherein, is the updated edge node model parameter; is the cloud global model parameter, for example ω CB , ω CF ; L edge is the local recommendation loss of the edge node; is the gradient of the local loss function of the edge node; τ is the learning rate, and the default value is 0.01; is the proximal item coefficient, which controls the deviation of the edge parameter and the cloud, and the default value is 0.001;

[0126] It should be noted that the regularization term ensures that the optimization direction of the recommendation model is consistent with the causal path of the knowledge graph, avoiding the disconnection between data-driven recommendation and physical mechanism;

[0127] The FedProx algorithm alleviates the parameter divergence problem caused by data heterogeneity of the edge node through the proximal term, for example, different device fault mode differences, which can effectively improve the synchronization stability.

[0128] It should be explained that in the FedProx algorithm, in order to cope with the problems of data heterogeneity and system heterogeneity in federated learning, a concept called proximal item proximal is introduced, which is mainly used to limit the difference between the client model update and the global model, thereby improving the stability and convergence of the training process.

[0129] In the embodiment of the application, through the three-layer architecture of edge high-frequency response-cloud global optimization-federated learning parameter synchronization, the contradiction between real-time processing of high-frequency data in industrial equipment maintenance and dependence on global knowledge graph can be solved, and the multi-dimensional optimization goal of low delay, high precision and privacy protection of the recommendation strategy can be realized.

[0130] In several embodiments of the present application, it should be understood that the disclosed method can be implemented in other manners. For example, the above described embodiments of the present application are merely illustrative, and for example, the division of the modules is merely logical function division, and there can be other division manners in actual implementation.

[0131] The modules illustrated as separate components can or can not be physically separate, and the components illustrated as modules can or can not be physical modules, and can be located in one place or distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0132] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The integrated module can be implemented in the form of hardware or in the form of hardware plus software functional module.

[0133] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the essential characteristics of the present application.

[0134] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application and are not limiting, and although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twins, characterized by: include: Multimodal feature processing module: Based on the real-time synchronized equipment operation data of the digital twin, the module predicts the remaining life of key components through a long short-term memory network, extracts vibration spectrum, temperature gradient, and image texture features that are strongly correlated with life attenuation, and constructs a time-varying feature matrix. Among them, an LSTM network integrating attention mechanism is constructed, the input is the preprocessed multimodal time series feature, and the output is the remaining life prediction value The loss function is designed as: Where MSE() is the mean square error function; ω1 is the weight coefficient of the MSE loss term; ω2 is the weight coefficient of the attenuation rate deviation term; Δλ=λ pred -λ true , is the predicted decay rate λ pred and the true decay rate λ true The deviation, Indicates the life decay rate; RUL true The actual remaining life of the equipment; Causal Relationship Enhancement Processing Module: This module integrates predictive features with historical maintenance and inspection data to construct a two-layer knowledge graph containing the fault causal chain. The bottom layer is the physical entity association layer, and the upper layer is the causal reasoning layer. The causal strength between entities is quantified through a causal Bayesian network. Hybrid recommendation strategy dynamic analysis module: Based on the device's current RUL value and knowledge graph causal path, it uses reinforcement learning to dynamically adjust the weight ratio of content-based causal reasoning and collaborative filtering, analyzes the RUL value and dynamically triggers the causal reasoning priority mode. Based on the calculated recommendation adoption degree, it actively monitors, evaluates, and controls the implementation effect of the dynamically triggered causal reasoning priority mode. Edge-cloud collaborative optimization module: The edge node processes high-frequency sensor data in real time and generates preliminary recommendations. The cloud performs causal path mining and recommendation model parameter optimization on the knowledge graph based on global maintenance and inspection big data, and achieves parameter synchronization through federated learning.

2. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 1 is characterized in that: Based on real-time data mapping between digital twins and physical devices, collect time-series sensor data, visual data, and text data; Preprocess and enhance the features of the collected multimodal data to obtain a multimodal time series feature set; Construct a long short-term memory network integrated with an attention mechanism, with preprocessed multimodal time series features as input and remaining life expectancy prediction as output; The model is trained using full life cycle failure data generated through digital twin simulation.

3. The multimodal equipment maintenance and inspection knowledge intelligent recommendation system combined with digital twin according to claim 2 is characterized in that: When constructing the time-varying feature matrix, the weighted feature vectors are arranged in time series with the time window Δt as the unit to construct the time-varying feature matrix X time-varying :X time-varying =[F weighted (t0)F weighted (t0+Δt)...F weighted (t0+nΔt)]; Among them, F weighted is the weighted eigenvector; n is the number of matrix columns; t0 is the initial time point.

4. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 3 is characterized in that: Based on the fusion data, physical entities and relationships are identified, and an entity-relationship network is constructed through entity extraction, relationship definition and entity association matrix construction.

5. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 4 is characterized in that: Based on the causal Bayesian network, the fault causal chain is mined, the causal strength between entities is quantified, and a causal reasoning layer is constructed.

6. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 5 is characterized in that: The average causal effect ACE is calculated through the conditional probability table of the causal Bayesian network, and the formula is: ACE(v i -v j )=P(v j =1|do(v i =1))-P(v j =1|do(v i =0)); Among them, do(v i =1) is an intervention operation, indicating the forced cause variable v i occurs, and takes the value as 1; do(v i =0), indicating the mandatory cause variable v i occurs, and takes its value as 0; P(v j =1|do(v i =1)) is intervention v i =1 when the result variable v j The probability of occurrence; P(v j =1|do(v i =0) is intervention v i =0 when the result variable v j Probability of occurrence; ACE(v i -v j ) is the causal strength.

7. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 6 is characterized in that: When fusing two layers of graphs, the upper-layer causal variables are associated with the underlying physical entities. Each time a new batch of maintenance and inspection data or prediction features is added, the causal strength is updated using the following formula: ACE new =α·ACE old +(1-α)·ACE new-data ; Among them, α is the historical weight coefficient; ACE old is the causal strength calculated before adding new data; ACE new-data is the causal strength calculated based on the newly added data.

8. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 7 is characterized in that: Define the reward function r t , quantify the effect of the recommendation strategy under the current weight combination, and guide the agent to learn the optimal weight adjustment strategy: r t =β·Acc pred -χ·C cost +δ·S path ·Π(RUL current ≤RUL threshold ); Among them, Acc pred is the fault prediction accuracy of the recommended maintenance strategy; C cost is the normalized value of maintenance cost; Π() is the indicator function, when RUL current ≤RUL threshold 1 when it is set, otherwise 0, RUL current is the current predicted remaining life, RUL threshold is the threshold that triggers the causal reasoning priority mode; S path is the causal path strength; β, χ, and δ are all weight coefficients.

9. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 8 is characterized in that: Extracting time-varying feature matrix X based on compressed data time-varying , call the lightweight CB model to generate recommendations: Among them, ω CB，edge 、ω CF，edge They are the fixed weights of the edge of the causal reasoning model and the collaborative filtering model respectively; Sim CB，k 、 are the causal reasoning similarity and collaborative filtering score of the k-th recommendation item, respectively.

10. The multimodal equipment maintenance knowledge intelligent recommendation system combined with digital twin according to claim 9 is characterized in that: When optimizing the cloud-based recommendation model parameters and synchronizing them with federated learning, the cloud optimizes the recommendation model parameters based on the global knowledge graph, and sends the optimized parameters to the edge nodes through federated learning, using the FedProx algorithm to achieve cloud-edge parameter synchronization.

Images