Cross-brand home appliance maintenance knowledge sharing system and method based on federated learning

Abstract

The present invention relates to the field of home appliance repair technology, and in particular to a cross-brand home appliance repair knowledge sharing system and method based on federated learning. The multimodal fault data collection and intelligent repair system integrates image, text, fault phenomenon and multimodal instruction data collection. After preprocessing, a feature decoupling model is established through a repair knowledge parsing module. A cross-brand repair knowledge base stores common and unique fault information to facilitate knowledge transfer. The federated learning training module adopts differential privacy homomorphic encryption to protect data privacy and realize gradient aggregation collaborative training. The augmented reality repair guidance terminal intuitively guides repairs and improves efficiency. The preventive maintenance module calculates the equipment maintenance cycle, generates prevention plans, and reduces failure rates and costs. The system significantly improves repair accuracy, quickly integrates into new brands, realizes knowledge sharing and utilization, and brings revolutionary changes to the home appliance repair industry.

Description

Technical Field

[0001] The present invention relates to the technical field of home appliance repair technology, and in particular to a cross-brand home appliance repair knowledge sharing system and method based on federated learning. Background Art

[0002] With the ever-increasing variety and brands of home appliances, the appliance repair industry faces increasingly complex technical challenges. Traditional repair methods, relying primarily on the repair technician's personal experience and single-brand repair manuals, struggle to adapt to the demands of modern appliance repair. In recent years, with the development of artificial intelligence and big data technologies, intelligent repair systems have emerged, attempting to improve repair efficiency and accuracy through data analysis and machine learning.

[0003] However, existing intelligent maintenance systems still have many limitations. First, most systems can only process maintenance knowledge for a single brand or a limited number of brands, making it difficult to achieve cross-brand knowledge sharing and transfer. This leads to the formation of knowledge silos, limiting the effective utilization and expansion of maintenance knowledge. Second, existing systems generally use centralized data processing methods, which not only faces challenges in data privacy and security, but also makes it difficult to fully utilize the large amount of maintenance data distributed across various brands and maintenance locations. Furthermore, existing systems have limited capabilities in processing multimodal maintenance data (such as text, images, sensor data, etc.), making it difficult to fully capture fault characteristics and maintenance knowledge.

[0004] In addition, the existing technology also has the following specific problems:

[0005] The lack of an effective feature decoupling mechanism makes it difficult to distinguish between common fault features and brand-specific features, which affects the generalization ability of maintenance knowledge.

[0006] Insufficient privacy protection measures may lead to the leakage of sensitive information in the process of sharing maintenance knowledge;

[0007] Weak preventive maintenance capabilities, focusing primarily on failures that have already occurred rather than predicting and preventing potential problems;

[0008] The lack of intuitive and effective remote maintenance guidance has limited the practical application of maintenance knowledge. Summary of the Invention

[0009] This invention aims to address the aforementioned technical issues by providing a cross-brand home appliance repair knowledge sharing system and method based on federated learning. This system effectively shares and transfers multi-brand repair knowledge while protecting data privacy, improving repair efficiency and accuracy, and providing strong technical support for preventive maintenance and remote guidance.

[0010] The present invention proposes a cross-brand home appliance repair knowledge sharing system and method based on federated learning, including:

[0011] Multimodal fault data collection module for:

[0012] Acquiring multimodal fault data, the multimodal fault data including image data, text data, fault phenomenon data, and multimodal instruction data;

[0013] Preprocessing the image data and text data;

[0014] The maintenance knowledge parsing module is in communication with the multimodal fault data collection module and is used to:

[0015] Receiving the preprocessed image data and text data sent by the multimodal fault data collection module;

[0016] Based on the preprocessed image data and text data, a feature decoupling model of multimodal fault data is established;

[0017] A cross-brand maintenance knowledge base, in communication with the maintenance knowledge parsing module, is used to:

[0018] Store common fault types and common features of brands, as well as unique fault types and unique features of home appliances;

[0019] A federated learning training module is communicatively connected to the maintenance knowledge parsing module and the cross-brand maintenance knowledge base, and is used to:

[0020] Adopting differential privacy homomorphic encryption privacy protection strategy, a federated learning training mechanism is established;

[0021] Adopting gradient aggregation mechanism to exchange gradients and conduct collaborative training among federated learning agents;

[0022] The augmented reality maintenance guidance terminal is connected to the cross-brand maintenance knowledge base and is used to:

[0023] Identify target parts and provide repair instructions;

[0024] A preventive maintenance module, in communication with the cross-brand maintenance knowledge base and the federated learning training module, is configured to:

[0025] Calculate the maintenance cycle of target equipment and generate preventive maintenance plans.

[0026] Preferably, the maintenance knowledge parsing module includes:

[0027] Image decoupling encoder for:

[0028] Input image samples and output the probability distribution of common fault types and the probability distribution of unique fault types;

[0029] Perform multi-label classification on the image feature vectors of each brand based on deep neural network;

[0030] Text decoupled encoder, used for:

[0031] Input text samples and output the probability distribution of common fault types and the probability distribution of unique fault types;

[0032] Perform multi-label classification on each brand text feature vector based on deep neural network;

[0033] Attention matrix, used for:

[0034] Record the correspondence between common fault characteristics and unique fault characteristics;

[0035] A probabilistic attention mechanism is used to separate common failure modes from brand-specific failure modes.

[0036] Preferably, the federated learning training module includes:

[0037] Data collectors, used for:

[0038] Collect maintenance data from the local equipment maintenance knowledge database;

[0039] Global server, used for:

[0040] Coordinate the federated learning process and perform model aggregation;

[0041] Federated Learning Agents, used for:

[0042] Perform model training locally and exchange gradients with the global server.

[0043] Preferably, the cross-brand maintenance knowledge base is constructed in the form of a knowledge graph, including:

[0044] Entity, representing the fault type, symptoms and repair plan;

[0045] Relationships represent connections between entities;

[0046] Attributes, which describe the characteristics of an entity;

[0047] The knowledge graph realizes the association and expansion of knowledge by linking the corpus with the knowledge base.

[0048] Preferably, the augmented reality maintenance guidance terminal includes:

[0049] Data sampler for:

[0050] Collect image and depth information of target parts;

[0051] Positioning feedback module for:

[0052] Obtaining the coordinate position of the located part from the cross-brand maintenance knowledge base;

[0053] Achieve image-to-part alignment;

[0054] Disassemble and install the navigation module for:

[0055] Obtaining three-dimensional disassembly and assembly steps from the cross-brand maintenance knowledge base;

[0056] Provide users with visual disassembly and assembly guidance.

[0057] Preferably, the preventive maintenance module includes:

[0058] Survival analysis models are used to:

[0059] Learning sample features;

[0060] Predict the remaining useful life of target equipment;

[0061] Feature selection unit, used to:

[0062] Screening important characteristic variables;

[0063] Machine learning models for:

[0064] Training prediction models based on the screened feature variables;

[0065] Maintenance plan generator for:

[0066] Generate preventive maintenance plans based on prediction results.

[0067] As an option, it also includes:

[0068] A multimodal data fusion module is communicatively connected to the maintenance knowledge parsing module and the cross-brand maintenance knowledge base, and is used to:

[0069] Taking maintenance knowledge and fault feature vectors as inputs to the graph neural network model;

[0070] A comprehensive maintenance plan is generated by combining the maintenance knowledge in the maintenance knowledge base, the maintenance plan obtained from the knowledge graph, and the voice and image information during the maintenance process.

[0071] As an option, it also includes:

[0072] A privacy loss assessment module, in communication with the federated learning training module, is used to:

[0073] Calculate the privacy loss during gradient aggregation;

[0074] Dynamically adjust the training strategy based on the privacy loss evaluation results.

[0075] As an option, it also includes:

[0076] A knowledge transfer module is communicatively connected to the cross-brand maintenance knowledge base and the federated learning training module, and is used to:

[0077] Based on the decoupled common features, a transferable maintenance knowledge base is constructed;

[0078] Realize the transfer learning of maintenance knowledge between different brands.

[0079] The cross-brand home appliance maintenance knowledge sharing method based on federated learning includes the following steps:

[0080] S1: Acquire multi-modal fault data;

[0081] S2: extracting features from the multimodal fault data;

[0082] S3: Matching a maintenance plan in a cross-brand maintenance knowledge base based on the multimodal fault data, and transmitting the fault data not in the cross-brand maintenance knowledge base to a federated learning training module for training;

[0083] S4: A cross-brand maintenance knowledge base is used to classify unmatched fault data. A general feature decoupling model is trained using a multi-label classification loss and a privacy-preserving regularization loss function for samples in specific categories.

[0084] S5: Use a cross-brand maintenance knowledge base to classify unmatched fault data. For common category samples, use multi-label classification loss and privacy-preserving regularization loss function to train a unique feature decoupling model.

[0085] S6: Perform differential privacy assessment based on the general feature decoupling model and the unique feature decoupling model using the differential privacy homomorphic encryption protection strategy;

[0086] S7: Evaluate the cross-brand maintenance knowledge base according to the differential privacy evaluation result, and update the cross-brand maintenance knowledge base on the differential privacy;

[0087] S8: Generate maintenance solutions based on the updated cross-brand maintenance knowledge base;

[0088] S9: Using augmented reality technology to provide remote maintenance guidance;

[0089] S10: Predict equipment lifespan and generate preventive maintenance recommendations based on survival analysis methods.

[0090] The cross-brand home appliance repair knowledge sharing system and method based on federated learning of the present invention demonstrates significant innovation and practical value in terms of overall architecture design and collaboration among modules, providing a new solution for the intelligent transformation of the home appliance repair industry.

[0091] From a macro perspective, this invention successfully integrates advanced technologies such as federated learning, multimodal data processing, and knowledge graphs to create a secure, efficient, and scalable cross-brand maintenance knowledge sharing platform. This innovative technological integration not only breaks down the knowledge silos of traditional maintenance systems but also achieves breakthroughs in data privacy protection and knowledge transfer. The system's distributed architecture enables each brand to participate in global model training while protecting its own data privacy, achieving a seamless integration of data localization and knowledge sharing.

[0092] At the micro level, the various functional modules of this invention demonstrate a high degree of synergy and complementarity. For example, the feature decoupling technology in the maintenance knowledge analysis module works in conjunction with the knowledge transfer module to extract common fault characteristics while retaining brand-specific maintenance knowledge, greatly improving the efficiency and accuracy of knowledge transfer. The combination of the multimodal data fusion module and the augmented reality maintenance guidance terminal provides users with intuitive and accurate maintenance guidance, transforming complex maintenance knowledge into easy-to-understand and easy-to-use visual information.

[0093] This invention also excels in resolving technical contradictions. A typical example is the balance between data utilization and privacy protection. By introducing a privacy loss assessment module and a dynamic adjustment strategy, the system maximizes data value while keeping privacy risks within acceptable limits. This balance not only meets increasingly stringent data protection requirements but also ensures continuous improvement in model performance.

[0094] In terms of synergistic effects, the combination of the preventive maintenance module and the federated learning training module in this invention generates value beyond simple fault diagnosis. By analyzing massive amounts of cross-brand repair data, the system can not only accurately diagnose current faults but also predict potential problems and provide users with proactive maintenance recommendations. This enhanced preventive maintenance capability is expected to significantly reduce unplanned downtime of home appliances, improve user satisfaction, and provide manufacturers with valuable information for product improvement.

[0095] In general, the beneficial effects of the present invention are embodied in the following aspects:

[0096] Significantly improved repair accuracy, especially when handling complex and cross-brand faults;

[0097] The efficiency of knowledge transfer has been greatly improved, allowing new brands to quickly integrate into the system and benefit from it;

[0098] While protecting data privacy, it enables full utilization of maintenance knowledge;

[0099] Reduce appliance failure rate and repair costs through preventive maintenance;

[0100] With the help of augmented reality technology, more intuitive and effective remote maintenance guidance is provided.

[0101] The combined effect of these effects not only improves the quality and efficiency of maintenance services, but also opens up new possibilities for the intelligent and personalized development of the home appliance industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 It is the top-level logic diagram of the system of the present invention;

[0103] Figure 2 This is the internal logic diagram of the maintenance knowledge parsing module of the present invention;

[0104] Figure 3 This is the internal logic diagram of the federated learning training module of the present invention;

[0105] Figure 4 This is the internal logic diagram of the preventive maintenance module of the present invention. DETAILED DESCRIPTION

[0106] Please refer to the attached Figure 1-4 The present invention provides a cross-brand home appliance repair knowledge sharing system and method based on federated learning. The system comprises multiple functional modules that work together to achieve secure sharing and efficient utilization of cross-brand home appliance repair knowledge. The specific implementation of the present invention will be described in detail below.

[0107] The system of the present invention includes a multimodal fault data collection module 1, a maintenance knowledge parsing module 2, a cross-brand maintenance knowledge base 3, a federated learning training module 4, an augmented reality maintenance guidance terminal 5, and a preventive maintenance module 6. These modules are connected through communication, forming a complete knowledge sharing and application system.

[0108] Multimodal fault data collection module 1 is the system's data input terminal, used to acquire multimodal fault data. Multimodal fault data here includes image data, text data, fault phenomenon data, and multimodal instruction data. For example, image data might be a photograph of the appliance's faulty part, text data might be a user's description of the fault phenomenon, fault phenomenon data might be abnormal data collected from sensors, and multimodal instruction data might be a maintenance technician's operational records during the repair process.

[0109] The multimodal fault data collection module 1 not only collects data, but also preprocesses the collected image data and text data. The purpose of preprocessing is to improve the efficiency and accuracy of subsequent analysis. For image data, preprocessing may include operations such as denoising, adjusting brightness and contrast, and cropping. Preferably, the present invention uses a median filtering algorithm for image denoising, which can effectively remove salt and pepper noise while maintaining image edge information. For text data, preprocessing may include operations such as word segmentation, removing stop words, and part-of-speech tagging. The present invention preferably uses the Jieba word segmentation library for Chinese word segmentation, which has high word segmentation accuracy and efficiency.

[0110] Maintenance Knowledge Parsing Module 2 communicates with Multimodal Fault Data Collection Module 1 to receive preprocessed image and text data. Based on this data, the core task of Maintenance Knowledge Parsing Module 2 is to establish a feature decoupling model for multimodal fault data. Feature decoupling is a key innovation of this invention, separating common fault features from brand-specific features, laying the foundation for subsequent knowledge transfer and sharing.

[0111] The process of establishing the feature decoupling model can be described as follows: First, for image data, a convolutional neural network (CNN) is used to extract features. This paper prefers to use ResNet50 as the backbone network because it can extract deep image features while maintaining low computational complexity. For text data, the BERT model is used for feature extraction. The advantage of the BERT model is its ability to capture contextual information, which is particularly important for understanding fault description text.

[0112] Next, the extracted features are input into an autoencoder. The structure of the autoencoder is as follows:

[0113] ,

[0114] ,

[0115] is the input feature, is the encoded feature, is the reconstructed feature, and is the weight matrix, and is the bias term, and In this paper, we choose ReLU as the activation function because it can effectively alleviate the gradient vanishing problem.

[0116] The training goal of the autoencoder is to minimize the reconstruction error:

[0117] ,

[0118] in, is the regularization coefficient, is the sparse regularization term. By adjusting The value of (preferably set to 0.01 in the present invention) can control the sparsity of features, thereby achieving feature decoupling.

[0119] The maintenance knowledge parsing module 2 includes an image decoupling encoder 21, a text decoupling encoder 22, and an attention matrix 23. These three submodules work together to complete the task of feature decoupling.

[0120] The main function of the image decoupling encoder 21 is to input image samples and output the probability distribution of common fault types and the probability distribution of unique fault types. Specifically, it uses a deep neural network to perform multi-label classification on the feature vectors of each brand image. The deep neural network structure used in this invention is as follows:

[0121] 1. Input layer: receives image feature vector, dimension is

[0122] 2. Fully connected layer 1: The number of neurons is 512, and the activation function is ReLU

[0123] 3. Dropout layer: dropout rate is 0.5 to prevent overfitting

[0124] 4. Fully connected layer 2: the number of neurons is 256, and the activation function is ReLU

[0125] 5. Output layer: The number of neurons is equal to the number of fault types, and the activation function is Sigmoid

[0126] The loss function of this network uses binary cross entropy:

[0127] ,

[0128] in, is the sample size, is the true label, is the predicted probability.

[0129] The function and structure of the text decoupling encoder 22 are similar to those of the image decoupling encoder 21, except that the input is a text feature vector. In this paper, we use the 768-dimensional feature vector extracted by the BERT model as input.

[0130] The attention matrix 23 is a key innovation of the maintenance knowledge parsing module 2. It is used to record the correspondence between common fault characteristics and unique fault characteristics, and uses a probabilistic attention mechanism to separate common fault modes from brand-specific fault modes. The calculation formula of the attention matrix is ​​as follows:

[0131] ,

[0132] in, are query matrix, key matrix and value matrix respectively, By adjusting these matrices, we can flexibly control the degree of attention paid to different features.

[0133] The federated learning training module 4 includes a data collector 41, a global server 42, and a federated learning agent 43. These three submodules constitute a complete federated learning system, enabling model training while protecting data privacy.

[0134] Data collector 41 is responsible for collecting maintenance data from the local device maintenance knowledge database. Local devices here can be after-sales service centers or smart home appliances of various brands. The frequency and scope of data collection can be adjusted based on actual needs. The present invention recommends daily data collection to ensure that the model can learn the latest maintenance knowledge in a timely manner.

[0135] The main task of the global server 42 is to coordinate the federated learning process and perform model aggregation. Model aggregation is a core step in federated learning, and this invention uses the FedAvg algorithm for model aggregation. Specifically, after each round of training, the global server 42 collects the model parameters uploaded by each federated learning agent 43 and then performs a weighted average according to the following formula:

[0136] ,

[0137] in, are the model parameters after aggregation, For the The model parameters of the agents, For the The amount of data per agent, is the total data volume.

[0138] The federated learning agent 43 is responsible for training the model locally and exchanging gradients with the global server 42. To protect data privacy, the present invention uses differential privacy technology during the gradient upload process. Specifically, before uploading the gradient, Laplace noise is added to the gradient:

[0139] ,

[0140] in, is the original gradient vector, is the gradient vector after adding noise, The mean is 0 and the scale parameter is The probability density function of the Laplace distribution is:

[0141] ,

[0142] scale parameter The choice of is directly related to the privacy protection strength, and its calculation formula is:

[0143] ,

[0144] in, Represents sensitivity, which is defined as the maximum impact of a single sample change on the gradient. For privacy budget. In this system, based on gradient clipping technology, we will Controlled to a constant (default value is 1.0), so The present invention proposes that the initial , corresponding to ,This strikes a good balance between protecting privacy and ,maintaining model performance.

[0145] From the above description, we can see that the system of this invention not only enables cross-brand home appliance repair knowledge sharing but also offers innovations in data privacy protection, feature decoupling, and multimodal data processing. These innovations together form an efficient, secure, and intelligent repair knowledge sharing platform, which is expected to significantly improve the efficiency and quality of home appliance repair.

[0146] The cross-brand maintenance knowledge base 3 of the present invention is constructed using a knowledge graph. This structured knowledge representation makes maintenance knowledge easier to store, retrieve, and update. The knowledge graph of the present invention primarily includes three core elements: entities, relationships, and attributes.

[0147] In the knowledge graph of the present invention, entities represent fault types, symptoms, and repair solutions. For example, "compressor failure" can be a fault type entity, "poor cooling effect" can be a symptom entity, and "replace compressor" can be a repair solution entity. By clearly defining and linking these entities, the system of the present invention can more accurately perform fault diagnosis and repair guidance.

[0148] Relationships represent connections between entities and are a core component of knowledge graphs. In this paper, relationships can include "cause," "solve," and "belong to." For example, a "cause" relationship can be established between "compressor failure" and "poor cooling effect," while a "solve" relationship can be established between "replace compressor" and "compressor failure." These relationships enable the system to reason and provide more intelligent repair recommendations.

[0149] Attributes are used to describe the characteristics of an entity. In embodiments of the present invention, attributes may include fault severity, frequency, and repair difficulty. For example, for the "compressor fault" entity, the "severity" attribute may be defined as "high" and the "repair difficulty" attribute as "medium." This attribute information can help the system consider more factors when making repair decisions, thereby providing more reasonable repair recommendations.

[0150] An important feature of the present invention is that the knowledge graph realizes the association and expansion of knowledge by linking the corpus to the knowledge base. This means that the system can not only utilize the structured knowledge graph, but also extract information from unstructured text data, continuously enriching and updating the knowledge base. Specifically, the present invention adopts entity linking technology to match the entities appearing in the text with the entities in the knowledge graph. Preferably, the present invention uses a bidirectional LSTM (Long Short-Term Memory) network for entity recognition and then performs entity linking through cosine similarity calculation. This method can effectively convert unstructured text information into structured knowledge and continuously expand the knowledge graph.

[0151] The augmented reality maintenance guidance terminal 5 of the present invention includes a data sampler, a positioning feedback module, and a disassembly and assembly navigation module. These three submodules work together to provide users with intuitive and efficient maintenance guidance.

[0152] The primary function of the data sampler is to collect image and depth information of the target part. In a preferred embodiment of the present invention, the data sampler uses an RGB-D camera, capable of simultaneously acquiring color images and depth maps. This dual-modal data acquisition provides the basis for subsequent precise positioning and navigation. Preferably, the RGB-D camera used in the present invention has a resolution of 1920x1080 and a depth accuracy of up to 1mm. This configuration can meet the needs of most household appliance repair scenarios.

[0153] The positioning feedback module is responsible for obtaining the coordinate position of the positioning parts from the cross-brand maintenance knowledge base 3 and realizing the alignment of the image to the parts. This process involves target detection and posture estimation technology in computer vision. The present invention preferably adopts the YOLOv5 algorithm for target detection. While maintaining high detection accuracy, the algorithm can realize real-time detection and is suitable for augmented reality scenarios. For posture estimation, the present invention adopts the PnP (Perspective-n-Point) algorithm, which can calculate the position and posture of the camera relative to the target object based on the feature points and corresponding 3D coordinates in the 2D image. Through these technologies, the positioning feedback module can accurately superimpose virtual information on actual parts and provide users with accurate maintenance guidance.

[0154] The function of the disassembly and assembly navigation module is to obtain the disassembly and assembly steps in three-dimensional space from the cross-brand maintenance knowledge base 3 and provide users with visual disassembly and assembly guidance. In an embodiment of the present invention, the disassembly and assembly steps are stored in the knowledge base in the form of animation sequences. The disassembly and assembly navigation module will retrieve the corresponding animation sequence from the knowledge base according to the current maintenance task, and then superimpose these animations on the user's actual field of view through augmented reality technology. In order to provide a more natural interactive experience, the present invention also adopts gesture recognition technology, allowing users to control the playback, pause and rewind of animations through gestures. Preferably, the present invention uses a gesture recognition algorithm based on MediaPipe, which can recognize 21 key points of the hand and support complex gesture interactions.

[0155] The preventive maintenance module 6 of the present invention includes a survival analysis model 61, a feature selection unit 62, a machine learning model 63, and a maintenance plan generator 64. These submodules work together to achieve the prediction of equipment life and the generation of a preventive maintenance plan.

[0156] The survival analysis model 61 is the core component of the preventive maintenance module 6, which is used to learn sample characteristics and predict the remaining useful life of the target equipment. In a preferred embodiment of the present invention, the Cox proportional hazards model is used as the survival analysis model. The hazard function of this model is defined as follows:

[0157] ,

[0158] in, is the benchmark hazard function, is the covariate vector, are regression coefficients. Model parameters are estimated by maximizing the partial likelihood function:

[0159] ,

[0160] in, is the event indicator, is the observed survival time, is an index variable, To satisfy The sample index collection of .

[0161] The feature selection unit 62 is used to select important feature variables. In the home appliance maintenance scenario, there may be a large number of features, but not all features have a significant impact on the device life prediction. The present invention uses the Lasso regularization method for feature selection, and its objective function is:

[0162] ,

[0163] in, is the target variable, is the feature matrix, is the regularization parameter. By adjusting The value of (preferably set to 0.01 in the present invention) can control the sparsity of the model, thereby selecting the most important features.

[0164] The machine learning model 63 trains a prediction model based on the filtered feature variables. Considering the complexity and nonlinear characteristics of appliance repair data, the present invention preferably employs the XGBoost (eXtreme Gradient Boosting) algorithm. XGBoost is an ensemble learning method that improves prediction accuracy by constructing multiple decision trees. Its objective function is defined as:

[0165] ,

[0166] in, is the loss function, is the regularization term, For the By optimizing this objective function, XGBoost can effectively prevent overfitting while maintaining high prediction accuracy.

[0167] The maintenance plan generator 64 generates a preventive maintenance plan based on the prediction results. It considers multiple factors, such as the predicted remaining life of the equipment, maintenance costs, and downtime losses, and generates an optimal maintenance plan using a dynamic programming algorithm. Specifically, the maintenance plan generator 64 solves the following optimization problem:

[0168] ,

[0169] in, for Maintenance decision at the moment (0 or 1), For maintenance costs, is the downtime loss per unit time, for By solving this optimization problem, the system can generate an optimal maintenance plan that balances maintenance cost and reliability.

[0170] The present invention also includes a multimodal data fusion module 7, which is in communication with the maintenance knowledge parsing module 2 and the cross-brand maintenance knowledge base 3. The main function of the multimodal data fusion module 7 is to integrate maintenance knowledge from different sources and in different forms to generate a more comprehensive and accurate maintenance plan.

[0171] In a preferred embodiment of the present invention, the multimodal data fusion module 7 first uses the maintenance knowledge and fault feature vectors as input to a graph neural network model. Here, a graph attention network (GAT) is used, and its hierarchical update formula is as follows:

[0172] ,

[0173] in, Indicates the Layer Node Features, Representation node The neighbor set of is the attention coefficient, is the weight matrix, is the activation function, These neighbor nodes are In this way, the model can adaptively learn the importance of nodes and thus better capture the relationship between maintenance knowledge.

[0174] Next, the multimodal data fusion module 7 combines the maintenance knowledge in the maintenance knowledge base, the maintenance plan obtained from the knowledge graph, and the voice and image information during the maintenance process to generate a comprehensive maintenance plan. This process uses a multimodal attention mechanism, and its formula is as follows:

[0175] ,

[0176] in, 、 、 Representing queries, keys, and values, respectively, they come from different data modalities. In this way, the system can automatically learn the importance of information from different modalities and perform effective fusion.

[0177] Finally, the fused features are processed by a multi-layer perceptron to generate a final maintenance plan. This plan not only includes specific repair steps but also comprehensive information such as required tools, estimated time, and precautions, providing detailed guidance for maintenance personnel.

[0178] From the above description, we can see that the system of this invention has made innovations in knowledge representation, augmented reality applications, preventive maintenance, and multimodal data fusion. These innovations together form a comprehensive and intelligent knowledge-sharing platform for home appliance repair, which is expected to significantly improve repair efficiency and service quality.

[0179] The present invention also includes a privacy loss assessment module 8, which is communicatively connected to the federated learning training module 4. The primary function of this module is to calculate the privacy loss during gradient aggregation and dynamically adjust the training strategy based on the privacy loss assessment results. This innovative design enables the system of the present invention to achieve a better balance between protecting data privacy and improving model performance.

[0180] In a preferred embodiment of the present invention, the privacy loss assessment module 8 adopts a differential privacy framework to quantify privacy loss. Specifically, the present invention uses the ε-differential privacy definition, where ε represents the privacy budget. For any two adjacent data sets, and , and any output ,satisfy:

[0181] ,

[0182] in, represents the randomization mechanism (in this case, the federated learning process). A smaller ε value results in higher privacy protection but may have a greater impact on model performance.

[0183] In order to accurately track the cumulative privacy loss during multiple rounds of training, this system uses the moment mechanism (MomentAccountant). Compared with the simple combination theorem, the moment mechanism provides a tighter privacy bound. The privacy loss function defined by the moment mechanism is:

[0184] ,

[0185] in, Represents output In the dataset The logarithmic probability of and For adjacent datasets, is a randomization mechanism (in this invention, it refers to the federated learning process with Laplace noise added). When the cumulative privacy loss approaches the preset threshold (the present invention is set to 5.0), the system triggers the dynamic adjustment strategy.

[0186] When the system detects increased privacy risks, it automatically activates the following adjustment policies:

[0187] 1. Dynamically adjust the noise amplitude based on the current accumulated privacy loss and the maximum tolerance threshold Dynamically adjust the noise scale parameter :

[0188] ,

[0189] in, is the adjustment factor (the default value is 0.2), as near , the noise amplitude will gradually increase, strengthening privacy protection. For example, when , , , hour, , indicating a 16% increase in noise amplitude.

[0190] 2. Learning rate decay strategy Learning rate decay can reduce the amplitude of each parameter update, thereby reducing the privacy sensitivity of a single round of training:

[0191] ,

[0192] in, is the learning rate, is the decay factor (default value is 0.1). Reducing the learning rate can directly reduce the impact of gradient updates on the model, thereby reducing the amount of noise required for the differential privacy mechanism.

[0193] 3. Sampling rate adjustment mechanism:

[0194] The system will dynamically adjust the client sampling rate based on privacy risks

[0195] ,

[0196] in, is the sampling rate adjustment coefficient (the default value is 0.15). Reducing the sampling rate can effectively reduce the privacy loss of each round of training, because subsampling itself has a privacy protection effect. For example, when , initial sampling rate When the new sampling rate is , that is, the sampling rate is reduced to 27.6%.

[0197] The implementation process of the entire privacy protection mechanism in the client-server architecture is as follows:

[0198] 1. Initialization phase: The global server 42 sets the initial privacy parameters ( ) and distributed to all federated learning agents 43;

[0199] 2. Local training phase: Each federated learning agent 43 uses local data to train the model and calculate the gradient ;

[0200] 3. Gradient processing stage: The agent clips the gradient to ensure , and then add Laplace noise ;

[0201] 4. Upload phase: The agent noisies the gradients Upload to the global server 42;

[0202] 5. Aggregation and evaluation phase: The server aggregates all gradients, calculates the privacy loss of the current round, and accumulates it to the total privacy loss ;

[0203] 6. Parameter adjustment stage: The server and The ratio of the dynamic calculation of the new noise parameters , learning rate and sampling rate ;

[0204] 7. Parameter distribution phase: The server distributes the updated privacy parameters and global model to the agents participating in the next round of training;

[0205] 8. Termination judgment: If , the system will suspend training or significantly increase the privacy protection strength to ensure that the overall privacy loss does not exceed the preset threshold;

[0206] This dynamic adjustment mechanism ensures that the system can accurately control privacy loss during the entire training process, while ensuring model performance and keeping privacy risks within the user's acceptable range. Experiments have shown that the privacy protection strategy of the present invention makes the system The value remains at 3.2, which is much lower than the 8.7 of the basic federated learning method. At the same time, the repair accuracy reaches 94.5%, which fully proves that this scheme has achieved an excellent balance between privacy protection and model performance.

[0207] The present invention also includes a knowledge transfer module 9, which is communicatively connected to the cross-brand maintenance knowledge base 3 and the federated learning training module 4. The primary function of this module is to construct a transferable maintenance knowledge base based on decoupled common features, enabling transfer learning of maintenance knowledge across different brands. This innovative design significantly improves the knowledge utilization efficiency and adaptability of the system.

[0208] In a preferred embodiment of the present invention, the knowledge transfer module 9 adopts the domain adaptation technology in transfer learning. Specifically, the present invention uses a deep adversarial network to achieve feature space alignment. The network includes a feature extractor , label predictor and domain discriminator There are three main components. Its objective function is defined as follows:

[0209] ,

[0210] in, is the classification loss, is the domain discrimination loss, Through this adversarial training, the system is able to learn feature representations that are shared between the source domain (existing brand) and the target domain (new brand).

[0211] In order to further improve the effect of knowledge transfer, the present invention also introduces an attention mechanism. During the feature extraction process, the system calculates the importance weights of different features:

[0212] ,

[0213] in, Indicates the Features, is a learnable parameter vector. In this way, the system can automatically identify and focus on the most important features for maintenance knowledge transfer.

[0214] Another important function of the knowledge transfer module 9 is to build a transferable maintenance knowledge base. This process involves knowledge distillation technology. Specifically, the present invention uses a teacher-student network architecture, where the teacher network is a model trained in the source domain, and the student network is optimized for the target domain. The loss function for knowledge distillation is defined as follows:

[0215] ,

[0216] in, is the cross entropy, is the KL divergence, and are the output logits of the teacher network and the student network respectively, is the temperature parameter, is the balancing factor. In this way, the knowledge in the source domain can be effectively transferred to the target domain, thereby quickly building a maintenance knowledge base for the new brand.

[0217] This paper also provides a method for sharing cross-brand home appliance repair knowledge based on federated learning. The method includes 10 main steps, and the specific implementation and technical key points of each step are detailed below.

[0218] Step S1: Acquire Multimodal Fault Data. In this step, the system collects multimodal data related to household appliance faults through various channels. This data may come from user feedback, maintenance records, sensor readings, and other sources. In a preferred embodiment of the present invention, a distributed data collection architecture is employed, with each brand or maintenance location equipped with a data collection terminal. These terminals can upload fault data in real time, ensuring the timeliness and diversity of the data.

[0219] Step S2: Feature extraction is performed on the multimodal fault data. The key to this step is how to effectively extract meaningful features from data in different modalities. The present invention uses modality-specific feature extractors, such as convolutional neural networks (CNNs) for image data and the BERT model for text data. Furthermore, the present invention introduces a cross-modal attention mechanism, which is mathematically expressed as follows:

[0220] ,

[0221] in, and Represent the characteristics of different modes, is the learnable parameter matrix, Represents the original feature vector The result after applying linear transformation. In this way, the system can capture the relationship between different modalities and extract more comprehensive and robust features.

[0222] Steps S3-S5: These steps involve maintenance solution matching and feature decoupling model training. This invention employs an innovative two-stage training strategy. In the first stage, the system uses the existing maintenance knowledge base for preliminary matching. Samples that fail to match are treated as hard examples for subsequent model training. In the second stage, the system simultaneously trains both the general feature decoupling model and the unique feature decoupling model. The training objective functions for these two models are as follows:

[0223] ,

[0224] in, is the multi-label classification loss, For adversarial loss (used to separate common features and unique features), is the regularization term (including privacy protection related regularization terms).

[0225] Steps S6-S7: These two steps involve privacy assessment and knowledge base update. The present invention adopts a differential privacy mechanism based on local sensitivity, and its privacy loss assessment formula is as follows:

[0226] ,

[0227] in, is a constant, For sensitivity, is the standard deviation of the added Gaussian noise, is the number of queries, is the probability of privacy failure, i.e., the upper bound of the probability that the differential privacy guarantee may not hold. Based on this evaluation result, the system dynamically adjusts the noise addition strategy and the knowledge base update frequency.

[0228] Steps S8-S10: These steps involve maintenance plan generation, remote guidance, and preventive maintenance. The present invention fully utilizes the previously trained model and constructed knowledge base in these steps. It is particularly noteworthy that when generating preventive maintenance recommendations, the present invention employs a multi-objective optimization algorithm, simultaneously considering multiple factors such as equipment lifespan, maintenance costs, and user convenience. Its objective function can be expressed as:

[0229] ,

[0230] in, Indicates the optimization goals, Indicates the By solving this multi-objective optimization problem, the system can generate the optimal maintenance recommendations that balance multiple factors.

[0231] The detailed description above demonstrates the innovative approach we've introduced in data processing, model training, privacy protection, and decision optimization. These innovations complement each other to create an efficient, secure, and intelligent cross-brand appliance repair knowledge sharing solution.

[0232] In order to verify the superiority of the present invention, the present invention conducted a series of simulation experiments. The experimental environment is set as follows:

[0233] Hardware environment: The server configuration is Intel Xeon E5-2680 v4 CPU, 256GB RAM, and NVIDIA Tesla V100 GPU.

[0234] Software environment: The operating system is Ubuntu 20.04 LTS, Python 3.8, PyTorch 1.9.0, and CUDA 11.2.

[0235] Experimental Dataset: This paper uses a comprehensive dataset of 100,000 home appliance repair records, covering data from five major appliance brands (A, B, C, D, and E). The dataset includes multimodal information such as text descriptions, fault images, and sensor data.

[0236] The embodiments of the present invention were compared with two comparative examples:

[0237] Example: The present invention proposes a cross-brand home appliance maintenance knowledge sharing system based on federated learning.

[0238] Comparative Example 1: Traditional centralized machine learning method, which brings together data from all brands for training.

[0239] Comparative Example 2: Basic federated learning method, which does not include innovations such as feature decoupling, knowledge transfer, and privacy loss assessment.

[0240] The experiment mainly focuses on the following indicators:

[0241] 1. Repair accuracy: The percentage of faults that are correctly diagnosed and effective repair solutions provided.

[0242] 2. Knowledge transfer efficiency: The amount of data required to achieve stable performance after a new brand joins the system.

[0243] 3. Privacy protection degree: evaluated using the differential privacy ε value.

[0244] 4. System response time: the time from receiving fault information to generating a maintenance plan.

[0245] 5. Preventive maintenance accuracy: the proportion of equipment failures accurately predicted.

[0246] The detection methods of various indicators are as follows:

[0247] 1. Repair accuracy: Using the holdout method, 20% of the data is randomly selected as the test set and the proportion of correct repair cases is calculated.

[0248] 2. Knowledge transfer efficiency: Simulate a new brand E joining the system and record the amount of data required to achieve a 90% repair accuracy rate.

[0249] 3. Privacy protection degree: Calculate the cumulative ε value of the entire training process.

[0250] 4. System response time: Calculate the average response time of 1,000 random fault diagnosis requests.

[0251] 5. Preventive maintenance accuracy: Use time series forecasting to evaluate the accuracy of failure predictions within the next 30 days.

[0252] The experimental results are shown in the following table:

[0253] index Example Comparative Example 1 Comparative Example 2 Maintenance accuracy 94.5% 89.2% 91.8% Knowledge transfer efficiency (amount of data required) 5000 20000 12000 Privacy protection degree (ε value) 3.2 N / A 8.7 System response time 0.8 1.5 1.2 Preventive maintenance accuracy 87.3% 79.6% 83.1%

[0254] By analyzing the above experimental results, we can draw the following conclusions:

[0255] 1. Repair Accuracy: The embodiment of the present invention significantly outperformed the two comparative examples, primarily due to the application of feature decoupling and multimodal data fusion technology. By separating common features from brand-specific features, the system can more accurately identify fault modes and provide more precise repair recommendations.

[0256] 2. Knowledge Transfer Efficiency: This invention demonstrates outstanding performance in transferring knowledge to new brands, achieving stable performance with only 5,000 pieces of data, significantly less than the amount required for the comparative example. This demonstrates that the knowledge transfer module of this invention can effectively leverage existing knowledge and quickly adapt to the repair characteristics of new brands.

[0257] 3. Privacy Protection: The ε value of the proposed method is significantly lower than that of the basic federated learning method (Comparative Example 2), indicating that it is more effective in protecting data privacy. This is due to the privacy loss assessment module and dynamic adjustment strategy introduced in the proposed method.

[0258] 4. System Response Time: This system offers the fastest response time, primarily due to the application of knowledge graphs and efficient feature extraction methods. Fast response times are crucial for real-world maintenance scenarios and can significantly improve maintenance efficiency.

[0259] 5. Preventive Maintenance Accuracy: This method achieves the best performance in predicting potential equipment failures, thanks to its combined use of survival analysis models and multimodal data. Highly accurate preventive maintenance can significantly reduce unplanned equipment downtime and improve user satisfaction.

[0260] In summary, the present invention demonstrates significant advantages across all key metrics. Its outstanding performance in knowledge transfer efficiency and privacy protection particularly demonstrates its innovative and practical value in cross-brand home appliance repair knowledge sharing. These results demonstrate that the present invention not only improves repair accuracy and efficiency but also enables effective knowledge sharing and transfer while protecting data privacy, providing strong support for the intelligent and digital transformation of the home appliance repair industry.

[0261] Best embodiment: Based on the above experimental results, the best embodiment of the present invention adopts the following configuration:

[0262] 1. Feature decoupling model: A 5-layer deep neural network is used, with the number of hidden layer neurons being 512, 256, 128, 64, and 32 respectively.

[0263] 2. Knowledge graph: Stored in the Neo4j graph database, it contains approximately 500,000 nodes and 2 million relationships.

[0264] 3. Federated learning parameters: 30% of the clients are selected to participate in each round of training, the number of local training rounds is 5, and the global aggregation frequency is once every 10 rounds.

[0265] 4. Privacy protection: A Gaussian mechanism is used, with the initial ε set to 0.1 and a dynamic adjustment step size of 0.05.

[0266] 5. Preventive maintenance model: Combining the Cox proportional hazards model and the LSTM network, the prediction window is set to 90 days.

[0267] This optimal embodiment not only ensures high performance but also fully considers the system's scalability and practical application needs. Through carefully adjusted parameter settings, the present invention can achieve optimal results in various complex home appliance repair scenarios, providing users with accurate, timely, and safe repair services.

[0268] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A cross-brand home appliance maintenance knowledge sharing system based on federated learning, characterized by , including: a multimodal fault data collection module, used to: obtain multimodal fault data, the multimodal fault data including image data, text data, fault phenomenon data and multimodal instruction data; preprocess the image data and text data; a maintenance knowledge parsing module, connected in communication with the multimodal fault data collection module, used to: receive the preprocessed image data and text data sent by the multimodal fault data collection module; establish a feature decoupling model of the multimodal fault data based on the preprocessed image data and text data; a cross-brand maintenance knowledge base, connected in communication with the maintenance knowledge parsing module, used to: store brand-wide common fault types and Common features, as well as household appliance-specific fault types and features; a federated learning training module, communicating with the maintenance knowledge parsing module and the cross-brand maintenance knowledge base, for: establishing a federated learning training mechanism using a differential privacy homomorphic encryption privacy protection strategy; and using a gradient aggregation mechanism to exchange gradients and conduct collaborative training between federated learning agents; an augmented reality maintenance guidance terminal, communicating with the cross-brand maintenance knowledge base, for: identifying target parts and providing maintenance guidance; a preventive maintenance module, communicating with the cross-brand maintenance knowledge base and the federated learning training module, for: calculating the maintenance cycle of target equipment and generating a preventive maintenance plan; The maintenance knowledge parsing module includes: an image decoupling encoder for inputting image samples and outputting probability distributions of common fault types and probability distributions of unique fault types; performing multi-label classification on image feature vectors of each brand based on a deep neural network; a text decoupling encoder for inputting text samples and outputting probability distributions of common fault types and probability distributions of unique fault types; performing multi-label classification on text feature vectors of each brand based on a deep neural network; an attention matrix for recording the correspondence between common fault features and unique fault features; and using a probabilistic attention mechanism to separate common fault modes from brand-specific fault modes. The federated learning training module includes: a data collector for collecting maintenance data from a local equipment maintenance knowledge database; a global server for coordinating the federated learning process and performing model aggregation; and a federated learning agent for performing model training locally and exchanging gradients with the global server. It also includes: a privacy loss assessment module, which is in communication with the federated learning training module and is used to: calculate the privacy loss in the gradient aggregation process; and dynamically adjust the training strategy according to the privacy loss assessment results.

2. The system according to claim 1, characterized in that The cross-brand maintenance knowledge base is constructed in the form of a knowledge graph, including: entities, which represent fault types, symptoms and maintenance solutions; relationships, which represent the connections between entities; attributes, which describe the characteristics of entities; wherein the knowledge graph realizes the association and expansion of knowledge by linking the knowledge base corpus.

3. The system according to claim 1, characterized in that The augmented reality maintenance guidance terminal includes: a data sampler for collecting image and depth information of the target part; a positioning feedback module for obtaining the coordinate position of the positioned part from the cross-brand maintenance knowledge base; and realizing alignment between the image and the part; and a disassembly and assembly navigation module for obtaining disassembly and assembly steps in three-dimensional space from the cross-brand maintenance knowledge base and providing visual disassembly and assembly guidance to the user.

4. The system according to claim 1, characterized in that The preventive maintenance module includes: a survival analysis model, which is used to learn sample features and predict the remaining service life of the target equipment; a feature selection unit, which is used to screen important feature variables; a machine learning model, which is used to train the prediction model based on the screened feature variables; and a maintenance plan generator, which is used to generate a preventive maintenance plan based on the prediction results.

5. The system according to claim 1, characterized in that , also includes: a multimodal data fusion module, which is in communication with the maintenance knowledge parsing module and the cross-brand maintenance knowledge base, and is used to: use maintenance knowledge and fault feature vectors as inputs of the graph neural network model; combine the maintenance knowledge in the maintenance knowledge base, the maintenance plan obtained by the knowledge graph, and the voice and image information during the maintenance process to generate a comprehensive maintenance plan.

6. The system according to claim 1, characterized in that , also includes: a knowledge transfer module, which is in communication connection with the cross-brand maintenance knowledge base and the federated learning training module, and is used to: build a transferable maintenance knowledge base based on the decoupled common features; and realize the transfer learning of maintenance knowledge between different brands.

7. A cross-brand home appliance maintenance knowledge sharing method based on federated learning, characterized by , including the following steps: S1: acquiring multimodal fault data; S2: performing feature extraction on the multimodal fault data; S3: matching maintenance plans in a cross-brand maintenance knowledge base according to the multimodal fault data, and transferring the fault data not in the cross-brand maintenance knowledge base to a federated learning training module for training; S4: using the cross-brand maintenance knowledge base to classify the unmatched fault data, and using multi-label classification loss and privacy-preserving regularization loss function to train a general feature decoupling model for specific category samples; S5: using the cross-brand maintenance knowledge base to classify the unmatched fault data, and using multi-label classification loss and privacy-preserving regularization loss function to train a unique feature decoupling model for general category samples; S6: performing differential privacy evaluation based on the universal feature decoupling model and the unique feature decoupling model through a differential privacy homomorphic encryption protection strategy; S7: evaluating the cross-brand maintenance knowledge base according to the differential privacy evaluation result, and the cross-brand maintenance knowledge base updates the differential privacy; S8: Generate maintenance plans based on the updated cross-brand maintenance knowledge base; S9: Use augmented reality technology to provide remote maintenance guidance; S10: Based on survival analysis methods, predict equipment life and generate preventive maintenance recommendations.

Images