Intelligent glasses fault early warning system and method based on big data analysis

Abstract

The application relates to the technical field of intelligent wearing equipment fault diagnosis, and particularly discloses a fault early warning system and method for intelligent glasses based on big data analysis. The system adopts an edge-cloud collaborative architecture, and comprises an edge processing module arranged locally on the glasses, a deep analysis module arranged in the cloud, and a collaborative communication and decision module. The edge processing module performs real-time feature extraction and abnormality scoring on sensor data; the deep analysis module performs deep fault diagnosis and trend prediction on the reported information; and the collaborative communication and decision module intelligently decides data reporting and early warning triggering through a dynamic threshold self-adaptive mechanism. The method ensures millisecond-level real-time response, reduces data transmission load, and improves the detection accuracy of early fault signs.

Description

Technical Field

[0001] This invention belongs to the field of fault diagnosis technology for smart wearable devices, specifically relating to a fault early warning system and method for smart glasses based on big data analysis. Background Technology

[0002] In the fields of IoT and wearable device technology, smart terminals integrate multiple sensors to perceive and interact with the physical world, and their data acquisition and processing capabilities are the foundation for supporting upper-layer intelligent applications. As a typical wearable device integrating display, computing, and sensing modules, the stable and reliable operation of smart glasses is crucial for user experience and functional implementation.

[0003] Equipment fault early warning based on big data analytics is a key technology for ensuring the long-term stable operation of equipment. This technology aims to continuously collect equipment operating data, use data analysis models to identify potential fault modes, and thus issue early warnings before faults occur, enabling predictive maintenance.

[0004] Current technologies typically upload all sensor data collected by smart glasses to a cloud platform for centralized analysis and processing. However, smart glasses sensor data is characterized by its large volume and high real-time requirements. Streaming all data to the cloud for analysis introduces significant network transmission latency and consumes a large amount of communication bandwidth, causing the early warning system's response speed to fail to meet millisecond-level real-time requirements. This centralized processing architecture struggles to effectively capture early, subtle anomalies that occur during device operation, and these subtle anomalies are often precursors to major failures. Therefore, how to achieve low-latency, high-precision early fault symptom detection in massive real-time data streams has become a pressing technical challenge in the field of predictive maintenance for smart glasses. Summary of the Invention

[0005] The purpose of this invention is to provide a fault early warning system and method for smart glasses based on big data analysis, so as to solve the technical contradiction in the prior art that the network latency and bandwidth consumption caused by uploading all sensor data to the cloud for centralized analysis are large, which makes it impossible to meet the millisecond-level real-time fault early warning requirements of smart glasses and make it difficult to effectively capture early signs of minor anomalies.

[0006] To achieve the above objectives, the present invention provides a fault early warning system for smart glasses based on big data analysis, comprising:

[0007] An edge processing module deployed locally on the smart glasses is used to perform real-time online processing of raw time-series data generated by the sensors built into the smart glasses.

[0008] A cloud-based analytics module, deployed on a remote server, receives and processes data reported by the edge processing module, performing in-depth fault mode diagnosis and fault evolution prediction.

[0009] The collaborative communication and decision-making module is used to formulate and dynamically adjust the data reporting strategy and early warning triggering mechanism between the edge processing module and the cloud analysis module.

[0010] Preferably, the edge processing module includes a data preprocessing submodule, a lightweight feature extraction submodule, and a local anomaly detection submodule;

[0011] The data preprocessing submodule is used to standardize and filter the raw sensor data for noise reduction.

[0012] The lightweight feature extraction submodule is used to simultaneously extract time-domain statistical features and frequency-domain energy features from the preprocessed data stream in fixed time windows.

[0013] The local anomaly detection submodule embeds a pre-trained single-class support vector machine model, which is used to construct the closed boundary of the normal operating state of the device in the high-dimensional feature space, and calculate the shortest distance from the feature vector extracted in each time window to the closed boundary, and define the distance as the real-time anomaly score.

[0014] Preferably, the cloud-based analysis module includes a feature fusion and enhancement submodule, a deep diagnostic model submodule, and a prediction and knowledge base submodule;

[0015] The feature fusion and enhancement submodule is used to receive condensed feature sequences and anomaly score sequences reported by edge processing modules of multiple smart glasses, and to align and splice historical feature sequences from the same device in the time dimension to construct enhanced feature vectors. At the same time, it performs spatial correlation and comparative analysis on similar sensor features from different devices in the same time period.

[0016] The deep diagnostic model submodule uses a bidirectional long short-term memory neural network based on an attention mechanism as the core diagnostic model. The input is a long sequence of enhanced feature vectors processed by the feature fusion and enhancement submodule, and the output is a probability distribution vector of the fault type corresponding to the current device state.

[0017] The prediction and knowledge base submodule is used to extrapolate and predict the probability of various faults occurring within a specific time window in the future based on the fault probability sequence output by the deep diagnostic model and the exponential smooth state space model, and to maintain a global fault knowledge graph.

[0018] Preferably, the collaborative communication and decision-making module includes a dynamic threshold adaptive unit; the dynamic threshold adaptive unit is used to continuously monitor the real-time anomaly score stream from the edge processing module and the fault probability feedback from the cloud analysis module, and run a two-layer threshold decision logic;

[0019] The first layer of the dual-layer threshold decision logic is the local real-time decision threshold. When the real-time anomaly score is greater than the threshold, the edge processing module is instructed to urgently report the original data snapshot and condensed feature vector of the current time window to the cloud analysis module, and simultaneously issue a primary alarm to the user on the glasses.

[0020] The second layer of the dual-layer threshold decision logic is the cloud-based cumulative decision threshold. The cloud-based cumulative decision threshold is dynamically calculated and distributed by the cloud analysis module based on the frequency of abnormal reports from similar device groups recently received, the predicted failure probability of the current device, and the case matching degree in the knowledge base. When the cumulative weighted value of the abnormal score reported by the edge processing module within a certain time period is greater than the cloud-based cumulative decision threshold, the edge processing module is instructed to start the periodic feature reporting mode.

[0021] Preferably, the kernel function of the single-class support vector machine model in the local anomaly detection submodule is a radial basis function;

[0022] The training process of the single-class support vector machine model is completed in the cloud, and the training data comes from the sensor feature vector set collected by a large number of normal smart glasses in various typical usage scenarios.

[0023] After the trained model parameters are compressed and optimized, they are deployed to the edge processing module of each smart glasses through a secure firmware upgrade channel.

[0024] Preferably, the process by which the dynamic threshold adaptive unit calculates the cumulative decision threshold in the cloud is as follows:

[0025] First, the cloud analysis module calculates the incidence rate of abnormal reporting events with the same device model and sensor type as the current device in the global device network over the past 24 hours, which serves as the group baseline risk coefficient.

[0026] Secondly, obtain the maximum predicted probability of the main failure modes of the current device in the future from the prediction and knowledge base submodule;

[0027] Next, query the global fault knowledge graph to calculate the matching similarity between the recent abnormal feature patterns of the current device and historical known fault cases;

[0028] Finally, the group baseline risk coefficient, the maximum probability of failure prediction, and the case matching similarity are weighted and fused together, and a dynamic threshold coefficient between 0.1 and 0.5 is generated through a preset S-shaped function mapping. This coefficient is multiplied by the basic threshold to obtain the cloud cumulative decision threshold issued to the specific device.

[0029] Preferably, it also includes an online model update module, which is deployed in the cloud and used to continuously learn from newly generated fault case data;

[0030] When the cloud analysis module confirms a new fault diagnosis case and the difference between the feature pattern of the case and the existing pattern in the knowledge base is greater than a preset threshold, the online model update module is activated. It uses an incremental learning algorithm to fine-tune the parameters of the bidirectional long short-term memory neural network in the deep diagnostic model sub-module, and at the same time updates the nodes and relationships in the global fault knowledge graph.

[0031] The updated model parameters and knowledge graph incremental packages are encrypted and verified before being pushed to online smart glasses devices in batches.

[0032] Preferably, the time-domain statistical features include the mean, variance, peak-to-peak value, and zero-crossing rate of the data within the window;

[0033] The frequency domain energy characteristics are calculated by performing a fast Fourier transform on the window data to determine the energy percentage within several preset key frequency bands.

[0034] Preferably, the feature fusion and enhancement submodule aligns and splices historical feature sequences from the same device in the time dimension to construct an enhanced feature vector with a longer time span;

[0035] This invention provides a fault warning method for smart glasses based on big data analysis, and uses the aforementioned fault warning system for smart glasses based on big data analysis to achieve fault warning.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] 1. This invention constructs a hierarchical processing architecture that integrates edge and cloud, decentralizing the initial processing and feature extraction of massive real-time data to the local edge of the smart glasses. Only condensed abnormal feature information or filtered snapshots of raw data are uploaded to the cloud, fundamentally solving the problems of network bandwidth congestion and excessive transmission latency caused by uploading the entire data stream. Millisecond-level anomaly score calculation at the edge enables the system to react instantly to minute fluctuations in device operation, meeting the real-time requirements of smart glasses fault warnings.

[0038] 2. This invention uses a single-class support vector machine model trained on normal data for anomaly detection at the edge. This unsupervised learning method does not require a large number of fault samples for training and is particularly suitable for the early deployment stage where fault samples are scarce. It can effectively detect unknown types of anomalies that deviate from the normal pattern and enhance the ability to discover early and new fault symptoms.

[0039] 3. The dynamic threshold adaptive collaborative mechanism designed in this invention achieves an intelligent balance between early warning sensitivity and system resource consumption through a two-layer decision-making process involving both local real-time thresholds and cloud-based dynamic cumulative thresholds. Emergency anomalies trigger immediate reporting and in-depth analysis, while suspected anomalies are filtered through cumulative evaluation and dynamic thresholds. This avoids missed reports and prevents increased device energy consumption and excessive cloud processing pressure caused by frequent data uploads, giving the system good scalability and practicality.

[0040] 4. The cloud-based deep diagnostic model of this invention integrates attention mechanisms and long short-term memory networks, enabling in-depth mining of enhanced features in long-term sequences, accurate identification of complex fault modes, and quantification of their occurrence probabilities. Combining fault evolution prediction and a global knowledge graph, the system can not only diagnose the current state but also predict fault development trends and provide maintenance decision support, achieving a leap from simple anomaly detection to intelligent predictive maintenance.

[0041] 5. The online model update module introduced in this invention enables the system to continuously evolve using constantly generated field data. New fault cases are quickly absorbed and used to optimize the diagnostic model and knowledge base, and synchronized to terminal devices through a security mechanism, forming a closed-loop learning and optimization system. This ensures that the early warning model can adapt to changes brought about by aging equipment hardware, software updates, and new usage scenarios, maintaining a high early warning accuracy rate over the long term. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the overall technical solution architecture of the fault early warning system for smart glasses based on big data analysis proposed in this invention;

[0043] Figure 2 This is a schematic diagram of the core principle framework of the hierarchical intelligent processing architecture with edge-cloud collaboration in this invention;

[0044] Figure 3 This is a logical flow diagram of the edge processing module in this invention;

[0045] Figure 4 This is a logical flow diagram of the cloud analysis module in this invention;

[0046] Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow of the collaborative communication and decision-making module in this invention. Detailed Implementation

[0047] This invention provides a fault warning system for smart glasses based on big data analytics. This system employs a hierarchical intelligent processing architecture that coordinates edge and cloud computing. Please refer to the appendix. Figures 1 to 5The diagram illustrates the overall technical architecture of the system. The entire system consists of an edge processing module deployed locally on the smart glasses, a cloud analysis module deployed on a remote server, and a collaborative communication and decision-making module that connects and coordinates the two modules.

[0048] These three modules interact with each other via a wireless communication network, forming a closed-loop intelligent early warning system. Please refer to the appendix. Figure 2 The diagram further reveals the core principle framework of the hierarchical intelligent processing architecture that coordinates edge and cloud. Its core lies in decomposing the fault early warning task into two logical layers: real-time anomaly perception on the edge side and deep diagnosis and prediction on the cloud side, and linking them through a dynamic and adaptive collaborative protocol.

[0049] The edge processing module is the system's front-end sensing and preliminary computing unit, directly integrated into the smart glasses' hardware system. This module is responsible for real-time online processing of raw time-series data generated by various sensors built into the smart glasses. These sensors typically include, but are not limited to, inertial measurement units, ambient light sensors, proximity sensors, temperature sensors, gyroscopes, accelerometers, and microphones. The edge processing module further consists of a data preprocessing submodule, a lightweight feature extraction submodule, and a local anomaly detection submodule. Please refer to the appendix for its internal logic flow. Figure 3 .

[0050] The data preprocessing submodule first receives raw data streams from each sensor. The raw data streams are typically discrete time series generated at a specific sampling rate; for example, the sampling rate of the inertial measurement unit might be 200 Hz, while the sampling rate of the temperature sensor might be 1 Hz.

[0051] The first step in the data preprocessing submodule is data standardization. This step is performed independently for each type of sensor data, aiming to eliminate the impact of differences in the dimensions and magnitudes of different sensors on subsequent analysis. Specifically, for each sensor data stream, the system maintains dynamically updated mean and standard deviation statistics. When a new data point arrives, standardization is performed using the following calculation: subtract the moving average of the sensor data stream over the past statistical period from the original data value, and then divide by the moving standard deviation over that statistical period. The length of the statistical period is configured based on the stability of the sensor data; for high-frequency dynamic data such as accelerometer data, the statistical period is typically set to 10 seconds; for low-frequency slowly varying data such as temperature data, the statistical period can be set to 300 seconds. The standardized data is mapped to a distribution interval with a mean of 0 and a variance of 1.

[0052] After standardization, the data preprocessing submodule performs filtering and noise reduction. Different filtering algorithms are used for different types of sensor data and noise characteristics. For data from inertial measurement units and accelerometers, which are susceptible to high-frequency random noise interference, a first-order low-pass digital filter is used. The cutoff frequency of this filter is set according to the main effective signal frequency band of the sensor; for example, for acceleration signals capturing human head motion, the cutoff frequency is set to 20 Hz. The filter's transfer function is implemented in the digital domain, and the input sequence is convolved in real time using a difference equation to effectively filter out noise components higher than the cutoff frequency. For data from ambient light sensors, which may be affected by pulse interference, a median filtering algorithm is used.

[0053] The algorithm sets a sliding window with a length of 5 sampling points, sorts the data within the window by size, and takes the median as the output value at that moment, thereby effectively suppressing isolated impulse noise points. Through standardization and filtering noise reduction, the data preprocessing submodule outputs a clean and normalized sensor time-series data stream, laying the foundation for subsequent feature extraction.

[0054] The lightweight feature extraction submodule receives a clean data stream from the data preprocessing submodule and simultaneously extracts temporal statistical features and frequency domain energy features within a fixed time window. The length of the time window is configured differently based on the sensor type and data sampling rate, which is crucial for ensuring the timeliness and representativeness of the features. For sensor data with a sampling rate greater than 100 Hz, such as angular velocity and linear acceleration data from an inertial measurement unit, the time window length is set to 256 milliseconds.

[0055] For an accelerometer data stream with a sampling rate of 200 Hz, a time window of 256 milliseconds will contain approximately 51 data points.

[0056] For sensor data with a sampling rate of less than 10 Hz, such as ambient light sensors or temperature sensors, the time window length is set to 5 seconds.

[0057] For a temperature sensor with a sampling rate of 1 Hz, a 5-second window contains 5 data points.

[0058] This differentiated configuration ensures that high-frequency data can capture transient features, while low-frequency data can accumulate sufficient information about changes.

[0059] Within each time window, the lightweight feature extraction submodule computes two sets of features in parallel. The first set consists of temporal statistical features, including four core metrics:

[0060] The arithmetic mean of all data points within the window is used to characterize the DC component or baseline level of the signal;

[0061] The variance of all data points within the window is used to characterize the intensity of the signal's fluctuation around the mean.

[0062] The difference between the maximum and minimum values ​​within the window is the peak-to-peak value, which is used to characterize the overall dynamic range of the signal.

[0063] The zero-crossing rate is the number of times a signal waveform crosses the zero level, and it is used to roughly estimate the frequency components of a signal.

[0064] These statistics are calculated using streaming algorithms, which update intermediate variables such as cumulative sum, sum of squares, maximum and minimum values ​​in real time as data points enter the window one by one. This allows the feature values ​​to be obtained immediately when the window ends, resulting in low computational complexity and meeting real-time requirements.

[0065] The second group consists of frequency domain energy features. The lightweight feature extraction submodule performs a Fast Fourier Transform (FFT) on the data sequence within each time window. Due to limited computational resources at the edge, the number of points in the FFT is adaptively determined based on the window length and sampling rate, typically using a length closest to an integer power of 2. For example, for 200 Hz data within a 256-millisecond window, a 64-point FFT is used. The frequency spectrum of the signal is obtained after the transform.

[0066] The system predefines several key frequency bands associated with potential equipment failure modes. For example, for acceleration signals reflecting motor vibration, the key frequency bands may be set to 50 to 100 Hz, 100 to 200 Hz, etc.

[0067] For each preset key frequency band, calculate the sum of the squares of the amplitudes of all frequency components within that frequency band, which is the energy of that frequency band.

[0068] Then, the proportion of energy in this frequency band to the total energy of the entire signal is calculated, which is used as the energy proportion characteristic of this frequency band.

[0069] Finally, for the time window and a class of sensor data, the lightweight feature extraction submodule outputs a feature vector that fuses time-domain and frequency-domain information. This feature vector serves as the input for subsequent anomaly detection.

[0070] The local anomaly detection submodule embeds a pre-trained single-class support vector machine model, which is trained on massive amounts of normal operating condition data before the equipment leaves the factory. Its core function is to construct a closed decision boundary for the normal operating status of the equipment in a high-dimensional feature space.

[0071] This model is trained using only a large number of normal samples, without requiring any faulty samples, making it particularly suitable for scenarios where samples are scarce in the early stages of a fault. The model's kernel function uses radial basis functions, whose mathematical expression focuses on calculating the similarity between two feature vectors. The training process is completed in the cloud, using feature vectors collected and extracted from tens of thousands of normal smart glasses under various typical usage scenarios to form the training set. The training objective is to find a hyperspherical boundary with the smallest volume that can enclose the vast majority of normal samples, or to find a separating hyperplane in a high-dimensional feature space that separates the origin from the normal samples.

[0072] The trained model parameters include the set of support vectors, their corresponding coefficients, and the parameters of the kernel function. These parameters undergo model compression and quantization optimization to transform them into a lightweight format suitable for embedded devices, such as converting floating-point numbers to fixed-point numbers. The optimized model parameters are then deployed as encrypted data packets to the local anomaly detection submodule of the edge processing module in each smart glasses via a secure firmware upgrade channel.

[0073] During runtime, for each feature vector output by the lightweight feature extraction submodule, the local anomaly detection submodule calculates its distance to the predefined normal state boundary. This distance is quantized into a real value and defined as a real-time anomaly score. The calculation process involves performing kernel function calculations on the feature vector and all support vectors stored in the model, and then performing weighted summation and offset adjustments based on the model parameters. The smaller the real-time anomaly score, the closer the feature vector is to the center of the normal state cluster; the larger the value, the further it deviates from the normal state. This real-time anomaly score calculation is completed within milliseconds, enabling instantaneous assessment of the device's operating status.

[0074] The cloud-based analytics module is deployed on a remote server cluster with powerful computing and storage capabilities. It receives and processes data reported by the edge processing module, performing in-depth fault mode diagnosis and fault evolution prediction. This module further comprises a feature fusion and enhancement submodule, a deep diagnostic model submodule, and a prediction and knowledge base submodule. Please refer to the appendix for its logical flow. Figure 4 .

[0075] The Feature Fusion and Enhancement submodule serves as the entry point and preprocessing hub for cloud-based data processing. It receives data reported from edge processing modules of thousands of smart glasses. The reported data is primarily divided into two categories: one is the raw data snapshot and corresponding condensed feature vectors that trigger emergency reporting; the other is higher-resolution feature sequences sent in periodic feature reporting modes. The submodule's primary task is to fuse and enhance the data in both the spatiotemporal dimensions. For feature sequences from the same device, the Feature Fusion and Enhancement submodule performs alignment and concatenation in the time dimension. The system maintains a time-series buffer for each device. When a new feature data packet arrives, the submodule inserts it into the correct time-series position in the corresponding buffer for that device based on the timestamp information in the packet. Subsequently, the system extracts continuous feature vector sequences from the buffer using longer analysis time windows, such as the past 30 minutes or 2 hours, and concatenates them in the feature dimension to form a longer-spanning, more information-rich enhanced feature vector. This long sequence vector reflects the evolution of fault characteristics.

[0076] Simultaneously, the feature fusion and enhancement submodule also performs spatial correlation analysis. It aggregates similar sensor features from different devices within the same time period. For example, for the X-axis variance feature of the accelerometer of a certain model of smart glasses, the submodule collects the feature values ​​of all online devices of that model in the past hour and calculates its statistical distribution, such as mean, standard deviation, and percentiles. By comparing the feature values ​​of an individual device with the statistical distribution of the group, it can be identified whether the device exhibits significant anomalies within the group, which helps distinguish between common environmental interference and individual hardware faults. The data after spatiotemporal fusion and enhancement is organized into a structured tensor format and used as input to the deep diagnostic model.

[0077] The deep diagnostic model submodule is the core of cloud-based analytics, employing a bidirectional long short-term memory neural network based on an attention mechanism as its core diagnostic model. This network is specifically designed to process long-term series data and capture long-term dependencies and key time points. The network's input is the long-sequence enhanced feature vector output from the feature fusion and enhancement submodule. Assuming the time step of the enhanced feature vector is... The feature dimension of each time step is The input is a shape of × The matrix.

[0078] The bidirectional Long Short-Term Memory (LSTM) neural network consists of two LSM layers: a forward layer and a backward layer. Each LSM layer has 128 hidden units. The forward layer is arranged chronologically from step 1 to step 2. The first step processes the sequence, capturing dependency information from the past to the future; the backward layer processes it in reverse order from the first step. The first step processes the sequence, capturing dependency information from the future to the past. For each time step, the forward and backward layers each output a 128-dimensional hidden state vector. The forward and backward hidden states at the same time step are concatenated to obtain a 256-dimensional fused hidden state vector, which contains the full sequence information of the context at that time step.

[0079] The attention mechanism is applied to all outputs of the network's final bidirectional long short-term memory layer. The fused hidden state sequence is calculated over 10 time steps. The purpose of the attention mechanism is to assign a weight to each time step in the sequence, and the magnitude of the weight represents the importance of the features of that time step to the final fault diagnosis decision.

[0080] The specific calculation process is as follows: First, the 256-dimensional hidden state at each time step is mapped to an attention energy value through a trainable fully connected layer; then, the Softmax function is used to apply the attention energy to all... The attention energy values ​​at each time step are normalized to obtain the attention weight for each time step, and the sum of all weights is 1. Finally, the hidden states of all time steps are weighted and summed according to their corresponding attention weights to obtain a 256-dimensional context vector. This context vector focuses on the most critical and anomalous time segments in the sequence, greatly improving the model's diagnostic sensitivity for early, intermittent anomalous patterns.

[0081] ;

[0082] ;

[0083] ;

[0084] In the above formula, Representing the The fused hidden state vector at each time step. and These are trainable weight matrices and bias vectors. These are trainable weight vectors that together form a fully connected layer used to compute the attention energy value at each time step. . For the first The attention energy value corresponding to each time step. It is the first one obtained after Softmax normalization. Attention weights for each time step. It is the final weighted context vector.

[0085] This context vector is then fed into a fully connected classification layer. The number of neurons in the classification layer equals the number of predefined potential fault modes in the system, for example, 10. The classification layer outputs a fault type probability distribution vector. Each element in this vector corresponds to the probability of a specific fault mode occurring, and the sum of all elements is 1. For example, the output vector might be represented as [0.02, 0.85, 0.01, 0.10, 0.02], indicating that the current device state has an 85% probability of being diagnosed as the second fault mode, with a 10% probability of being diagnosed as the fourth fault mode. This probabilistic output provides a quantitative basis for subsequent risk assessment and decision-making.

[0086] The prediction and knowledge base submodule is responsible for trend prediction and knowledge management based on the fault probability sequence output by the deep diagnostic model. This submodule uses an exponentially smoothed state-space model to extrapolate and predict the probability time series of various faults. The model receives a sequence of observed fault probabilities over a past period, such as 24 hours. It models and updates the horizontal and trend components of the sequence using state-space equations. The model can estimate the probability of various faults occurring within a specific future time window, such as the next 2 hours or 6 hours. This achieves a leap from current state diagnosis to future risk prediction.

[0087] In addition, the prediction and knowledge base submodule maintains a global fault knowledge graph. This knowledge graph stores historical diagnostic cases, fault solutions, and causal relationship chains between different fault modes in a graph structure. Nodes in the graph represent entities, such as specific fault modes, symptom characteristics, affected hardware components, solution steps, and device models. Edges between nodes represent relationships, such as "cause," "manifests as," "can be resolved by," and "common in..." models. Whenever a new fault diagnosis case is confirmed in the cloud, the system structures the case's characteristic patterns, diagnostic results, handling measures, and time context, and adds them as new nodes and edges to the knowledge graph.

[0088] Knowledge graphs support complex graph queries. For example, when the system detects that features A and B are both abnormal, it can query which fault modes in the graph are associated with both features, thereby assisting in diagnosis. The continuous accumulation of the graph enables the system to possess increasingly rich domain knowledge.

[0089] The collaborative communication and decision-making module serves as an intelligent hub connecting the edge and the cloud, responsible for formulating and dynamically adjusting data reporting strategies and early warning triggering mechanisms between the two. The core of this module is the dynamic threshold adaptive unit; its multi-level interaction relationships and data flow are detailed in the appendix. Figure 5 This unit continuously monitors two core data streams: one is the real-time anomaly score stream from the edge processing module; the other is the fault probability feedback and group statistics information from the cloud analysis module.

[0090] The dynamic threshold adaptive unit internally operates a two-layer threshold decision logic, which is crucial for achieving a balance between resources and sensitivity. The first layer of decision is based on a local real-time decision threshold. This threshold is a fixed value preset according to the device model and sensor type and stored in the edge processing module. When the real-time anomaly score calculated by the local anomaly detection submodule exceeds this local real-time decision threshold, it indicates that the device may have experienced a significant transient anomaly. At this time, the collaborative communication and decision module immediately sends a command to the edge processing module through the control channel.

[0091] After receiving the instruction, the edge processing module performs the following actions:

[0092] First, cache and package the original sensor data snapshot corresponding to the time window that triggered the exception;

[0093] Secondly, the condensed feature vector extracted from this time window is appended to the data packet; finally, the data packet is urgently reported to the feature fusion and enhancement submodule of the cloud analysis module via a high-speed data channel, such as when a WiFi network is available.

[0094] Simultaneously, the collaborative communication and decision-making module instructs the human-computer interaction unit of the smart glasses to flash specific warning icons on the glasses' miniature display screen or emit brief alert sounds via bone conduction headphones, providing the user with a basic visual or auditory alarm. This mechanism ensures an extremely rapid response to serious and sudden anomalies.

[0095] The second-level decision is based on a cloud-accumulated decision threshold. This threshold is not a fixed value, but is dynamically calculated by the cloud analysis module based on multi-dimensional information and then sent to specific devices. The process of the dynamic threshold adaptive unit calculating the second-level threshold is a complex multi-factor fusion process.

[0096] The specific steps are as follows:

[0097] First, the cloud analytics module calculates the frequency of abnormal reporting events with the same device model and sensor type as the current device across the entire global device network over the past 24 hours. This frequency value is then normalized to obtain a baseline risk coefficient for the group, ranging from 0 to 1. This coefficient reflects the overall health status of this type of device group during the current period.

[0098] Secondly, the maximum predicted probability of the main failure modes for the current device within the next 2 hours is obtained from the prediction and knowledge base submodule. For example, the probability of a "display driver chip overheating" failure occurring within the next 2 hours is predicted to be 30%.

[0099] Next, the global fault knowledge graph is queried. The recent abnormal feature patterns reported by the current device are compared with the feature patterns of historical known fault cases stored in the knowledge graph to calculate similarity. Similarity calculation typically uses cosine similarity or a distance metric based on graph embedding to obtain a case matching similarity score, ranging from 0 to 1.

[0100] Finally, the three factors mentioned above—the group baseline risk coefficient, the maximum probability of failure prediction, and the case matching similarity—are weighted and fused. The weighting coefficients are determined based on experience or offline optimization, for example, assigned weights of 0.3, 0.4, and 0.3 respectively. The weighted sum yields a preliminary comprehensive risk index. To map this index into a coefficient suitable for threshold comparison, it is input into a preset S-shaped function. The S-shaped function maps the input to an output range of 0.1 to 0.5. This output value is the dynamic threshold coefficient. The system presets a basic cumulative threshold, for example, a baseline value of 50 for the cumulative sum of anomaly scores. Multiplying the dynamic threshold coefficient by the basic cumulative threshold yields the final second-layer cloud-based cumulative decision threshold issued to that specific device. For example, if the dynamic coefficient is 0.3, then the dynamic threshold is 15.

[0101] The edge-side collaborative communication and decision-making module agent continuously calculates the cumulative weighted value of real-time anomaly scores over a certain time period. This cumulative weighted value is not a simple summation, but rather an exponentially decaying weighted sum of anomaly scores over a past period, giving higher weight to recent anomaly scores. When this cumulative weighted value exceeds the dynamic threshold issued by the cloud, it indicates that although the device has not experienced a drastic instantaneous anomaly, it remains in a sub-healthy or suspected faulty state. At this point, the collaborative communication and decision-making module instructs the edge processing module to initiate a periodic feature reporting mode.

[0102] In this mode, the edge processing module no longer simply reports data that triggers events, but instead uploads higher-resolution feature sequences to the cloud at a lower frequency, such as once every 5 minutes. This data provides material for the cloud to conduct more in-depth long-term trend analysis and pattern mining, while avoiding the energy consumption and bandwidth pressure caused by continuous full-speed reporting.

[0103] Furthermore, the system also includes an online model update module. This module is also deployed in the cloud and tightly integrated with the cloud analysis module. Its core function is to continuously learn from newly generated fault case data, enabling the system to self-evolve. When the cloud analysis module confirms a new and valid fault diagnosis case through the deep diagnostic model and the expert system, and after feature comparison, finds that the difference between the feature pattern of the case and all existing patterns in the knowledge base is greater than a preset threshold, the online model update module will be triggered.

[0104] The online model update module first adds new case data to the incremental learning data pool. Then, it uses an incremental learning algorithm to fine-tune the parameters of the bidirectional long short-term memory neural network in the deep diagnostic model submodule. This fine-tuning process is not training from scratch, but rather iteratively training with new case data and some historical data based on the existing model parameters. The aim is to enable the model to learn to recognize this new failure mode while avoiding catastrophic forgetting of previously learned knowledge. Commonly used algorithms include elastic weight consolidation or incremental learning based on replay mechanisms.

[0105] Simultaneously, the online model update module updates the global fault knowledge graph. It adds new fault modes as new nodes to the graph and establishes edges between these nodes and related symptom feature nodes, hardware component nodes, and solution nodes. Furthermore, it may update the weights of relationships between existing nodes.

[0106] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0107] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A failure warning system for smart glasses based on big data analysis, characterized in that, include: An edge processing module deployed locally on the smart glasses is used to perform real-time online processing of raw time-series data generated by the sensors built into the smart glasses. A cloud-based analytics module, deployed on a remote server, receives and processes data reported by the edge processing module, performing in-depth fault mode diagnosis and fault evolution prediction. The collaborative communication and decision-making module is used to formulate and dynamically adjust the data reporting strategy and early warning triggering mechanism between the edge processing module and the cloud analysis module. The edge processing module includes a data preprocessing submodule, a lightweight feature extraction submodule, and a local anomaly detection submodule; The data preprocessing submodule is used to standardize and filter the raw sensor data for noise reduction. The lightweight feature extraction submodule is used to simultaneously extract time-domain statistical features and frequency-domain energy features from the preprocessed data stream in fixed time windows. The local anomaly detection submodule embeds a pre-trained single-class support vector machine model, which is used to construct the closed boundary of the normal operating state of the device in the high-dimensional feature space, and calculate the shortest distance from the feature vector extracted in each time window to the closed boundary, and define the distance as the real-time anomaly score. The cloud-based analysis module includes a feature fusion and enhancement submodule, a deep diagnostic model submodule, and a prediction and knowledge base submodule. The feature fusion and enhancement submodule is used to receive condensed feature sequences and anomaly score sequences reported by edge processing modules of multiple smart glasses, and to align and splice historical feature sequences from the same device in the time dimension to construct enhanced feature vectors. At the same time, it performs spatial correlation and comparative analysis on similar sensor features from different devices in the same time period. The deep diagnostic model submodule uses a bidirectional long short-term memory neural network based on an attention mechanism as the core diagnostic model. The input is a long sequence of enhanced feature vectors processed by the feature fusion and enhancement submodule, and the output is a probability distribution vector of the fault type corresponding to the current device state. The prediction and knowledge base submodule is used to extrapolate and predict the probability of various faults occurring within a specific time window in the future based on the fault probability sequence output by the deep diagnostic model and the exponential smooth state space model, and to maintain a global fault knowledge graph. The collaborative communication and decision-making module includes a dynamic threshold adaptive unit; The dynamic threshold adaptive unit is used to continuously monitor the real-time anomaly score stream from the edge processing module and the fault probability feedback from the cloud analysis module, and to run a two-layer threshold decision logic. The first layer of the dual-layer threshold decision logic is the local real-time decision threshold. When the real-time anomaly score is greater than the threshold, the edge processing module is instructed to urgently report the original data snapshot and condensed feature vector of the current time window to the cloud analysis module, and simultaneously issue a primary alarm to the user on the glasses. The second layer of the dual-layer threshold decision logic is the cloud-based cumulative decision threshold. The cloud-based cumulative decision threshold is dynamically calculated and distributed by the cloud analysis module based on the frequency of recent abnormal reports from similar device groups, the predicted failure probability of the current device, and the case matching degree in the knowledge base. When the cumulative weighted value of the abnormal score reported by the edge processing module within a certain time period is greater than the cloud-based cumulative decision threshold, the edge processing module is instructed to start the periodic feature reporting mode.

2. The big data analytics based intelligent glasses failure warning system as claimed in claim 1, wherein, The kernel function of the single-class support vector machine model in the local anomaly detection submodule adopts the radial basis function; The training process of the single-class support vector machine model is completed in the cloud, and the training data comes from the sensor feature vector set collected by a large number of normal smart glasses in various typical usage scenarios. After the trained model parameters are compressed and optimized, they are deployed to the edge processing module of each smart glasses through a secure firmware upgrade channel.

3. The big data analytics based smart glasses failure warning system as claimed in claim 2, wherein, The process by which the dynamic threshold adaptive unit calculates the cumulative decision threshold in the cloud is as follows: First, the cloud analysis module calculates the incidence rate of abnormal reporting events with the same device model and sensor type as the current device in the global device network over the past 24 hours, which serves as the group baseline risk coefficient. Secondly, obtain the maximum predicted probability of the main failure modes of the current device in the future from the prediction and knowledge base submodule; Next, query the global fault knowledge graph to calculate the matching similarity between the recent abnormal feature patterns of the current device and historical known fault cases; Finally, the group baseline risk coefficient, the maximum probability of failure prediction, and the case matching similarity are weighted and fused together, and a dynamic threshold coefficient between 0.1 and 0.5 is generated through a preset S-shaped function mapping. This coefficient is multiplied by the basic threshold to obtain the cloud cumulative decision threshold issued to the specific device.

4. The big data analytics based smart glasses failure warning system as claimed in claim 3, wherein, It also includes an online model update module, which is deployed in the cloud and used to continuously learn from newly generated fault case data; When the cloud analysis module confirms a new fault diagnosis case and the difference between the feature pattern of the case and the existing pattern in the knowledge base is greater than a preset threshold, the online model update module is activated. It uses an incremental learning algorithm to fine-tune the parameters of the bidirectional long short-term memory neural network in the deep diagnostic model sub-module, and at the same time updates the nodes and relationships in the global fault knowledge graph. The updated model parameters and knowledge graph incremental packages are encrypted and verified before being pushed to online smart glasses devices in batches. 5.The big data analytics based smart glasses failure warning system as claimed in claim 1, wherein, The time-domain statistical features include the mean, variance, peak-to-peak value, and zero-crossing rate of the data within the window; The frequency domain energy characteristics are calculated by performing a fast Fourier transform on the window data to determine the energy percentage within several preset key frequency bands. 6.The big data analytics based smart glasses failure warning system as claimed in claim 1, wherein, The feature fusion and enhancement submodule aligns and splices historical feature sequences from the same device in the time dimension to construct an enhanced feature vector with a longer time span.

7. A method for failure warning of smart glasses based on big data analysis, characterized in that, Fault warning is achieved using the fault warning system for smart glasses based on big data analysis as described in any one of claims 1 to 6.

Images