Fault early warning system and method for intelligent glasses based on big data analysis

Abstract

The invention relates to the technical field of fault diagnosis of intelligent wearable equipment, and particularly discloses a fault early warning system and method for intelligent glasses based on big data analysis. The system adopts an edge-cloud collaborative architecture, and comprises an edge processing module deployed at the local of the glasses, a deep analysis module deployed at the cloud and a collaborative communication and decision module. The edge processing module performs real-time feature extraction and anomaly scoring on the sensor data; the deep analysis module performs deep fault diagnosis and trend prediction on the reported information; and the cooperative communication and decision module intelligently decides data reporting and early warning triggering through a dynamic threshold adaptive mechanism. According to the method, millisecond-level real-time response is ensured, meanwhile, the data transmission load is reduced, and the detection precision of early fault symptoms is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent wearable device fault diagnosis, and particularly relates to a fault early warning system and method for intelligent glasses based on big data analysis. BACKGROUND

[0002] In the field of Internet of Things and wearable devices, intelligent terminals realize the perception and interaction of the physical world by integrating multiple sensors, and the data acquisition and processing capability is the basis to support the upper layer intelligent application. As a typical wearable device integrating display, computing and sensing modules, the stable and reliable operation of intelligent glasses is crucial for user experience and function implementation.

[0003] Device fault early warning based on big data analysis is a key technical direction to ensure long-term stable operation of the device. This technology aims to identify potential fault patterns by continuously collecting device operation data and using data analysis models, so as to issue an early warning before the fault occurs and realize predictive maintenance.

[0004] The prior art usually uploads all sensor data collected by the intelligent glasses to the cloud platform for centralized analysis and processing. However, the sensor data of the intelligent glasses has the characteristics of large data volume and high real-time requirement. Streaming all data to the cloud for analysis will introduce significant network transmission delay and occupy a large amount of communication bandwidth, resulting in that the response speed of the early warning system cannot meet the millisecond-level real-time requirement. This centralized processing architecture is difficult to effectively capture early and small abnormal fluctuations that occur during device operation, and these small abnormalities are often precursors of major faults. Therefore, how to realize low-delay and high-precision early fault symptom detection in a large amount of real-time data stream has become a technical problem to be solved in the field of predictive maintenance of intelligent glasses. SUMMARY

[0005] The purpose of the present application is to provide a fault early warning system and method for intelligent glasses based on big data analysis, to solve the technical contradiction in the prior art that the network delay is large and the bandwidth occupation is high due to uploading all sensor data to the cloud for centralized analysis, which cannot meet the millisecond-level real-time fault early warning requirement of intelligent glasses, and it is difficult to effectively capture early and small abnormal symptoms.

[0006] To achieve the above purpose, the present application provides a fault early warning system for intelligent glasses based on big data analysis, comprising:

[0007] An edge processing module deployed locally in the intelligent glasses, for real-time online processing of raw time series data generated by the sensors built-in the intelligent glasses,

[0008] A cloud analysis module deployed in a remote server, for receiving and processing the data reported by the edge processing module, performing deep fault pattern diagnosis and fault evolution prediction,

[0009] a collaborative communication and decision module for formulating and dynamically adjusting data reporting strategy and early warning triggering mechanism between the edge processing module and the cloud analysis module.

[0010] Preferably, the edge processing module comprises a data preprocessing submodule, a lightweight feature extraction submodule, and a local anomaly detection submodule.

[0011] The data preprocessing submodule is configured to perform standardization and filtering noise reduction processing on the original sensor data.

[0012] The lightweight feature extraction submodule is configured to extract time domain statistical features and frequency energy features from the preprocessed data stream in fixed time windows.

[0013] The local anomaly detection submodule is embedded with a pre-trained one-class support vector machine model, which is configured to construct a closed boundary of the device normal operating state in a high-dimensional feature space, calculate the shortest distance from the feature vector extracted in each time window to the closed boundary, and define the distance as a real-time anomaly score.

[0014] Preferably, the cloud analysis module comprises a feature fusion and enhancement submodule, a deep diagnosis model submodule, and a prediction and knowledge base submodule.

[0015] The feature fusion and enhancement submodule is configured to receive condensed feature sequences and anomaly score sequences reported from multiple intelligent glasses edge processing modules, align and splice historical feature sequences from the same device in the time dimension to construct an enhanced feature vector, and simultaneously correlate and compare the same type of sensor features from different devices in the same time period in the spatial dimension.

[0016] The deep diagnosis model submodule adopts a bidirectional long short-term memory neural network based on an attention mechanism as the core diagnosis model, with the long sequence enhanced feature vector processed by the feature fusion and enhancement submodule as the input, and a fault type probability distribution vector corresponding to the current device state as the output.

[0017] The prediction and knowledge base submodule is configured to use an exponential smoothing state space model to extrapolate and predict the probability of each type of fault occurring in a specific future time window based on the fault probability sequence output by the deep diagnosis model, and maintain a global fault knowledge graph.

[0018] Preferably, the collaborative communication and decision module comprises a dynamic threshold adaptive unit, which is configured to continuously monitor the real-time anomaly score stream from the edge processing module and the fault probability feedback from the cloud analysis module, and run a double-layer threshold decision logic.

[0019] The first layer of the double-layer threshold decision logic is a local instant decision threshold. When the real-time anomaly score is greater than the threshold, the edge processing module urgently reports the original data snapshot and the condensed feature vector of the current time window to the cloud analysis module, and simultaneously sends a primary alert to the user on the glasses side;

[0020] The second layer of the double-layer threshold decision logic is a cloud cumulative decision threshold. The cloud cumulative decision threshold is dynamically calculated and issued by the cloud analysis module according to the abnormal report frequency of the same type of device group recently received, the failure probability prediction value of the current device, and the case matching degree in the knowledge base. When the cumulative weighted value of the anomaly score reported by the edge processing module within a certain time period is greater than the cloud cumulative decision threshold, the edge processing module starts a periodic feature reporting mode.

[0021] Preferably, the kernel function of the one-class support vector machine model in the local anomaly detection submodule adopts a radial basis function;

[0022] The training process of the one-class support vector machine model is completed in the cloud. The training data comes from a large number of normal intelligent glasses collecting sensor feature vectors in multiple typical use scenarios;

[0023] After the model parameters are trained, compressed and optimized, they are deployed to the edge processing module of each intelligent glasses through a secure firmware upgrade channel.

[0024] Preferably, the process of calculating the cloud cumulative decision threshold by the dynamic threshold adaptive unit is as follows:

[0025] First, the cloud analysis module calculates the abnormal report event rate of the same type of device and the same type of sensor in the global device network within the past 24 hours as the group baseline risk coefficient;

[0026] Secondly, the maximum prediction probability of the prediction and knowledge base submodule for the future main failure mode of the current device is obtained;

[0027] Thirdly, the global fault knowledge graph is queried to calculate the matching similarity of the recent abnormal feature mode of the current device and the historical known fault cases;

[0028] Finally, the group baseline risk coefficient, the maximum fault prediction probability, and the case matching similarity are weighted and fused, and a dynamic threshold coefficient between 0.1 and 0.5 is generated through a pre-set S-shaped function mapping. The coefficient is multiplied by the basic threshold to obtain the cloud cumulative decision threshold issued to the specific device.

[0029] Preferably, it further includes a model online updating module, which is deployed in the cloud for continuous learning of newly generated fault case data;

[0030] When the cloud analysis module confirms a new fault diagnosis case and the difference between the feature mode of the case and the existing mode in the knowledge base is greater than the preset threshold, the model online updating module is started, the bidirectional long short-term memory neural network parameters in the deep diagnosis model submodule are fine-tuned by using an incremental learning algorithm, and the nodes and relations in the global fault knowledge graph are updated at the same time;

[0031] The updated model parameters and the knowledge graph incremental package are encrypted and verified, and then are pushed to the online intelligent glasses device in batches.

[0032] Preferably, the time domain statistical features include mean, variance, peak-to-peak value and zero-crossing rate of the data in the window.

[0033] The frequency domain energy features are calculated by performing fast Fourier transform on the window data to calculate the energy proportion in the preset plurality of key frequency bands.

[0034] Preferably, the feature fusion and enhancement submodule aligns and splices the historical feature sequences from the same device in the time dimension to construct an enhanced feature vector with a longer time span.

[0035] The application provides a fault early warning method of intelligent glasses based on big data analysis.

[0036] Compared with the prior art, the application has the following beneficial effects:

[0037] 1. The application constructs a hierarchical processing architecture of edge-cloud cooperation, and sinks the preliminary processing and feature extraction of massive real-time data to the local edge side of the intelligent glasses, so that only the condensed abnormal feature information or the filtered original data snapshot is uploaded to the cloud, thereby fundamentally solving the problems of network bandwidth congestion and high transmission delay caused by uploading all data streams.

[0038] 2. The application uses a single-class support vector machine model based on normal data training for abnormal perception on the edge side.

[0039] 3、The dynamic threshold self-adaptive coordination mechanism designed in the application realizes the intelligent balance of early warning sensitivity and system resource consumption through the double-layer decision of local instant threshold and cloud dynamic cumulative threshold. The emergency exception is triggered for instant reporting and deep analysis, and the suspected exception is filtered through cumulative evaluation and dynamic threshold, which avoids missing reports and prevents the increase of device energy consumption and the excessive processing pressure of the cloud caused by frequent data uploading, so that the system has good scalability and practicality.

[0040] 4、The cloud deep diagnosis model of the application fuses the attention mechanism and the long short-term memory network, can deeply mine the enhanced features of long time sequence, accurately identifies complex fault modes and quantifies the occurrence probability. Combined with fault evolution prediction and global knowledge graph, the system can not only diagnose the current state, but also predict the fault development trend and provide maintenance decision support, realizing the leap from simple anomaly detection to intelligent predictive maintenance.

[0041] 5、The model online updating module introduced in the application enables the system to continuously evolve by using the field data generated continuously. New fault cases are quickly absorbed and used to optimize the diagnosis model and knowledge base, and are synchronized to the terminal device through a safety mechanism, forming a closed-loop learning and optimization system, which ensures that the early warning model can adapt to changes caused by device hardware aging, software updates and new use scenarios, and long-term maintain high early warning accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is the overall technical scheme architecture schematic diagram of the intelligent glasses fault early warning system based on big data analysis provided by the application;

[0043] Figure 2 is the core principle framework schematic diagram of the hierarchical intelligent processing architecture of edge-cloud cooperation in the application;

[0044] Figure 3 is the logic flow framework diagram of the edge processing module in the application;

[0045] Figure 4 is the logic flow framework diagram of the cloud analysis module in the application;

[0046] Figure 5 is the multi-level interaction relationship and data flow schematic diagram of the cooperative communication and decision module in the application. DETAILED DESCRIPTION

[0047] The application provides an intelligent glasses fault early warning system based on big data analysis, which adopts a hierarchical intelligent processing architecture of edge and cloud cooperation. Please refer to the accompanying drawings Figures 1 to 5, which shows the overall technical architecture of the system. The entire system is composed of an edge processing module deployed locally on the smart glasses, a cloud analysis module deployed on a remote server, and a collaborative communication and decision-making module that connects and coordinates the above two modules.

[0048] The three modules interact with each other through a wireless communication network to form a closed-loop intelligent early warning system. Please refer to the attached Figure 2 , which further reveals the core principle framework of the hierarchical intelligent processing architecture of edge and cloud collaboration. The core lies in decomposing the fault early warning task into two logical levels: real-time anomaly perception on the edge side and deep diagnosis prediction on the cloud side, and linking them through a dynamic adaptive collaborative protocol.

[0049] The edge processing module is the front-end perception and preliminary calculation unit of the system, directly integrated into the hardware system of the smart glasses. This module is responsible for real-time online processing of raw time series data generated by various sensors built into the smart glasses. These sensors typically include but are not limited to inertial measurement units, ambient light sensors, proximity sensors, temperature sensors, gyroscopes, accelerometers, and microphones, etc. The edge processing module is further composed of a data preprocessing submodule, a lightweight feature extraction submodule, and a local anomaly detection submodule. Its internal logic flow is described in the attached Figure 3 .

[0050] The data preprocessing submodule first receives raw data streams from various sensors. Raw data streams are usually discrete time series generated at a specific sampling rate. For example, the data sampling rate of an inertial measurement unit may be 200 Hz, while the sampling rate of a temperature sensor may be 1 Hz.

[0051] The first step of the data preprocessing submodule is data standardization. This step is performed independently for each type of sensor data, and its purpose is to eliminate the influence of different sensor dimensions and magnitudes on subsequent analysis. Specifically, for each sensor data stream, the system maintains dynamically updated mean and standard deviation statistics. When a new data point arrives, the following calculation process is used for standardization: subtract the sliding mean of the sensor data stream in the past statistical period from the original data value, and divide by the sliding standard deviation in the statistical period. The length of the statistical period is configured according to the stability of the sensor data. For high-frequency dynamic data such as accelerometer data, the statistical period is usually set to 10 seconds; for low-frequency slowly varying data such as temperature data, the statistical period can be set to 300 seconds. The standardized data is mapped to a distribution interval with a mean of 0 and a variance of 1.

[0052] After standardization, the data preprocessing submodule performs filtering and noise reduction processing. Different filtering algorithms are used for different types of sensor data and noise characteristics. For data such as inertial measurement units and accelerometers that are easily disturbed by high-frequency random noise, a first-order low-pass digital filter is used. The cutoff frequency of this filter is set according to the main effective signal frequency band of the sensor, for example, for acceleration signals capturing human head movement, the cutoff frequency is set to 20 Hz. The transfer function of the filter is implemented in the digital domain, and the input sequence is convolved in real time through the difference equation, effectively filtering out noise components greater than the cutoff frequency. For data such as ambient light sensors that may be subject to pulse interference, a median filter algorithm is used.

[0053] The algorithm sets a sliding window with a length of 5 sampling points, sorts the data in the window by size, and takes the median as the output value at that time, effectively suppressing isolated pulse noise points. Through standardization and filtering and noise reduction processing, the data preprocessing submodule outputs clean, normalized sensor time series data, laying the foundation for subsequent feature extraction.

[0054] The lightweight feature extraction submodule receives clean data streams from the data preprocessing submodule and synchronously extracts time domain statistical features and frequency domain energy features in fixed time windows. The length of the time window is differentially configured according to the sensor type and data sampling rate, which is key to ensuring the timeliness and representativeness of the features. For sensor data with a sampling rate greater than 100 Hz, such as angular velocity and linear acceleration data from an inertial measurement unit, the time window length is set to 256 milliseconds.

[0055] For an accelerometer data stream with a sampling rate of 200 Hz, a 256 ms time window will contain approximately 51 data points.

[0056] For sensor data with a sampling rate less than 10 Hz, such as ambient light sensors or temperature sensors, the time window length is set to 5 seconds.

[0057] For a temperature sensor with a sampling rate of 1 Hz, a 5-second window contains 5 data points.

[0058] This differential configuration ensures that high-frequency data can capture transient features, while low-frequency data can accumulate enough change information.

[0059] Within each time window, the lightweight feature extraction submodule calculates two sets of features in parallel. The first set is the time domain statistical features, including four core indicators:

[0060] The arithmetic mean of all data points in the window, representing the DC component or baseline level of the signal;

[0061] Variance of all data points in the window, to represent the intensity of fluctuation around the mean value of the signal.

[0062] Difference between the maximum and minimum value in the window, i.e. the peak-to-peak value, to represent the overall dynamic range of the signal.

[0063] The number of times the signal waveform crosses the zero level, i.e. the zero-crossing rate, to roughly estimate the frequency component of the signal.

[0064] The calculation of these statistics all adopts a streaming algorithm, which updates the intermediate variables such as cumulative sum, square sum, maximum and minimum value in real time as the data points enter the window one by one, so that the feature values are obtained immediately at the end of the window, with low computational complexity and meeting the real-time requirement.

[0065] The second group is the frequency domain energy features. The lightweight feature extraction submodule performs a fast Fourier transform on the data sequence in each time window. Due to the limited computing resources on the edge side, the number of points of the fast Fourier transform is adaptively determined according to the window length and the sampling rate, usually using the length closest to an integer power of 2, for example, for 200 Hz data of a 256 ms window, a 64-point fast Fourier transform is used. After the transformation, the frequency spectrum of the signal is obtained.

[0066] The system pre-defines multiple key frequency bands related to potential failure modes of the device. For example, for an acceleration signal reflecting the vibration of a motor, the key frequency bands can be set to 50-100 Hz, 100-200 Hz, etc.

[0067] For each pre-set key frequency band, the sum of the squared amplitudes of all frequency components in the frequency band, i.e. the energy of the frequency band, is calculated.

[0068] Then, the proportion of the energy of the frequency band in the total energy of the signal is calculated as the energy proportion feature of the frequency band.

[0069] Finally, for a time window and a type of sensor data, the lightweight feature extraction submodule outputs a feature vector that integrates the time domain and frequency domain information. This feature vector is the input of the subsequent anomaly detection.

[0070] The local anomaly detection submodule embeds a pre-trained one-class support vector machine model, which is trained based on a large amount of normal working condition data before the device is shipped, and its core function is to construct a closed decision boundary of the normal operating state of the device in a high-dimensional feature space.

[0071] This model is trained only with a large number of normal samples, without any failure samples, especially suitable for the scene where the initial failure sample is scarce. The kernel function of the model uses the radial basis function, and its mathematical expression focuses on calculating the similarity between two feature vectors. The training process is completed in the cloud, using feature vectors collected and extracted from thousands of normal smart glasses in various typical use scenarios to form the training set. The training goal is to find the smallest volume of hyperspherical boundary that can surround most normal samples, or to find a separating hyperplane in the high-dimensional feature space that separates the origin from the normal samples.

[0072] The trained model parameters include the set of support vectors, corresponding coefficients, and kernel function parameters. These parameters are optimized through model compression and quantization, converted into a lightweight format suitable for running on embedded devices, such as converting floating-point numbers to fixed-point numbers. Optimized model parameters are deployed to the local anomaly detection submodule of the edge processing module of each smart glass in the form of encrypted data packets through the secure firmware upgrade channel.

[0073] In the running phase, for each feature vector output by the lightweight feature extraction submodule, the local anomaly detection submodule calculates its distance to the pre-defined normal state boundary. This distance is quantified as a real number value, defined as a real-time anomaly score. The calculation process involves kernel function calculation of the feature vector with all support vectors stored in the model, and weighted summation and offset adjustment according to the model parameters. The smaller the real-time anomaly score value, the closer the feature vector is to the center of the normal state cluster; the larger the value, the more it deviates from the normal state. This real-time anomaly score calculation is completed within milliseconds, achieving instantaneous evaluation of the device running state.

[0074] The cloud analysis module is deployed on a remote server cluster with powerful computing and storage capabilities, used to receive and process data reported by the edge processing module, perform deep fault mode diagnosis and fault evolution prediction. This module is further composed of feature fusion and enhancement submodule, deep diagnosis model submodule, and prediction and knowledge base submodule, please refer to the attached Figure 4 .

[0075] Feature fusion and enhancement module is the entry and preprocessing hub of cloud data processing. It receives data reported from thousands of intelligent glasses edge processing modules. The reported data is mainly divided into two categories: one is the original data snapshot and the corresponding condensed feature vector triggered by emergency reporting; the other is the higher resolution feature sequence sent in the periodic feature reporting mode. The primary task of this submodule is to fuse and enhance the data in time and space dimensions. For feature sequences from the same device, the feature fusion and enhancement submodule performs alignment and splicing in the time dimension. The system maintains a time series buffer for each device. When a new feature data packet arrives, the submodule inserts it into the correct time sequence position of the buffer corresponding to the device according to the timestamp information in the data packet. Subsequently, the system extracts continuous feature vector sequences from the buffer in longer analysis time windows, such as the past 30 minutes or 2 hours, and splices them in the feature dimension to form enhanced feature vectors with longer time spans and richer information. This long sequence vector can reflect the evolution process of fault features.

[0076] At the same time, the feature fusion and enhancement submodule also performs correlation analysis in the spatial dimension. It aggregates the same type of sensor features from different devices within the same time period. For example, for the accelerometer X-axis variance feature of a certain model of intelligent glasses, the submodule collects the feature values of all online devices of this model within the past 1 hour, calculates the statistical distribution such as mean, standard deviation, and percentile. By comparing the feature value of a single device with the statistical distribution of the group, it can identify whether the device is significantly abnormal in the group, which helps to distinguish common environmental interference from individual hardware failure. After time and space fusion and enhancement processing, the data is organized into a structured tensor format as input for the deep diagnostic model.

[0077] The deep diagnostic model submodule is the core of cloud analysis, using a bidirectional long short-term memory neural network based on attention mechanism as the core diagnostic model. This network is specifically designed to process long time series data and capture long-term dependencies and key time points. The input of the network is the long sequence enhanced feature vector output by the feature fusion and enhancement submodule. Assuming that the time step of the enhanced feature vector is , and the feature dimension of each time step is , then the input is a matrix with a shape of × .

[0078] The bidirectional long short-term memory neural network is composed of two long short-term memory layers in the forward and backward directions. The number of hidden layer units in each long short-term memory layer is set to 128. The forward layer processes the sequence from the first step to the step in time order, capturing the dependency information from the past to the future; the backward layer processes the sequence in reverse order from the The first step of the processing sequence captures the future-to-past dependency information. For each time step, the forward layer and the backward layer output 128-dimensional hidden state vectors, respectively. The forward hidden state and the backward hidden state of the same time step are concatenated to obtain a 256-dimensional fused hidden state vector, which contains the full sequence information of the context at this time step.

[0079] The attention mechanism is applied to the fused hidden state sequence output by the last bidirectional LSTM layer of the network. The purpose of the attention mechanism is to assign a weight to each time step in the sequence, and the weight size represents the importance of the feature of this time step to the final fault diagnosis decision.

[0080] The specific calculation process is as follows: first, the 256-dimensional hidden state of each time step is mapped to an attention energy value through a trainable fully connected layer; then, the attention energy values of all time steps are normalized using the Softmax function to obtain the attention weight of each time step, and the sum of all weights is 1; finally, the hidden states of all time steps are weighted and summed according to their corresponding attention weights to obtain a 256-dimensional context vector. This context vector focuses on the most critical and abnormal time segment in the sequence, greatly improving the model's sensitivity to early and intermittent abnormal patterns.

[0081] ;

[0082] ;

[0083] ;

[0084] In the above formula, represents the fused hidden state vector of the time step. and are trainable weight matrices and bias vectors, is a trainable weight vector, which together constitute a fully connected layer for calculating the attention energy value of each time step . is the attention energy value corresponding to the time step. is the attention weight of the time step after Softmax normalization. is the final generated weighted context vector.

[0085] This context vector is then fed into a fully connected classification layer. The number of neurons in the classification layer equals the number of predefined potential failure modes, e.g. 10. The classification layer outputs a failure type probability distribution vector. Each element in this vector corresponds to the probability of occurrence of a specific failure mode, and the sum of all elements is 1. For example, the output vector can be represented as [0.02, 0.85, 0.01, 0.10, 0.02], indicating that the current device state is diagnosed to be the second failure mode with a probability as high as 85%, while there is a 10% probability of being the fourth failure mode. This probabilistic output provides a quantitative basis for subsequent risk assessment and decision-making.

[0086] The prediction and knowledge base submodule is responsible for trend prediction and knowledge management based on the failure probability sequence output by the deep diagnostic model. This submodule uses an exponential smoothing state space model to extrapolate the probability time series of each type of failure. The model receives a sequence of observed values of a certain failure probability over a period of time, e.g. 24 hours. The model models and updates the level component and trend component of the sequence through state space equations. The model can estimate the probability values of each type of failure occurring in a specific time window in the future, e.g. 2 hours, 6 hours in the future. This realizes the leap from current state diagnosis to future risk prediction.

[0087] In addition, the prediction and knowledge base submodule maintains a global failure knowledge graph. This knowledge graph stores historical diagnosis cases, failure solutions, and causal relationship chains between different failure modes in a graph structure. The nodes in the graph represent entities, such as specific failure modes, symptom features, affected hardware components, solution steps, device models, etc. The edges between nodes represent relationships, such as "causes", "manifests as", "can be solved by", "common in … model", etc. Whenever a new failure diagnosis case is confirmed by the cloud, the system will structure the case's feature patterns, diagnosis results, handling measures, and temporal context information, and add them as new nodes and edges to the knowledge graph.

[0088] The knowledge graph supports complex graph queries, such as when the system detects that features A and B are both abnormal, it can query the graph to find out which failure modes are associated with both features, thereby assisting in diagnosis. The continuous accumulation of the graph enables the system to have increasingly rich domain knowledge.

[0089] The collaborative communication and decision-making module is an intelligent hub connecting the edge and the cloud, responsible for formulating and dynamically adjusting the data reporting strategy and early warning triggering mechanism between the two. The core of this module is the dynamic threshold adaptive unit, which has a multi-level interaction relationship and data flow please refer to the attached Figure 5 This unit continuously monitors two core data streams: one is the real-time anomaly score stream from the edge processing module; the other is the failure probability feedback and group statistical information from the cloud analysis module.

[0090] The dynamic threshold adaptive unit internally runs a double-layer threshold decision logic, which is the key to achieving the balance between resource and sensitivity. The first layer decision is based on the local real-time decision threshold. This threshold is a fixed value preset according to the device model and sensor type, stored in the edge processing module. When the real-time anomaly score calculated by the local anomaly detection submodule is greater than this local real-time decision threshold, it indicates that the device may have a significant transient anomaly. At this time, the collaborative communication and decision module will immediately issue an instruction to the edge processing module through the control channel.

[0091] After receiving the instruction, the edge processing module performs the following actions:

[0092] First, the original sensor data snapshot corresponding to the time window that triggered the anomaly is cached and packaged;

[0093] Second, the condensed feature vector extracted from the time window is appended to the data packet; finally, through a high-speed data channel, such as when WiFi is available, the data packet is urgently reported to the feature fusion and enhancement submodule of the cloud analysis module.

[0094] At the same time, the collaborative communication and decision module instructs the human-computer interaction unit of the smart glasses to flash a specific warning icon on the micro display screen of the glasses, or issue a short prompt sound through the bone conduction earphone, to send a primary visual or auditory alarm to the user. This mechanism ensures an extremely fast response to serious sudden anomalies.

[0095] The second layer decision is based on the cloud cumulative decision threshold. This threshold is not a fixed value, but is dynamically calculated by the cloud analysis module according to multi-dimensional information and delivered to specific devices. The process of calculating the second layer threshold by the dynamic threshold adaptive unit is a complex multi-factor fusion process.

[0096] The specific steps are as follows:

[0097] First, the cloud analysis module counts the frequency of abnormal event reporting within the past 24 hours in the entire global device network for devices of the same model and sensor type as the current device. This frequency value is normalized to obtain a group baseline risk coefficient between 0 and 1. This coefficient reflects the overall health situation of the device group in the current period.

[0098] Second, obtain the maximum prediction probability of the main failure mode within the next 2 hours from the prediction and knowledge base submodule. For example, the probability of "display driver chip overheating" failure within the next 2 hours is 30%.

[0099] Thirdly, query the global fault knowledge graph. The abnormal feature pattern recently reported by the current device is matched with the feature pattern of the historical known fault cases stored in the knowledge graph for similarity calculation. The similarity calculation usually uses cosine similarity or distance measurement based on graph embedding to obtain a case matching similarity score ranging from 0 to 1.

[0100] Finally, the three factors, the group baseline risk coefficient, the maximum fault prediction probability, and the case matching similarity, are fused by weighting. The weight coefficients are determined according to experience or offline optimization, for example, the weights are 0.3, 0.4, and 0.3, respectively. The weighted sum obtains a preliminary comprehensive risk index. In order to map the index to a coefficient suitable for threshold comparison, it is input into a preset S-shaped function. The S-shaped function maps the input to an output interval of 0.1 to 0.5. This output value is the dynamic threshold coefficient. The system presets a basic cumulative threshold, for example, the baseline value of the abnormal score cumulative sum is 50. Multiplying the dynamic threshold coefficient by the basic cumulative threshold obtains the final second-layer cloud cumulative decision threshold issued to the specific device. For example, if the dynamic coefficient is 0.3, the dynamic threshold is 15.

[0101] The edge-side collaborative communication and decision module agent continuously calculates the cumulative weighted value of the real-time abnormal score within a certain time period. The cumulative weighted value is not a simple sum, but an exponentially decaying weighted sum of the abnormal scores in the past period, so that the recent abnormal scores have higher weights. When this cumulative weighted value is greater than the dynamic threshold issued by the cloud, it indicates that the device is in a sub-healthy or suspected fault state although no severe instantaneous abnormality has occurred. At this time, the collaborative communication and decision module instructs the edge processing module to start the periodic feature reporting mode.

[0102] In this mode, the edge processing module no longer reports only the trigger event data, but uploads higher-resolution feature sequences to the cloud at a lower frequency, for example, once every 5 minutes. These data provide materials for the cloud to conduct more in-depth long-term trend analysis and pattern mining, while avoiding the energy consumption and bandwidth pressure caused by continuous full-speed reporting.

[0103] Further, the system also includes a model online updating module. The module is also deployed in the cloud and is closely integrated with the cloud analysis module. Its core function is to continuously learn new fault case data to realize self-evolution of the system. When the cloud analysis module confirms a new and effective fault diagnosis case through the deep diagnosis model and the expert system, and finds that the feature pattern of the case has a difference greater than a preset threshold from all existing patterns in the knowledge base through feature comparison, the model online updating module will be triggered to start.

[0104] The model online updating module first adds the new case data into the incremental learning data pool. Then, it fine-tunes the parameters of the bi-directional long short-term memory neural network in the deep diagnostic model submodule using the incremental learning algorithm. The fine-tuning process is not a complete retraining from scratch, but rather a small number of iterative training using the new case data and part of the historical data based on the existing model parameters, aiming to let the model learn to recognize this new failure mode, while avoiding catastrophic forgetting of the already learned knowledge. Commonly used algorithms include elastic weight consolidation or incremental learning based on the replay mechanism.

[0105] At the same time, the model online updating module updates the global fault knowledge graph. It adds the new failure mode as a new node to the graph, and establishes the edges between this node and the related symptom feature nodes, hardware component nodes, and solution nodes. In addition, it may also update the relationship weights between existing nodes.

[0106] It should be noted that the relational terms herein, such as first and second, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any such actual relationship or order between or among the entities or actions. Moreover, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0107] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A fault early warning system for smart glasses based on big data analysis, characterized in that, include: An edge processing module deployed locally on the smart glasses is used to perform real-time online processing of raw time-series data generated by the sensors built into the smart glasses. A cloud-based analytics module, deployed on a remote server, receives and processes data reported by the edge processing module, performing in-depth fault mode diagnosis and fault evolution prediction. The collaborative communication and decision-making module is used to formulate and dynamically adjust the data reporting strategy and early warning triggering mechanism between the edge processing module and the cloud analysis module.

2. The fault early warning system for smart glasses based on big data analysis according to claim 1, characterized in that, The edge processing module includes a data preprocessing submodule, a lightweight feature extraction submodule, and a local anomaly detection submodule; The data preprocessing submodule is used to standardize and filter the raw sensor data for noise reduction. The lightweight feature extraction submodule is used to simultaneously extract time-domain statistical features and frequency-domain energy features from the preprocessed data stream in fixed time windows. The local anomaly detection submodule embeds a pre-trained single-class support vector machine model, which is used to construct the closed boundary of the normal operating state of the device in the high-dimensional feature space, and calculate the shortest distance from the feature vector extracted in each time window to the closed boundary, and define the distance as the real-time anomaly score.

3. The fault early warning system for smart glasses based on big data analysis according to claim 2, characterized in that, The cloud-based analysis module includes a feature fusion and enhancement submodule, a deep diagnostic model submodule, and a prediction and knowledge base submodule. The feature fusion and enhancement submodule is used to receive condensed feature sequences and anomaly score sequences reported by edge processing modules of multiple smart glasses, and to align and splice historical feature sequences from the same device in the time dimension to construct enhanced feature vectors. At the same time, it performs spatial correlation and comparative analysis on similar sensor features from different devices in the same time period. The deep diagnostic model submodule uses a bidirectional long short-term memory neural network based on an attention mechanism as the core diagnostic model. The input is a long sequence of enhanced feature vectors processed by the feature fusion and enhancement submodule, and the output is a probability distribution vector of the fault type corresponding to the current device state. The prediction and knowledge base submodule is used to extrapolate and predict the probability of various faults occurring within a specific time window in the future based on the fault probability sequence output by the deep diagnostic model and the exponential smooth state space model, and to maintain a global fault knowledge graph.

4. The fault early warning system for smart glasses based on big data analysis according to claim 3, characterized in that, The collaborative communication and decision-making module includes a dynamic threshold adaptive unit; the dynamic threshold adaptive unit is used to continuously monitor the real-time anomaly score stream from the edge processing module and the fault probability feedback from the cloud analysis module, and run a two-layer threshold decision logic. The first layer of the dual-layer threshold decision logic is the local real-time decision threshold. When the real-time anomaly score is greater than the threshold, the edge processing module is instructed to urgently report the original data snapshot and condensed feature vector of the current time window to the cloud analysis module, and simultaneously issue a primary alarm to the user on the glasses. The second layer of the dual-layer threshold decision logic is the cloud-based cumulative decision threshold. The cloud-based cumulative decision threshold is dynamically calculated and distributed by the cloud analysis module based on the frequency of recent abnormal reports from similar device groups, the predicted failure probability of the current device, and the case matching degree in the knowledge base. When the cumulative weighted value of the abnormal score reported by the edge processing module within a certain time period is greater than the cloud-based cumulative decision threshold, the edge processing module is instructed to start the periodic feature reporting mode.

5. The fault early warning system for smart glasses based on big data analysis according to claim 4, characterized in that, The kernel function of the single-class support vector machine model in the local anomaly detection submodule adopts the radial basis function; The training process of the single-class support vector machine model is completed in the cloud, and the training data comes from the sensor feature vector set collected by a large number of normal smart glasses in various typical usage scenarios. After the trained model parameters are compressed and optimized, they are deployed to the edge processing module of each smart glasses through a secure firmware upgrade channel.

6. The fault early warning system for smart glasses based on big data analysis according to claim 5, characterized in that, The process by which the dynamic threshold adaptive unit calculates the cumulative decision threshold in the cloud is as follows: First, the cloud analysis module calculates the incidence rate of abnormal reporting events with the same device model and sensor type as the current device in the global device network over the past 24 hours, which serves as the group baseline risk coefficient. Secondly, obtain the maximum predicted probability of the main failure modes of the current device in the future from the prediction and knowledge base submodule; Next, query the global fault knowledge graph to calculate the matching similarity between the recent abnormal feature patterns of the current device and historical known fault cases; Finally, the group baseline risk coefficient, the maximum probability of failure prediction, and the case matching similarity are weighted and fused together, and a dynamic threshold coefficient between 0.1 and 0.5 is generated through a preset S-shaped function mapping. This coefficient is multiplied by the basic threshold to obtain the cloud cumulative decision threshold issued to the specific device.

7. The fault early warning system for smart glasses based on big data analysis according to claim 6, characterized in that, It also includes an online model update module, which is deployed in the cloud and used to continuously learn from newly generated fault case data; When the cloud analysis module confirms a new fault diagnosis case and the difference between the feature pattern of the case and the existing pattern in the knowledge base is greater than a preset threshold, the online model update module is activated. It uses an incremental learning algorithm to fine-tune the parameters of the bidirectional long short-term memory neural network in the deep diagnostic model sub-module, and at the same time updates the nodes and relationships in the global fault knowledge graph. The updated model parameters and knowledge graph incremental packages are encrypted and verified before being pushed to online smart glasses devices in batches.

8. The fault early warning system for smart glasses based on big data analysis according to claim 2, characterized in that, The time-domain statistical features include the mean, variance, peak-to-peak value, and zero-crossing rate of the data within the window; The frequency domain energy characteristics are calculated by performing a fast Fourier transform on the window data to determine the energy percentage within several preset key frequency bands.

9. The fault early warning system for smart glasses based on big data analysis according to claim 1, characterized in that, The feature fusion and enhancement submodule aligns and splices historical feature sequences from the same device in the time dimension to construct an enhanced feature vector with a longer time span.

10. A fault early warning method for smart glasses based on big data analysis, characterized in that, Fault warning is achieved using the fault warning system for smart glasses based on big data analysis as described in any one of claims 1 to 9.

Images