Abnormal data information diagnosis method in Internet of Things table detection

Abstract

The invention discloses an abnormal data information diagnosis method in Internet-of-Things meter detection, and relates to the technical field of electric energy meter abnormity diagnos.The method comprises the steps that firstly, data collection and preliminary judgment are conducted, and then a cloud data center platform constructs an abnormal data information diagnosis database through a distributed graph database; then abnormal data information fine identification is carried out, abnormal data information diagnosis is carried out according to a fine abnormal detection result, automatic reasoning is carried out on a fault reason, a fault severity degree and an influence range through a reasoning engine, and abnormal data feedback and restoration are carried out; the method solves the problems that an existing statistical method is suitable for a single variable and cannot describe the relation between the variables, and a method based on machine learning is unbalanced in sample. According to the method, the conditions of large data volume and high data dimension can be effectively handled, and the efficiency and the flexibility of processing the abnormal data information detected by the Internet of Things table are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal diagnosis of electric energy meters, and more specifically to a method for diagnosing abnormal data information in detection of an Internet of Things meter. Background Art

[0002] With the development and popularization of the Internet of Things, the number and application scenarios of IoT devices are becoming more and more extensive, such as smart homes, smart factories, smart cities, etc. The Internet of Things refers to a network that connects various objects through the Internet to achieve intelligent management and control. In the Internet of Things, devices such as sensors, actuators, and embedded systems continuously collect data and transmit and process them through the Internet. However, due to the complexity of IoT devices and the generation of a large amount of data, the detection and diagnosis of abnormal data information has become an important research topic. Abnormal data information may cause performance degradation, system crashes and failures of IoT devices, affecting user experience and business operations. There are some abnormal data in a large amount of data, which may affect the normal operation of the entire system. Therefore, it is particularly important to detect and diagnose abnormal data in the Internet of Things.

[0003] At present, the methods for detecting abnormal data in the Internet of Things are mainly divided into two categories: statistical methods and machine learning methods. The statistical methods mainly use indicators such as mean, variance, and standard deviation to determine whether a sample is abnormal. However, this method is often only applicable to a single variable and cannot describe the relationship between variables, so it performs poorly in complex environments. The machine learning method can model samples from multiple dimensions and is more suitable for anomaly detection in complex environments. However, in actual applications, due to the large number of abnormal data types, the difficulty in abstracting features, and the lack of effective information diagnosis methods, the machine learning method cannot give full play to its advantages. Due to the information of meter data characteristics, such as how to effectively detect and diagnose abnormal data with voltage, current, power and other features, the accuracy of the data and the normal operation of the system are ensured. This will help improve the operation and maintenance efficiency of the meter system, reduce maintenance costs, and provide users with more accurate and reliable services.

[0004] Therefore, the present invention discloses a method for diagnosing abnormal data information in IoT meter detection, which can effectively process abnormal data information of electric energy meters with large data volume and high data dimension, thereby improving the ability to diagnose abnormal data information in IoT meter detection. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention discloses a method for diagnosing abnormal data information in the detection of the Internet of Things table. By using a wireless sensor network and an embedded preprocessing chip to make a preliminary judgment on the collected data, the judgment efficiency and accuracy are greatly improved. By using a distributed graph database to classify, store and clean the abnormal data information, and perform cluster analysis and association analysis on the data, the efficiency and accuracy of data processing and analysis are greatly improved; by using a multivariate principal component PCA analysis and a cross-modal feature fusion method, the abnormal data can be analyzed from multiple angles and aspects, greatly improving the accuracy of detection; by introducing an inference engine, the abnormal data can be automatically inferred, and the cause of the fault, the severity of the fault and the scope of influence can be found, thereby improving the depth and breadth of abnormal diagnosis; by using a wireless communication network and an automatic recovery mechanism, the abnormal data can be quickly and effectively fed back and repaired, thereby improving the efficiency and accuracy of abnormal processing; by using a multivariate principal component PCA analysis and a cross-modal feature fusion, the problem of numerous abnormal data types and difficult feature abstraction is solved. At the same time, the SVM-DS data selection model and the abnormal data information diagnosis database are used to perform refined detection on the fused abnormal data information features, thereby improving the diagnostic effect of the machine learning method in the Internet of Things environment. In addition, in the process of abnormal data diagnosis, cluster analysis and association analysis are used to discover the implicit relationships and similarities between the input abnormal data, so as to better understand the rules and trends in the overall sample; the degree of automation and intelligence is high.

[0006] The present invention adopts the following technical solutions: A method for diagnosing abnormal data information in IoT table detection comprises the following steps: Step 1: Data collection and preliminary judgment: The IoT table detects and collects environmental parameters and device status parameters through the wireless sensor network, and performs preliminary abnormality judgment on the collected data through the embedded pre-processing chip. The environmental parameters and device status parameters at least include temperature, humidity, pressure, flow and working power. The embedded pre-processing chip performs preliminary judgment and marking of abnormal data information in the IoT table detection through the automatic screening model, and uploads the preliminary judged abnormal data information to the cloud data center platform through the wireless communication network; Step 2: construct an abnormal data information diagnosis database. The cloud data center platform uses a distributed graph database to construct the abnormal data information diagnosis database. The abnormal data information diagnosis database classifies and stores the input abnormal data information according to the type, time and location factors, and configures storage strategy and cleanup strategy according to the data volume and storage time. The abnormal data information diagnosis database discovers the implicit relationship and similarity between the input abnormal data information through cluster analysis and association analysis, and diagnoses and processes the abnormal data information based on the analysis results; Step 3: Refined identification of abnormal data information: the cloud data center platform extracts multimodal features for preliminary judgment of abnormal data information through multivariate principal component PCA analysis, and performs cross-modal feature fusion. The cloud data center platform performs refined anomaly detection on the fused abnormal data information features based on the SVM-DS data selection model and the abnormal data information diagnosis database; Step 4: abnormal data information diagnosis: the cloud data center platform performs abnormal data information diagnosis based on the refined anomaly detection results, and automatically infers the cause of the fault, the severity of the fault, and the scope of impact through the inference engine; Step 5: abnormal data feedback and repair: the cloud data center platform sends abnormal data information processing instructions and abnormal data information diagnosis results to the IoT meter detection control system and management personnel through the wireless communication network to automatically trigger the abnormal recovery mechanism.

[0007] As a further technical solution of the present invention, the automatic screening model performs preliminary judgment and marking on abnormal data information in the detection of the IoT table based on variational modal decomposition combined with wavelet semi-soft threshold. The variational modal decomposition uses a non-negative matrix to decompose the environmental parameters and equipment state parameters into multiple local modes. The non-negative matrix is ​​adaptively decomposed through variation, and the decomposition formula is: (1) In formula (1), is the kth local mode, is the residual term, K is the total number of decomposed local modes; decompose the kth local mode into wavelet coefficients , and perform semi-soft threshold denoising on the wavelet coefficients through the wavelet semi-soft threshold to obtain the corrected wavelet coefficients , the modified wavelet coefficients are recombined to obtain the local mode after denoising, which is expressed as: (2) In formula (2), is the transposed matrix of the cross wavelet transform, which is used to convert the modified wavelet coefficients into the modified local modes , judging whether the environmental parameters and the equipment status parameters are abnormal according to the waveform characteristics and the spectrum characteristics of the corrected local mode.

[0008] As a further technical solution of the present invention, the abnormal data information diagnosis database adopts a graph structure for distributed storage and calculation, and organizes data in the form of nodes and edges. The abnormal data information diagnosis database adopts data backup and fault-tolerant mechanisms to ensure the security and reliability of data storage and application. The abnormal data information diagnosis database records the number of times the cache information item is called through an access counter, and uses a timer to trigger a clearing operation. The timer clears the cache information item based on a cache strategy. The cache strategy sorts the cache information item weights from large to small according to the number of times the cache information item is called and the calling time, and clears them in reverse order.

[0009] As a further technical solution of the present invention, the working method of the multivariate principal component PCA analysis comprises the following steps: Step 1: Data standardization: convert the preliminary abnormal data information into a standard normal distribution according to the time series, and perform multi-parameter sampling on the preliminary abnormal data information at the time node to form a high-dimensional vector space; Step 2: Calculate the covariance matrix, and use the covariance matrix to describe the correlation and variance between different variables in the standardized data to obtain the correlation structure between multivariate data; Step 3: Calculate the eigenvalues ​​and eigenvectors, and use the covariance matrix to perform eigenvalue decomposition to obtain the eigenvalues ​​and eigenvectors; Step 4: Select the principal components, sort the eigenvalues ​​from bottom to top, and select the first k eigenvalues ​​as the principal components; Step 5: Calculate the principal component score, and calculate the initial judgment of abnormal data information in the principal component score according to the definition of the principal component to form a principal component score matrix; Step 6: Identify the principal components, classify or cluster the data through the principal component score matrix, and fuse the principal component score matrices of different modal data to comprehensively judge whether the abnormal data information is abnormal.

[0010] As a further technical solution of the present invention, the SVM-DS data selection model combines the vector machine classifier SVM with the data screening DS to perform refined anomaly detection on the fused abnormal data information features. The standardized processing formula for the fused abnormal data information features is: (3) In formula (3), express The mean column vector of each dimension of , represents the information features of the abnormal data after the i-th fusion, express -dimensional full column vector; is the variance of the eigenvalue of the fused abnormal data information; The fused data is screened using the data screening DS method, and the output formula is: (4) In formula (4), represents the filtered data set, Represents the density value of the fused abnormal data information feature, Represents the density threshold; the support vector machine classifier is used to classify the filtered data, and the output formula is: (5) In formula (5), express The data classification results are represents the kernel function, represents the Lagrange multiplier, represents the bias term, express The sample labels of .

[0011] As a further technical solution of the present invention, the reasoning engine includes a rule base, an inference engine, a proof evaluator and an interpreter. The rule base performs automatic reasoning through rule matching. The inference engine draws conclusions through logical deduction based on the rules and facts defined in the rule base. The proof evaluator calculates the severity and impact range of the fault based on the abnormal characteristic values, frequency and duration generated by the occurrence of different types of faults. The interpreter interprets and presents the reasoning results based on known information and data. The output end of the rule base is connected to the input end of the inference engine, the output end of the inference engine is connected to the input end of the proof evaluator, and the output end of the proof evaluator is connected to the input end of the interpreter.

[0012] As a further technical solution of the present invention, the abnormal recovery mechanism sends alarm information and fault diagnosis information to management personnel and maintenance personnel via mobile phone text messages, emails and APP push, and automatically performs corresponding service restart, backup node switching and system restoration operations according to the abnormal diagnosis results. The abnormal recovery mechanism adjusts the load balancing strategy according to the abnormal situation to distribute the request to the normally operating nodes.

[0013] Positive beneficial effects: The invention discloses a method for diagnosing abnormal data information in the detection of an Internet of Things table, which gives full play to the advantages of a machine learning method in an Internet of Things environment, can effectively deal with various problems that occur in the actual application process, and improves the efficiency and flexibility of abnormal data information processing; by using a wireless sensor network and an embedded pre-processing chip to make a preliminary judgment on the collected data, the judgment efficiency and accuracy are greatly improved; by using a distributed graph database to classify, store and clean the abnormal data information, and to perform cluster analysis and association analysis on the data, the efficiency and accuracy of data processing and analysis are greatly improved; by using a multivariate principal component PCA analysis and a cross-modal feature fusion method, the abnormal data can be analyzed from multiple angles and aspects, and the detection accuracy is greatly improved; by introducing an inference engine, the abnormal data is automatically inferred, and the information such as the cause of the fault, the severity of the fault and the scope of influence can be discovered, and the depth and breadth of abnormal diagnosis can be improved; by using a wireless communication network and an automatic recovery mechanism, the abnormal data can be quickly and effectively fed back and repaired, and the efficiency and accuracy of abnormal processing are improved; by using a multivariate principal component PCA analysis and a cross-modal feature fusion, the problem of numerous abnormal data types and difficult feature abstraction is solved. At the same time, the SVM-DS data selection model and the abnormal data information diagnosis database are used to perform refined detection on the fused abnormal data information features, which improves the diagnostic effect of machine learning methods in the Internet of Things environment. In addition, in the abnormal data diagnosis process, cluster analysis and association analysis are also used to discover the implicit relationship and similarity between the input abnormal data, so as to better understand the rules and trends in the overall sample; the degree of automation and intelligence is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the overall process of a method for diagnosing abnormal data information in IoT table detection according to the present invention; Figure 2 It is a flow chart of multivariate principal component PCA analysis in a method for diagnosing abnormal data information in a table of things detection of the present invention; Figure 3 The model architecture of the inference engine in the abnormal data information diagnosis method in the detection of the table of things of the present invention; Figure 4 It is a schematic diagram of the architecture of refined identification of abnormal data information in a method for diagnosing abnormal data information in detection of a connected table of things according to the present invention; Figure 5 The present invention is a schematic diagram of the principle of an abnormal data information diagnosis database in a method for diagnosing abnormal data information in a connected table detection. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Figure 1 As shown, a method for diagnosing abnormal data information in IoT table detection includes the following steps: Step 1: Data collection and preliminary judgment: The IoT table detects and collects environmental parameters and device status parameters through the wireless sensor network, and performs preliminary abnormality judgment on the collected data through the embedded pre-processing chip. The environmental parameters and device status parameters at least include temperature, humidity, pressure, flow and working power. The embedded pre-processing chip performs preliminary judgment and marking of abnormal data information in the IoT table detection through the automatic screening model, and uploads the preliminary judged abnormal data information to the cloud data center platform through the wireless communication network; In the application process of embedded pre-processing chip, the chip can receive raw data from sensors, such as temperature, humidity, pressure, etc. These data may exist in the form of analog signals or digital signals. Depending on the type of sensor and the data format, the embedded pre-processing chip converts the raw data into a unified digital signal. This may include operations such as analog-to-digital conversion (ADC) and digital-to-analog conversion (DAC). The embedded pre-processing chip also includes a control circuit and a power supply circuit. The control circuit is responsible for receiving, converting, filtering, adjusting and other operations of the data, while the power supply circuit is responsible for providing a stable power supply for the entire chip.

[0017] Step 2: construct an abnormal data information diagnosis database. The cloud data center platform uses a distributed graph database to construct the abnormal data information diagnosis database. The abnormal data information diagnosis database classifies and stores the input abnormal data information according to the type, time and location factors, and configures storage strategy and cleanup strategy according to the data volume and storage time. The abnormal data information diagnosis database discovers the implicit relationship and similarity between the input abnormal data information through cluster analysis and association analysis, and diagnoses and processes the abnormal data information based on the analysis results; Step 3: Refined identification of abnormal data information: the cloud data center platform extracts multimodal features for preliminary judgment of abnormal data information through multivariate principal component PCA analysis, and performs cross-modal feature fusion. The cloud data center platform performs refined anomaly detection on the fused abnormal data information features based on the SVM-DS data selection model and the abnormal data information diagnosis database; Step 4: abnormal data information diagnosis: the cloud data center platform performs abnormal data information diagnosis based on the refined anomaly detection results, and automatically infers the cause of the fault, the severity of the fault, and the scope of impact through the inference engine; Step 5: abnormal data feedback and repair: the cloud data center platform sends abnormal data information processing instructions and abnormal data information diagnosis results to the IoT meter detection control system and management personnel through the wireless communication network to automatically trigger the abnormal recovery mechanism.

[0018] In the above embodiment, the automatic screening model performs preliminary judgment and marking on abnormal data information in the IoT table detection based on variational modal decomposition combined with wavelet semi-soft threshold. The variational modal decomposition uses a non-negative matrix to decompose the environmental parameters and device state parameters into multiple local modes. The non-negative matrix is ​​adaptively decomposed through variation, and the decomposition formula is: (1) In formula (1), is the kth local mode, is the residual term, K is the total number of decomposed local modes; In order to improve the separation effect, the variational adaptive non-negative matrix factorization method can be used. The specific steps are as follows: In the automatic screening model, a method based on the combination of variational mode decomposition (VMD) and wavelet semi-soft thresholding is used to make preliminary judgments and mark abnormal data information in the IoT table detection. The following are the specific methods and steps: 1. Data preprocessing: First, the IoT table detection data is preprocessed, including removing noise, filling missing values, etc., to improve the data quality.

[0019] 2. Variational Mode Decomposition: The preprocessed data is decomposed into multiple Intrinsic Mode Functions (IMFs) using the VMD method. The VMD method is an adaptive signal decomposition method that can effectively separate the frequency components of the signal.

[0020] 3. Wavelet semi-soft threshold denoising: Perform wavelet semi-soft threshold denoising on each IMF component. The wavelet semi-soft threshold denoising method uses wavelet transform to convert the signal to the frequency domain, and performs threshold processing on the signal in the frequency domain, setting the high-frequency components less than the threshold to zero to suppress the noise components.

[0021] 4. Feature extraction: Extract the features of the denoised IMF components, such as energy, entropy, etc. These features can be used to characterize the intrinsic structure of the signal.

[0022] 5. Automatic screening model: The extracted features are input into the automatic screening model to make preliminary judgments and mark abnormal data. The automatic screening model can be based on machine learning or deep learning methods, such as support vector machines (SVMs), artificial neural networks (ANNs), etc.

[0023] 6. Model training and optimization: Use the labeled normal data and the denoised IMF component features to train the automatic screening model. During the training process, cross-validation, adjustment of model parameters and other methods can be used to optimize the model.

[0024] 7. Abnormal data detection: Apply the trained automatic screening model to the actual IoT table detection data and make abnormal judgments on each data. Based on the judgment results, the abnormal data is preliminarily marked.

[0025] 8. Post-processing: Further analysis of the initially marked abnormal data, such as manual review, statistical analysis, etc., to obtain the final abnormal data list.

[0026] Through the above steps, the automatic screening model based on variational mode decomposition and wavelet semi-soft threshold can effectively make preliminary judgments and mark abnormal data information in the detection of the Internet of Things table. In practical applications, model parameters and optimization algorithms can be adjusted according to actual conditions to improve the screening effect.

[0027] In a specific embodiment, the method of adaptively decomposing a non-negative matrix through variation plays a key role in the automatic screening model, which helps to mine the internal hidden structure in the process of detecting abnormal data information in the association table and improve the accuracy of abnormal data detection. At the same time, combined with the wavelet semi-soft threshold denoising technology, it can effectively suppress noise and further improve the reliability of the detection results.

[0028] Local modes are local features of data in the detection of the Internet of Things table. The vibration characteristics of these local areas can be described by local modes. These local data are converted into microscopic function characteristics to improve the calculation and application capabilities of the data. The residual term parameters are incorporated into the calculation model to find the difference between the parameter data response and the theoretical predicted structural response. The residual term can be used to evaluate the accuracy and reliability of the analysis method. The application process is then integrated with the total number of local modes. This data can reflect the diversity of local vibration characteristics in a certain frequency band in the detected Internet of Things table data, which can help evaluate the quality of the optimization design of abnormal data and avoid poor optimization results caused by local modal phenomena.

[0029] In specific applications, the kth local mode is decomposed into wavelet coefficients , and perform semi-soft threshold denoising on the wavelet coefficients through the wavelet semi-soft threshold to obtain the corrected wavelet coefficients , the modified wavelet coefficients are recombined to obtain the local mode after denoising, which is expressed as: (2) In formula (2), is the transposed matrix of the cross wavelet transform, which is used to convert the modified wavelet coefficients into the modified local modes , judging whether the environmental parameters and the equipment status parameters are abnormal according to the waveform characteristics and the spectrum characteristics of the corrected local mode.

[0030] In a specific embodiment, non-negative matrix adaptive decomposition (NMAD) is a matrix decomposition method based on non-negative matrix factorization (NMF). NMF is a linear transformation method that can decompose a non-negative matrix into the product of two non-negative matrices, thereby revealing potential structures and new features in the matrix.

[0031] When it works, two non-negative matrices W and H are randomly set, and a small positive number is set as the initial iteration step size. Then in each iteration process, W, H and step size are updated according to the following steps, and the product W * H is calculated based on W and H. Then the update direction of W and H is calculated, and according to the pre-set convergence condition (such as Euclidean distance or other distance indicators), it is checked whether the update of W and H meets the convergence condition. If it meets, the iteration is stopped; otherwise, the step size is updated and returned to step a. When the preset number of iterations is reached or the convergence condition is met, the iteration is stopped.

[0032] In a specific embodiment, the embedded pre-processing chip can make a preliminary judgment and mark the abnormal data information in the IoT table detection through an automatic screening model. The model is constructed by combining variational mode decomposition with wavelet semi-soft threshold, and is specifically implemented as follows: the embedded pre-processing chip is a small processor with a built-in automatic screening model. It can receive the raw data acquired by the sensor and make a preliminary abnormal judgment and mark it through the automatic screening model. Variational mode decomposition uses a non-negative matrix decomposition method to decompose the raw data into a series of basic mode functions (BMFs) to reflect the change characteristics of different time scales and frequency scales. After this step, a new multidimensional feature vector is obtained. Wavelet semi-soft threshold uses wavelet semi-soft threshold to remove noise in the feature vector and extract more obvious features for subsequent anomaly detection. Anomaly detection uses an automatic screening model for anomaly detection. The automatic screening model classifies the feature vector by learning a large amount of known abnormal data to obtain the marks of abnormal and normal data. Abnormal processing processes the abnormal data according to the data marking results output by the automatic screening model, such as recording, alarming, or analyzing and processing the abnormal data. Through this method, the embedded pre-processing chip can quickly and accurately make preliminary abnormal judgments on the collected data, and can realize embedded real-time abnormal detection and processing in the IoT application scenario, which has high practical application value. The hardware working environment includes: 1. Sensor: used to collect environmental parameter and equipment status parameter signals.

[0033] 2. Data acquisition card: converts the signals collected by the sensor into digital signals and transmits them to the computer.

[0034] 3. Computer: Software environment for running variational mode decomposition and wavelet semi-soft thresholding algorithms.

[0035] 4. Storage device: used to store the collected signal data and algorithm processing results.

[0036] 5. Display and input devices: used for human-computer interaction and result display.

[0037] 6. Communication equipment: such as network interface cards, routers, etc., used for application scenarios such as remote monitoring and data transmission.

[0038] The WISDM dataset on the UCI Machine Learning Repository was used, which includes accelerometer data during human motion, including walking, going upstairs, and going downstairs. The three types of data, walking, going upstairs, and going downstairs, were taken as normal data, and other types of data were taken as abnormal data. The data was denoised and standardized, and the time domain and frequency domain features were extracted using signal spectrum analysis. The automatic screening model based on variational mode decomposition and wavelet semi-soft threshold (Group A) was adopted, and the total number of local modes was set to 10. The accuracy, recall rate, and F1 score were selected as evaluation indicators. The traditional local anomaly factor algorithm (Group B) was selected as the comparison algorithm. The experiment was repeated five times, and the results are recorded in Table 1.

[0039] Table 1 Results Statistics

[0040] The results show that the automatic screening model has higher accuracy, recall and F1 score. In comparison, the local anomaly factor algorithm performs poorly. The well-performing automatic screening model can provide strong support for anomaly detection in the Internet of Things.

[0041] In the above embodiment, the abnormal data information diagnosis database adopts a graph structure for distributed storage and calculation, and organizes data in the form of nodes and edges. The abnormal data information diagnosis database adopts data backup and fault-tolerant mechanisms to ensure the security and reliability of data storage and application. The abnormal data information diagnosis database records the number of times the cache information item is called through an access counter, and uses a timer to trigger a clearing operation. The timer clears the cache information item based on a cache strategy. The cache strategy sorts the cache information item weights from large to small according to the number of times the cache information item is called and the calling time, and clears them in reverse order.

[0042] In a specific embodiment, Figure 5As shown in the figure, the abnormal data information diagnosis database adopts a graph structure for distributed storage and calculation. It organizes data in the form of nodes and edges, and adopts data backup and fault tolerance mechanisms to ensure the security and reliability of data storage and application. The database also uses access counters to record the number of times the cache information items are called, and uses timers to trigger clearing operations to clear cache information items according to cache strategies, making it more efficient. Graph database storage: A graph structure is used to store heterogeneous data, including nodes and edges, and the data is organically combined to form a graph database. At the same time, distributed storage and calculation are adopted, and different nodes can communicate with each other. By graphically representing the data, the data can be easily analyzed. Data backup and fault tolerance mechanism: Data backup and fault tolerance mechanism are adopted to ensure the security and reliability of data. When a node fails, the data maintenance and repair can be completed through the backup node, and the data will not be lost. Access counter: The access counter records the number of times the cache information items are called, which can store commonly used data in the cache, thereby reducing its access time. At the same time, the validity period of the cache can also be determined by recording the call time of the cache point. Timer: Use a timer to trigger the clearing operation to clear expired cache information items, avoid excessive data storage, and make the database more efficient. Cache strategy: Use the number of times the cache information items are called and the call time to perform weighted storage, making it faster when accessed. When the cache expires or the database storage space reaches capacity, the cache information items can be cleared by sorting them from large to small according to weight. Through the above implementation, the abnormal data information diagnosis database can manage and store data more efficiently, and improve the reliability and security of the data, thereby meeting users' needs for rapid query and accurate diagnosis of abnormal data information.

[0043] In the above embodiment, the working method of the multivariate principal component PCA analysis includes the following steps: Step 1: Data standardization: convert the preliminary abnormal data information into a standard normal distribution according to the time series, and perform multi-parameter sampling on the preliminary abnormal data information at the time node to form a high-dimensional vector space; Step 2: Calculate the covariance matrix, and use the covariance matrix to describe the correlation and variance between different variables in the standardized data to obtain the correlation structure between multivariate data; Step 3: Calculate the eigenvalues ​​and eigenvectors, and use the covariance matrix to perform eigenvalue decomposition to obtain the eigenvalues ​​and eigenvectors; Step 4: Select the principal components, sort the eigenvalues ​​from bottom to top, and select the first k eigenvalues ​​as the principal components; Step 5: Calculate the principal component score, and calculate the initial judgment of abnormal data information in the principal component score according to the definition of the principal component to form a principal component score matrix; Step 6: Identify the principal components, classify or cluster the data through the principal component score matrix, and fuse the principal component score matrices of different modal data to comprehensively judge whether the abnormal data information is abnormal.

[0044] In a specific embodiment, multivariate principal component PCA analysis is a multivariate statistical analysis method that can be used to determine the main relationship between a set of variables. It converts the original data into a set of new variables through linear transformation. These new variables are called principal components, and each principal component explains a certain proportion of the variance in the original data. In the refined identification of error information, multivariate principal component PCA analysis can be used to extract multimodal features of abnormal data and provide preliminary judgment for subsequent feature fusion and identification.

[0045] In the above embodiment, the SVM-DS data selection model combines the vector machine classifier SVM with the data screening DS to perform refined anomaly detection on the fused abnormal data information features. The standardized processing formula for the fused abnormal data information features is: (3) In formula (3), express The mean column vector of each dimension of , represents the information features of the abnormal data after the i-th fusion, express -dimensional full column vector; is the variance of the eigenvalue of the fused abnormal data information; In a specific embodiment, when the features of the fused abnormal data information are standardized, the specific working method is as follows: data preprocessing: preprocess the fused abnormal data, including removing noise, filling missing values, etc., to improve the quality of the data, and then extract features of the abnormal data information type in the IoT table detection: extract meaningful features such as mean, standard deviation, energy, etc. from the preprocessed data. These features help to characterize the abnormal nature of the data. Then, the abnormal data information is standardized, that is, the extracted features are standardized so that different features are comparable. In a further embodiment, the standardization processing method mainly includes the following two: a. Zero mean standardization: Set the mean of the feature to 0 and the standard deviation to 1. This allows the standardized feature to fluctuate around 0, and the degree of fluctuation is proportional to the degree of abnormality of the original data.

[0046] b. Standard deviation standardization: Set the mean of the feature to 0 and the standard deviation to 1. Compared with zero mean standardization, the feature value distribution range after standard deviation standardization is wider, but it can still reflect the degree of abnormality of the data.

[0047] 4. Anomaly detection: Input the standardized features into the anomaly detection model to judge the abnormal data. The anomaly detection model can be based on statistical methods, machine learning methods, or deep learning methods.

[0048] 5. Model training and optimization: Use the labeled normal data and standardized features to train the anomaly detection model. During the training process, you can use cross-validation, adjust model parameters and other methods to optimize the model.

[0049] 6. Abnormal data identification: Apply the trained anomaly detection model to the actual data to identify anomalies for each data. Based on the judgment results, further analyze and manage the abnormal data.

[0050] In the detection of the IoT table, the abnormal data diagnosis types include data information such as numerical anomaly, fluctuation anomaly, association anomaly, pattern anomaly, missing anomaly, and clustering anomaly. Among them, numerical anomaly refers to the situation where the data in the IoT table is significantly different from the normal data range, and there may be errors or abnormalities. For example, abnormal values ​​caused by sensor failure, data transmission errors, etc. Fluctuation anomaly refers to abnormal fluctuations in the IoT table data in time or space. This may be caused by environmental factors, equipment failure, or malicious attacks. For example, the data suddenly fluctuates violently over a period of time, or the data fluctuates abnormally in a specific area. Association anomaly refers to the abnormal relationship between data from different sensors or different areas in the IoT table. This may be caused by factors such as failures between multiple sensors, errors in the data transmission process, and inconsistencies within the system. For example, the data of two interrelated sensors show significant inconsistency at a certain moment. Pattern anomaly refers to a pattern in the IoT table data that is significantly different from the normal pattern. This may be caused by abnormal behavior, abnormal events, or malicious attacks. For example, under normal circumstances, the data in a certain area should show a specific change trend within a specific time range. If a data pattern that does not match this appears, there may be an abnormality. Missing anomalies refer to situations where data in the IoT table is missing or incomplete. This may be caused by factors such as sensor failure, data transmission interruption, or system error. Abnormal clustering refers to the clustering of abnormal data in the IoT table in space or time. This may be caused by equipment failure, environmental factors, or malicious attacks. For example, the prevalence of abnormal values ​​in the data in a certain area or time period may indicate that there is a problem in the area or time period. In other embodiments, there is also other abnormal data information. For these abnormal data diagnosis types, the IoT table detection system needs to adopt corresponding algorithms and strategies for anomaly detection. In a specific embodiment, the fused data is screened using a data screening DS method, and the output formula is: (4) In formula (4), represents the filtered data set, Represents the density value of the fused abnormal data information feature, represents the density threshold; the support vector machine classifier is used to classify the filtered data. In specific applications, the original data related to the target problem that needs to be collected is processed, and then the correlation between the features is analyzed, the features with strong correlation are retained, and the features with weak correlation are eliminated. Based on the importance indicators of the features, such as the density value of the filtered data set, the density threshold of the abnormal data information features mentioned above, the data information is processed. In data communication, data feature selection is also performed, and then the result information is evaluated. When evaluating the data information, the filtered features are evaluated to check the screening effect. According to the evaluation results, the data screening method is iteratively optimized. The abnormal data information is diagnosed and studied, and the filtered features are used for subsequent modeling and analysis.

[0051] In the specific steps, the output formula for abnormal data information is: (5) In formula (5), express The abnormal data information data classification results, represents the kernel function, represents the Lagrange multiplier, represents the bias term, express The sample labels of .

[0052] In a specific embodiment, the SVM-DS data selection model combines the vector machine classifier SVM with the data screening DS to perform refined anomaly detection on the fused abnormal data information features, standardizes the fused abnormal data information features, and calculates the standardized features through formula (3). The standardized fused features are subjected to data screening DS, and the filtered data set is calculated through formula (4). According to the set density threshold value, the data with density less than the threshold value is eliminated, thereby obtaining the filtered data set. The filtered data set is classified using the support vector machine classifier, and the data classification result is calculated through formula (5). Among them, the kernel function can be selected according to different application scenarios, and the Lagrange multiplier and the bias term are important parameters of the SVM classifier. By combining the SVM classifier with the data screening DS method, the SVM-DS data selection model can effectively improve the accuracy and efficiency of anomaly detection, can process multi-dimensional abnormal data information features, and has good adaptability to large-scale data sets. The hardware working environment of the model includes the following: 1. Computer: An operating system environment that can be used for writing model algorithms and processing data.

[0053] 2. Server: can be used to store large-scale data sets and provide high-performance computing capabilities.

[0054] 3. GPU: can be used to accelerate the training and reasoning process of machine learning algorithms and improve model efficiency.

[0055] 4. Storage devices: such as hard disks, SSDs, etc., used to store data and model parameters.

[0056] 5. Memory: Sufficiently large memory can improve the speed and performance of the algorithm.

[0057] 6. Input and output devices: such as keyboard, mouse, monitor, etc., used for human-computer interaction and result display.

[0058] A dataset from the Numenta Anomaly Benchmark (NAB) dataset was selected for the experiment. NAB is a public anomaly detection dataset that contains various types of anomalies, such as server load, network traffic, etc., and performs preprocessing and feature extraction. The proposed fusion method is used to fuse multi-source data, and the fused data is standardized. The fused data is filtered using the data screening method DS to obtain the filtered dataset. The filtered dataset is input into the SVM classifier for classification to obtain the classification results. The detection accuracy of the SVM-DS data selection model (Group A) is evaluated, including precision, recall, and F1 score, and compared with the traditional SVM algorithm (Group B). The experiment is repeated five times and recorded in Table 2.

[0059] Table 2 Results Statistics

[0060] According to the experimental results, the detection accuracy of the SVM-DS data selection model is better than that of the traditional SVM algorithm. Its average accuracy, recall rate and F1 score are 0.943, 0.891 and 0.915 respectively, which are significantly higher than the performance of the traditional SVM algorithm. This shows that the proposed fusion method and data screening method can obtain more accurate anomaly detection results. These methods are applicable to various types of abnormal data and can be applied in practical scenarios to improve the accuracy and efficiency of anomaly detection.

[0061] In the above embodiment, the reasoning engine includes a rule base, an inference engine, a proof evaluator and an interpreter. The rule base performs automatic reasoning through rule matching. The inference engine draws conclusions through logical deduction based on the rules and facts defined in the rule base. The proof evaluator calculates the severity and impact range of the fault based on the abnormal characteristic values, frequency and duration generated by the occurrence of different types of faults. The interpreter interprets and presents the reasoning results based on known information and data. The output end of the rule base is connected to the input end of the inference engine, the output end of the inference engine is connected to the input end of the proof evaluator, and the output end of the proof evaluator is connected to the input end of the interpreter.

[0062] In a specific embodiment, the inference engine is a system that can perform automatic reasoning, and a system that draws new conclusions and information by reasoning and inferring existing knowledge. In the diagnosis of abnormal data information, the inference engine can use the abnormal detection results to further perform reasoning analysis to obtain information such as the cause of the fault, the severity of the fault and the scope of impact, so as to achieve rapid processing and resolution of abnormal situations. Specifically, the inference engine mainly includes the following units: Rule base: contains known rules and knowledge, and can use prior knowledge to analyze and reason data. Inference engine: It is the core component of the inference engine, which can process various forms of knowledge representation and generate new conclusions and information through reasoning and inference. Proof evaluator: It is used to prove and verify in the reasoning process to ensure the correctness and accuracy of the reasoning results. Interpreter: The reasoning results can be interpreted and presented for easy understanding and use by users. By using the inference engine, the cloud data center platform can quickly diagnose abnormal data, predict the cause of the fault and the scope of impact, and further improve the availability and stability of the system.

[0063] In the above embodiment, the abnormal recovery mechanism sends alarm information and fault diagnosis information to management personnel and maintenance personnel through mobile phone text messages, emails and APP push, and automatically performs corresponding service restart, backup node switching and system restoration operations according to the abnormal diagnosis results. The abnormal recovery mechanism adjusts the load balancing strategy according to the abnormal situation and distributes the request to the normally operating nodes.

[0064] In a specific embodiment, abnormal data feedback and repair are important processes for the cloud data center platform to handle abnormal situations. The abnormal situation can be quickly repaired and recovered by sending abnormal information and processing instructions to the IoT table detection control system and management personnel, and automatically triggering the abnormal recovery mechanism. Specifically, the abnormal recovery mechanism includes the following aspects: Data backup and recovery: In abnormal situations, the cloud data center platform can ensure the security and integrity of system data by backing up data and resources. At the same time, when recovering data, technologies and methods such as data replication and data recovery can be used to achieve data transmission and sharing between multiple nodes, so as to quickly restore the affected data resources. Emergency response: When an abnormal situation occurs, the cloud data center platform can quickly take measures through a preset emergency response plan to ensure the normal operation of the system. Specific measures may include repairing hardware equipment, security vulnerabilities, software defects, etc., and at the same time, technologies such as failover and load balancing can be used to ensure the availability and stability of the system. Security monitoring and maintenance: The cloud data center platform can also ensure the security and stability of the system through security monitoring and maintenance. Specifically, the system can be monitored and closely monitored by adopting security policies and mechanisms, and abnormal situations can be discovered and handled in a timely manner to ensure the stable operation of the system. By adopting the above-mentioned abnormal recovery mechanism, the cloud data center platform can effectively deal with various abnormal situations, thereby ensuring the stability and security of the data center system.

[0065] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that these specific embodiments are only illustrative, and those skilled in the art may omit, replace, and change the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, merging the above method steps so as to perform substantially the same functions in substantially the same manner to achieve substantially the same results is within the scope of the present invention. Therefore, the scope of the present invention is limited only by the appended claims.

Claims

1. A method for diagnosing abnormal data information in IoT table detection, characterized by: The following steps are involved: Step 1: Data collection and preliminary judgment: The IoT table detects and collects environmental parameters and device status parameters through the wireless sensor network, and performs preliminary abnormality judgment on the collected data through the embedded pre-processing chip. The environmental parameters and device status parameters at least include temperature, humidity, pressure, flow and working power. The embedded pre-processing chip performs preliminary judgment and marking of abnormal data information in the IoT table detection through the automatic screening model, and uploads the preliminary judged abnormal data information to the cloud data center platform through the wireless communication network; Step 2: construct an abnormal data information diagnosis database. The cloud data center platform uses a distributed graph database to construct the abnormal data information diagnosis database. The abnormal data information diagnosis database classifies and stores the input abnormal data information according to the type, time and location factors, and configures storage strategy and cleanup strategy according to the data volume and storage time. The abnormal data information diagnosis database discovers the implicit relationship and similarity between the input abnormal data information through cluster analysis and association analysis, and diagnoses and processes the abnormal data information based on the analysis results; Step 3: Refined identification of abnormal data information: the cloud data center platform extracts multimodal features for preliminary judgment of abnormal data information through multivariate principal component PCA analysis, and performs cross-modal feature fusion. The cloud data center platform performs refined anomaly detection on the fused abnormal data information features based on the SVM-DS data selection model and the abnormal data information diagnosis database; Step 4: abnormal data information diagnosis: the cloud data center platform performs abnormal data information diagnosis based on the refined anomaly detection results, and automatically infers the cause of the fault, the severity of the fault, and the scope of impact through the inference engine; Step 5: abnormal data feedback and repair: the cloud data center platform sends abnormal data information processing instructions and abnormal data information diagnosis results to the IoT meter detection control system and management personnel through the wireless communication network to automatically trigger the abnormal recovery mechanism.

2. The method for diagnosing abnormal data information in the detection of a IoT table according to claim 1, characterized in that: The automatic screening model performs preliminary judgment and marking of abnormal data information in the IoT table detection based on variational modal decomposition combined with wavelet semi-soft threshold. The variational modal decomposition uses a non-negative matrix to decompose the environmental parameters and device state parameters into multiple local modes. The non-negative matrix is ​​adaptively decomposed through variation. The decomposition formula is: （1） In formula (1), is the kth local mode, is the residual term, K is the total number of decomposed local modes; decompose the kth local mode into wavelet coefficients , and perform semi-soft threshold denoising on the wavelet coefficients through the wavelet semi-soft threshold to obtain the corrected wavelet coefficients , the modified wavelet coefficients are recombined to obtain the local mode after denoising, which is expressed as: （2） In formula (2), is the transposed matrix of the cross wavelet transform, which is used to convert the modified wavelet coefficients into the modified local modes , judging whether the environmental parameters and the equipment status parameters are abnormal according to the waveform characteristics and the spectrum characteristics of the corrected local mode.

3. The method for diagnosing abnormal data information in the detection of a IoT table according to claim 1, characterized in that: The abnormal data information diagnosis database adopts a graph structure for distributed storage and calculation, and organizes data in the form of nodes and edges. The abnormal data information diagnosis database adopts data backup and fault-tolerant mechanisms to ensure the security and reliability of data storage and application. The abnormal data information diagnosis database records the number of times the cache information item is called through an access counter, and uses a timer to trigger a clearing operation. The timer clears the cache information item based on a cache strategy. The cache strategy sorts the cache information item weights from large to small according to the number of times the cache information item is called and the calling time, and clears them in reverse order.

4. The method for diagnosing abnormal data information in the detection of a IoT table according to claim 1, characterized in that: The working method of the multivariate principal component PCA analysis comprises the following steps: Step 1: Data standardization: convert the preliminary abnormal data information into a standard normal distribution according to the time series, and perform multi-parameter sampling on the preliminary abnormal data information at the time node to form a high-dimensional vector space; Step 2: Calculate the covariance matrix, and use the covariance matrix to describe the correlation and variance between different variables in the standardized data to obtain the correlation structure between multivariate data; Step 3: Calculate the eigenvalues ​​and eigenvectors, and use the covariance matrix to perform eigenvalue decomposition to obtain the eigenvalues ​​and eigenvectors; Step 4: Select the principal components, sort the eigenvalues ​​from bottom to top, and select the first k eigenvalues ​​as the principal components; Step 5: Calculate the principal component score, and calculate the initial judgment of abnormal data information in the principal component score according to the definition of the principal component to form a principal component score matrix; Step 6: Identify the principal components, classify or cluster the data through the principal component score matrix, and fuse the principal component score matrices of different modal data to comprehensively judge whether the abnormal data information is abnormal.

5. The method for diagnosing abnormal data information in the detection of a IoT table according to claim 1, characterized in that: The SVM-DS data selection model combines the vector machine classifier SVM with the data screening DS to perform refined anomaly detection on the fused abnormal data information features. The standardized processing formula for the fused abnormal data information features is: （3） In formula (3), express The mean column vector of each dimension of , represents the information features of the abnormal data after the i-th fusion, express -dimensional full column vector; is the variance of the eigenvalue of the fused abnormal data information; The fused data is screened using the data screening DS method, and the output formula is: （4） In formula (4), represents the filtered data set, Represents the density value of the fused abnormal data information feature, Represents the density threshold; the support vector machine classifier is used to classify the filtered data, and the output formula is: （5） In formula (5), express The data classification results are represents the kernel function, represents the Lagrange multiplier, represents the bias term, express The sample labels of .

6. The method for diagnosing abnormal data information in the detection of a IoT table according to claim 1, characterized in that: The reasoning engine includes a rule base, an inference engine, a proof evaluator and an interpreter. The rule base performs automatic reasoning through rule matching. The inference engine draws conclusions through logical deduction based on the rules and facts defined in the rule base. The proof evaluator calculates the severity and impact range of the fault based on the abnormal characteristic values, frequency and duration generated by the occurrence of different types of faults. The interpreter interprets and presents the reasoning results based on known information and data. The output end of the rule base is connected to the input end of the inference engine, the output end of the inference engine is connected to the input end of the proof evaluator, and the output end of the proof evaluator is connected to the input end of the interpreter.

7. The method for diagnosing abnormal data information in the detection of a IoT table according to claim 1, characterized in that: The abnormal recovery mechanism sends alarm information and fault diagnosis information to management personnel and maintenance personnel through mobile phone text messages, emails and APP push, and automatically performs corresponding service restart, backup node switching and system restoration operations according to the abnormal diagnosis results. The abnormal recovery mechanism adjusts the load balancing strategy according to the abnormal situation and distributes the request to the normally operating nodes.

Images