Data analysis method, device, equipment and storage medium based on intelligent gateway

Abstract

The present application discloses a data analysis method, device, equipment and storage medium based on intelligent gateway, which relates to the field of artificial intelligence and edge computing technology. The method includes: obtaining sensor data, video surveillance data, control instruction data and other data from various intelligent devices; performing time alignment and format conversion on the data to obtain standardized gateway data; performing sampling diffusion and gain update on the standardized gateway data to obtain fused gateway data; performing feature extraction and weighted adjustment on the fused gateway data to obtain fused gateway data features; performing linear dimension reduction on the fused gateway data features to obtain a low-dimensional gateway data feature matrix; detecting abnormal features from the low-dimensional gateway data feature matrix by fitting the normal data range, and determining the corresponding abnormal gateway data according to the abnormal features. The present application can effectively integrate, process and analyze a large amount of data from different devices, and identify abnormal data and behaviors of the devices.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and edge computing technology, and in particular to a data analysis method, device, equipment and storage medium based on an intelligent gateway. Background Art

[0002] With the development of Internet of Things technology, smart gateways, as key nodes connecting the physical and digital worlds, are responsible for processing massive and diverse data from sensors and devices in fields such as industrial automation, smart cities, and smart homes, including physical quantities and multimedia information. Real-time and efficient analysis of this data is essential to improving system performance, optimizing resources, and ensuring security. At present, data processing of smart gateways relies on traditional methods and basic data analysis technologies, and occasionally uses machine learning for classification and prediction, but usually requires uploading data to the cloud for processing, resulting in network delays and causing privacy and security issues. There are several major problems with existing data processing methods. First, network congestion may occur during data transmission to the cloud or central server, resulting in increased data processing delays, which is unacceptable for application scenarios that require real-time response. Second, large amounts of data transmission will also increase the burden on network bandwidth and increase operating costs. In addition, storing data in the cloud or central server may increase the risk of data leakage and attacks. Finally, traditional data analysis methods often lack sufficient intelligence and cannot effectively identify anomalies or important patterns from complex data. Therefore, how to identify abnormal data from a large amount of gateway data in real time and efficiently has become an urgent problem to be solved. Summary of the invention

[0003] The purpose of this application is to provide a data analysis method, device, equipment and storage medium based on an intelligent gateway, aiming to solve the technical problem of how to identify abnormal data from a large amount of gateway data in real time and efficiently.

[0004] To achieve the above objectives, the present application proposes a data analysis method based on an intelligent gateway, the method comprising:

[0005] Obtain sensor data, video surveillance data, control command data, network communication data, and device log data from various smart devices;

[0006] Performing time alignment and format conversion on the sensor data, the video surveillance data, the control instruction data, the network communication data, and the device log data to obtain standardized gateway data;

[0007] Performing sampling diffusion and gain updating on the standardized gateway data to obtain fused gateway data;

[0008] Extracting features and performing weighted adjustment on the fusion gateway data to obtain fusion gateway data features;

[0009] Performing linear dimensionality reduction on the fusion gateway data features to obtain a low-dimensional gateway data feature matrix;

[0010] Abnormal features are detected from the low-dimensional gateway data feature matrix by fitting the normal data range, and corresponding abnormal gateway data are determined according to the abnormal features.

[0011] One or more technical solutions proposed in this application have at least the following technical effects:

[0012] The sensor data, video surveillance data, control instruction data, network communication data and device log data from various intelligent devices are obtained; the sensor data, video surveillance data, control instruction data, network communication data and device log data are time aligned and format converted to obtain standardized gateway data; the standardized gateway data is sampled and diffused and gain updated to obtain fused gateway data; the fused gateway data is feature extracted and weighted adjusted to obtain fused gateway data features; the fused gateway data features are linearly reduced to obtain a low-dimensional gateway data feature matrix; abnormal features are detected from the low-dimensional gateway data feature matrix by fitting the normal data range, and the corresponding abnormal gateway data is determined according to the abnormal features. The system first collects multiple types of data including sensors, video surveillance, control instructions, network communication and device logs from intelligent devices. After time alignment and format conversion, these data are unified into a standardized format for easy integration and analysis. Subsequently, the data quality and representativeness are improved through sampling diffusion and gain update, and feature extraction and weighted adjustment are performed to ensure the importance of key features in the analysis. Then, the linear dimensionality reduction technology is used to simplify the data structure while retaining the core information and reducing the computational complexity. Finally, the system compares the normal data range, efficiently identifies abnormal data points, and promptly discovers and handles potential problems in equipment operation, thereby ensuring the stability and reliability of the system. Through the above steps, the AI ​​smart gateway system can effectively integrate, process and analyze large amounts of data from different devices, identify abnormal behavior of devices, and ensure reliable operation of devices and efficient use of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0015] Figure 1 A flowchart of the first embodiment of the data analysis method based on the intelligent gateway of the present application;

[0016] Figure 2 A flow chart of the second embodiment of the data analysis method based on the intelligent gateway provided in this application;

[0017] Figure 3 A brief flowchart of the data analysis method based on the intelligent gateway provided in Example 2 of the present application.

[0018] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are only used to explain the technical solution of the present application and are not used to limit the present application. In order to better understand the technical solution of the present application, the following will be described in detail in conjunction with the accompanying drawings and specific implementation methods.

[0020] It should be noted that the execution subject of the embodiment of the present application may be a computing service device with data processing, network communication and program running functions, or an electronic device, AI smart gateway device, AI smart gateway system, etc. capable of implementing the above functions. The following takes the AI ​​smart gateway system as an example to illustrate this embodiment and the following embodiments.

[0021] Based on this, the embodiment of the present application provides a data analysis method based on an intelligent gateway, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the data analysis method based on the intelligent gateway of the present application.

[0022] In this embodiment, the data analysis method based on the intelligent gateway includes steps S10 to S60:

[0023] Step S10, acquiring sensor data, video surveillance data, control instruction data, network communication data, and device log data from various smart devices;

[0024] It should be noted that in this embodiment, the hardware platform of the AI ​​smart gateway generally adopts an efficient processor with an embedded ARM or RISC-V architecture, equipped with dedicated acceleration units such as GPU and NPU for real-time AI computing and deep learning reasoning. The communication module supports multiple protocols, such as Wi-Fi, 5G, Ethernet, ZigBee, LoRa, etc., to achieve high-speed connection and low-power communication between devices and with the cloud. The operating system and middleware are usually based on a lightweight Linux kernel or a real-time operating system (RTOS), and have built-in edge computing frameworks (such as EdgeX Foundry) and multimodal data processing middleware to ensure flexible scalability and task scheduling. The AI ​​computing framework generally integrates TensorFlow Lite, ONNX Runtime or OpenVINO to support local model deployment and reasoning. The security module covers hardware-level encryption units (TPM or Secure Enclave), end-to-end communication encryption (TLS / SSL), as well as authentication and access control to fully protect data and network security.

[0025] Smart devices refer to devices that can exchange data with the AI ​​smart gateway system through built-in sensors or actuators, and can collect data or execute specific control instructions through sensors. These devices can be thermostats and security cameras in smart homes, sensors and controllers in industrial automation, or monitors in medical equipment, etc. They are connected to the gateway wirelessly or wired to send and receive data. Sensor data refers to real-time information collected by various sensors for monitoring and analyzing physical phenomena such as temperature, pressure, humidity, speed and position. These data are output in the form of electrical signals and can reflect environmental changes, the state of people and objects. Sensors convert the measured physical phenomena into data that can be understood and processed by computers. Video surveillance data refers to images and video stream data captured by video surveillance equipment. These data can be used for security monitoring, behavior analysis, face recognition and other applications. In the smart gateway system, video surveillance data can be analyzed in real time to identify abnormal behavior or specific events. Control instruction data refers to commands and parameters used to control the behavior of smart devices. These instructions can be turning lights on and off, adjusting temperatures, changing device settings, etc. Control instruction data are usually sent by users to corresponding smart devices through the smart gateway system to achieve remote control or automatic control. Network communication data refers to the data transmitted between the smart gateway system and smart devices, including device status information, user instructions, system notifications, etc. These data ensure the interconnection and timely transmission of information between devices, and are the basis for the normal operation of the smart gateway system. Device log data refers to the process event record data generated by the IT system, including user operations, system events, sensor data, etc. By viewing the log data, you can understand which user, at what time, on which device or in which application system, did what specific operation.

[0026] It can be understood that, first of all, the AI ​​smart gateway system acquires data by establishing connections with various smart devices. Smart devices refer to those devices that can communicate directly with the gateway, such as temperature and humidity sensors, cameras, industrial controllers, etc., which send data to the gateway wirelessly or wired. Secondly, the acquisition of sensor data is achieved through sensors on smart devices, which can detect environmental variables such as temperature, humidity, etc., and convert these physical quantities into electrical signals and send them to the gateway for further processing and analysis. Then, the acquisition of video surveillance data is achieved through video surveillance devices connected to the gateway, which capture real-time video streams and transmit them to the gateway for storage and analysis. Next, the acquisition of control instruction data involves commands sent by users or automation systems to smart devices, which are transmitted through the gateway to control the behavior of smart devices, such as switching devices or adjusting settings. The acquisition of network communication data is to record and monitor the communication process between smart devices and between smart devices and gateways, including data transmission, status updates, and system responses, which is crucial to maintaining the stability and security of the system. Finally, the acquisition of device log data involves recording the operation history and system events of smart devices, which helps fault diagnosis and performance monitoring, and is usually completed by implementing logging functions on the device side. Through these steps, the AI ​​smart gateway system is able to comprehensively manage and analyze data from smart devices.

[0027] Step S20, performing time alignment and format conversion on the sensor data, the video surveillance data, the control instruction data, the network communication data, and the device log data to obtain standardized gateway data;

[0028] It should be noted that time alignment refers to the process of matching and aligning data points from different data sources according to timestamps. This is done to ensure that data from different data sources are comparable in the time dimension, even if their sampling frequencies or timestamp recording methods are different. For example, in a multi-sensor system, one sensor may record data at a faster frequency, while another may be slower. Time alignment dynamically adjusts the time axis so that the corresponding points in the two time series are aligned as much as possible, so that effective data analysis and comparison can be performed. Format conversion refers to the process of converting data from one format to another. This may include converting data from a CSV file to an Excel file, or from one encoding format to another. Format conversion is to ensure that data can be compatible and processed by different systems and applications. For example, some systems may only be able to process data in a specific format, so the data needs to be converted into a format that the system can recognize for further analysis and processing. Standardized gateway data refers to converting data from different devices and sensors into a unified and standardized format for easy storage, processing and analysis. This includes unifying the measurement unit, data structure and encoding method of the data. The purpose of standardization is to reduce data inconsistency and improve data interoperability and analysis efficiency. For example, the temperature data of different devices can be unified from Celsius and Fahrenheit to Celsius, or the communication protocols of different devices can be unified into a standard protocol to facilitate data integration and analysis.

[0029] It can be understood that, first of all, time alignment refers to synchronizing data from different sources according to a unified time base to ensure the consistency of data in the time dimension. This is done to accurately compare and associate information from different data sources during data analysis and improve the accuracy of analysis. Secondly, format conversion refers to converting data from one encoding or structure to another so that different systems or applications can recognize and process it. This usually involves changing the storage format, encoding method or data structure of the data, for example, converting video files from MP4 format to AVI format, or converting data from JSON format to XML format. This improves the compatibility and interchangeability of data and enables data to be used on different platforms and devices. Finally, standardized gateway data refers to further processing the data after time alignment and format conversion into a unified and standardized format, which includes unified measurement units, data structure and encoding methods. The effect of standardization is to make the data more consistent, convenient for storage, processing and analysis, and also improve the interoperability of data, so that data from different sources can be integrated for comprehensive analysis.

[0030] As an example, the steps of time aligning and format converting the sensor data, the video surveillance data, the control instruction data, the network communication data and the device log data to obtain standardized gateway data include: cleaning, denoising and standardizing the sensor data, the video surveillance data, the control instruction data, the network communication data and the device log data to obtain initial data; extracting timestamp information from the initial data; constructing time series data based on the timestamp information and the initial data; calculating the distance matrix between each of the time series data; starting from the lower right corner of the distance matrix, backtracking to find the minimum distance path; time aligning the time series data according to the minimum distance path to obtain reference data; identifying the data format and data structure of the reference data through JSON format matching expression, XML format matching expression, CSV format matching expression, log format matching expression and network communication data format matching expression; converting the reference data into a preset data format according to the data format and the data structure to obtain standardized gateway data.

[0031] Cleaning is the process of removing errors, duplications, or irrelevant information from a data set. This includes removing or correcting erroneous data records, deleting duplicate entries, and removing irrelevant or incomplete data. This is done to improve the quality of the data and ensure the accuracy of the analysis results. Denoising is the process of removing random noise or outliers from the data. Noise may be caused by sensor errors, data transmission errors, or other unexpected factors. Denoising helps improve the signal quality of the data and reduce interference during analysis. Standardization refers to converting data into a unified format or scale for comparison and analysis. This may include converting all numerical data to the same unit or encoding categorical data into numerical form. Standardization helps ensure that data from different data sources can be compared fairly. Initial data refers to the raw data collected before any processing. These data may contain noise, errors, and inconsistencies and need to be preprocessed through steps such as cleaning, denoising, and standardization. Timestamp information refers to the date and time stamp associated with a data point. This information is used to determine the time when the data point was collected and is essential for time series analysis and data synchronization. Time series data refers to a collection of data points arranged in chronological order. These data are often used to analyze trends and patterns over time. A distance matrix is ​​a table used to represent the similarity or difference between time series data. Each element in the matrix represents the distance between two time series, usually obtained by calculating the difference between them. In the distance matrix, the lower right corner refers to the last element of the matrix, that is, the intersection of the last row and the last column. In the process of time alignment, backtracking from the lower right corner is to find the best alignment path between time series. Backtracking refers to the process of checking step by step from the current position in the opposite direction. In time alignment, backtracking is used to start from the lower right corner of the distance matrix and gradually go up and left to find the minimum distance path. The minimum distance path refers to the path connecting the starting point and the end point in the distance matrix. The total distance (or difference) on this path is the smallest. This path represents the best alignment between time series data. Reference data refers to the data after time alignment. It serves as the basis for analysis and further processing. These data have been adjusted according to the minimum distance path to ensure temporal consistency. JSON format matching expressions refer to patterns or rules used to identify and parse JSON format data, such as "{\s*"[^"]*"\s*:\s*[^,]+(,\s*"[^"]*"\s*:\s*[^,]+)*}". XML format matching expressions refer to patterns or rules used to identify and parse XML format data, such as "<([^>]+)>([^<]+)< / \1> . A CSV format matching expression refers to a pattern or rule used to recognize and parse CSV (comma separated value) format data, such as "([^,]*)(,\s*([^,]*))*".Log format matching expressions refer to patterns or rules used to identify and parse data in log files, such as "\[(.*?)\] (.*) : (.*)". Log files usually contain operation records of systems or applications. Network communication data format matching expressions refer to patterns or rules used to identify and parse network communication data, such as "(GET|POST|PUT|DELETE)\s([^\s]+)\sHTTP\ / (1\.

[01] )". These data may follow specific protocols, such as HTTP, FTP, etc. Data format refers to the organization and encoding method of data, including the type, structure and representation method of data. Data structure refers to the organization of data in memory, which defines the relationship between data elements and the storage method of data. Preset data format refers to a pre-defined data format used to ensure the consistency and compatibility of data. In this embodiment, it is the JSON format. In data processing, converting data into a preset format can facilitate subsequent analysis and storage.

[0032] First, the AI ​​smart gateway system performs cleaning operations on sensor data, video surveillance data, control instruction data, network communication data, and device log data, which includes automatically identifying and removing erroneous or inconsistent data records, as well as identifying and replacing duplicate data entries. This is done to ensure the accuracy and completeness of the data and improve the reliability of subsequent analysis. Secondly, the system performs denoising operations, using algorithms to identify and filter out random fluctuations or outliers in the data, such as identifying and removing extreme values ​​that exceed the normal range in sensor data through statistical analysis methods, which helps reduce noise in the data and improve the quality of the signal. Then, the system standardizes the data, converting data in different formats and units into a unified format and unit, such as converting temperature data to degrees Celsius, in order to make the data comparable between different devices and sensors, and facilitate integration and analysis. Next, the AI ​​smart gateway system extracts timestamp information from the initial data after cleaning, denoising, and standardization, which usually involves parsing the date and time fields in the data record and converting them into a unified time format, so that the data can be sorted and associated in chronological order during data analysis. Then, the system constructs time series data based on the timestamp information and the initial data, and arranges the data points in chronological order to form a continuous time series. This can reveal the trends and patterns of data changes over time. Next, the system calculates the distance matrix between each time series data, and calculates a distance value for each pair of time series by comparing the differences between different time series. This can quantify the similarity or difference between different time series and provide a basis for subsequent time alignment. Starting from the lower right corner of the distance matrix, the system uses a dynamic programming algorithm to backtrack to find the minimum distance path. This can determine the best time series alignment to minimize the overall difference. According to the minimum distance path, the AI ​​smart gateway system time-aligns the time series data and adjusts the data points in different time series so that they correspond in time. This can obtain temporally consistent reference data, providing a basis for further data analysis. Finally, the system uses predefined regular expressions to identify the format and structure of reference data, such as the key-value pair structure of JSON data, the tag structure of XML data, the comma-delimited structure of CSV data, etc. This can automatically determine the format of the data and convert the data into a preset data format to obtain standardized gateway data. This allows data to be processed and analyzed consistently between different systems and applications, improving data interoperability and analysis efficiency.

[0033] Step S30, performing sampling diffusion and gain update on the standardized gateway data to obtain fused gateway data;

[0034] It should be noted that sampling diffusion refers to the process of simulating the data sampled from the real data distribution to gradually superimpose Gaussian noise in the data fusion process of the unscented Kalman filter (UKF), and finally obtaining a series of samples after noise superposition. This process can be regarded as gradually transforming the data from an ordered state to an unordered noise state, simulating the diffusion process of data from clear to completely random noise. Gain update refers to the key step of correcting the prediction results according to the new measurement value in the Kalman filtering process to obtain the final state estimate. Specifically, the Kalman gain is a weight coefficient that measures the difference between the predicted value and the actual measurement value, which is used to update the state estimate and the covariance matrix to minimize the mean square error. The purpose of gain update is to adjust the estimated value so that the predicted value is closer to the true value and improve the accuracy of the estimate. Fusion gateway data refers to the optimal state estimate that integrates multiple sensor data after sampling diffusion and gain update. Such data fusion can effectively integrate redundant or complementary information from different sensors and improve the accuracy, reliability and robustness of system state estimation.

[0035] It can be understood that, first, the AI ​​intelligent gateway system performs a sampling diffusion step on the standardized gateway data. Specifically, the system samples a set of sample points from the probability distribution of the nonlinear system state through the unscented Kalman filter (UKF) algorithm. These sample points capture the statistical characteristics of the state distribution without linearizing the system model. The purpose of this is to approximate the statistical characteristics of the nonlinear function without losing the nonlinear characteristics of the system, and to provide a set of samples that can represent the real data distribution for subsequent data fusion. Secondly, the system performs a gain update step. After obtaining new measurement data, the system calculates the Kalman gain, which is a weight coefficient that weighs the prediction error and the measurement error. The system uses this gain to adjust the state prediction, tilting the predicted value toward the measured value, thereby reducing the estimation error and improving the accuracy of the state estimation. The effect of this is to make the predicted value closer to the true value and optimize the accuracy of the state estimation. Finally, through the above steps, the system obtains the fused gateway data. These data are the state estimation results that integrate multiple sensor information after sampling diffusion and gain update. Fusion gateway data can more accurately reflect the true state of the observed system, improve the accuracy and reliability of state estimation, enable data to be consistently processed and analyzed between different systems and applications, and improve data interoperability and analysis efficiency.

[0036] As an example, the step of sampling, diffusing and gain updating the standardized gateway data to obtain fused gateway data includes: setting an initial data state vector and an initial data state covariance matrix according to the standardized gateway data; calculating diffusion parameters according to preset hyperparameters, preset weight parameters, preset calibration parameters and the dimension of the initial data state vector; calculating data state mean weights and data state covariance weights according to the diffusion parameters, the dimension, the preset hyperparameters and the preset weight parameters; calculating diffusion factors according to the dimension, the diffusion parameters and the initial data state covariance matrix; calculating observation prediction covariance and cross covariance according to at least four of the diffusion factor, the initial data state vector, the dimension, the data state mean weight, the process noise covariance matrix, the data state covariance weight, the nonlinear observation function and the observation noise covariance matrix; obtaining Kalman gain according to the observation prediction covariance and the inverse calculation of the observation prediction covariance; updating the state estimate and state covariance of the standardized gateway data according to the cross covariance, the Kalman gain and the observation prediction covariance to obtain fused gateway data.

[0037] Initial data state vector (0) is the initial guess of the system state, which is used to represent the initial estimate of the system. Specifically, for the AI ​​smart gateway, it may include the initial values ​​read from sensors (such as temperature sensors and pressure sensors), the preliminary analysis results of video surveillance data (such as whether there is abnormal activity in a certain area), and the initial execution status of control instruction data. For example, the initial data state vector may include: the initial temperature, humidity, pressure and other readings of sensor data; the execution status of control instructions (for example, whether the device is started or shut down); network communication status (whether it is connected normally). The initial data state covariance matrix P(0) represents the uncertainty of the initial estimate, that is, the credibility of the initial guess of the system state. In the AI ​​smart gateway, this can represent the uncertainty of the initial state of sensor data, video surveillance data, control instruction data, etc. For example, the temperature sensor may be more accurate than the video surveillance data, so its covariance is smaller, while the video surveillance data may be more affected by factors such as illumination and noise, and the covariance is larger. The hyperparameter α is a parameter in the UKF that controls the degree of expansion of the sampling points of the distribution. It is a very small positive number, which affects the ability of the Kalman filter algorithm to capture the state changes of nonlinear systems. Specifically, α in the AI ​​smart gateway can be set to affect the accuracy of feature extraction from sensors or video surveillance data, especially in complex or noisy environments. The weight parameter β controls the weight distribution of the target state estimation, especially in UKF, which is used to balance the importance of different modal data. In the AI ​​smart gateway, β can affect the fusion weight between sensor data and video surveillance data, network communication data, and control instruction data, which is usually 2. The calibration parameter k is an adjustment factor used to control the scalability in UKF, especially when dealing with nonlinear systems. In the AI ​​smart gateway, k may affect the responsiveness to changes in system behavior in device log data, and is 0 in this embodiment. Dimension refers to the number of elements contained in the state vector, which represents the complexity of the system state. For the AI ​​smart gateway, the dimensions of the initial data state vector may include readings of multiple sensors, video surveillance features, control instruction status, and network communication performance indicators. For example, temperature, humidity, pressure, device status, bandwidth, latency, etc. can all be vector dimensions. The diffusion parameter λ is calculated based on the initial data state covariance matrix and hyperparameters. It is used to describe how the uncertainty of state estimation propagates during the filtering process. In the AI ​​smart gateway, λ controls the accuracy and stability of the fusion of multimodal data (such as sensor data and video surveillance data) to ensure that the contributions of different data sources are reasonably integrated. The data state mean weight is the weight assigned to the average value of state estimates of different data sources when data is fused. For example, sensor data may be more accurate than device log data, so when fusing data, the mean weight of sensor data may be larger. The data state covariance weight represents the impact of the covariance matrix of different data sources on state estimation.For multimodal data in AI smart gateways, inputs with higher covariance (such as noise in video surveillance data) may produce greater uncertainty in the final estimation results, so their covariance weights need to be adjusted appropriately. The diffusion factor is calculated by combining the initial data state covariance matrix and the diffusion parameter, and is used to describe how to diffuse each part of the state variable. For AI smart gateways, the diffusion factor reflects how different data (such as sensor data and network communication data) propagate to the final state estimate. The process noise covariance matrix describes the external influences and noise on the system during dynamic changes, such as sensor drift or interference in network communication. For AI smart gateways, this usually involves the instability of the device itself or the interference of the external environment on the sensor. The nonlinear observation function is used to describe how to derive the state of the system from the measurement data. In AI smart gateways, sensor data, video surveillance data, etc. often involve nonlinear relationships. Especially when multiple data modalities are fused, how to convert these data into system states is a key task. The observation noise covariance matrix describes the impact of noise in the observation data. For example, in AI smart gateways, the quality of video surveillance data may be affected by factors such as lighting changes and viewing angle limitations, so the observation noise is large. The observation prediction covariance is the covariance calculated based on the predicted state and observation model, which represents the prediction error of the observed data. The AI ​​smart gateway uses this covariance to measure the deviation between the actual observation and the predicted value, thereby optimizing the data fusion process. The cross covariance measures the correlation between different data sources. For example, in the AI ​​smart gateway, the cross covariance between sensor data and video surveillance data can reflect whether the change in environmental state affects multiple data sources at the same time. The Kalman gain determines the degree of influence of the observation value on the system state update. The AI ​​smart gateway uses the Kalman gain to decide whether to trust sensor data, video surveillance data or control instruction data more, and then adjusts the final state estimate. State estimation is to infer the optimal state of the system based on current and past observation data. In the AI ​​smart gateway, state estimation may include real-time evaluation of device health status, environmental changes (such as temperature, humidity), etc. The state covariance reflects the credibility of the state estimate and represents the uncertainty of the system state. In the AI ​​smart gateway, the size and shape of the state covariance matrix help determine the contribution of each data source to the state estimate.

[0038] First, in the initialization step, the sensor data, control instructions, video surveillance and other information are extracted from the standardized gateway data to set the initial data state vector (representing the initial estimate of the current state) and the initial data state covariance matrix (representing the uncertainty of the initial estimate). These initial values ​​provide a starting point for the subsequent Kalman filter. For example, the temperature sensor reading and the current state of the controller instruction are used as vector elements, and the covariance matrix represents the error range between the data sources by previous experience or preset error values.

[0039] Next, the diffusion parameter λ is calculated. This step is implemented based on the preset hyperparameters (α), weight parameters (β), calibration parameters (k), and the dimensions of the initial data state vector. The dimensions of the initial state need to be determined, and then λ is calculated based on these parameters (usually values ​​obtained through experience or tuning). The diffusion parameter λ controls the degree of "expansion" of the data during the filtering process. It quantifies the contribution of each data source and ensures that nonlinear transformations are correctly handled when multi-source data is fused.

[0040] λ=α 2 (dim(X)+κ)−dim(X)

[0041] Here, α is a small positive number, usually around 10 −3 to 10 −1 , β is usually 2, indicating that the prior distribution is symmetric, κ is usually 0 unless nonlinear prior knowledge is required, and dim(x) is the dimension of the state vector.

[0042] Next, the data state mean weight and covariance weight are calculated using the diffusion parameter λ, data dimension, preset hyperparameters, and weight parameters. The data state mean weight represents the influence of different data sources in state estimation, and the covariance weight represents the impact of the uncertainty of the data source on the update of state estimation. For example, sensor data is usually more accurate, so its covariance weight is smaller, while video surveillance data may be more affected by environmental factors and have a larger covariance weight. This step ensures that the contribution of each data source is appropriately adjusted according to its reliability during fusion. Weight calculation formula:

[0043] ,

[0044] Where λ is the diffusion parameter, n is the dimension of the state vector, is the data state mean weight of the first data state sampling point, is the covariance weight of the first data state sampling point.

[0045] For the remaining 2n data state sampling points, the weights are calculated as follows:

[0046]

[0047] Where i=1,…,2n.

[0048] Then, the diffusion factor is calculated, specifically, the diffusion factor is obtained by the initial data state covariance matrix, the diffusion parameter λ and the data dimension. The role of the diffusion factor is to describe how the data state expands across different dimensions. It ensures that the system's response to different features is properly adjusted to avoid over-expanding or shrinking the impact of a data source on the system state, especially in the interaction of multi-source data.

[0049] Diffusion Factor =

[0050] Where dim(x) is the dimension of the state vector, λ is the diffusion parameter, and P is the covariance matrix of the current state estimate, reflecting the uncertainty of the estimate.

[0051] Next, the observation prediction covariance and cross covariance are calculated by combining the diffusion factor, the initial data state vector, the process noise covariance matrix, the data state mean weight, the data state covariance weight, the nonlinear observation function and the observation noise covariance matrix. The observation prediction covariance is used to measure the difference between the system prediction state and the observed data, and the cross covariance describes the relationship between different data sources. The calculation of these covariances helps us understand the mutual influence and potential error propagation paths between different data sources, preparing for the final Kalman gain calculation.

[0052] Then, the Kalman gain is obtained by calculating the inverse of the observation-prediction covariance and combining it with the cross covariance. The Kalman gain determines the importance of each observation in the state update. The larger the Kalman gain, the greater the influence of the observation on the state estimate. This step ensures that the signals obtained from multiple data sources can be weighted and integrated according to their reliability. Kalman gain: .

[0053] Finally, the Kalman gain is used to update the state estimate and state covariance of the standardized gateway data to obtain the final fused gateway data. Specifically, the Kalman gain is combined with the deviation of the observed data to adjust the value of the state estimate and reduce the inaccurate part, thereby improving the accuracy of the state estimate. At the same time, by updating the state covariance, the assessment of the uncertainty of future data can be adjusted to ensure that the subsequent data processing is more accurate. Update state estimate:

[0054]

[0055] in, is the state estimate updated at time k+1 based on all available information (including the latest observations), is the state estimate at time k+1 based only on the information predicted at time kk, K is the Kalman gain, which is a weight matrix used to determine the degree of influence of new observations on the state estimate, and Z(k+1) is the data actually observed at time k+1. is the observed value predicted at time k+1 based on the information at time k, which is obtained by transforming the predicted state Obtained by transforming the observation model h. is the observation residual, that is, the difference between the actual observation and the predicted observation, which represents the error between the prediction and the actual observation.

[0056] Update state covariance:

[0057]

[0058] Where P(k+1|k+1) is the state covariance matrix at time k+1 after updating based on all available information (including the latest observations). It reflects the uncertainty of the updated state estimate. P(k+1|k) is the state covariance matrix predicted at time k+1 based only on information at time k. It reflects the uncertainty of the predicted state estimate, P zz is the observation prediction covariance matrix, which reflects the uncertainty of the predicted observation value and is calculated by predicting the observation data state sampling points. T is the transpose of the Kalman gain matrix.

[0059] Through this series of steps, the AI ​​smart gateway can combine multi-source data, eliminate noise and errors, improve the accuracy of equipment status estimation, and provide more reliable information support in intelligent decision-making.

[0060] As an example, the step of calculating the observation prediction covariance and the cross covariance according to at least four of the diffusion factor, the initial data state vector, the dimension, the data state mean weight, the process noise covariance matrix, the data state covariance weight, the nonlinear observation function and the observation noise covariance matrix includes: calculating a first data state sampling point according to the diffusion factor and the initial data state vector; propagating the first data state sampling point to the next moment through a nonlinear state transfer function to obtain a second data state sampling point; calculating a second data state sampling point according to the dimension, the data state mean weight and the second data state The state prediction mean is calculated according to the state sampling points; the state prediction covariance is calculated according to the process noise covariance matrix, the dimension, the data state covariance weight and the second data state sampling points; the observation prediction mean is calculated by the nonlinear observation function, the data state mean weight and the state prediction mean; the observation prediction covariance is calculated according to the observation noise covariance matrix, the nonlinear observation function, the data state covariance weight, the observation prediction mean and the state prediction covariance; the cross covariance is calculated according to the data state covariance weight, the observation prediction mean and the state prediction mean.

[0061] The first data state sampling point refers to the initial state sample point generated according to the diffusion factor and the initial data state vector. This sampling point starts from the current initial estimated state and generates multiple candidate state estimates by adjusting the diffusion factor to represent the uncertainty or volatility of the data. The nonlinear state transfer function refers to a mathematical model used to describe the evolution of the system state over time. The function takes into account the influence of nonlinear factors. For example, sensor data may not satisfy a linear relationship in some cases, so nonlinear functions (such as nonlinear filtering, neural network models, etc.) are needed to accurately simulate state changes. The nonlinear state transfer function f is usually defined as:

[0062] x k =f(x k−1 ,u k )+w k

[0063] Among them, x k is the state vector at the current moment, x k−1 is the state vector at the previous moment, u k is the control input, w k is the process noise, usually assumed to be Gaussian white noise.

[0064] The next moment refers to the time point after the system state is updated. It is usually based on the observed data at the current moment and the time step to predict the state at the next moment. In AI smart gateway applications, the state at the next moment may be closely related to information such as sensor updates and device control instructions. The second data state sampling point refers to the new state sample point obtained after the first data state sampling point is propagated through the nonlinear state transfer function. This sampling point represents the state estimate at the next moment. Usually, considering noise and other uncertain factors, the second data state sampling point generated can better reflect the real state.

[0065] Data state sampling point calculation formula:

[0066] X (i) (k|k) =

[0067] Among them, X (i) Refers to the data state sampling point.

[0068] The state prediction mean is the predicted value of the system state at the next moment. By taking the weighted average of the second data state sampling points and combining the data state mean weight, the result is the central value of the predicted state. This prediction mean represents the most likely state of the system at the next moment. State prediction mean:

[0069]

[0070] is the state mean predicted at time k+1 based on the information at time k. 2dim(X) represents the total number of data state sampling points, where dim(X) is the dimension of the state vector. Because 2dim(X)+1 data state sampling points are used, the upper limit of the sum is 2dim(X). is the mean weight of the i-th data state sampling point, It is the state value predicted by the i-th data state sampling point at time k+1 based on the information at time k.

[0071] The process noise covariance matrix represents the size and correlation of the noise or error that may occur in the system during the state transition process. This matrix describes the uncertainty of the system state caused by measurement errors, external environment changes and other factors during the time advancement. The process noise covariance matrix Q=E[wwT].

[0072] The state prediction covariance is a measure of the uncertainty of the system state prediction. The state prediction covariance matrix is ​​calculated based on the process noise covariance matrix and the second data state sampling points, which represents the possible deviation of the predicted state. A larger covariance means a higher uncertainty in the state prediction.

[0073]

[0074] The nonlinear observation function h represents a function that maps the system state to the observation data, which is usually a nonlinear relationship. For example, the camera data may be affected by nonlinear factors such as illumination and viewing angle, and a nonlinear observation function is needed to establish the relationship between the system state and the actual observation data. The nonlinear observation function h can be expressed as:

[0075] z k =h(x k ,v k )

[0076] Among them, z k is the observed value at time k, x k is the state vector at time k, v k is the observation noise, usually assumed to be Gaussian white noise with covariance R.

[0077] The observation prediction mean is the prediction result of the observation value at the next moment calculated by a nonlinear observation function based on the prediction state mean. It represents the most likely value of the observation data based on the current prediction state.

[0078]

[0079] The observation noise covariance matrix R represents the size and correlation of the noise or error that may occur during the observation process. The observation noise covariance matrix describes the error characteristics of the data obtained from sensors, cameras, or other devices, such as the accuracy of a temperature sensor or the error of a camera under different conditions. Mathematically, R=E[vv T ], where v is the observation noise and E is the expected value. The different components of the observation noise are independent of each other, so R is usually a diagonal matrix with the elements on the diagonal representing the noise variance of each observation component. For example, if the observation vector z contains two observations, then R can be expressed as:

[0080]

[0081] in, and are the noise variances of the two observations respectively.

[0082] The observation prediction covariance is a measure of the uncertainty of the observation prediction. It describes the uncertainty of the observation prediction mean calculated by the nonlinear observation function. It is usually calculated jointly by the observation noise covariance matrix and the state prediction covariance matrix.

[0083]

[0084] Cross-covariance describes the degree of correlation between the state of the system and the observed data. It measures the correlation between the state prediction and the observation prediction, indicating how the errors between the two affect each other. The larger the cross-covariance, the closer the relationship between the state estimate and the observed data. Cross-covariance:

[0085]

[0086] In these steps, all predictions, estimates, and covariance calculations are designed to integrate information from multiple data sources (such as sensor data, video surveillance data, etc.), reduce noise, enhance accuracy, and optimize the state estimation of the AI ​​smart gateway, ensuring that the final output data accurately reflects the state of the device or system in the real environment. These steps reflect the complex process of optimizing multi-source data fusion through unscented Kalman filtering.

[0087] First, the first data state sampling point is calculated based on the diffusion factor and the initial data state vector. This sampling point is simulated by combining the initial data state vector and the weight of the diffusion factor to simulate different possible initial states, thereby providing diversity and uncertainty information for subsequent predictions. Then, the first data state sampling point is propagated to the next moment using the nonlinear state transfer function. This step is to update the state according to the dynamic model of the system, that is, to obtain the second data state sampling point by considering the impact of time changes on the system state. This sampling point reflects the transition of the system state at a given time step. Then, the state prediction mean is calculated based on the second data state sampling point, dimension, and data state mean weight. This is a weighted average method that combines the information of different state sampling points to obtain a most likely state prediction, ensuring that the estimation of the system state is more accurate. Then, the state prediction covariance is calculated using the process noise covariance matrix, dimension, data state covariance weight, and the second data state sampling point. This process evaluates the uncertainty or deviation of the predicted state and provides a basis for subsequent filtering adjustments. Then, the observation prediction mean is calculated through the nonlinear observation function, data state mean weight and state prediction mean. This step is to map the predicted system state to the actual observation space through the observation model to obtain the predicted observation value, so as to prepare for subsequent verification and adjustment. Next, the observation prediction covariance is calculated using the observation noise covariance matrix, nonlinear observation function, data state covariance weight, observation prediction mean and state prediction covariance. This step is to evaluate the error between the observed data and the predicted state, so as to further adjust the accuracy of the model. Finally, the cross covariance is calculated based on the data state covariance weight, observation prediction mean and state prediction mean. This process is used to measure the correlation between state estimation and observation prediction, so as to effectively reduce the error during data fusion, so as to make the final state estimation more accurate and optimize the performance of the entire system.

[0088] Step S40, extracting features and weighting the fused gateway data to obtain fused gateway data features;

[0089] It should be noted that feature extraction is to identify and extract information that is important to the system state or behavior from the fused gateway data. In the application of AI smart gateways, feature extraction usually refers to extracting representative feature values ​​from multiple data sources (such as sensor data, video surveillance data, control instruction data, etc.), such as temperature, humidity, motion state, network delay, etc. These features can effectively characterize the current state of the device or environment and provide key information for subsequent analysis and decision-making. Feature extraction methods can include statistical analysis, frequency domain analysis, time series analysis, etc., aiming to extract key information that can reflect important trends or anomalies from the original data. Weighted adjustment is to weight the extracted features to reflect the importance of different features in the comprehensive analysis. Since data from different sources may have different importance or reliability, weighted adjustment assigns different weights to different features to strengthen or weaken these features accordingly. For example, if the sensor data is more reliable, the feature may be given a higher weight; if the video surveillance data is greatly disturbed by environmental factors, it can be given a lower weight. Weighted adjustment can help ensure that the comprehensive effect of the final features more accurately and effectively reflects the real system state. Fusion gateway data features refer to the feature set obtained by fusing multiple data sources (such as sensor data, control instruction data, video surveillance data, etc.) after feature extraction and weighted adjustment. These features can not only reflect the key information of each data source, but also fully reflect the priority of each data source after weighted adjustment. Fusion gateway data features are the core input data ultimately used for further analysis, prediction or decision-making. They have high accuracy and representativeness, and can provide AI smart gateways with an accurate understanding of the environment and device status.

[0090] It can be understood that, first of all, different data types in the fusion gateway data (such as sensor data, video surveillance data, control instruction data, etc.) are processed one by one, and corresponding methods are used to extract features for each data type. For example, statistical features such as mean and variance can be calculated for sensor data, image features can be extracted by convolutional neural network for video surveillance data, and key operation frequencies and sequence patterns can be extracted for control instruction data, so that the main information of the data can be obtained from multiple dimensions. Then, according to the extracted features, weights are assigned to each feature, and these weights are set through model training or empirical rules, focusing on features related to key decisions, while reducing the impact on noise or unimportant features. Finally, the weighted features are combined into a unified feature vector to form high-quality features that can represent the overall characteristics of the fusion gateway data, which can provide more accurate and efficient input for subsequent data analysis and model optimization, and improve the performance and responsiveness of the system.

[0091] As an example, the step of extracting features and weighting the fused gateway data to obtain fused gateway data features includes: extracting text features, statistical features, image features, time series features, state behavior features and periodic features from the fused gateway data; performing linear transformation on the text features, the statistical features, the image features, the time series features, the state behavior features and the periodic features to map to obtain a query vector, a key vector and a value vector; performing dot product on each of the query vectors and the key vector, and normalizing them to obtain an attention weight; and concatenating and weighting-adjusting the text features, the statistical features, the image features, the time series features, the state behavior features and the periodic features according to the attention weight to obtain fused gateway data features.

[0092] Text features refer to key information extracted from text data (such as logs, reports, etc.) in the fusion gateway data, such as word frequency, word embeddings, TF-IDF value, etc. These features reflect the distribution and semantic characteristics of vocabulary in text data. Statistical features refer to features extracted from numerical data that describe data distribution and trends, such as mean, variance, maximum value, minimum value, standard deviation, skewness, kurtosis, etc. These features usually reflect the central trend and dispersion of data. Image features refer to features extracted from image data (such as video surveillance images) that describe image content, such as edge detection, texture analysis, color distribution, deep features extracted by convolutional neural network (CNN), etc. Time series features refer to features extracted from time series data, such as trend, seasonality, periodicity, lag relationship, volatility, etc. These features reflect the regularity and periodicity of data changes over time. State behavior features refer to features extracted from the state and behavior data of a device or system, usually including information such as device operation status (such as power on / off, faults, etc.), user behavior (such as clicks, operation sequence, etc.). Periodic features refer to the identification of periodic or recurring patterns from data, for example, some equipment runs or stops regularly within a fixed time period, or the readings of some sensors have periodic changes. Linear transformation refers to the mathematical transformation of the original feature data, usually by weighted summation or matrix multiplication, etc., to map the features to a new space or dimension for subsequent processing. Linear transformation helps to convert the original features into a format that is more suitable for model processing. For each type of feature (text features, statistical features, image features, time series features, state behavior features, periodic features), a linear transformation is performed. Usually, the linear transformation is in the following form:

[0093] X′=W*X+b

[0094] Among them, X is the original feature matrix, W is the transformation matrix (weight matrix), b is the bias term, and X′ is the new feature matrix after linear transformation (i.e., query vector, key vector, and value vector).

[0095] In the attention mechanism, the query vector (Query) represents the information to be retrieved, the key vector (Key) represents the content related to the query, and the value vector (Value) contains the actual information. By calculating the relationship between the query vector and the key vector, the importance or weight of the value vector can be determined. Dot product When calculating the relationship between the query vector and the key vector, the dot product is a common operation. By calculating the inner product of the two vectors (that is, multiplying and summing the corresponding elements), a scalar value is obtained to represent the correlation or matching degree of the two vectors. Normalization refers to adjusting the data to a uniform scale or range. It is usually to transform the data according to some standardization method (such as maximum normalization, Z-score normalization, etc.) to eliminate the influence of different feature dimensions or numerical ranges. In the attention mechanism, in order to make the range of weight values ​​stable and easy to calculate, the dot product results are usually normalized. The softmax function is used here, which can convert each dot product result into a probability distribution:

[0096]

[0097] Among them, z i is the input of the dot product result, is the exponential of the dot product result, ∑ j is the exponential sum of all dot product results.

[0098] The attention weight is a value obtained by dot product calculation, which indicates the degree of correlation between the query vector and each key vector. The higher the weight, the higher the match between the query vector and the key vector. Through normalization, the sum of all weights is ensured to be 1, which is convenient for subsequent weighted calculations. Splicing refers to connecting different types of features (such as text features, image features, etc.) in a certain way. Usually, multiple feature vectors are merged into a longer vector in element order. Splicing can help integrate multiple types of features into a comprehensive representation. Weighted adjustment refers to weighting different features according to the attention weight, emphasizing those important features and reducing the impact of irrelevant or noise features, so that the final fused features are more effective and accurate. According to the calculated attention weight, each feature is spliced ​​and weighted. For each type of feature:

[0099] F final = α i *F i

[0100] Among them, Fi is the result of linear transformation of each feature (text feature, statistical feature, etc.), α i is the attention weight obtained by softmax normalization, F final is the final weighted feature, which is the weighted sum of all features.

[0101] The fused gateway data feature refers to the comprehensive feature vector obtained through feature extraction, weighted adjustment and splicing. It integrates different types of data (such as text, images, time series, etc.) and performs weighted processing between multiple features to provide an efficient input for subsequent model analysis.

[0102] First, extract text features, statistical features, image features, time series features, state behavior features, and periodic features from the fusion gateway data. Text features extract key information from text data such as logs and reports through natural language processing techniques (such as word frequency analysis, TF-IDF, or pre-trained models); statistical features are calculated from numerical data, such as summarizing data distribution through statistics such as mean and variance; image features extract visual features from surveillance video or image data through convolutional neural networks (CNN); time series features analyze data trends, periodicity, and change patterns based on the time dimension; state behavior features extract status information of devices or systems, such as on / off status or fault data; periodic features extract regular changes by analyzing the repetitive patterns of data. Then, all extracted features are mapped to query vectors, key vectors, and value vectors through linear transformations. This process is usually implemented through matrix multiplication, using a pre-trained linear mapping model or a custom weight matrix to transform each feature to ensure that the features are comparable and consistent in the same space. This is done to prepare for the subsequent attention mechanism so that different features can be compared and weighted in the same dimension. Finally, attention weights are obtained based on the calculated dot product between the query vector and the key vector, and normalized so that the sum of all weights is 1. Then, various features are weighted according to these attention weights, that is, each feature is multiplied by the corresponding weight value, so that important features have a larger proportion in the final result, while unimportant features are weakened. Next, the weighted features are concatenated, and various features are merged into a comprehensive feature vector, which contains the essence of the fusion gateway data for subsequent processing and analysis.

[0103] Step S50, performing linear dimensionality reduction on the fused gateway data features to obtain a low-dimensional gateway data feature matrix;

[0104] It should be noted that linear dimensionality reduction refers to the use of linear algebra methods to project high-dimensional data into a low-dimensional space to reduce the complexity of the data while retaining as much important information as possible. In the AI ​​smart gateway scenario, commonly used linear dimensionality reduction methods include principal component analysis (PCA) and singular value decomposition (SVD). Specifically, this step maps the fusion gateway data features (such as multidimensional features such as text, statistics, and images) to a feature space with lower dimensions by constructing a linear transformation matrix. The construction of this transformation matrix is ​​based on the importance of the features, such as selecting the main components by calculating the variance or correlation of the features. The purpose of dimensionality reduction is to reduce data redundancy and computational complexity and improve analysis efficiency. The low-dimensional gateway data feature matrix refers to the feature data matrix generated after linear dimensionality reduction. Each row of the matrix represents a data sample and each column represents a feature after dimensionality reduction. The dimension of this matrix is ​​significantly smaller than the dimension of the original feature, while retaining the core information of the data as much as possible. For example, in an AI smart gateway, the low-dimensional feature matrix may contain simplified device operation modes, time series trends, or key status features, which facilitates subsequent analysis tasks such as anomaly detection, classification, or clustering. By reducing the dimensionality, the matrix can improve processing speed and reduce storage resource consumption while reducing the impact of noise.

[0105] It can be understood that, first, the numerical representation of the fusion gateway data features is obtained, a high-dimensional data matrix with features as columns and samples as rows is constructed, and the covariance matrix of this matrix is ​​calculated to quantify the correlation between the features and the data distribution. Next, the covariance matrix is ​​subjected to eigenvalue decomposition or singular value decomposition to obtain a set of eigenvalues ​​and corresponding eigenvectors, and then sorted according to the size of the eigenvalues, and the eigenvalues ​​with higher cumulative variance contribution rates and their corresponding eigenvectors are selected to form the transformation matrix required for dimensionality reduction. Then, the original high-dimensional data matrix is ​​multiplied by this transformation matrix, and the data is projected into a low-dimensional space to generate a low-dimensional gateway data feature matrix. Finally, through this dimensionality reduction method, the redundant information in the original data is removed, and the retained low-dimensional features contain both the main information and facilitate subsequent calculations and analysis, while reducing the computational complexity and noise impact, making the analysis process more efficient and accurate.

[0106] As an example, the step of performing linear dimensionality reduction on the fused gateway data features to obtain a low-dimensional gateway data feature matrix includes: obtaining a high-dimensional feature matrix and a label vector based on the fused gateway data features; calculating the mean of each feature in the fused gateway data features based on the label vector; calculating the intra-class divergence matrix and the inter-class divergence matrix of different categories of features based on the high-dimensional feature matrix and the mean; calculating the eigenvalues ​​and corresponding eigenvectors based on the intra-class divergence matrix and the inter-class divergence matrix; sorting the eigenvalues ​​to obtain a feature sequence; selecting a preset number of the eigenvectors from the feature sequence to form a transformation matrix; and projecting the high-dimensional feature matrix onto the transformation matrix to obtain a low-dimensional gateway data feature matrix.

[0107] The high-dimensional feature matrix is ​​a matrix extracted from the fusion gateway data features, in which each row represents a sample and each column represents a feature. The matrix has a high dimension and may contain different types of data features such as text, images, and time series. Its function is to provide a complete data representation for subsequent analysis. The label vector is a one-dimensional vector used to represent the category to which each sample belongs, and is often used in supervised learning scenarios. In the AI ​​smart gateway, each sample data may be labeled as normal or abnormal. These labels form a label vector to distinguish sample categories. The relationship between the high-dimensional feature matrix and the label vector:

[0108] X=[x1,x2,…,x n ],Y=[y1,y2,…,y n ]

[0109] X is a high-dimensional feature matrix, each column x i represents the feature vector of a sample, Y is the label vector, y i represents the category label of the i-th sample. The data is divided into C categories {c1, c2, …, c c}.

[0110] The mean of each feature refers to the average value of all sample feature values ​​by column in the high-dimensional feature matrix. It is used to describe the center position of each feature and provide a benchmark for the subsequent calculation of the divergence matrix. The formula for calculating the mean of each feature is:

[0111]

[0112] Among them, μ c is the mean vector of category c, and the feature mean of each sample in the category is calculated. c is the number of samples in category c, μ is the global mean vector, and all feature means of the entire data set are calculated. These means are used to subsequently calculate the intra-class scatter matrix and the inter-class scatter matrix.

[0113] The intra-class scatter matrix is ​​a matrix used to measure the density of feature distribution of samples of the same category. When calculating, it is obtained by summing the squares of the deviations between each sample and its category mean. The size of the matrix is ​​the same as the dimension of the feature. The smaller the intra-class scatter, the more concentrated the samples within the category. The calculation formula of the intra-class scatter matrix is:

[0114]

[0115] Among them, S W is the intra-class scatter matrix, which indicates the degree of dispersion of the sample distribution within the same category. (x i −μ c ) is the deviation of the sample from its class center mean, S W It is the sum of the intra-class divergences of all classes.

[0116] The inter-class scatter matrix is ​​a matrix used to measure the distribution differences between the means of different categories. It is calculated by the sum of squares of the deviations between the means of each category and the overall mean. The larger the inter-class scatter, the higher the distinction between different categories. The calculation formula of the inter-class scatter matrix is:

[0117]

[0118] S B is the inter-class scatter matrix, which indicates the degree of dispersion of the distribution between the means of each category, (μ c −μ) is the deviation N between the mean vector of class c and the global mean c is the number of samples of category c, which is used to adjust the category weight.

[0119] The eigenvalue is a scalar calculated from the intra-class scatter matrix and the inter-class scatter matrix, which is used to measure the contribution of the eigenvector. The larger the eigenvalue, the stronger the ability to distinguish sample data. The eigenvector is the direction vector of the eigenvalue, which represents the projection of the data in a certain linear direction. Each eigenvalue has one or more corresponding eigenvectors. The calculation formula of eigenvalue and eigenvector is:

[0120]

[0121] in, is the target matrix, w is the eigenvector, indicating the new dimensionality reduction direction, and λ is the eigenvalue, indicating the discriminative ability corresponding to each eigenvector.

[0122] A feature sequence is an ordered list formed by sorting the feature vectors from large to small according to the corresponding eigenvalues. The top-ranked feature vectors are the most important directions for the data. The preset number is the target dimension selected manually in linear dimensionality reduction, that is, the number of feature vectors selected from the feature sequence. When selecting, it is necessary to weigh the information retention and computational efficiency after dimensionality reduction. The transformation matrix consists of the selected eigenvectors, each column is an eigenvector, which is used to project high-dimensional data into low-dimensional space, retaining the main information and eliminating redundancy. Transformation matrix:

[0123] W=[w1,w2,…,w k ]

[0124] Where W is the transformation matrix, and the eigenvalues ​​correspond to the largest k eigenvectors w1, w2, ..., w k Composition, k is the target dimension after dimensionality reduction, usually the number of categories C-1.

[0125] Projection refers to combining the high-dimensional feature matrix with the transformation matrix through matrix multiplication to map the high-dimensional data into a low-dimensional space, thereby generating a low-dimensional gateway data feature matrix. Projection formula:

[0126] X′=W T X

[0127] Among them, X′ is a low-dimensional feature matrix, W T is the transpose of the transformation matrix, and X is the high-dimensional feature matrix.

[0128] First, the fusion gateway data features are arranged into vectors according to the various features of each sample (such as text features, statistical features, image features, etc.). These vectors form a matrix, which is a high-dimensional feature matrix. At the same time, the category of each sample is extracted from the classification annotation of the data to form a label vector. Then, using the label vector, the samples in the high-dimensional feature matrix are grouped by category, and the mean of each feature in each category and the global mean of all samples are calculated to measure the distribution of features within and between classes. Next, the intra-class scatter matrix (reflecting the compactness of the distribution of samples of the same category) and the inter-class scatter matrix (reflecting the distinction between different categories) are calculated using these means, and the eigenvalues ​​and corresponding eigenvectors of the two scatter matrices are calculated using linear algebra methods, and the feature sequence is generated by sorting the eigenvalues. Subsequently, the preset number of eigenvectors are selected from the feature sequence and combined into a transformation matrix by column. Finally, the sample vectors in the high-dimensional feature matrix are projected into the space of the transformation matrix by matrix multiplication to obtain the low-dimensional gateway data feature matrix after dimensionality reduction, thereby retaining the most important feature information and reducing computational complexity.

[0129] Step S60, detecting abnormal features from the low-dimensional gateway data feature matrix by fitting a normal data range, and determining corresponding abnormal gateway data according to the abnormal features.

[0130] It should be noted that fitting the normal data range means modeling the low-dimensional features of normal gateway data to determine the range or boundary of its distribution. Specifically, historical data or labeled normal gateway data are used to build statistical or machine learning models based on low-dimensional feature matrices, such as Gaussian distribution fitting, support vector machine (SVM) or density estimation methods (such as kernel density estimation). These methods can determine the distribution center, distribution shape and boundary of normal data in the feature space (for example, the normal range is defined by a threshold or confidence interval), so that any feature that deviates from this range can be regarded as an abnormal feature. The significance of this step is to provide a reference standard for anomaly detection to ensure that normal and abnormal data can be accurately distinguished. Abnormal features refer to those feature values ​​or feature combinations in the low-dimensional feature matrix that have significant deviations from the fitted normal data range. They may be manifested in numerical anomalies in a specific dimension (for example, a sudden surge in time series features) or the overall feature vector is far from the normal range (for example, the Euclidean distance is greater than a certain threshold). During specific detection, the distance between each sample feature vector and the center of the normal distribution or the relative position to the boundary is calculated to determine whether it is an abnormal feature. Abnormal features often reveal the nature of potential problems, for example, anomalies in sensor data may be related to device failure, while anomalies in network communication features may be related to network attacks or configuration errors. Abnormal gateway data refers to specific original gateway data entries or sampling points that are located and traced back through abnormal features. These data may correspond to sensor readings (such as sudden temperature changes), video surveillance data (such as picture loss), control command data (such as invalid command repetitions), network communication data (such as data packet anomalies), or device log data (such as frequent error logs). The determination of abnormal gateway data is completed by mapping the abnormal features back to the original data set. For example, when the time series features are abnormal, the corresponding sensor data record can be found by the timestamp. The analysis of abnormal gateway data can help locate the source of the problem, such as hardware failure, data pollution, or network security threats.

[0131] It can be understood that, first, by analyzing the low-dimensional gateway data feature matrix, the mean and covariance matrix of the data are calculated to determine the center and shape of the data, thereby constructing an elliptical decision boundary to represent the distribution range of normal data. Then, based on this fitting result, it is determined whether the data point exceeds the range of the ellipse. Specifically, for each data point, the algorithm calculates the distance between the point and the fitted normal data range (i.e., the ellipse). If the point falls outside the ellipse, it is considered to be an abnormal feature. Finally, these abnormal features are mapped back to the original gateway data to find the specific gateway data points corresponding to the abnormal features. These data points usually indicate some abnormal behaviors or data errors, such as equipment failures, sensor misreadings, etc., thereby identifying potential problems in the system. In this way, abnormal points can be efficiently screened out from complex data and their sources can be traced for subsequent processing and optimization.

[0132] This embodiment provides a data analysis method based on an intelligent gateway, which obtains sensor data, video surveillance data, control instruction data, network communication data, and device log data from various intelligent devices; performs time alignment and format conversion on the sensor data, the video surveillance data, the control instruction data, the network communication data, and the device log data to obtain standardized gateway data; performs sampling diffusion and gain update on the standardized gateway data to obtain fused gateway data; performs feature extraction and weighted adjustment on the fused gateway data to obtain fused gateway data features; performs linear dimension reduction on the fused gateway data features to obtain a low-dimensional gateway data feature matrix; detects abnormal features from the low-dimensional gateway data feature matrix by fitting the normal data range, and determines the corresponding abnormal gateway data according to the abnormal features. The system first collects multiple types of data including sensors, video surveillance, control instructions, network communications, and device logs from intelligent devices. After time alignment and format conversion, these data are unified into a standardized format for easy integration and analysis. Subsequently, the data quality and representativeness are improved through sampling diffusion and gain update, and feature extraction and weighted adjustment are performed to ensure the importance of key features in the analysis. Next, linear dimensionality reduction technology is used to simplify the data structure while retaining core information and reducing computational complexity. Finally, the system compares the normal data range to efficiently identify abnormal data points, and promptly discovers and handles potential problems in equipment operation, thereby ensuring the stability and reliability of the system. Through the above steps, the AI ​​smart gateway system can effectively integrate, process and analyze large amounts of data from different devices, identify abnormal behavior of devices, and ensure reliable operation of devices and efficient use of data.

[0133] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 , Figure 2 This is a flow chart of the second embodiment of the data analysis method based on the intelligent gateway of the present application. Step S60 of the data analysis method based on the intelligent gateway includes steps S61 to S65:

[0134] Step S61, setting a confidence threshold according to the low-dimensional gateway data feature matrix;

[0135] It should be noted that the confidence threshold refers to the boundary value used to determine whether a data point belongs to the normal range in the analysis based on the low-dimensional gateway data feature matrix. Assuming that normal data is distributed within an ellipse defined by a Gaussian distribution, an optimal ellipse model is constructed by fitting normal data points, and the distance or score of each data point to the ellipse boundary is calculated based on the ellipse equation. The confidence threshold is determined according to the set confidence level (for example, 95% or 99%), indicating the proportion of normal data covered in the ellipse. If the distance or score of a data point exceeds the confidence threshold, it is considered an abnormal point; if it is within the threshold, it is considered a normal point. This can effectively distinguish normal data from abnormal data.

[0136] It can be understood that, first, according to the distribution of data points in the low-dimensional gateway data feature matrix, a Gaussian distribution model is fitted to establish a mathematical description of the normal data range, such as defining the distribution relationship between data points by calculating the center position and variance of each data point. Then, the confidence threshold is determined according to the set confidence level (such as 95% or 99%), and the maximum proportion of data points covered within the normal data distribution range is calculated, while ensuring that the threshold can tolerate a certain degree of normal fluctuations. Finally, the confidence threshold is used as a judgment criterion for classifying new data points. If a data point exceeds the threshold range, it is regarded as an abnormal data point. This can help to efficiently distinguish normal data from abnormal data and provide a basis for subsequent anomaly detection of smart devices.

[0137] Step S62, calculating the mean and covariance of the low-dimensional gateway data feature matrix to obtain a feature mean vector and a feature covariance matrix;

[0138] It should be noted that the mean refers to the average value of all data points in each column (feature) in the low-dimensional gateway data feature matrix, which is used to reflect the overall trend of the feature. For example, if a column represents temperature data, its mean is the average temperature of all records. Covariance is used to measure the strength and direction of the correlation between two features. A positive value of covariance indicates that the two features increase or decrease together, a negative value indicates that one of the two features increases while the other decreases, and a zero value indicates that the two are unrelated. Covariance not only considers the correlation between the two features, but also their amplitude of change. The feature mean vector refers to a vector in which the means of each column of the low-dimensional gateway data feature matrix are arranged in order. For example, if the matrix has three columns, corresponding to temperature, humidity, and pressure features, and the means are 20, 50, and 100, respectively, then the feature mean vector is [20, 50, 100]. Feature mean vector formula:

[0139]

[0140] Among them, X i is the i-th sample in the low-dimensional gateway data feature matrix, N is the number of samples, and μ represents the average feature vector of all samples, that is, the mean of each feature dimension.

[0141] The feature covariance matrix is ​​a symmetric matrix that is used to comprehensively represent the covariance between all features. The diagonal elements of the matrix are the variance of each feature, and the off-diagonal elements are the covariance between features. Covariance matrix formula:

[0142]

[0143] Among them, Σ is the covariance matrix, which reflects the correlation and variance between features. Each element Σ ij It represents the covariance between the i-th feature and the j-th feature.

[0144] It can be understood that, first, all data values ​​of each column (i.e., each feature) in the low-dimensional gateway data feature matrix are added one by one, and then divided by the total number of data points to calculate the mean of each feature, and these means are arranged into feature mean vectors in feature order; then, for any two columns of features in the matrix, their covariance is calculated: for each pair of data values, the average value of the product of their deviations from their respective means is calculated to obtain the covariance value, and this is used as an element to fill the corresponding position of the covariance matrix; the elements on the diagonal are the variances of each column of features (i.e., the special case where two features are the same in the covariance), and the other positions of the matrix represent the covariance between different features; finally, the feature mean vector and feature covariance matrix are output as the results, which respectively represent the central trend of the overall data distribution and the correlation structure between features, providing a statistical basis for subsequent anomaly detection and distribution fitting. This operation ensures the accurate representation of data distribution and helps to improve the accuracy and reliability of analysis.

[0145] Step S63, calculating the anomaly detection distance from each feature in the low-dimensional gateway data feature matrix to the feature mean vector according to the covariance matrix;

[0146] It should be noted that each feature refers to each column of data in the low-dimensional gateway data feature matrix. Each column represents a feature (such as a state variable in a time series or a measurement value output by a device sensor). These features are obtained from the original high-dimensional data after feature extraction and dimensionality reduction. They contain key information about the operation of the device. The anomaly detection distance refers to a measure used to measure the degree of deviation between a feature data point and the feature mean vector. Specifically, the anomaly detection distance reflects the relative distance between the position of a feature in the feature space and the central distribution (ie, the mean). This distance can be used to measure the correlation of multidimensional data based on the covariance matrix. The larger the distance, the more likely the data point is an anomaly. The calculation formula for the anomaly detection distance is:

[0147]

[0148] Among them, D(X i ) is each sample X i Anomaly detection distance to feature mean vector μ, Σ -1 It is the inverse of the covariance matrix, reflecting the correlation between features and their "scale".

[0149] It can be understood that, first, the data point of each feature is taken out from the low-dimensional gateway data feature matrix, and the corresponding feature mean vector is subtracted to calculate the difference between each feature data point and its mean, which is used to indicate the degree to which the feature deviates from the center. Then, these differences are subjected to matrix operations with the inverse matrix of the covariance matrix. Specifically, the differences are first multiplied by the inverse of the covariance matrix on the left and then multiplied by its transpose on the right to obtain the weighted sum of squares of each feature difference. This step comprehensively considers the correlation between features. Finally, the square root operation is performed on the weighted sum of squares of each feature to obtain its anomaly detection distance to the feature mean vector. The larger the distance values, the more obvious the degree to which the corresponding feature deviates from the mean, which is used to further identify abnormal feature points. This method helps to efficiently identify abnormal features by accurately calculating the relationship and distance between features.

[0150] Step S64, marking the features in the low-dimensional gateway data feature matrix whose abnormal detection distance is greater than the confidence threshold as abnormal features;

[0151] It should be noted that abnormal features refer to those feature data points in the low-dimensional gateway data feature matrix whose abnormal detection distance exceeds the confidence threshold. These features reflect the data points in the operation data of smart devices that are significantly deviated from the normal data pattern, which may be caused by device failure, network anomaly or other abnormal behavior. Specifically, abnormal features represent the characteristics of certain specific sensors, control instructions, network communications, etc. that cannot be covered by the normal range in data performance. These features can help locate problem data, thereby providing a basis for subsequent analysis or troubleshooting.

[0152] It can be understood that, first, the anomaly detection distance of each feature is taken out from the low-dimensional gateway data feature matrix and compared with the previously calculated confidence threshold one by one; if the anomaly detection distance of a feature is greater than the confidence threshold, the feature is marked as an abnormal feature. Then, these marked abnormal features are sorted out, and their specific locations and corresponding feature values ​​are recorded to distinguish normal data from abnormal data. Finally, by screening out abnormal features, it is possible to identify which data may represent abnormal conditions in device operation, providing a basis for further analysis and troubleshooting, thereby achieving more efficient and accurate intelligent device monitoring and management.

[0153] Step S65: determining corresponding abnormal gateway data according to the abnormal characteristics.

[0154] It should be noted that abnormal gateway data refers to the raw data associated with features that are marked as abnormal. When certain features are identified as abnormal, the raw gateway data corresponding to these features (such as sensor data, log records, etc.) are considered abnormal gateway data. Specifically, abnormal gateway data may be real-time sensor readings from devices, device operation records, control instructions, or other types of data that indicate that the behavior of a device or system at a certain moment deviates from the normal operating mode, and therefore requires further investigation or action.

[0155] It can be understood that, first, the system will match the abnormal features in the low-dimensional gateway data feature matrix with the features in the original data. Each low-dimensional feature represents a certain form of information in the original data (such as the measurement value of the sensor or the device status). After the dimensionality reduction process, these low-dimensional features can still be associated with each feature in the original data through a specific mapping relationship (usually through feature selection or encoding methods). Secondly, using the marking of abnormal features (such as features where the abnormal detection distance exceeds the threshold) as key clues, the system will trace back to the original data and extract the original data items corresponding to these features. For example, if the measurement value of a sensor is abnormal, the system will find the specific record of the sensor in the original data through the known mapping relationship and mark it as abnormal data. Finally, the original data items corresponding to the abnormal features will be selected and stored, thereby completing the mapping from the low-dimensional feature space to the high-dimensional original data space, ensuring the accurate identification of abnormal gateway data.

[0156] This embodiment sets a confidence threshold according to the low-dimensional gateway data feature matrix; calculates the mean and covariance of the low-dimensional gateway data feature matrix to obtain a feature mean vector and a feature covariance matrix; calculates the abnormal detection distance from each feature in the low-dimensional gateway data feature matrix to the feature mean vector according to the covariance matrix; marks the feature in the low-dimensional gateway data feature matrix whose abnormal detection distance is greater than the confidence threshold as an abnormal feature; and determines the corresponding abnormal gateway data according to the abnormal feature. First, the system sets a confidence threshold according to the low-dimensional gateway data feature matrix to distinguish normal features from abnormal features. The choice of this threshold affects the sensitivity of the system in identifying abnormalities, thus playing a key role in abnormality detection. Next, the system calculates the mean and covariance of the low-dimensional gateway data feature matrix to obtain a feature mean vector and a feature covariance matrix. The feature mean vector represents the average value of each feature in the data, while the feature covariance matrix measures the correlation between the features. Through these two statistics, the system can understand the overall trend of feature distribution and provide a basis for subsequent abnormality detection. Then, based on the covariance matrix, the system calculates the anomaly detection distance from each feature to the mean vector, which measures the degree of deviation of each feature in the data space. A large anomaly detection distance means that the feature deviates from the normal range and may be abnormal. Next, the system marks those features in the low-dimensional gateway data feature matrix whose anomaly detection distance is greater than the confidence threshold as abnormal features, so that potential abnormal data points can be automatically screened out. Finally, based on the features marked as abnormal, the system can trace back and identify abnormal gateway data related to these features, thereby providing a basis for problem diagnosis and processing. Through these steps, the system can effectively identify and mark abnormal gateway data, find abnormal points from massive gateway data in real time, improve data quality and monitoring efficiency, and provide valuable support for subsequent intelligent device maintenance and fault warning.

[0157] For example, in order to help understand the implementation process of the data analysis method based on the intelligent gateway obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 3 , Figure 3 A brief flow chart of a data analysis method based on an intelligent gateway is provided, specifically:

[0158] The figure shows the data analysis process based on the intelligent gateway. First, sensor data, video surveillance data, control instruction data, etc. are obtained from data sources such as sensors, smart devices 1 and smart devices 2. These data are transmitted to the AI ​​intelligent gateway, and then time alignment and format conversion are performed to ensure the consistency of the data in the time dimension and the uniformity of the format to generate standardized gateway data. Next, sampling diffusion and gain update are performed on the standardized gateway data to obtain fused gateway data. Then, text features, statistical features, image features, time series features, state behavior features, and periodic features are extracted from the fused gateway data, and weighted adjustment is performed through linear transformation and attention mechanism to form fused gateway data features. After that, linear dimension reduction is performed on the fused gateway data features, and the eigenvalues ​​and eigenvectors are calculated using the intra-class scatter matrix and the inter-class scatter matrix. The main components are selected for dimensionality reduction to obtain a low-dimensional gateway data feature matrix. Finally, by fitting the normal data range (ellipse fitting), abnormal features are detected from the low-dimensional gateway data feature matrix, and the corresponding abnormal gateway data is determined to achieve real-time and efficient recognition of abnormal behaviors of smart devices. The entire process embodies the core steps of the data analysis method disclosed in this application, and can identify abnormal data from a large amount of gateway data in real time and efficiently.

[0159] The present application also provides a data analysis device based on an intelligent gateway, the data analysis device based on the intelligent gateway comprising:

[0160] The data acquisition module 10 is used to acquire sensor data, video surveillance data, control instruction data, network communication data and device log data from various smart devices;

[0161] A data conversion module 20, configured to perform time alignment and format conversion on the sensor data, the video surveillance data, the control instruction data, the network communication data, and the device log data to obtain standardized gateway data;

[0162] A data fusion module 30, configured to perform sampling diffusion and gain update on the standardized gateway data to obtain fused gateway data;

[0163] A feature fusion module 40 is used to extract features and perform weighted adjustment on the fused gateway data to obtain fused gateway data features;

[0164] A feature dimension reduction module 50 is used to perform linear dimension reduction on the fusion gateway data features to obtain a low-dimensional gateway data feature matrix;

[0165] The anomaly detection module 60 is used to detect abnormal features from the low-dimensional gateway data feature matrix by fitting the normal data range, and determine the corresponding abnormal gateway data according to the abnormal features.

[0166] The present application provides a data analysis device based on an intelligent gateway, and the data analysis device based on an intelligent gateway includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the data analysis method based on the intelligent gateway in the above-mentioned embodiment 1. The data analysis device based on the intelligent gateway provided by the present application adopts the data analysis method based on the intelligent gateway in the above-mentioned embodiment, and can solve the technical problem of how to identify abnormal data from a large amount of gateway data in real time and efficiently.

[0167] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the data analysis method based on the intelligent gateway in the above embodiment. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the data analysis device based on the intelligent gateway, the data analysis device based on the intelligent gateway: obtains sensor data, video surveillance data, control instruction data, network communication data, and device log data from various types of intelligent devices; performs time alignment and format conversion on the sensor data, the video surveillance data, the control instruction data, the network communication data, and the device log data to obtain standardized gateway data; performs sampling diffusion and gain update on the standardized gateway data to obtain fused gateway data; performs feature extraction and weighted adjustment on the fused gateway data to obtain fused gateway data features; performs linear dimensionality reduction on the fused gateway data features to obtain a low-dimensional gateway data feature matrix; detects abnormal features from the low-dimensional gateway data feature matrix by fitting the normal data range, and determines the corresponding abnormal gateway data according to the abnormal features. The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions for executing the above-mentioned data analysis method based on the intelligent gateway, and can solve the technical problem of how to identify abnormal data from a large amount of gateway data in real time and efficiently.

[0168] The present application also provides a computer program product, including a computer program, which implements the steps of the data analysis method based on the intelligent gateway as described above when the computer program is executed by a processor. The computer program product provided by the present application can solve the technical problem of how to identify abnormal data from a large amount of gateway data in real time and efficiently. The beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the data analysis method based on the intelligent gateway of the above embodiment.

Claims

1. A data analysis method based on an intelligent gateway, characterized in that: The method comprises: Obtain sensor data, video surveillance data, control command data, network communication data, and device log data from various smart devices; Performing time alignment and format conversion on the sensor data, the video surveillance data, the control instruction data, the network communication data, and the device log data to obtain standardized gateway data; Performing sampling diffusion and gain updating on the standardized gateway data to obtain fused gateway data; Extracting features and performing weighted adjustment on the fusion gateway data to obtain fusion gateway data features; Performing linear dimensionality reduction on the fusion gateway data features to obtain a low-dimensional gateway data feature matrix; Setting a confidence threshold according to the low-dimensional gateway data feature matrix; Calculating the mean and covariance of the low-dimensional gateway data feature matrix to obtain a feature mean vector and a feature covariance matrix; Eigenmean vector formula: Among them, X i is the i-th sample in the low-dimensional gateway data feature matrix, N is the number of samples, and μ represents the average feature vector of all samples, that is, the mean of each feature dimension; Feature covariance matrix formula: Among them, Σ is the feature covariance matrix, which reflects the correlation and variance between features. Each element Σ ij Represents the covariance between the i-th feature and the j-th feature; Calculate the anomaly detection distance from each feature in the low-dimensional gateway data feature matrix to the feature mean vector according to the covariance matrix; The calculation formula of anomaly detection distance is: Among them, D(X i ) is each sample X i Anomaly detection distance to feature mean vector μ, Σ -1 is the inverse of the covariance matrix; Marking the features in the low-dimensional gateway data feature matrix whose anomaly detection distance is greater than the confidence threshold as abnormal features; Corresponding abnormal gateway data is determined according to the abnormal characteristics.

2. The method according to claim 1, characterized in that The step of extracting features and weighting the fusion gateway data to obtain fusion gateway data features includes: Extracting text features, statistical features, image features, time series features, state behavior features, and periodic features from the fusion gateway data; Performing linear transformation on the text features, the statistical features, the image features, the time series features, the state behavior features, and the periodic features to map and obtain a query vector, a key vector, and a value vector; Performing a dot product on each of the query vectors and the key vector, and normalizing the result to obtain an attention weight; The text features, the statistical features, the image features, the time series features, the state behavior features and the periodic features are spliced ​​and weighted according to the attention weights to obtain fused gateway data features.

3. The method according to claim 1, characterized in that The step of performing linear dimensionality reduction on the fusion gateway data features to obtain a low-dimensional gateway data feature matrix comprises: Obtaining a high-dimensional feature matrix and a label vector according to the fusion gateway data features; Calculating the mean of each feature in the fusion gateway data feature according to the label vector; Calculating intra-class scatter matrices and inter-class scatter matrices of features of different categories according to the high-dimensional feature matrix and the mean; Calculating eigenvalues ​​and corresponding eigenvectors according to the intra-class scatter matrix and the inter-class scatter matrix; Sorting the eigenvectors according to the eigenvalues ​​to obtain a eigensequence; Selecting a preset number of the feature vectors from the feature sequence to form a transformation matrix; The high-dimensional feature matrix is ​​projected onto the transformation matrix to obtain a low-dimensional gateway data feature matrix.

4. The method according to claim 1, characterized in that The step of performing sampling diffusion and gain updating on the standardized gateway data to obtain fused gateway data comprises: Setting an initial data state vector and an initial data state covariance matrix according to the standardized gateway data; Calculating a diffusion parameter according to a preset hyperparameter, a preset weight parameter, a preset calibration parameter, and the dimension of the initial data state vector; The data state mean weight and the data state covariance weight are calculated according to the diffusion parameter, the dimension, the preset hyperparameter and the preset weight parameter; A diffusion factor is calculated according to the dimension, the diffusion parameter, and the initial data state covariance matrix; Calculate the observation prediction covariance and the cross covariance according to at least four of the diffusion factor, the initial data state vector, the dimension, the data state mean weight, the process noise covariance matrix, the data state covariance weight, the nonlinear observation function and the observation noise covariance matrix; Obtaining a Kalman gain according to the observation prediction covariance and an inverse calculation of the observation prediction covariance; The state estimation and state covariance of the standardized gateway data are updated according to the cross covariance, the Kalman gain, and the observation prediction covariance to obtain fused gateway data.

5. The method according to claim 4, characterized in that The step of calculating the observation prediction covariance and the cross covariance according to at least four of the diffusion factor, the initial data state vector, the dimension, the data state mean weight, the process noise covariance matrix, the data state covariance weight, the nonlinear observation function and the observation noise covariance matrix comprises: Calculate a first data state sampling point according to the diffusion factor and the initial data state vector; Propagate the first data state sampling point to the next moment through a nonlinear state transfer function to obtain a second data state sampling point; Calculate the state prediction mean according to the dimension, the data state mean weight and the second data state sampling point; Calculate the state prediction covariance according to the process noise covariance matrix, the dimension, the data state covariance weight and the second data state sampling point; The observation prediction mean is calculated by using the nonlinear observation function, the data state mean weight and the state prediction mean; The observation prediction covariance is calculated according to the observation noise covariance matrix, the nonlinear observation function, the data state covariance weight, the observation prediction mean and the state prediction covariance; The cross covariance is calculated based on the data state covariance weight, the observation prediction mean and the state prediction mean.

6. The method according to any one of claims 1 to 5, characterized in that The step of performing time alignment and format conversion on the sensor data, the video surveillance data, the control instruction data, the network communication data, and the device log data to obtain standardized gateway data comprises: Cleaning, denoising and standardizing the sensor data, the video surveillance data, the control instruction data, the network communication data and the device log data to obtain initial data; Extracting timestamp information from the initial data; Constructing time series data according to the timestamp information and the initial data; Calculating the distance matrix between each of the time series data; Starting from the lower right corner of the distance matrix, backtrack to find the minimum distance path; Performing time alignment on the time series data according to the minimum distance path to obtain reference data; Identify the data format and data structure of the reference data through a JSON format matching expression, an XML format matching expression, a CSV format matching expression, a log format matching expression, and a network communication data format matching expression; The reference data is converted into a preset data format according to the data format and the data structure to obtain standardized gateway data.

7. A data analysis device based on an intelligent gateway, characterized in that: The device comprises: Data acquisition module, used to obtain sensor data, video surveillance data, control instruction data, network communication data and device log data from various smart devices; A data conversion module, used for performing time alignment and format conversion on the sensor data, the video monitoring data, the control instruction data, the network communication data and the device log data to obtain standardized gateway data; A data fusion module, used for performing sampling diffusion and gain update on the standardized gateway data to obtain fused gateway data; A feature fusion module, used for extracting features and weighting the fused gateway data to obtain fused gateway data features; A feature dimension reduction module, used for performing linear dimension reduction on the fusion gateway data features to obtain a low-dimensional gateway data feature matrix; Anomaly detection module, used to set the confidence threshold according to the low-dimensional gateway data feature matrix; calculate the mean and covariance of the low-dimensional gateway data feature matrix to obtain the feature mean vector and feature covariance matrix; feature mean vector formula: Among them, X i is the i-th sample in the low-dimensional gateway data feature matrix, N is the number of samples, μ represents the average feature vector of all samples, that is, the mean of each feature dimension; the feature covariance matrix formula is: Among them, Σ is the feature covariance matrix, which reflects the correlation and variance between features. Each element Σ ij Represents the covariance between the i-th feature and the j-th feature; calculates the anomaly detection distance from each feature in the low-dimensional gateway data feature matrix to the feature mean vector according to the covariance matrix; the calculation formula of the anomaly detection distance is: Among them, D(X i ) is each sample X i Anomaly detection distance to feature mean vector μ, Σ -1 is the inverse matrix of the covariance matrix; the features in the low-dimensional gateway data feature matrix whose abnormal detection distance is greater than the confidence threshold are marked as abnormal features; and corresponding abnormal gateway data are determined according to the abnormal features.

8. A data analysis device based on an intelligent gateway, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the intelligent gateway-based data analysis method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data analysis method based on the intelligent gateway as described in any one of claims 1 to 6 are implemented.

Images