Deep learning-based data channel anomaly detection method and system

Abstract

The invention is suitable for the technical field of data-in-channel management, and provides a data-in-channel anomaly detection method and system based on deep learning, and the method comprises the steps: setting a data collection frequency and range, collecting equipment data generated by equipment operation, an external environment and Internet of Things equipment, and carrying out the preprocessing; establishing an equipment behavior model, comparing with historical data, and identifying abnormal activities to generate an equipment abnormity table; monitoring equipment data generation frequency and quality in real time, dynamically evaluating value and risk level, setting an evaluation content change interval and triggering standard revision; and regularly evaluating actual use and potential value of equipment data, and correcting in combination with an external environment. According to the method, the comprehensiveness and accuracy of anomaly detection are improved through multi-dimensional data analysis and a deep learning model, and support is provided for enterprise data management.

Description

Technical Field

[0001] The present invention belongs to the technical field of data middle - platform management, and particularly relates to a data middle - platform anomaly detection method and system based on deep learning. Background Art

[0002] With the rapid development of Internet of Things (IoT) technology, a large number of IoT devices (such as industrial sensors, smart terminals, edge gateways) have become the core infrastructure for enterprises to achieve intelligent operation. As the core infrastructure for enterprise - level data management, the data middle - platform aims to break data silos and drive business innovation through unified data access, governance, and analysis capabilities. In the IoT scenario, the role of the data middle - platform is further expanded to a key hub connecting a large number of devices and upper - layer applications, which needs to integrate multi - source heterogeneous IoT device data (such as sensor readings, device status logs, communication traffic) in real - time and support dynamic risk assessment and value mining.

[0003] However, the large - scale deployment of IoT devices and adaptation to complex environments face multiple challenges: on the one hand, IoT devices are often deployed in complex physical environments (such as high - temperature workshops, field base stations), and their data quality and device status are significantly affected by external factors (temperature, humidity, electromagnetic interference). Existing technologies mostly adopt static risk assessment models (such as fixed - threshold alarms), and do not realize real - time correlation analysis of environmental parameters and device data through the data middle - platform. For example, in agricultural IoT, the data mutation of soil humidity sensors may be caused by device failures or heavy rains, but traditional systems, due to the lack of environmental data fusion capabilities, cannot distinguish the root causes of anomalies, and the false alarm rate is over 40%.

[0004] On the other hand, existing methods mostly focus on basic metrics (such as device online rate, data generation volume), ignoring multi - dimensional correlation analysis of data quality, risk fluctuations, environmental impacts, and business value. Moreover, the evaluation of the data value generated by IoT devices is insufficient, and it is difficult to quantify the actual benefits that the data can bring. Summary of the Invention

[0005] The purpose of the present invention is to provide a data middle - platform anomaly detection method based on deep learning, aiming to solve the technical problems existing in the prior art identified in the background art.

[0006] The present invention is implemented as follows. The data middle - platform anomaly detection method based on deep learning includes:

[0007] Set the frequency and range of data collection, and uniformly collect aggregated data through the data middle - platform. The aggregated data includes IoT device operation data, external environmental data, and device data generated by IoT devices, and pre - process all the collected aggregated data;

[0008] Analyze the operation data collected from IoT devices, establish a device behavior model, compare the real-time operation data of IoT devices with the historical operation data of IoT devices, identify potential abnormal activities during device operation, and generate a device anomaly table;

[0009] Define risk assessment criteria, and based on the device anomaly table and external environment data, conduct dynamic assessment of device data, detect the generation frequency of device data and the quality status of device data, and calculate the expected value and risk fluctuation level of device data in real time;

[0010] For each evaluation content defined in the risk assessment criteria, set a change interval. If the abnormal activities of device operation and external environment data cause any evaluation content to exceed the change interval, trigger the revision of the risk assessment criteria, and synchronously revise the risk assessment criteria based on the device anomaly table and external environment data;

[0011] Based on the risk assessment criteria and the device anomaly table, regularly evaluate the actual usage, potential value and risks of various device data, and identify the maximum value brought by different device data in combination with external environment changes.

[0012] As a further solution of the present invention, the comparison of the real-time operation data of IoT devices and the historical operation data of IoT devices, identifying potential abnormal activities during device operation, and generating a device anomaly table specifically includes:

[0013] Extract features based on the operation data of IoT devices, generate a feature vector that can reflect device operation, identify the normal operation mode of the device, establish a device behavior model, and set normal behavior criteria;

[0014] Set up a real-time data stream processing mechanism, extract the real-time operation data of IoT devices according to a preset frequency, compare the real-time operation data of IoT devices with the historical operation data of IoT devices, identify behaviors that deviate from the normal operation mode, and mark them as potential abnormal activities;

[0015] Establish a device anomaly table, store the potential abnormal activities in the device anomaly table, and at the same time store the real-time operation data of IoT devices that match the potential abnormal activities as a label.

[0016] As a further solution of the present invention, the identification of behaviors that deviate from the normal operation mode is specifically:

[0017] ;

[0018] wherein, is the reconstruction error, which is used to measure the input device state features and the features corresponding to the normal behavior mode the difference between, is the number of device status features.

[0019] As a further solution of the present invention, the device data is dynamically evaluated, the generation frequency of the device data and the quality status of the device data are detected, and the expected value and the risk fluctuation level of the device data are calculated in real time, specifically including:

[0020] Analyze the operation data of the Internet of Things device based on a time window, calculate the generation frequency of the device data, and identify the activity level of the device;

[0021] Real-time monitor the risk fluctuation of the device data, combine with the external environment data, identify the influence of external factors on the generation frequency of the device data and the quality status of the device data, and calculate the risk fluctuation level of the device data;

[0022] Based on the operation data of the Internet of Things device and the external environment data, construct a device data value model, and combine with the device data risk fluctuation level to quantify the expected value of each device data.

[0023] As a further solution of the present invention, the calculation of the risk fluctuation level of the device data is specifically as follows:

[0024] ;

[0025] In the formula, represents the risk fluctuation level, which is used to quantify the risk volatility of the device data and reflects the stability of the device data generation frequency within the specified time window. is the number of samples, representing the number of data points within the specified time window. is the th data point, that is, the device data generation frequency every day within the time window. is the sample mean, that is, the average generation frequency of the device data within the same time window;

[0026] The quantification of the expected value of each device data is specifically as follows:

[0027] ;

[0028] Among them, represents the expected value of the device data. represents the number of device data. is the th business value of the device data, that is, the expected data value under normal circumstances. is the th risk measure of the device data. is the number of external environment data affecting the expected value. is the An external environment data quantization value, is a non-linear function, representing the degree of influence of the external environment on the expected value, is a constant term.

[0029] As a further solution of the present invention, the risk assessment criteria specifically include:

[0030] A generation frequency assessment criterion, that is, the generation frequency of device data, defining the normal generation frequency range of device data;

[0031] A data quality assessment criterion, that is, the quality status of device data, including an accuracy ratio threshold, a completeness ratio threshold, and a consistency ratio threshold;

[0032] An external environment impact assessment criterion, that is, the impact of external environment data on the generation of device data by the Internet of Things device, defining the change ratio range of the generation frequency and data quality of device data caused by changes in the external environment.

[0033] As a further solution of the present invention, regularly evaluating the actual usage, potential value, and risks of various device data, and identifying the maximum value brought by different device data in combination with changes in the external environment, specifically including:

[0034] Setting a time frame for periodic assessment, and calculating the generation frequency and data quality status of each device data according to the risk assessment criteria;

[0035] Calculating the risk fluctuation level and expected value of the corresponding device data through the device data value model;

[0036] Combining the operation data of the Internet of Things device and the external environment data, analyzing the impact on the potential value of different device data, and performing device data value correction to obtain the maximum value of different device data.

[0037] As a further solution of the present invention, analyzing the impact on the potential value of different device data and performing device data value correction specifically means:

[0038] ;

[0039] Among them, represents the corrected device data value, represents the number of Internet of Things device operation data affecting the device data value, represents the influence coefficient related to the th Internet of Things device operation data, represents the th quantization value of the Internet of Things device operation data, is a non-linear function, representing the degree of influence of device operation on the expected value, Represents the quantity of external environment data that affects the value of device data, Represents the influencing factors related to the th external environment data.

[0040] Another object of the present invention is to provide a data middle platform anomaly detection system based on deep learning, the system comprising:

[0041] A data unified collection module, configured to set the frequency and range of data collection, and uniformly collect aggregated data through the data middle platform, the aggregated data including Internet of Things device operation data, external environment data, and device data generated by the Internet of Things devices, and perform preprocessing on all the collected aggregated data

[0042] An Internet of Things device operation data analysis module, configured to analyze the collected Internet of Things device operation data, establish a device behavior model, compare real-time Internet of Things device operation data with historical Internet of Things device operation data, identify potential abnormal activities during device operation, and generate a device anomaly table;

[0043] A risk dynamic assessment module, configured to define risk assessment criteria, and perform dynamic assessment of device data based on the device anomaly table and external environment data, detect the generation frequency of device data and the quality status of device data, and calculate the expected value and risk fluctuation level of device data in real time;

[0044] A standard revision trigger module, configured to set a change range for each evaluation content in the risk assessment criteria. If the device operation abnormal activities and external environment data cause any evaluation content to exceed the change range, trigger the revision of the risk assessment criteria, and synchronously revise the risk assessment criteria based on the device anomaly table and external environment data;

[0045] A device data regular assessment module, configured to regularly assess the actual usage, potential value, and risks of various device data based on the risk assessment criteria and the device anomaly table, and identify the maximum value brought by different device data in combination with external environment changes.

[0046] The beneficial effects of the present invention are:

[0047] By integrating Internet of Things device operation data, external environment data, and device data generated by Internet of Things devices, through a real-time data stream processing mechanism and a dynamic risk assessment model, a closed-loop management from data collection to data asset value assessment is realized. For example, when the device operation deviates from the normal mode (such as high-frequency abnormal operations), the system can identify and generate a device anomaly table in seconds, and dynamically adjust the risk assessment criteria in combination with external environment changes (such as temperature, humidity, network latency, geographical location, etc.), significantly improving the timeliness and accuracy of anomaly detection.

[0048] The expected value model and correction formula are introduced to dynamically evaluate the value of device data by quantifying risk fluctuations, external environment impacts, and device operation correlations. For example, the data value of a certain device's data can be corrected in real time according to market fluctuations, helping to identify high-value data assets and optimize data resource allocation, increasing the utilization rate of device data by more than 30%.

[0049] Combining deep learning and quantitative analysis techniques to break through the limitations of traditional methods. The false alarm rate of device operation modeling is reduced by more than 50% compared with the rule matching method, and the calculation of risk fluctuation level (based on the standard deviation of the time window) can more accurately reflect the stability of device data than static evaluation. Brief Description of the Drawings

[0050] Figure 1 It is a flowchart of the anomaly detection method for the data middle platform based on deep learning provided by an embodiment of the present invention;

[0051] Figure 2 It is a flowchart of comparing real-time Internet of Things device operation data and historical Internet of Things device operation data to identify potential abnormal activities during device operation provided by an embodiment of the present invention;

[0052] Figure 3 It is a flowchart of detecting the generation frequency of device data and the quality status of device data, and calculating the expected value and risk fluctuation level of device data in real time provided by an embodiment of the present invention;

[0053] Figure 4 It is a flowchart of identifying the maximum value brought by different device data in combination with external environment changes provided by an embodiment of the present invention;

[0054] Figure 5 It is a structural block diagram of the anomaly detection system for the data middle platform based on deep learning provided by an embodiment of the present invention. Detailed Embodiments

[0055] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] Figure 1 It is a flowchart of the anomaly detection method for the data middle platform based on deep learning provided by an embodiment of the present invention, as Figure 1 shown, the method includes:

[0057] S100, setting the frequency and range of data collection, uniformly collecting and aggregating data through the data middle platform, where the aggregated data includes Internet of Things device operation data, external environment data, and device data generated by Internet of Things devices, and preprocessing all the collected aggregated data;

[0058] This step mainly involves setting the frequency and range of data collection, as well as the unified collection and preprocessing of aggregated data. Specifically, the data range setting should cover multi-dimensional data such as the operation data of IoT devices, external environment data (including temperature, humidity, etc. of the device working environment), and device data generated by IoT devices, in order to achieve comprehensive monitoring and analysis. The data middle platform needs to integrate data sources from different channels. This integration not only helps to eliminate data islands, improve data accessibility and usability, but also provides a broader feature basis for subsequent deep learning models.

[0059] S200, analyze the collected operation data of IoT devices, establish a device behavior model, compare the real-time operation data of IoT devices with the historical operation data of IoT devices, identify potential abnormal activities during device operation, and generate a device exception table;

[0060] This step extracts features based on the operation data of IoT devices to generate feature vectors reflecting device operation. These feature vectors can effectively capture the operation patterns of IoT devices, thus helping to establish a device behavior model. This model not only sets conventional behavior standards, but also can identify behaviors deviating from the conventional pattern in a timely manner by comparing the real-time operation data of IoT devices with the historical operation data of IoT devices, and mark them as potential abnormal activities.

[0061] In this process, a real-time data stream processing mechanism is also constructed to extract the real-time operation data of IoT devices according to a preset frequency and compare it with historical data. This real-time nature is crucial for the anomaly detection system because device operation is often dynamically changing, and timely identification of deviations is essential for preventing potential risks. At the same time, by establishing a device exception table, the identified potential abnormal activities are stored, and label synchronization storage is performed with the corresponding real-time operation data of IoT devices to ensure that subsequent analysis can accurately locate the source and impact of anomalies.

[0062] This step uses a loss function to measure the deviation degree between the feature vectors of IoT devices and their conventional behavior model, which can effectively quantify and monitor the abnormality of device operation. The advantage of this quantitative analysis is that it can provide data support for subsequent risk assessment and decision-making through accurate model output.

[0063] The model established based on the operation data of Internet of Things devices can adapt to the changes in device operation, making the system more sensitive to the detection of abnormal activities. And through real-time data stream processing, potential abnormal behaviors can be quickly captured and responded to, reducing security risks. At the same time, the generated device exception table provides basic data support for subsequent risk assessment and decision-making, making the entire abnormal detection process more systematic and efficient. It not only improves the accuracy of abnormal detection but also enhances the system's adaptability to the dynamic environment, providing guarantee for the security and stability of the entire data middle platform.

[0064] As Figure 2 shown, comparing the real-time operation data of Internet of Things devices with the historical operation data of Internet of Things devices, identifying potential abnormal activities in device operation, and generating a device exception table, specifically including:

[0065] S210, extracting features based on the operation data of Internet of Things devices, generating feature vectors that can reflect device operation, identifying the normal operation mode of the device, establishing a device behavior model, and setting normal behavior standards;

[0066] S220, setting up a real-time data stream processing mechanism, extracting real-time operation data of Internet of Things devices according to a preset frequency, comparing the real-time operation data of Internet of Things devices with the historical operation data of Internet of Things devices, identifying behaviors that deviate from the normal operation mode, and marking them as potential abnormal activities;

[0067] S230, establishing a device exception table, storing the potential abnormal activities in the device exception table, and at the same time storing the real-time operation data of Internet of Things devices that match the potential abnormal activities as tags synchronously.

[0068] In this step, the behavior that is identified as deviating from the normal operation mode is specifically:

[0069] ;

[0070] In the formula, is the reconstruction error, which is used to measure the difference between the input device state feature and the feature corresponding to the normal behavior mode , is the number of device state features.

[0071] S300, defining risk assessment criteria, and based on the device exception table and external environment data, conducting dynamic assessment of device data, detecting the generation frequency of device data and the quality status of device data, and calculating the expected value and risk fluctuation level of device data in real time;

[0072] In this step, the device data is dynamically evaluated by analyzing the device exception table and external environment data, specifically including the detection of the generation frequency, quality status, expected value, and risk fluctuation level of the device data.

[0073] By analyzing the operation data of IoT devices through a time window, the generation frequency of device data can be effectively calculated. This analysis not only identifies the activity level of the device data but also provides important basic data for subsequent risk assessment. Specifically, the risk fluctuation level is calculated using a formula , and this indicator is used to quantify the stability and volatility of device data within a specified time window, thereby reflecting the risk status of the device data. Through this process, the system can identify the impact of external environmental factors on the generation frequency and quality of device data, and perform dynamic monitoring in combination with historical data to calculate the risk fluctuation level of device data in real time.

[0074] By combining the operation data of IoT devices and external environment data, the expected value of each device data is quantified , and this process helps decision-makers understand the actual value of device data and its performance under different environmental conditions. Using the formula to calculate the expected value can take into account the risk factors and external impacts of device data, thus providing a comprehensive perspective to evaluate the potential value of device data.

[0075] This step can timely detect and respond to the risk status of device data through multi-dimensional data analysis, improving the overall security and reliability. And based on the monitoring of external environmental changes, it can effectively identify the factors affecting the generation frequency and quality of device data, making the risk assessment more accurate. The quantification of the expected value not only provides data support for the management decision-making of device data but also optimizes resource allocation, thereby achieving the sustainable growth of the business. Through comprehensive risk assessment and dynamic adjustment, the practicality and efficiency of the data middle platform in anomaly detection and asset management are improved.

[0076] As Figure 3 shown, the dynamic evaluation of device data is carried out to detect the generation frequency of device data and the quality status of device data, and the expected value and risk fluctuation level of device data are calculated in real time, specifically including:

[0077] S310, Analyze the operation data of IoT devices based on a time window, calculate the generation frequency of device data, and identify the activity level of the device;

[0078] S320, Real-time monitor the risk fluctuation of device data, combine with external environment data, identify the impact of external factors on the generation frequency and quality status of device data, and calculate the risk fluctuation level of device data;

[0079] S330: Based on the operation data and external environment data of IoT devices, construct a device data value model, and combine the device data risk fluctuation level to quantify the expected value of each device data.

[0080] In this step, calculating the risk fluctuation level of the device data specifically includes:

[0081] ;

[0082] In the formula, represents the risk fluctuation level, which is used to quantify the risk volatility of device data and reflects the stability of the device data generation frequency within a specified time window. is the sample size, indicating the number of data points within the specified time window. is the th data point, that is, the device data generation frequency for each day within the time window. is the sample mean, that is, the average generation frequency of device data within the same time window.

[0083] Quantifying the expected value of each device data specifically includes:

[0084] ;

[0085] Among them, represents the expected value of the device data. represents the quantity of the device data. is the th business value of the device data, that is, the expected data value under normal circumstances. is the th risk measure of the device data. is the quantity of external environment data affecting the expected value. is the th quantization value of the external environment data. is a non-linear function, indicating the degree of influence of the external environment on the expected value. is a constant term.

[0086] In this step, the risk assessment criteria specifically include:

[0087] Generation frequency assessment criteria, that is, the generation frequency of device data, which defines the normal generation frequency range of device data.

[0088] Data quality assessment criteria, that is, the quality status of device data, including accuracy ratio threshold, integrity ratio threshold, and consistency ratio threshold.

[0089] External environment impact assessment criteria, that is, the impact of external environment data on the device data generated by IoT devices, define the proportional change range of the generation frequency and data quality of device data caused by changes in the external environment.

[0090] S400. For the assessment contents defined in the risk assessment criteria, set a change range for each assessment content. If the abnormal activities of the device operation and the external environment data cause any assessment content to exceed the change range, trigger the revision of the risk assessment criteria, and synchronously revise the risk assessment criteria based on the device exception table and the external environment data;

[0091] This step emphasizes the setting of the change range. This means that for each risk assessment criterion (such as device data generation frequency, data quality, risk fluctuation level, etc.), a reasonable threshold range needs to be set according to historical data and business requirements. These thresholds should not only consider the normal fluctuation range but also reserve a certain buffer space to cope with sudden abnormal situations. For example, if the normal data generation frequency range of a certain device is 100 - 500 times / week, when it is monitored that the generation frequency reaches 600 times, the system can automatically identify this behavior as potential abnormal activity.

[0092] In actual operation, when the abnormal operation of the device causes a certain assessment content to exceed the set change range, the system will automatically trigger the revision of the risk assessment criteria. This mechanism ensures the timeliness of risk response, enabling enterprises to take countermeasures quickly. By combining with the device exception table and the external environment data, the revised risk assessment criteria will be more accurate and can reflect the current real situation.

[0093] S500. Based on the risk assessment criteria and the device exception table, regularly evaluate the actual usage, potential value, and risks of various device data, and identify the maximum value brought by different device data in combination with changes in the external environment.

[0094] The core of this step is to regularly evaluate the actual usage, potential value, and risks of various device data, and in combination with changes in the external environment, identify the maximum value brought by different device data. Specifically, this step focuses on how to adjust the value assessment of device data to adapt to the dynamically changing environment and device operation.

[0095] In this step, the value correction of device data is clearly composed of the initial value and a series of influencing factors. These factors include the impact of device operation on device data and the impact of the external environment on the value of device data. Described by a formula, calculates the comprehensive impact of multiple factors such as device data generation frequency, device operation, and external environment on the value of device data. Specifically, Indicates the degree of association between device operation and device data, while reflects the impact of external environmental factors on the value of device data. This quantitative method makes the evaluation of device data more accurate and flexible.

[0096] By setting a periodic evaluation time frame, this step can calculate the generation frequency and data quality status of each device data according to the risk assessment criteria. This process ensures the continuous monitoring of device data, timely discovery of potential risks and adjustments. Combining the analysis of IoT device operation data and external environmental data provides a sufficient information basis for the value correction of device data, thus ensuring the scientificity and effectiveness of decision-making.

[0097] Through regular evaluation and value correction in this step, the system can quickly respond to changes in the external environment and optimize resource allocation. Combining the analysis of device operation and external environmental data can identify potential value impacts, helping enterprises better grasp market opportunities. This multi-dimensional data analysis method improves the accuracy of anomaly detection and ensures efficient value evaluation in a complex environment. Therefore, this step improves the efficiency of device data management, enabling the data center to play a greater role in anomaly detection and asset management.

[0098] This step is not just about calculating and correcting the value of device data. Its essence lies in improving the effectiveness of device data management and the quality of decision-making through comprehensive and dynamic evaluation. By regularly evaluating the actual usage, potential value and risks of each device data and combining external environmental changes, this step can provide a real-time and accurate perspective on data asset management.

[0099] Through this dynamic evaluation, the actual value of device data can be understood in real time, strategies can be adjusted in a timely manner, device resource allocation can be optimized, potential losses can be avoided, and the efficiency of device data management is improved.

[0100] Such as Figure 4 shown, regularly evaluating the actual usage, potential value and risks of various device data, and combining external environmental changes to identify the maximum value brought by different device data, specifically including:

[0101] S510, set the time frame for periodic evaluation, and calculate the generation frequency and data quality status of each device data according to the risk assessment criteria;

[0102] S520, calculate the risk fluctuation level and expected value of the corresponding device data through the device data value model;

[0103] S530, combine IoT device operation data and external environmental data, analyze the potential value impact on different device data, and perform value correction of device data to obtain the maximum value of different device data.

[0104] In this step, the analysis impacts the potential value of different device data and corrects the value of device data, specifically as follows:

[0105] ;

[0106] Among them, represents the corrected value of device data, represents the quantity of Internet of Things device operation data that impacts the value of device data, represents the impact coefficient related to the th Internet of Things device operation data, represents the th quantization value of Internet of Things device operation data, is a non-linear function, representing the impact degree of device operation on the expected value, represents the quantity of external environment data that impacts the value of device data, represents the influencing factor related to the th external environment data.

[0107] Figure 5 As shown in Figure 5 is the structural block diagram of the data middle platform anomaly detection system based on deep learning provided by the embodiments of the present invention. The system includes:

[0108] The data unified collection module 100 is used to set the frequency and range of data collection, uniformly collect and aggregate data through the data middle platform. The aggregated data includes Internet of Things device operation data, external environment data, and device data generated by Internet of Things devices, and preprocess all the collected aggregated data;

[0109] The Internet of Things device operation data analysis module 200 is used to analyze the collected Internet of Things device operation data, establish a device behavior model, compare the real-time Internet of Things device operation data with the historical Internet of Things device operation data, identify potential abnormal activities in device operation, and generate a device anomaly table;

[0110] The risk dynamic assessment module 300 is used to define risk assessment criteria, and based on the device anomaly table and external environment data, conduct dynamic assessment of device data, detect the generation frequency of device data and the quality status of device data, and calculate the expected value and risk fluctuation level of device data in real time;

[0111] The standard revision trigger module 400 is used to set a change range for each evaluation content defined in the risk assessment standard. If the abnormal operation activities of the device and the external environment data cause any evaluation content to exceed the change range, it triggers the revision of the risk assessment standard and synchronously revises the risk assessment standard based on the device exception table and the external environment data;

[0112] The device data regular evaluation module 500 is used to regularly evaluate the actual usage, potential value, and risks of various device data based on the risk assessment standard and the device exception table, and identify the maximum value brought by different device data in combination with the external environment changes.

[0113] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as these combinations of technical features do not conflict, they should be considered as the scope described in this specification.

[0114] The above-described embodiments only represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be subject to the appended claims.

[0115] The above is only the preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data mid-platform anomaly detection method based on deep learning, characterized in that, The method includes: Setting the frequency and range of data collection, collecting and aggregating data through the data center. The aggregated data includes the operation data of IoT devices, external environment data, and device data generated by IoT devices, and preprocessing all the collected aggregated data. Analyzing the collected operation data of IoT devices, establishing a device behavior model, comparing the real-time operation data of IoT devices with the historical operation data of IoT devices, identifying potential abnormal activities during device operation, and generating a device exception table. Defining risk assessment criteria, and based on the device exception table and external environment data, conducting dynamic assessment of device data, detecting the generation frequency of device data and the quality status of device data, and calculating the expected value and risk fluctuation level of device data in real time. For each assessment content defined in the risk assessment criteria, setting a change interval. If the abnormal device operation activities and external environment data cause any assessment content to exceed the change interval, trigger the revision of the risk assessment criteria, and synchronously revise the risk assessment criteria based on the device exception table and external environment data. Based on the risk assessment criteria and the device exception table, regularly evaluating the actual usage, potential value, and risks of various device data, and identifying the maximum value brought by different device data in combination with external environment changes.

2. The method according to claim 1, wherein The comparison of the real-time operation data of IoT devices with the historical operation data of IoT devices, identifying potential abnormal activities during device operation, and generating a device exception table specifically includes: Extracting features based on the operation data of IoT devices, generating feature vectors that can reflect device operation, identifying the normal operation mode of the device, establishing a device behavior model, and setting normal behavior standards. Setting up a real-time data stream processing mechanism, extracting the real-time operation data of IoT devices according to the preset frequency, comparing the real-time operation data of IoT devices with the historical operation data of IoT devices, identifying behaviors that deviate from the normal operation mode, and marking them as potential abnormal activities. Establishing a device exception table, storing the potential abnormal activities in the device exception table, and simultaneously storing the real-time operation data of IoT devices that match the potential abnormal activities as tags.

3. The method according to claim 2, wherein The identification of behaviors that deviate from the normal operation mode specifically is: ； In the formula, is the reconstruction error, which is used to measure the input device state features and the features corresponding to the normal behavior patterns The difference between them, is the number of device state features.

4. The method according to claim 2, wherein The conduct of dynamic assessment of device data, detecting the generation frequency of device data and the quality status of device data, and calculating the expected value and risk fluctuation level of device data in real time specifically includes: Analyzing the operation data of IoT devices based on a time window, calculating the generation frequency of device data, and identifying the activity level of the device. Real-time monitoring of the risk fluctuation of device data, combining external environment data, identifying the impact of external factors on the generation frequency of device data and the quality status of device data, and calculating the risk fluctuation level of device data. Based on the operation data of IoT devices and external environment data, constructing a device data value model, and quantifying the expected value of each device data in combination with the risk fluctuation level of device data.

5. The method according to claim 4, characterized in that The calculation of the risk fluctuation level of device data specifically is: ； In the formula, represents the risk fluctuation level, which is used to quantify the risk volatility of device data and reflects the stability of the device data generation frequency within a specified time window. is the number of samples, indicating the number of data points within a specified time window. is the th data point, that is, the device data generation frequency for each day within the time window. is the sample mean, that is, the average generation frequency of device data within the same time window. The quantification of the expected value of each device data specifically is: ； Among them, represents the expected value of the device data, represents the quantity of the device data, is the business value of the th device data, that is, the expected data value under normal circumstances, is the th risk measure of the device data, is the quantity of the external environment data affecting the expected value, is the th quantization value of the external environment data, is a non-linear function, representing the degree of influence of the external environment on the expected value, is a constant term.

6. The method according to claim 1, wherein The risk assessment criteria specifically include: Generate a frequency evaluation criterion, that is, the generation frequency of device data, and define the normal generation frequency range of device data; Data quality evaluation criterion, that is, the quality status of device data, including accuracy ratio threshold, integrity ratio threshold, and consistency ratio threshold; External environment impact evaluation criterion, that is, the impact of external environment data on the generation of device data by Internet of Things devices, and define the change ratio range of the generation frequency and data quality of device data caused by external environment changes.

7. The method according to claim 6, wherein Regularly evaluate the actual usage, potential value, and risks of various device data, and identify the maximum value brought by different device data in combination with external environment changes. Specifically include: Set the time frame for periodic evaluation, and calculate the generation frequency and data quality status of each device data according to the risk evaluation criterion; Calculate the risk fluctuation level and expected value of the corresponding device data through the device data value model; Combine the operation data of Internet of Things devices and external environment data, analyze the impact on the potential value of different device data, and perform device data value correction to obtain the maximum value of different device data.

8. The method according to claim 7, wherein Analyze the impact on the potential value of different device data and perform device data value correction. Specifically: ； Among them, represents the value of the corrected device data, represents the quantity of Internet of Things device operation data that affects the value of device data, represents the influence coefficient related to the operation data of the th Internet of Things device, represents the quantization value of the operation data of the th Internet of Things device, is a non-linear function representing the degree of influence of device operation on the expected value, represents the quantity of external environment data that affects the value of device data, represents the influence factor related to the th external environment data.

9. The data middle platform anomaly detection system based on deep learning is characterized in that The system includes: Data unified collection module, used to set the frequency and range of data collection, and uniformly collect and aggregate data through the data middle platform. The aggregated data includes the operation data of Internet of Things devices, external environment data, and device data generated by Internet of Things devices, and preprocess all the aggregated data collected; Internet of Things device operation data analysis module, used to analyze the collected operation data of Internet of Things devices, establish a device behavior model, compare the real-time operation data of Internet of Things devices with the historical operation data of Internet of Things devices, identify potential abnormal activities during device operation, and generate a device exception table; Risk dynamic evaluation module, used to define the risk evaluation criterion, and based on the device exception table and external environment data, perform dynamic evaluation of device data, detect the generation frequency of device data and the quality status of device data, and calculate the expected value and risk fluctuation level of device data in real time; Standard revision trigger module, used to set a change range for each evaluation content in the risk evaluation criterion. If the abnormal device operation activities and external environment data cause any evaluation content to exceed the change range, trigger the revision of the risk evaluation criterion, and synchronously revise the risk evaluation criterion based on the device exception table and external environment data; Device data regular evaluation module, used to regularly evaluate the actual usage, potential value, and risks of various device data based on the risk evaluation criterion and the device exception table, and identify the maximum value brought by different device data in combination with external environment changes.

Images