Internet of Things data quality screening method and system based on multistage intelligent evaluation

Abstract

The invention relates to the field of Internet of Things data processing, and discloses an Internet of Things data quality screening method and system based on multistage intelligent evaluation, and the method comprises the following steps: (1) carrying out the primary screening of a sample based on data energy distribution: dividing windows for an input Internet of Things data flow, calculating the energy value of each window, dynamically adjusting a screening threshold value according to the energy distribution, and carrying out the initial screening of the sample; high-energy samples are reserved; and (2) sample screening based on time sequence information consistency: carrying out frequency domain analysis on the primarily screened samples, and determining main frequency and co-frequency bands in the samples through signal analysis. According to the method, data energy distribution is automatically analyzed through an iterative algorithm, the screening threshold value is dynamically determined, manual rule presetting is not needed, the method adapts to energy heterogeneity under different equipment models, sampling frequencies and transmission conditions, the threshold value is periodically updated in a real-time data stream scene, the method adapts to environment changes, and high-value data missing or noise misjudgment is avoided.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things data processing, and in particular to a method and system for screening Internet of Things data quality based on multi-level intelligent evaluation. Background Art

[0002] With the rapid development of IoT technology, massive numbers of IoT devices (such as sensors, drones, and wearable devices) continuously generate heterogeneous data streams, encompassing a variety of modalities, including structured time-series data and unstructured video streams. This data is crucial for intelligent analysis, value identification, and data pricing, but its quality is highly dependent on factors such as device performance, sampling frequency, and transmission conditions, resulting in significant data heterogeneity.

[0003] Currently, IoT data quality assessment mainly relies on the following two methods:

[0004] Threshold filtering and rule presetting: For example, patent CN108874876A proposes filtering abnormal data using static thresholds, while patent CN110598495A uses a rule engine to clean data. These methods rely on manual experience to set rules and are difficult to adapt to the dynamic changes and heterogeneous characteristics of data streams.

[0005] Model-biased evaluation methods, such as the ICML 2019 paper "Data Shapley," use model contribution to assess data value. However, these methods rely on task-specific labeled data and incur significant computational overhead from multiple rounds of iteration. Furthermore, while statistical sampling methods (IEEE Internet of Things Journal 2019) can reduce data size, they cannot guarantee semantic consistency in the screening results.

[0006] Existing data quality screening methods lack dynamic adaptability. Fixed thresholds and preset rules are difficult to cope with the heterogeneity of data streams, which can easily lead to omissions of high-value data or misjudgment of noise. Model preference-based methods need to be bound to specific downstream tasks, and multiple rounds of iterative evaluation significantly increase computing resource consumption. They do not address the differentiated evaluation needs of structured and unstructured data, and are difficult to support standardized data pricing and cross-scenario applications. Summary of the Invention

[0007] In order to solve the above-mentioned problems, the present invention is implemented through the following technical solutions.

[0008] In one aspect, the present invention provides a method for screening IoT data quality based on multi-level intelligent evaluation, comprising the following steps:

[0009] (1) Sample screening based on data energy distribution: divide the input IoT data stream into windows, calculate the energy value of each window, dynamically adjust the screening threshold according to the energy distribution, and retain high-energy samples;

[0010] (2) Sample screening based on consistency of time series information: Perform frequency domain analysis on the samples after initial screening, determine the main frequency and harmonic frequency bands in the samples through signal analysis, calculate the proportion of their energy to the overall energy of the samples, calculate the consistency index, and screen out samples with a ratio higher than the preset threshold;

[0011] (3) High-quality evaluation based on variational autoencoder: The samples after secondary screening are input into the pre-trained variational autoencoder model, the reconstruction loss is calculated, and the samples with loss below the threshold are retained as high-quality data.

[0012] Preferably, in step (1), the energy value is calculated as follows:

[0013]

[0014] Among them, x i is the i-th sampling point, and N is the window size.

[0015] Preferably, in step (2), the calculation formula of the consistency index is:

[0016]

[0017] Among them E main is the main frequency band energy, E co is the harmonic band energy, E total is the total energy of the sample.

[0018] Preferably, the specific steps of dynamically adjusting the screening threshold in step (1) are:

[0019] S1: Data Windowing and Energy Calculation: Divide the IoT data stream into multiple data segments according to a fixed window size, and calculate the energy value for each window;

[0020] S2: Construct an energy distribution histogram: Count the energy value distribution of all windows and generate an energy histogram. The horizontal axis is the energy interval and the vertical axis is the number of windows in the corresponding interval.

[0021] S3: Iterative threshold determination: Set the initial threshold T0 based on the global mean or median of the histogram and calculate the current threshold T k The proportion of high energy windows, that is, energy value ≥ T k The ratio of the number of windows to the total windows. If the ratio is lower than the preset target value, the threshold is lowered. If the ratio is higher than the target value, the threshold is increased. Repeat the iteration until the error between the ratio and the target value is less than the tolerance range.

[0022] S4: Real-time dynamic adaptation: For real-time incoming data, the energy distribution statistics are regularly updated and the thresholds are recalculated to adapt to changes in the data collection environment.

[0023] Preferably, in step (2), the signal analysis means includes Fourier transform, which is used to convert the time domain signal into a frequency domain signal and identify the main frequency band and the harmonic frequency band.

[0024] Preferably, in step (2), the frequency domain analysis is based on the following assumptions:

[0025] The main frequency band exists within a preset frequency range;

[0026] The harmonic band is the harmonic component with the largest energy;

[0027] The total energy of the main frequency and harmonics is significantly higher than that of the noise frequency band.

[0028] Preferably, in step (3), the variational autoencoder model is trained through self-supervised learning, and the training data includes unlabeled samples of multiple IoT modalities.

[0029] On the other hand, the present invention also provides an IoT data quality screening system based on multi-level intelligent evaluation, comprising:

[0030] The energy distribution screening module receives IoT data streams, divides the data into windows, calculates the energy value of each window, and screens high-energy samples based on a dynamically adjusted energy threshold.

[0031] The time series consistency screening module performs frequency domain analysis on the samples after the initial screening, extracts the energy of the main frequency band and the energy of the harmonic frequency band, calculates the consistency index, and screens samples whose information consistency is higher than the preset threshold;

[0032] The variational autoencoder quality assessment module uses a pre-trained variational autoencoder model to calculate the reconstruction loss of the samples after secondary screening, and selects high-quality data with a reconstruction loss lower than the set threshold.

[0033] Preferably, the energy distribution preliminary screening module includes an energy calculation unit and a dynamic threshold adjustment unit.

[0034] Preferably, the timing consistency screening module includes a frequency domain analysis unit and a consistency index calculation unit.

[0035] The present invention provides a method and system for IoT data quality screening based on multi-level intelligent evaluation. Compared with existing technologies, the method has the following advantages: it automatically analyzes data energy distribution through an iterative algorithm, dynamically determines the screening threshold, eliminates the need for manually pre-set rules, adapts to energy heterogeneity under different device models, sampling frequencies, and transmission conditions, regularly updates the threshold in real-time data stream scenarios, adapts to environmental changes, and avoids the omission of high-value data or noise misjudgment. By introducing frequency domain energy distribution analysis and utilizing triple hypotheses to screen samples with strong temporal regularity, it effectively identifies periodic signals and compensates for the inability of traditional time domain threshold filtering to capture signal frequency domain characteristics. By training the model with unlabeled data, it utilizes the characteristics of high-dimensional semantics that are easy to reconstruct and noise that is difficult to reconstruct to screen high-quality data. The method is universal for various IoT time series modes and is applicable to all IoT scenarios, reducing the cost of cross-task adaptation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the screening method proposed in the present invention.

[0037] Figure 2 This is a block diagram of the energy distribution preliminary screening module proposed in the present invention.

[0038] Figure 3 This is a block diagram of the timing consistency screening module proposed in the present invention.

[0039] Figure 4 Block diagram of the variational autoencoder quality assessment module proposed in this invention. DETAILED DESCRIPTION

[0040] The present invention is further described below with reference to specific examples. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of protection of the present invention.

[0041] The following describes the embodiments of the present invention through specific examples. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention.

[0042] Example 1

[0043] Reference Figure 1 ,The IoT data quality screening method based on multi-level ,intelligent evaluation includes the following steps:

[0044] (1) Sample screening based on data energy distribution: Divide the input IoT data stream into windows, calculate the energy value of each window, dynamically adjust the screening threshold according to the energy distribution, and retain high-energy samples. The energy value calculation formula is:

[0045]

[0046] Among them, x i is the i-th sampling point, N is the window size;

[0047] (2) Sample screening based on consistency of time series information: Perform frequency domain analysis on the samples after initial screening, determine the main frequency and harmonic frequency bands in the sample through signal analysis, calculate the proportion of their energy to the overall energy of the sample, calculate the consistency index, and screen samples with a ratio higher than the preset threshold. The calculation formula of the consistency index is:

[0048]

[0049] Among them E main is the main frequency band energy, E co is the harmonic band energy, E total is the total energy of the sample;

[0050] (3) High-quality evaluation based on variational autoencoder: The samples after secondary screening are input into the pre-trained variational autoencoder model, the reconstruction loss is calculated, and the samples with loss below the threshold are retained as high-quality data.

[0051] In this embodiment, in view of the sparse distribution of events presented by the continuously collected IoT data stream, the screening threshold is automatically determined through an iterative algorithm based on the energy distribution in the data samples, high-energy samples with active events are retained, and low-energy samples dominated by noise are filtered out. Furthermore, information-intensive events in the continuous data stream should present information regularity consistency in a short period of time. Therefore, the main frequency and harmonic frequency bands in the sample are determined by signal analysis means, and the proportion of their energy to the overall energy of the sample is counted. Samples with a high proportion represent high-quality samples with high information consistency, which are screened. Finally, the variational autoencoder algorithm (VAE) in deep learning is used to perform model self-supervision training on the above-mentioned screened samples, and then the reconstruction loss of the samples is counted separately. Taking advantage of the fact that high-dimensional semantics of information are easy to restore while random noise is difficult to reconstruct, high-quality data samples with low reconstruction difficulty are screened out. Through the three-level linkage architecture of energy distribution initial screening, time series consistency screening and unsupervised VAE evaluation, the screening problem caused by data heterogeneity is solved and the computational overhead is reduced. Through sliding window updates and mutation detection, it adapts to changes in data stream characteristics and supports multimodal data such as structured sensor data and unstructured video streams. The three-level screening mechanism reduces invalid data input.

[0052] In step (1), the specific steps for dynamically adjusting the screening threshold are:

[0053] S1: Data Windowing and Energy Calculation: Divide the IoT data stream into multiple data segments according to a fixed window size, and calculate the energy value for each window;

[0054] S2: Construct an energy distribution histogram: Count the energy value distribution of all windows and generate an energy histogram. The horizontal axis is the energy interval and the vertical axis is the number of windows in the corresponding interval.

[0055] S3: Iterative threshold determination: Set the initial threshold T0 based on the global mean or median of the histogram and calculate the current threshold T k The proportion of high energy windows, that is, energy value ≥ T k The ratio of the number of windows to the total windows. If the ratio is lower than the preset target value, the threshold is lowered. If the ratio is higher than the target value, the threshold is increased. Repeat the iteration until the error between the ratio and the target value is less than the tolerance range.

[0056] S4: Real-time dynamic adaptation: For real-time incoming data, the energy distribution statistics are regularly updated and the thresholds are recalculated to adapt to changes in the data collection environment.

[0057] In step (2), the signal analysis method includes Fourier transform, which is used to convert the time domain signal into the frequency domain signal and identify the main frequency band and harmonic frequency band. The frequency domain analysis is based on the following assumptions:

[0058] The main frequency band exists within a preset frequency range;

[0059] The harmonic band is the harmonic component with the largest energy;

[0060] The total energy of the main frequency and harmonics is significantly higher than that of the noise frequency band.

[0061] In step (3), the variational autoencoder model is trained through self-supervised learning, and the training data includes unlabeled samples of multiple IoT modalities.

[0062] This solution uses the collaborative processing of three-level intelligent evaluation modules to achieve full-process screening from multimodal IoT data input to high-quality data output. The specific workflow is as follows:

[0063] 1. Data Access and Preprocessing

[0064] 1. Multimodal data input

[0065] Supported types: low-altitude drone video stream (RGB frame sequence), vehicle-road cooperative sensor time series data (radar / lidar point cloud), user wearable device physiological signals (heart rate / blood oxygen time series data), etc.

[0066] Data interface: Receives data streams through protocols such as TCP / IP and MQTT, supporting real-time access from the edge or batch import from the cloud.

[0067] 2. Preprocessing Adaptation

[0068] Unified format:

[0069] Video stream: grayscale / normalization processing, conversion to a fixed size (e.g. 1080p → 224×224 pixels);

[0070] Time series data: Resample to a uniform frequency (e.g., sensor data from 100 Hz to 50 Hz), and merge multi-channel signals into a multidimensional array.

[0071] Preliminary noise filtering: Remove high-frequency random noise (such as sensor glitch signals) through sliding average filtering or median filtering.

[0072] 2. Primary screening: initial screening based on data energy distribution

[0073] 1. Sliding window splitting

[0074] Window partitioning: Split the data stream into a fixed window size N (e.g., every 10 frames of video stream is a window, and every 1000 sampling points of sensor data is a window) to generate continuous data segments.

[0075] 2. Energy value calculation

[0076] Perform energy calculations on each window;

[0077] Dynamic threshold adjustment and screening;

[0078] Count the energy value distribution of all windows and generate an energy histogram (horizontal axis: energy range, vertical axis: number of windows);

[0079] Iterative screening: The initial threshold setting is based on the mean / median setting T, and the proportion of high-energy windows P is calculated;

[0080] Threshold optimization: If P deviates from the target value, gradually adjust the threshold until the proportion is stable;

[0081] Real-time adaptation: regularly update thresholds to adapt to environmental changes.

[0082] Output: retain high-energy samples with energy values ≥ the final threshold and filter out low-energy noise.

[0083] Secondary Screening: Fine Screening Based on Time Series Information Consistency

[0084] Frequency domain analysis and feature extraction

[0085] Signal conversion: Perform Fourier transform on high-energy samples to convert the time domain signal into a frequency domain amplitude spectrum to obtain the energy distribution of each frequency component.

[0086] Three assumptions drive the analysis:

[0087] Main frequency band positioning: preset the frequency range according to the signal type and filter the frequency components within the range;

[0088] Harmonic energy search: Identify the main harmonic component with the largest energy and accumulate the second and third harmonic energies;

[0089] Energy significance verification: Ensure that the total energy of the sample is significantly higher than the noise baseline.

[0090] Consistency index calculation and screening.

[0091] Level 3 evaluation: semantic-level verification based on variational autoencoders (VAEs)

[0092] 1. VAE model self-supervised training

[0093] Data: Multimodal unlabeled samples (e.g., historical sensor data, mixed sets of drone videos);

[0094] Loss function: reconstruction loss (such as mean square error) + KL divergence to ensure that the distribution of latent variables is close to normal distribution.

[0095] Reconstruction loss calculation and screening

[0096] Loss assessment: Calculate the reconstruction loss (such as video frame pixel difference and time series data point error) for the samples after secondary screening. High-value data has low loss due to the complete semantic information, while noise data has high loss.

[0097] Output decision: retain samples with reconstruction loss ≤ preset threshold (such as 0.05) to form the final high-quality dataset.

[0098] 5. High-quality data output and application

[0099] 1. Data output format

[0100] Structured data: sensor time series data is saved in CSV / JSON format with quality scores (energy value, consistency index, reconstruction loss);

[0101] Unstructured data: Video streams are saved as key frame sequences and labeled with high-quality segment timestamps.

[0102] 2. Downstream application scenarios

[0103] Data pricing: Develop differentiated pricing strategies based on the three-level evaluation indicators (e.g., data with high consistency and low reconstruction loss are priced higher);

[0104] AI training: used for model training (such as vehicle-road collaboration prediction models and health monitoring algorithms) to improve the accuracy of downstream tasks;

[0105] Storage optimization: Eliminate low-quality data to reduce storage costs and transmission bandwidth usage.

[0106] Example 2

[0107] Reference Figure 2-4 , an IoT data quality screening system based on multi-level intelligent evaluation, including:

[0108] The energy distribution screening module is used to receive IoT data streams, divide the data into windows, calculate the energy value of each window, and screen high-energy samples based on the dynamically adjusted energy threshold. The energy distribution screening module includes an energy calculation unit and a dynamic threshold adjustment unit.

[0109] The time series consistency screening module performs frequency domain analysis on the samples after the initial screening, extracts the main frequency band energy and the harmonic frequency band energy, calculates the consistency index, and screens out samples whose information consistency is higher than the preset threshold; the time series consistency screening module includes a frequency domain analysis unit and a consistency index calculation unit;

[0110] The variational autoencoder quality assessment module uses a pre-trained variational autoencoder model to calculate the reconstruction loss of the samples after secondary screening, and screens out high-quality data with a reconstruction loss below a set threshold; the variational autoencoder quality assessment module includes a variational autoencoder model loading unit and a reconstruction loss judgment unit.

[0111] The IoT data streams mentioned above include at least one of the following types:

[0112] Video streams collected by low-altitude drones;

[0113] Vehicle-road cooperative sensor time series data;

[0114] Physiological signal data from the user's wearable device.

[0115] Interaction process of system modules:

[0116] Data preprocessing - energy distribution screening module: The preprocessed data is input into the screening module, and the high-energy candidate set is output through energy calculation and threshold adjustment.

[0117] Energy distribution initial screening module - time series consistency screening module: high-energy samples enter the frequency domain analysis, and highly regular samples are screened out through consistency index calculation.

[0118] Temporal consistency screening module - Variational autoencoder quality assessment module: Pre-train VAE with highly regular sample input and filter out semantically complete, high-quality data through reconstruction loss.

[0119] Variational Autoencoder Quality Assessment Module - Output: The final high-quality dataset enters the application layer or is fed back to the preprocessing module to optimize the input adaptation strategy.

[0120] Heterogeneous adaptation: The preprocessing module unifies the data format, and the dynamic threshold adjustment adapts to device differences.

[0121] Capturing regularity: Frequency domain analysis combined with triple assumptions effectively identifies the signal's main frequency and harmonics, and screens periodic / regular data.

[0122] Semantic-level verification: VAE reconstructs data through unsupervised learning and removes noise from the high-dimensional feature level.

[0123] Through the above process, this solution realizes the automation of the entire process from data access to quality screening, forming a multi-level intelligent evaluation system of "coarse screening-fine screening-semantic verification", effectively solving the screening problems caused by the massive heterogeneity of IoT data.

[0124] Example 3

[0125] The difference between this embodiment and the first embodiment is that the timing information consistency screening method is replaced by a timing similarity evaluation method based on dynamic time warping (DTW).

[0126] To meet the nonlinear alignment requirements of time series data, the DTW algorithm is used to calculate the similarity distance between the sample and the template signal, replacing the frequency domain energy ratio analysis. It is suitable for consistency screening of non-stationary signals (such as physiological signals of wearable devices and sudden motion video streams of drones).

[0127] In summary, compared with the existing technology, it has the following beneficial effects:

[0128] Through iterative algorithms, data energy distribution is automatically analyzed and screening thresholds are dynamically determined without the need for manual preset rules. This adapts to energy heterogeneity under different device models, sampling frequencies, and transmission conditions. Thresholds are regularly updated in real-time data stream scenarios to adapt to environmental changes and avoid missing high-value data or misjudgment of noise.

[0129] By introducing frequency domain energy distribution analysis and using triple hypotheses to screen samples with strong time series regularity, we can effectively identify periodic signals and make up for the defect that traditional time domain threshold filtering cannot capture the frequency domain characteristics of signals. By training the model with unlabeled data, we can use the characteristics of high-dimensional semantics that are easy to reconstruct and noise that is difficult to reconstruct to screen high-quality data. This method is universal for various time series modes of the Internet of Things, suitable for all scenarios of the Internet of Things, and reduces the cost of cross-task adaptation.

[0130] The initial energy screening quickly filters low-energy noise, reducing the amount of subsequent data processing. The temporal consistency screening focuses on frequency domain feature analysis to avoid deep processing of the entire data. The VAE evaluation only performs high-dimensional semantic verification on the candidate samples after the secondary screening, reducing the overall computational overhead.

[0131] A three-level evaluation indicator system is constructed to quantify data quality layer by layer from underlying signal energy to high-level semantic features, providing a unified evaluation framework for structured and unstructured data.

[0132] Thus, although the invention has been described herein with reference to specific embodiments thereof, freedom of modification, various changes and substitutions are contemplated within the foregoing disclosure, and it should be understood that in some cases, some features of the invention will be employed without the corresponding use of other features without departing from the scope and spirit of the claimed invention. Thus, many modifications may be made to adapt a particular environment or material to the true scope and spirit of the invention. The invention is not intended to be limited to the specific terminology used in the claims below and / or to the specific embodiments disclosed as the best mode contemplated for carrying out the invention, but the invention is intended to include any and all embodiments and equivalents falling within the scope of the appended claims. Thus, the scope of the invention will be determined solely by the appended claims.

Claims

1. The IoT data quality screening method based on multi-level intelligent evaluation is characterized by: The following steps are involved: (1) Sample screening based on data energy distribution: divide the input IoT data stream into windows, calculate the energy value of each window, dynamically adjust the screening threshold according to the energy distribution, and retain high-energy samples; (2) Sample screening based on consistency of time series information: Perform frequency domain analysis on the samples after initial screening, determine the main frequency and harmonic frequency bands in the samples through signal analysis, calculate the proportion of their energy to the overall energy of the samples, calculate the consistency index, and screen out samples with a ratio higher than the preset threshold; (3) High-quality evaluation based on variational autoencoder: The samples after secondary screening are input into the pre-trained variational autoencoder model, the reconstruction loss is calculated, and the samples with loss below the threshold are retained as high-quality data.

2. The method for screening Internet of Things data quality based on multi-level intelligent evaluation according to claim 1 is characterized in that: In step (1), the energy value is calculated as follows: Among them, x i is the i-th sampling point, and N is the window size.

3. The method for screening Internet of Things data quality based on multi-level intelligent evaluation according to claim 1 is characterized in that: In step (2), the calculation formula of the consistency index is: Among them E main is the main frequency band energy, E co is the harmonic band energy, E total is the total energy of the sample.

4. The method for screening Internet of Things data quality based on multi-level intelligent evaluation according to claim 1 is characterized in that: The specific steps of dynamically adjusting the screening threshold in step (1) are as follows: S1: Data Windowing and Energy Calculation: Divide the IoT data stream into multiple data segments according to a fixed window size, and calculate the energy value for each window; S2: Construct an energy distribution histogram: Count the energy value distribution of all windows and generate an energy histogram. The horizontal axis is the energy interval and the vertical axis is the number of windows in the corresponding interval. S3: Iterative threshold determination: Set the initial threshold T0 based on the global mean or median of the histogram and calculate the current threshold T k The proportion of high energy windows, that is, energy value ≥ T k The ratio of the number of windows to the total windows. If the ratio is lower than the preset target value, the threshold is lowered. If the ratio is higher than the target value, the threshold is increased. Repeat the iteration until the error between the ratio and the target value is less than the tolerance range. S4: Real-time dynamic adaptation: For real-time incoming data, the energy distribution statistics are regularly updated and the thresholds are recalculated to adapt to changes in the data collection environment.

5. The method for screening Internet of Things data quality based on multi-level intelligent evaluation according to claim 1 is characterized in that: In the step (2), the signal analysis means includes Fourier transform, which is used to convert the time domain signal into the frequency domain signal and identify the main frequency band and the harmonic frequency band.

6. The method for screening Internet of Things data quality based on multi-level intelligent evaluation according to claim 1 is characterized in that: In step (2), the frequency domain analysis is based on the following assumptions: The main frequency band exists within a preset frequency range; The harmonic band is the harmonic component with the largest energy; The total energy of the main frequency and harmonics is significantly higher than that of the noise frequency band.

7. The method for screening Internet of Things data quality based on multi-level intelligent evaluation according to claim 1, characterized in that: In step (3), the variational autoencoder model is trained through self-supervised learning, and the training data includes unlabeled samples of multiple IoT modalities.

8. The IoT data quality screening system based on multi-level intelligent evaluation is characterized by: include: The energy distribution screening module receives IoT data streams, divides the data into windows, calculates the energy value of each window, and screens high-energy samples based on a dynamically adjusted energy threshold. The time series consistency screening module performs frequency domain analysis on the samples after the initial screening, extracts the energy of the main frequency band and the energy of the harmonic frequency band, calculates the consistency index, and screens samples whose information consistency is higher than the preset threshold; The variational autoencoder quality assessment module uses a pre-trained variational autoencoder model to calculate the reconstruction loss of the samples after secondary screening, and selects high-quality data with a reconstruction loss lower than the set threshold.

9. The Internet of Things data quality screening system based on multi-level intelligent evaluation according to claim 8 is characterized in that: The energy distribution primary screening module includes an energy calculation unit and a dynamic threshold adjustment unit.

10. The Internet of Things data quality screening system based on multi-level intelligent evaluation according to claim 8, characterized in that: The timing consistency screening module includes a frequency domain analysis unit and a consistency index calculation unit.

Images