An internet of things big data fusion processing platform

Abstract

The application belongs to the technical field of data processing, and discloses a big data fusion processing platform of Internet of Things; comprising: a data acquisition module, which is used for collecting operation data of Internet of Things equipment to obtain Internet of Things data; a data transmission module, which is used for transmitting the Internet of Things data to a data preprocessing module; the data preprocessing module, which is used for receiving the Internet of Things data transmitted by the data transmission module, and preprocessing the Internet of Things data to obtain perfect Internet of Things data; a data analysis module, which is used for constructing an Internet of Things database to store the perfect Internet of Things data, and performing data analysis on the perfect Internet of Things data to obtain an optimal operation parameter combination; the optimal operation parameter combination is optimized by using an optimization algorithm to obtain a final operation parameter combination; and the Internet of Things equipment is controlled and optimized based on the final operation parameter combination; the platform improves the operation efficiency of the Internet of Things equipment and reduces the operation cost.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to an Internet of Things (IoT) big data fusion processing platform. Background Technology

[0002] Patent application CN111787066A discloses an IoT data platform based on big data and AI, including a data acquisition node server cluster (Godzilla-Agent), a relay node server cluster (Godzilla-Broker), a central coordination node server cluster (Godzilla-Coordinator), a data parsing node server cluster (Godzilla-Processor), an API open platform server cluster (Godzilla-API), an alarm notification server cluster (Godzilla-Notifier), an AI server cluster (Godzilla-AI), and a platform management server cluster (Godzilla-Admin). The system is compatible with IoT terminal devices using various communication protocols, can quickly access point data, and combines data from different dimensions for fusion processing. It links IoT data from different systems at the underlying level to form new data samples for machine learning and AI decision analysis, opening up data channels for multi-system integration of IoT, increasing the data correlation between independent systems, providing data support for upper-layer business systems, and forming an integrated solution for IoT data acquisition, storage, processing, and AI decision-making.

[0003] In the realm of existing data processing technologies, it is difficult to guarantee the accuracy and integrity of data transmission. Using traditional encoding methods to encode data can easily lead to excessively large data packets and high transmission costs during data packaging. During data transmission, factors such as network latency and packet loss can easily result in incorrect or missing data. Traditional preprocessing methods lack consideration for different data types, and using the same preprocessing method leads to low data processing efficiency and poor data quality after preprocessing. There is a lack of suitable data analysis methods for IoT data, particularly for IoT devices in industrial and transportation sectors that require rapid response. Furthermore, the operating environment of IoT devices is mostly dynamic, and in such cases, the operating parameters of the devices often cannot be adjusted in a timely manner, resulting in low operating efficiency.

[0004] In view of this, the present invention proposes an Internet of Things big data fusion processing platform to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: an Internet of Things (IoT) big data fusion processing platform, comprising:

[0006] The data acquisition module is used to collect operational data from IoT devices to obtain IoT data.

[0007] The data transmission module is used to transmit IoT data to the data preprocessing module;

[0008] The data preprocessing module is used to receive IoT data transmitted by the data transmission module and preprocess the IoT data to obtain complete IoT data.

[0009] The data analysis module is used to build an IoT database to store complete IoT data, and to analyze the complete IoT data to obtain the optimal combination of operating parameters; it uses optimization algorithms to optimize the optimal combination of operating parameters to obtain the final combination of operating parameters; and it optimizes the control of IoT devices based on the final combination of operating parameters; the various modules are connected to each other via wired and / or wireless means.

[0010] Furthermore, IoT data is obtained by collecting operational data from sensors in IoT devices; feature codes are added to the IoT data, and the IoT data is matched with IoT devices; the IoT data includes time-series data and visualized data.

[0011] Furthermore, the method of transmitting IoT data to the data preprocessing module includes:

[0012] Define data packets;

[0013] Arrange the time-series data in chronological order to obtain a time-series data sequence; compress the visualization data to obtain compressed visualization data;

[0014] The data packet includes a header, a time-series data sequence, compressed and visualized data, a data checksum, and a packet trailer;

[0015] The process involves several steps: First, the time-series data sequence is encoded to obtain the time-series data code. Second, the compressed visualization data is encoded to obtain the visualization data code. Third, the time-series data code and the visualization data code are packaged into a data packet. Before sending the data packet, a data checksum is calculated. This checksum is then used as the sending checksum, and the data packet is included in the data packet by the data transmission module. When the preprocessing module receives the data packet, it recalculates the data checksum. This recalculation becomes the receiving checksum. The sending checksum is compared with the receiving checksum. If they match, the transmitted IoT data is complete; otherwise, it is incomplete, and the data transmission module retransmits the data. Finally, the preprocessing module unpacks the received data packet, extracts the time-series data code and the visualization data code, and restores them to the time-series data sequence and the compressed visualization data, respectively.

[0016] Furthermore, the method of encoding the time-series data sequence includes:

[0017] The time-series data sequence is standardized to obtain a standard time-series data sequence;

[0018] The formula for standardization is:

[0019] Among them, X i This represents the i-th data point in a standard temporal data sequence. X represents the i-th data point in a time-series data sequence; max X represents the maximum value in a time-series data sequence; min Represents the minimum value in a time-series data sequence;

[0020] Define the i-th data X in a standard temporal data sequence i In the standard time-series data sequence, each digit after the decimal point is denoted as num. A sequence of digits containing num zeros and ending with a 1 is used as the encoding for num. This encoding process is then applied to each digit after the decimal point to obtain the i-th data element X. i Data encoding; encode all data in a standard time-series data sequence to obtain the time-series data encoding.

[0021] Furthermore, the method of encoding the compressed visualization data includes:

[0022] P(x0,y0) = [R(x0,y0),G(x0,y0),B(x0,y0)]; where R(x0,y0) represents the red channel intensity of pixel (x0,y0); G(x0,y0) represents the green channel intensity of pixel (x0,y0); B(x0,y0) represents the blue channel intensity of pixel (x0,y0); and P(x0,y0) represents the pixel value of any pixel (x0,y0) in any image of the compressed visualization data.

[0023] Extract the red channel intensity, green channel intensity, and blue channel intensity of each pixel, and represent the pixel value of each pixel by forming a three-dimensional vector that matches the corresponding pixel.

[0024] The pixel values ​​of all pixels are combined into a W×3 matrix, which is the pixel matrix; where W represents the number of pixels in any image of the compressed visualization data. The pixel matrix is ​​then centered to obtain a centered pixel matrix. A pixel covariance matrix is ​​constructed based on the pixel matrix and the centered pixel matrix. The pixel covariance matrix is ​​then orthogonally transformed to obtain an orthogonal pixel covariance matrix. The eigenvalues ​​and eigenvectors of each element in the orthogonal covariance matrix are calculated, and the top eigenvalues ​​and eigenvectors are selected. The eigenvectors corresponding to the largest eigenvalues ​​form the feature matrix; based on the orthogonal pixel covariance matrix and the feature matrix, a projection matrix is ​​constructed, which is the encoding result, thus obtaining the visual data encoding;

[0025] Pixel covariance matrix Where V represents the pixel matrix; V center Represents a centered pixel matrix;

[0026] The projection matrix V0 = C0·T0; where C0 represents the orthogonal pixel covariance matrix; and T0 represents the feature matrix.

[0027] Methods for calculating data checksums include:

[0028] Divide the IoT data into N0 equal parts and calculate the information entropy of the IoT data. Among them, W0 j This represents the probability distribution of the j-th IoT data set;

[0029] Data checksum Where HW represents the sum of parameters for each IoT data set; A represents an adaptive adjustment function; and V0 represents environmental parameters.

[0030] Adaptive adjustment function Where ki is an adjustment coefficient; S represents the size of any data packet during IoT data transmission; S maxP0 represents the size of the largest data packet during IoT data transmission; P0 represents the sum of environmental parameters.

[0031] Furthermore, the methods for preprocessing IoT data include:

[0032] Denoising the time-series data sequence yields denoised time-series data; filtering the compressed visualization data yields filtered visualization data; combining the denoised time-series data and the filtered visualization data yields complete IoT data.

[0033] Furthermore, the method for denoising the time-series data sequence includes:

[0034] Remove outliers from the time-series data sequence and fill in missing data to obtain coarse-processed time-series data;

[0035] Define wavelet detail coefficients;

[0036] A pre-defined wavelet basis function is used, and an adaptive layering function is employed to adaptively select the number of wavelet packet decomposition layers, resulting in the wavelet decomposition layer number. Based on the wavelet basis function and the wavelet decomposition layer number, coarsely processed time-series data is decomposed using wavelet decomposition, and the noise standard deviation is estimated using minimum absolute deviation. A threshold is determined based on the noise standard deviation, and the wavelet detail coefficients of each layer are thresholded to obtain denoised detail coefficients. Based on the denoised detail coefficients, the signal is reconstructed through inverse wavelet transform to obtain denoised time-series data.

[0037] The method of adaptively selecting the number of layers in wavelet packet decomposition using an adaptive layering function includes:

[0038] Define the length of the coarse-processed temporal data and the filter length of the wavelet basis function, and calculate the maximum number of decomposition layers based on the length of the coarse-processed temporal data and the filter length of the wavelet basis function;

[0039] Calculate the energy of the wavelet detail coefficients in layer l; sum the energies of the wavelet detail coefficients in the first l layers to obtain the total energy of the wavelet detail coefficients in the first l layers; sum the energies of the wavelet detail coefficients in all layers to obtain the total energy of the wavelet detail coefficients in all layers; calculate the cumulative energy ratio based on the total energy of the wavelet detail coefficients in the first l layers and the total energy of the wavelet detail coefficients in all layers, and determine the adaptive decomposition layer based on the cumulative energy ratio; obtain the wavelet decomposition layer based on the maximum decomposition layer and the adaptive decomposition layer.

[0040] Maximum number of decomposition levels Where N1 represents the length of the coarse-processed temporal data; length represents the filter length of the wavelet basis function;

[0041] The formula for calculating the energy of the wavelet detail coefficients of the l-th layer is:

[0042] Among them, EJ l Len represents the energy of the wavelet detail coefficients of the l-th layer. l cD represents the length of the wavelet detail coefficients of the l-th layer. l cA represents the wavelet detail coefficients of the l-th layer; l Represents the wavelet approximation coefficients of the l-th layer;

[0043] Cumulative energy percentage Among them, E total E(cA) represents the total energy of the wavelet detail coefficients across all layers. l ) represents the energy of the wavelet approximation coefficients of the l-th layer; This represents the total energy of the wavelet detail coefficients in the first l layers;

[0044] If the cumulative energy percentage EB is greater than or equal to the preset cumulative energy percentage threshold, then the current layer number l is taken as the adaptive decomposition layer number; the minimum value between the adaptive decomposition layer number and the maximum decomposition layer number is selected as the wavelet decomposition layer number.

[0045] The method for thresholding the wavelet detail coefficients of each layer includes:

[0046] An adaptive threshold is determined for the wavelet detail coefficients of each layer based on the noise standard deviation, and the wavelet detail coefficients of each layer are processed using a soft thresholding method based on the adaptive threshold to obtain the denoised detail coefficients.

[0047] The formula for determining the adaptive threshold is:

[0048] Where, λ l σ represents the adaptive threshold of the l-th layer; σ represents the noise standard deviation; σ l This represents the local variance of the wavelet detail coefficients of the l-th layer.

[0049] Furthermore, the method for filtering the compressed visualization data includes:

[0050] Initialize the filter;

[0051] The compressed visualization data is sharpened using a sharpening function to obtain sharpened visualization data. An automatic grayscale adjustment framework is constructed, which includes: initializing the framework parameters, including the gamma value γ and the dynamic adjustment coefficient f; dynamically adjusting the grayscale value of any pixel in the sharpened visualization data by adjusting these parameters; and filtering the sharpened visualization data based on a filter and the automatic grayscale adjustment framework to obtain filtered visualization data.

[0052] The formula for sharpening compressed and visualized data is as follows:

[0053] Sharpening function Where PB represents the pixel value of any pixel in the compressed visualization data; This represents the contrast adjustment function; α represents the sharpening intensity coefficient. Represents the Laplace operator;

[0054] Contrast adjustment function a represents the contrast control coefficient; b represents the brightness control coefficient; zscore(PB) represents the pixel value after normalization of any pixel in the compressed visualization data.

[0055] The mechanism for setting up an automatic grayscale value adjustment framework is as follows: if the grayscale value of any pixel in the sharpened visualization data is lower than or higher than the preset grayscale value range, then the gamma value γ and the dynamic adjustment coefficient f are adjusted to perform a gamma transformation on the sharpened visualization data, so as to maintain the grayscale value of any pixel in the sharpened visualization data within the preset grayscale value range.

[0056] The formula for calculating the gamma transform is: Among them, Gr L This represents the grayscale value of any pixel in the sharpened visualization data after gamma transformation; Gr B This represents the grayscale value of any single pixel in the original sharpened visualization data.

[0057] The formula for filtering sharpened visualization data is as follows:

[0058] Where PL represents the pixel value of any pixel in the filtered visualization data; ω r Represents grayscale weight; ω s Represents spatial weights; PR(m,n) represents the pixel value of the pixel located at (m,n) in the sharpened visualization data;

[0059] Grayscale weight Where, σ r Gr represents the standard deviation of the Gaussian function. B (m,n) represents the grayscale value of the pixel located at (m,n) in the original sharpened visualization data;

[0060] Spatial weight Where m represents the x-coordinate of a pixel and n represents the y-coordinate of a pixel.

[0061] Furthermore, the methods for constructing an IoT database to store comprehensive IoT data include:

[0062] Define primary keys, foreign keys, and indexes;

[0063] The IoT database includes an IoT device ID data table, a time-series data table, and a visualization data table.

[0064] Define the IoT device ID, the denoised temporal data number, and the filtered visualization data number;

[0065] Based on the feature code, the IoT device ID is matched with the denoised temporal data number and the filtered visualization data number;

[0066] Construct an IoT device ID data table, including IoT device ID and feature code; use IoT device ID as the primary key of the IoT device ID data table; use feature code as a foreign key of the IoT device ID data table, pointing to the time-series data table and the visualization data table;

[0067] Construct a time-series data table, including IoT device ID, denoised time-series data number, time-series data type, time-series data measurement value, and time-series data timestamp; use the denoised time-series data number and time-series data timestamp as the primary key of the time-series data table; use the IoT device ID as the foreign key of the time-series data table, pointing to the IoT device ID data table;

[0068] Construct a visualization data table, including IoT device ID, filtered visualization data number, visualization data resolution, and visualization data storage path; use the filtered visualization data number as the primary key of the visualization data table; use the IoT device ID as the foreign key of the visualization data table, pointing to the IoT device ID data table;

[0069] The improved IoT data will be imported into an IoT database for storage, and an index will be added to the IoT database. The index includes IoT device ID, feature code, denoised time-series data number, and filtered visualization data number.

[0070] Furthermore, the methods for performing data analysis on improved IoT data include:

[0071] Construct a neural network model, using deep neural networks as the model framework;

[0072] Methods for constructing neural network models include:

[0073] Historical IoT data is collected and preprocessed to obtain complete historical IoT data; machine learning algorithms are used to process the complete historical IoT data to obtain excellent IoT device operating parameters; and the excellent IoT device operating parameters are used as training labels.

[0074] The historical complete IoT data is normalized to obtain normalized IoT data, and the normalized IoT data is used as the IoT data training set.

[0075] Define a loss function, train the neural network model using IoT data training set, perform forward propagation and calculate the function value of the loss function; calculate the gradient of the weight and threshold of each neuron in the neural network through backpropagation, and adjust the weight and threshold according to the gradient until the function value of the loss function no longer decreases. At this point, fix the parameters to obtain the trained neural network model.

[0076] The trained neural network model is used to process and refine IoT data to obtain the optimal combination of operating parameters for IoT devices.

[0077] The method of optimizing the optimal combination of operating parameters using optimization algorithms includes:

[0078] Initialize the population, with a preset population size of To, and any individual F in the population. q Let represent an optimal combination of operating parameters, where q = 1, 2, 3, ..., To;

[0079] Define fitness function Among them, EN(F q ) indicates the use of the running parameter combination F q Energy consumption generated by IoT devices; TE(F) q ) indicates the use of the running parameter combination F q The response time of IoT devices; δ represents the penalty coefficient; ε represents the population coefficient; q represents the number of individuals;

[0080] The optimal individual in the population is selected using the roulette wheel selection method; any individual in the population other than the optimal individual is randomly selected and cross-crossed with the optimal individual to obtain the first and second new individuals.

[0081] New Individual No. 1

[0082] New Individual No. 2

[0083] Where no represents the number of parameters in any individual; z represents the exchange of parameters between the two individuals at point z; F1 q [1],F1 q [2],...,F2 q [no] represents the first new individual. Each running parameter in F2; q [1],F2 q [2],...,,F1 q [no] represents the second new individual. Each running parameter in;

[0084] Mutation is performed on new individuals 1 and 2. Any parameter of new individual 1 is randomly selected and its value is randomly changed to obtain new individual 1. Any parameter of new individual 2 is randomly selected and its value is randomly changed to obtain new individual 2. The fitness function values ​​of new individuals 1 and 2 are calculated respectively. The individual with the largest value is taken as the best individual in the population. The crossover and mutation processes are repeated on the best individual in the population until the preset number of iterations is reached. The best individual in the population at this time is the final combination of running parameters.

[0085] The technical effects and advantages of the IoT big data fusion processing platform of the present invention are as follows:

[0086] By collecting operational data from IoT devices, a platform for efficient IoT data processing is created. This platform utilizes a customized data transmission protocol for transmission, performs meticulous preprocessing, efficiently stores the data, and employs optimization algorithms to intelligently optimize IoT devices. Compared to existing experience, the customized data transmission protocol makes data transmission more secure and efficient, ensuring data integrity and accuracy. Noise reduction and filtering improve the accuracy of the IoT data, facilitating subsequent processing. A database stores the data, with corresponding tables for different data types, enabling rapid retrieval and improving database query efficiency. Finally, by using models to process and analyze the data, and optimizing IoT devices based on the analysis results, the platform improves device operating efficiency, indirectly increasing the company's economic benefits. Attached Figure Description

[0087] Figure 1 This is a schematic diagram of an IoT big data fusion processing platform according to the present invention;

[0088] Figure 2 This is a schematic diagram of an IoT big data fusion processing method according to the present invention. Detailed Implementation

[0089] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0090] Example 1

[0091] Please see Figure 1 As shown in this embodiment, an IoT big data fusion processing platform includes:

[0092] The data acquisition module is used to collect operational data from IoT devices to obtain IoT data.

[0093] The data transmission module is used to transmit IoT data to the data preprocessing module;

[0094] The data preprocessing module is used to receive IoT data transmitted by the data transmission module and preprocess the IoT data to obtain complete IoT data.

[0095] The data analysis module is used to build an IoT database to store complete IoT data, and to analyze the complete IoT data to obtain the optimal combination of operating parameters; it uses optimization algorithms to optimize the optimal combination of operating parameters to obtain the final combination of operating parameters; and it optimizes the control of IoT devices based on the final combination of operating parameters; the various modules are connected to each other via wired and / or wireless means.

[0096] IoT data is obtained by collecting operational data from sensors in IoT devices. Feature codes (character sequences that match IoT devices with their generated data) are added to this IoT data, allowing for matching between the IoT data and the IoT devices. The sensors include temperature, humidity, pressure, air pressure, and speed sensors. These sensors convert changes in temperature and humidity into electrical signals that the sensors can recognize and transmit these signals back to the sensors, resulting in corresponding operational data. The IoT data includes time-series data and visualized data. Time-series data represents a set of data points measured by the sensors of the IoT devices and arranged in chronological order, such as temperature, humidity, air pressure, and speed, which change over time. Visualized data represents data generated by the IoT devices that can be represented visually, such as waveforms, tables, bar charts, and line graphs.

[0097] The method of transmitting IoT data to the data preprocessing module includes:

[0098] Define a data packet; a data packet represents the basic transmission unit in the network communication field and contains certain important information (such as destination address, source address, and data packet length) to ensure that the data can be received correctly.

[0099] Arranging time-series data in chronological order yields a time-series data sequence; compressing visualization data yields compressed visualization data; compressed visualization data means reducing the resolution and size of the original visualization data through compression, facilitating data transmission, reducing data transmission costs, and improving data transmission efficiency.

[0100] A data packet includes a header, a time-series data sequence, compressed and visualized data, a data checksum, and a packet trailer. The header contains essential information about the data packet, such as the protocol type, sequence number, and port number, which describe the characteristics and attributes of the data packet. The data checksum is used to detect whether errors occur during data transmission, ensuring the integrity and correctness of the data. The packet trailer contains additional essential information besides that included in the header.

[0101] The process involves encoding the time-series data sequence to obtain time-series data encoding; encoding the compressed visualization data to obtain visualization data encoding; packaging the time-series data encoding and visualization data encoding into a data packet; adding a timestamp to the time-series data encoding to match the time-series data encoding with the timestamp (to ensure that the order of the time-series data is not disrupted during transmission); adding a separator between the time-series data encoding and visualization data encoding to distinguish the two data types; and calculating a data checksum (a value calculated using a specific method to detect whether errors have occurred during data transmission) before sending the data packet. The checksum is the sent data checksum, which is included in the data packet and sent by the data transmission module. When the preprocessing module receives the data packet, it recalculates the data checksum, which is now the received data checksum. The sent data checksum is compared with the received data checksum. If they are the same, the transmitted IoT data is complete; otherwise, it is incomplete, and the data transmission module retransmits it. The preprocessing module unpacks the received data packet, extracts the temporal data encoding and visualization data encoding from the data packet, and restores the temporal data encoding and visualization data encoding to temporal data sequence and compressed visualization data, respectively.

[0102] The methods for encoding time-series data sequences include:

[0103] The time-series data sequence is standardized to obtain a standard time-series data sequence;

[0104] The formula for standardization is:

[0105] Among them, X i This represents the i-th data point in a standard temporal data sequence. x represents the i-th data point in a time-series data sequence; maxX represents the maximum value in a time-series data sequence; min Represents the minimum value in a time-series data sequence;

[0106] Define the i-th data X in a standard temporal data sequence i In this example, each digit after the decimal point is denoted as num. A sequence of digits containing num zeros and ending with a 1 is used as the encoding for num (e.g., if 0.123 is a data point in a standard time-series data sequence, its encoding is 010010001). The same encoding process is then applied to each digit after the decimal point to obtain the i-th data point X in the standard time-series data sequence. i Data encoding; Encoding all data in a standard time-series data sequence to obtain time-series data encoding;

[0107] The methods for encoding compressed visualization data include:

[0108] P(x0,y0) = [R(x0,y0),G(x0,y0),B(x0,y0)]; where R(x0,y0) represents the red channel intensity of pixel (x0,y0); G(x0,y0) represents the green channel intensity of pixel (x0,y0); B(x0,y0) represents the blue channel intensity of pixel (x0,y0); and P(x0,y0) represents the pixel value of any pixel (x0,y0) in any image of the compressed visualization data.

[0109] Extract the red channel intensity, green channel intensity, and blue channel intensity of each pixel, and represent the pixel value of each pixel by forming a three-dimensional vector that matches the corresponding pixel.

[0110] The pixel values ​​of all pixels are combined into a W×3 matrix, which is the pixel matrix; where W represents the number of pixels in any image of the compressed visualization data. The pixel matrix is ​​then centered (each element in the first column is subtracted from the mean intensity of the red channel, each element in the second column from the mean intensity of the green channel, and each element in the third column from the mean intensity of the blue channel), resulting in a centered pixel matrix. A pixel covariance matrix is ​​constructed based on the pixel matrix and the centered pixel matrix. The pixel covariance matrix is ​​then orthogonally transformed to obtain an orthogonal pixel covariance matrix. The eigenvalues ​​and eigenvectors of each element in the orthogonal covariance matrix are calculated, and the top eigenvalues ​​and eigenvectors are selected. The eigenvectors corresponding to the largest eigenvalues ​​form the feature matrix; based on the orthogonal pixel covariance matrix and the feature matrix, a projection matrix is ​​constructed, which is the encoding result, thus obtaining the visual data encoding;

[0111] Pixel covariance matrix Where V represents the pixel matrix; V center Represents a centered pixel matrix;

[0112] The projection matrix V0 = C0·T0; where C0 represents the orthogonal pixel covariance matrix; and T0 represents the feature matrix.

[0113] Methods for calculating data checksums include:

[0114] Divide the IoT data into N0 equal parts and calculate the information entropy of the IoT data. Among them, W0 j This represents the probability distribution of the j-th IoT data set;

[0115] Data checksum Where HW represents the sum of parameters for each piece of IoT data (for time-series data sequences, the sum of parameters refers to the sum of values ​​of several data points; for compressed visualization data, the sum of parameters refers to the sum of pixel values ​​of several pixels); A represents an adaptive adjustment function; V0 represents environmental parameters (such as network latency, network packet loss rate);

[0116] Adaptive adjustment function Where ki is an adjustment coefficient (ki is a variable constant, ki∈(-∞,+∞)); S represents the size of any data packet in the IoT data transmission process; S max P0 represents the size of the largest data packet during IoT data transmission; P0 represents the sum of environmental parameters.

[0117] By adding an adaptive adjustment function to the data checksum, the computational complexity of the function is adjusted according to the value of the adjustment coefficient, and a value that varies with environmental parameters and packet size is output, thereby reducing the impact of network conditions on the data checksum and improving the accuracy of the data checksum.

[0118] The methods for preprocessing IoT data include:

[0119] The time-series data sequence is denoised to obtain denoised time-series data. The denoising method is a denoising algorithm. The compressed visualization data is filtered to obtain filtered visualization data. The filtering method is to use a filter. The denoised time-series data and the filtered visualization data are combined to obtain complete IoT data.

[0120] Methods for denoising time-series data sequences using denoising algorithms include:

[0121] Remove outliers from the time-series data sequence (e.g., use spline interpolation to handle outliers) and fill in missing data in the time-series data sequence to obtain coarse-processed time-series data;

[0122] Define wavelet detail coefficients; each wavelet transform generates a new set of wavelet detail coefficients to represent certain local features of the data;

[0123] A pre-defined wavelet basis function (e.g., Daubechies function) is used, and an adaptive layering function is employed to adaptively select the number of wavelet packet decomposition layers, resulting in the wavelet decomposition layer number. Based on the wavelet basis function and the wavelet decomposition layer number, coarsely processed time-series data is decomposed using wavelet decomposition, and the noise standard deviation is estimated using minimum absolute deviation. A threshold is determined based on the noise standard deviation, and the wavelet detail coefficients of each layer are thresholded to obtain denoised detail coefficients. Based on the denoised detail coefficients, the signal is reconstructed through inverse wavelet transform to obtain denoised time-series data.

[0124] The method of adaptively selecting the number of layers in wavelet packet decomposition using an adaptive layering function includes:

[0125] Define the length of the coarse-processed temporal data and the filter length of the wavelet basis function (representing the number of coefficients of the digital filter used for filtering in digital signal processing), and calculate the maximum number of decomposition layers based on the length of the coarse-processed temporal data and the filter length of the wavelet basis function;

[0126] Calculate the energy of the wavelet detail coefficients in layer l; sum the energies of the wavelet detail coefficients in the first l layers to obtain the total energy of the wavelet detail coefficients in the first l layers; sum the energies of the wavelet detail coefficients in all layers to obtain the total energy of the wavelet detail coefficients in all layers; calculate the cumulative energy ratio based on the total energy of the wavelet detail coefficients in the first l layers and the total energy of the wavelet detail coefficients in all layers, and determine the adaptive decomposition layer based on the cumulative energy ratio; obtain the wavelet decomposition layer based on the maximum decomposition layer and the adaptive decomposition layer.

[0127] Maximum number of decomposition levels Where N1 represents the length of the coarse-processed temporal data; length represents the filter length of the wavelet basis function;

[0128] The formula for calculating the energy of the wavelet detail coefficients of the l-th layer is:

[0129] Among them, EJ l Len represents the energy of the wavelet detail coefficients of the l-th layer. l This represents the length of the wavelet detail coefficients of the l-th layer (the length of the wavelet detail coefficients refers to the number of elements in the wavelet detail coefficient array of that layer); cD l cA represents the wavelet detail coefficients of the l-th layer; l This represents the wavelet approximation coefficient of the l-th layer (a constant used to reflect the periodic changes in the data);

[0130] Cumulative energy percentage Among them, E total E(cA) represents the total energy of the wavelet detail coefficients across all layers. l ) represents the energy of the wavelet approximation coefficients of the l-th layer; This represents the total energy of the wavelet detail coefficients in the first l layers;

[0131] If the cumulative energy percentage EB is greater than or equal to the preset cumulative energy percentage threshold, then the current layer number l is taken as the adaptive decomposition layer number; the minimum value between the adaptive decomposition layer number and the maximum decomposition layer number is selected as the wavelet decomposition layer number.

[0132] The adaptive selection of the number of wavelet packet decomposition layers using an adaptive layering function has a significant impact on the denoising effect. If too many decomposition layers are selected, it will lead to data overfitting, with a large amount of noise being retained and poor denoising effect. If too few decomposition layers are selected, high-frequency noise cannot be removed, resulting in incomplete denoising. Selecting an appropriate number of decomposition layers is beneficial to improving the denoising effect and facilitating subsequent processing.

[0133] The method for thresholding the wavelet detail coefficients of each layer includes:

[0134] An adaptive threshold is determined for the wavelet detail coefficients of each layer based on the noise standard deviation. Then, based on the adaptive threshold, a soft thresholding method is used to process the wavelet detail coefficients of each layer (for wavelet detail coefficients whose absolute value is less than the adaptive threshold, they are set to zero; for wavelet detail coefficients whose absolute value is greater than the adaptive threshold, the adaptive threshold is subtracted from the wavelet detail coefficient to reduce the risk of overfitting) to obtain the denoised detail coefficients.

[0135] The formula for determining the adaptive threshold is:

[0136] Where, λ l σ represents the adaptive threshold of the l-th layer; σ represents the noise standard deviation; σ l The local variance of the wavelet detail coefficients in the l-th layer is represented.

[0137] The methods for filtering compressed visualization data using filters include:

[0138] Initialize the filter;

[0139] The compressed visualization data is sharpened using a sharpening function to obtain sharpened visualization data. An automatic grayscale adjustment framework is constructed, which includes: initializing the framework parameters, including the gamma value γ and the dynamic adjustment coefficient f; dynamically adjusting the grayscale value of any pixel in the sharpened visualization data by adjusting these parameters; and filtering the sharpened visualization data based on a filter and the automatic grayscale adjustment framework to obtain filtered visualization data.

[0140] The formula for sharpening compressed and visualized data is as follows:

[0141] Sharpening function Where PB represents the pixel value of any pixel in the compressed visualization data; This represents the contrast adjustment function, used to adjust the contrast of compressed visualization data; α represents the sharpening intensity coefficient (used to control the degree of sharpening processing); Represents the Laplace operator;

[0142] Contrast adjustment function a represents the contrast control coefficient (when a<1, the contrast is reduced; when a>1, the contrast is increased); b represents the brightness control coefficient (b>0); zscore(PB) represents the pixel value after normalization of any pixel in the compressed visualization data.

[0143] The mechanism for setting up an automatic grayscale adjustment framework is as follows: if the grayscale value of any pixel in the sharpened visualization data is lower than or higher than the preset grayscale value range, then a gamma transform is performed on the sharpened visualization data by adjusting the gamma value γ and the dynamic adjustment coefficient f, so that the grayscale value of any pixel in the sharpened visualization data is maintained within the preset grayscale value range; gamma transform is often used in the field of image processing to enhance the visual effect of an image by adjusting its grayscale values;

[0144] The formula for calculating the gamma transform is: Among them, Gr L This represents the grayscale value of any pixel in the sharpened visualization data after gamma transformation; Gr B This represents the grayscale value of any single pixel in the original sharpened visualization data.

[0145] The formula for filtering sharpened visualization data is as follows:

[0146] Where PL represents the pixel value of any pixel in the filtered visualization data; ω r Represents grayscale weight; ω sRepresents spatial weights; PR(m,n) represents the pixel value of the pixel located at (m,n) in the sharpened visualization data;

[0147] Grayscale weight Where, σ r Gr represents the standard deviation of the Gaussian function. B (m,n) represents the grayscale value of the pixel located at (m,n) in the original sharpened visualization data;

[0148] Spatial weight Where m represents the x-coordinate of the pixel and n represents the y-coordinate of the pixel;

[0149] The methods for constructing an IoT database to store comprehensive IoT data include:

[0150] Define primary keys, foreign keys, and indexes;

[0151] The IoT database includes an IoT device ID data table, a time-series data table, and a visualization data table.

[0152] Define the IoT device ID, the denoised temporal data number, and the filtered visualization data number;

[0153] Based on the feature code, the IoT device ID is matched with the denoised temporal data number and the filtered visualization data number;

[0154] Construct an IoT device ID data table, including IoT device ID and feature code; use IoT device ID as the primary key of the IoT device ID data table; use feature code as a foreign key of the IoT device ID data table, pointing to the time-series data table and the visualization data table;

[0155] Construct a time-series data table, including IoT device ID, denoised time-series data number, time-series data type (including integer, floating-point, or other types), time-series data measurement value, and time-series data timestamp (to ensure the time uniqueness of each data point); use the denoised time-series data number and time-series data timestamp as the primary key of the time-series data table (by using the denoised time-series data number and time-series data timestamp as the primary key, the denoised time-series data at a certain point in time or time period can be uniquely identified); use the IoT device ID as the foreign key of the time-series data table, pointing to the IoT device ID data table (as an identifier to distinguish different data generated by different devices);

[0156] Construct a visualization data table, including IoT device ID, filtered visualization data number, visualization data resolution, and visualization data storage path; use the filtered visualization data number as the primary key of the visualization data table; use the IoT device ID as the foreign key of the visualization data table, pointing to the IoT device ID data table;

[0157] Complete IoT data is imported into an IoT database for storage, and indexes are added to the IoT database. The indexes include IoT device IDs, feature codes, denoised time-series data numbers, and filtered visualization data numbers. These fields are frequently used when querying the IoT database, so using them as indexes improves the query efficiency of the IoT database.

[0158] The methods for intelligent control optimization of IoT devices using optimization algorithms include:

[0159] Construct a neural network model, using a deep neural network (such as a DNN) as the model framework;

[0160] Methods for constructing neural network models include:

[0161] Collect historical IoT data and preprocess it to obtain complete historical IoT data; use machine learning algorithms (such as decision tree or random forest algorithm) to process the complete historical IoT data to obtain excellent IoT device operating parameters; use the excellent IoT device operating parameters as training labels.

[0162] The historical complete IoT data is normalized to obtain normalized IoT data, and the normalized IoT data is used as the IoT data training set.

[0163] Define a loss function, train the neural network model using IoT data training set, perform forward propagation and calculate the function value of the loss function; calculate the gradient of the weight and threshold of each neuron in the neural network through backpropagation, and adjust the weight and threshold according to the gradient until the function value of the loss function no longer decreases. At this point, fix the parameters to obtain the trained neural network model.

[0164] The trained neural network model is used to process and refine IoT data to obtain the optimal combination of operating parameters for IoT devices.

[0165] The method of optimizing the optimal combination of operating parameters using optimization algorithms includes:

[0166] Initialize the population, with a preset population size of To, and any individual F in the population. q Let represent an optimal combination of operating parameters, where q = 1, 2, 3, ..., To;

[0167] Define fitness function Among them, EN(F q ) indicates the use of the running parameter combination F q Energy consumption generated by IoT devices; TE(F) q ) indicates the use of the running parameter combination F q The response time of the IoT device; δ represents the penalty coefficient (δ∈(0.5,1)); ε represents the population coefficient (ε is an integer); q represents the number of individuals;

[0168] The optimal individual in the population is selected using the roulette wheel selection method (used in genetic algorithms to select individuals); any individual in the population other than the optimal individual is randomly selected and cross-crossed with the optimal individual (the parameters of any two individuals in the population are swapped) to obtain the first new individual and the second new individual.

[0169] New Individual No. 1

[0170] New Individual No. 2

[0171] Where no represents the number of parameters in any individual; z represents the exchange of parameters between the two individuals at point z; F1 q [1],F1 q [2],...,F2 q [no] represents the first new individual. Each running parameter in F2; q [1],F2 q [2],...,,F1 q [no] represents the second new individual. Each running parameter in;

[0172] Mutation is performed on new individuals 1 and 2 (to increase population diversity and enhance the algorithm's exploration ability). Any parameter of new individual 1 is randomly selected and its value is randomly changed to obtain a new individual 1. Any parameter of new individual 2 is randomly selected and its value is randomly changed to obtain a new individual 2. The fitness function values ​​of new individuals 1 and 2 are calculated respectively. The individual with the largest value is taken as the optimal individual in the population. Crossover and mutation are repeated on the optimal individual in the population until the preset number of iterations is reached. The optimal individual obtained at this time is the final combination of running parameters.

[0173] Optimizing the control of IoT devices reduces energy consumption and improves their operating efficiency.

[0174] This embodiment collects operational data from IoT devices to obtain IoT data. It then uses a custom data transmission protocol to transmit this data, performs refined preprocessing, efficiently stores the data, and uses optimization algorithms to intelligently optimize IoT devices, forming a platform for efficient IoT data processing. Compared to existing experience, the custom data transmission protocol makes IoT data transmission more secure and efficient, ensuring the integrity and accuracy of data transmission. Noise reduction and filtering of the IoT data improves its accuracy, facilitating subsequent data processing. A database is built to store the IoT data, and corresponding data tables are designed for different data types, enabling rapid data retrieval and improving database query efficiency. By using models to process and analyze the IoT data, and optimizing IoT devices based on the data analysis results using optimization algorithms, the operating efficiency of IoT devices is improved, indirectly increasing the company's economic benefits.

[0175] Example 2

[0176] Please see Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. This embodiment provides an IoT big data fusion processing method, including:

[0177] S1. Collect operational data from IoT devices to obtain IoT data;

[0178] S2. Transmit IoT data to the data preprocessing module;

[0179] S3. Receive IoT data transmitted by the data transmission module and preprocess the IoT data to obtain complete IoT data;

[0180] S4. Construct an IoT database to store complete IoT data, and perform data analysis on the complete IoT data to obtain the optimal combination of operating parameters; use optimization algorithms to optimize the optimal combination of operating parameters to obtain the final combination of operating parameters; and optimize the control of IoT devices based on the final combination of operating parameters.

[0181] Example 3

[0182] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the operation mode of the IoT big data fusion processing method described above.

[0183] Since the electronic device described in this embodiment is an electronic device used to implement the IoT big data fusion processing method described in this application embodiment, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the IoT big data fusion processing method described in this application embodiment. Therefore, how the electronic device implements the method in this application embodiment will not be described in detail here. Any electronic device used by those skilled in the art to implement the IoT big data fusion processing method in this application embodiment falls within the scope of protection of this application.

[0184] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0185] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An Internet of Things (IoT) big data fusion processing platform, characterized in that, include: The data acquisition module is used to collect operational data from IoT devices to obtain IoT data. The data transmission module is used to transmit IoT data to the data preprocessing module; The data preprocessing module is used to receive IoT data transmitted by the data transmission module and preprocess the IoT data to obtain complete IoT data. The data analysis module is used to build an IoT database to store comprehensive IoT data and to perform data analysis on this comprehensive IoT data, including: Construct a neural network model, using deep neural networks as the model framework; Methods for constructing neural network models include: Historical IoT data is collected and preprocessed to obtain complete historical IoT data; machine learning algorithms are used to process the complete historical IoT data to obtain excellent IoT device operating parameters; and the excellent IoT device operating parameters are used as training labels. The historical complete IoT data is normalized to obtain normalized IoT data, and the normalized IoT data is used as the IoT data training set. Define a loss function, train the neural network model using IoT data training set, perform forward propagation and calculate the function value of the loss function; calculate the gradient of the weight and threshold of each neuron in the neural network through backpropagation, and adjust the weight and threshold according to the gradient until the function value of the loss function no longer decreases. At this point, fix the parameters to obtain the trained neural network model. The trained neural network model is used to process and refine IoT data to obtain the optimal combination of operating parameters for IoT devices. Optimize the optimal combination of operating parameters using optimization algorithms, including: Initialize the population, with a preset population size of [value missing]. Any individual in the population This represents an optimal combination of operating parameters, where ; Define fitness function ;in, Indicates the use of a combination of runtime parameters Energy consumption generated by IoT devices; Indicates the use of a combination of runtime parameters Response time of IoT devices; Indicates the penalty coefficient; Indicates the population coefficient; Indicates the number of individuals; The optimal individual in the population is selected using the roulette wheel selection method; any individual in the population other than the optimal individual is randomly selected and cross-crossed with the optimal individual to obtain the first and second new individuals. New Individual No. 1 ; New Individual No. 2 ; in, This represents the number of runtime parameters in any given instance; Indicating two types of individuals in Exchange parameters at the location; They represent the first new individual. Each running parameter in; These represent the second new individual. Each running parameter in; Mutation is performed on new individuals 1 and 2. Any parameter of new individual 1 is randomly selected and its value is randomly changed to obtain a new individual 1. Similarly, any parameter of new individual 2 is randomly selected and its value is randomly changed to obtain a new individual 2. The fitness function values ​​of new individuals 1 and 2 are calculated, and the individual with the largest value is selected as the optimal individual in the population. Crossover and mutation processes are repeated on the optimal individual until a preset number of iterations is reached. The optimal individual obtained at this point is the final combination of operating parameters. The control of the IoT device is optimized based on this final combination of operating parameters. All modules are connected via wired and / or wireless means.

2. The IoT big data fusion processing platform according to claim 1, characterized in that, IoT data is obtained by collecting operational data from sensors in IoT devices; feature codes are added to the IoT data, and the IoT data is matched with IoT devices; the IoT data includes time-series data and visualized data.

3. The IoT big data fusion processing platform according to claim 2, characterized in that, The method of transmitting IoT data to the data preprocessing module includes: Define data packets; Arrange the time-series data in chronological order to obtain a time-series data sequence; compress the visualization data to obtain compressed visualization data; The data packet includes a header, a time-series data sequence, compressed and visualized data, a data checksum, and a packet trailer; The process involves several steps: First, the time-series data sequence is encoded to obtain the time-series data code. Second, the compressed visualization data is encoded to obtain the visualization data code. Third, the time-series data code and the visualization data code are packaged into a data packet. Before sending the data packet, a data checksum is calculated. This checksum is then used as the sending checksum, and the data packet is included in the data packet by the data transmission module. When the preprocessing module receives the data packet, it recalculates the data checksum. This recalculation becomes the receiving checksum. The sending checksum is compared with the receiving checksum. If they match, the transmitted IoT data is complete; otherwise, it is incomplete, and the data transmission module retransmits the data. Finally, the preprocessing module unpacks the received data packet, extracts the time-series data code and the visualization data code, and restores them to the time-series data sequence and the compressed visualization data, respectively.

4. The IoT big data fusion processing platform according to claim 3, characterized in that, The methods for encoding time-series data sequences include: The time-series data sequence is standardized to obtain a standard time-series data sequence; The formula for standardization is: , ;in, Represents the first in a standard time-series data sequence One data point; Represents the first in a time-series data sequence One data point; This represents the maximum value in a time-series data sequence. Represents the minimum value in a time-series data sequence; Defined in the standard temporal data sequence, the first Data In Chinese, any digit after the decimal point is... , utilizing the existence indivual And only the last digit is The sequence of numbers as numbers The encoding is performed, and each digit after the decimal point is encoded in the same way to obtain the first digit in the standard time-series data sequence. Data Data encoding; encode all data in a standard time-series data sequence to obtain the time-series data encoding.

5. The IoT big data fusion processing platform according to claim 4, characterized in that, The methods for encoding compressed visualization data include: ;in, Represents pixels The intensity of the red channel; Represents pixels The intensity of the green channel; Represents pixels The intensity of the blue channel; Represents any pixel in any image of compressed visualization data. Pixel values; Extract the red channel intensity, green channel intensity, and blue channel intensity of each pixel, and represent the pixel value of each pixel by forming a three-dimensional vector that matches the corresponding pixel. Combine the pixel values ​​of all pixels into one The matrix is ​​the pixel matrix; where, This represents the number of pixels in any image within the compressed visualization data; the pixel matrix is ​​centered to obtain a centered pixel matrix; a pixel covariance matrix is ​​constructed based on the pixel matrix and the centered pixel matrix; the pixel covariance matrix is ​​orthogonally transformed to obtain an orthogonal pixel covariance matrix; the eigenvalues ​​and eigenvectors of each element in the orthogonal covariance matrix are calculated, and the top eigenvalues ​​and eigenvectors are selected. The eigenvectors corresponding to the largest eigenvalues ​​form the feature matrix; based on the orthogonal pixel covariance matrix and the feature matrix, a projection matrix is ​​constructed, which is the encoding result, thus obtaining the visual data encoding; Pixel covariance matrix ;in, Represents a pixel matrix; Represents a centered pixel matrix; Projection matrix ;in, Represents the orthogonal pixel covariance matrix; Represents the characteristic matrix; Methods for calculating data checksums include: Divide the IoT data into equal parts Calculate the information entropy of IoT data. ;in, Indicates the first The probability distribution of IoT data; Data checksum ;in, This represents the parameters and sum of each piece of IoT data; This represents an adaptive adjustment function; Indicates environmental parameters; Adaptive adjustment function ;in, It is an adjustment coefficient; This represents the size of any data packet during IoT data transmission. This indicates the size of the largest data packet during IoT data transmission. This represents the sum of environmental parameters.

6. The IoT big data fusion processing platform according to claim 5, characterized in that, The methods for preprocessing IoT data include: Denoising the time-series data sequence yields denoised time-series data; filtering the compressed visualization data yields filtered visualization data; combining the denoised time-series data and the filtered visualization data yields complete IoT data.

7. The IoT big data fusion processing platform according to claim 6, characterized in that, The methods for denoising time-series data sequences include: Remove outliers from the time-series data sequence and fill in missing data to obtain coarse-processed time-series data; Define wavelet detail coefficients; A pre-defined wavelet basis function is used, and an adaptive layering function is employed to adaptively select the number of wavelet packet decomposition layers, resulting in the wavelet decomposition layer number. Based on the wavelet basis function and the wavelet decomposition layer number, coarsely processed time-series data is decomposed using wavelet decomposition, and the noise standard deviation is estimated using minimum absolute deviation. A threshold is determined based on the noise standard deviation, and the wavelet detail coefficients of each layer are thresholded to obtain denoised detail coefficients. Based on the denoised detail coefficients, the signal is reconstructed through inverse wavelet transform to obtain denoised time-series data. The method of adaptively selecting the number of layers in wavelet packet decomposition using an adaptive layering function includes: Define the length of the coarse-processed temporal data and the filter length of the wavelet basis function, and calculate the maximum number of decomposition layers based on the length of the coarse-processed temporal data and the filter length of the wavelet basis function; Calculate the first Energy of wavelet detail coefficients of the layer; for the previous The energy summation of the wavelet detail coefficients of the layer yields the previous layer. The total energy of wavelet detail coefficients of all layers; summing the energies of the wavelet detail coefficients of all layers to obtain the total energy of the wavelet detail coefficients of all layers; based on the previous... The cumulative energy ratio is calculated from the total energy of the wavelet detail coefficients of the layer and the total energy of the wavelet detail coefficients of all layers, and the adaptive decomposition layer number is determined based on the cumulative energy ratio; the wavelet decomposition layer number is obtained based on the maximum decomposition layer number and the adaptive decomposition layer number. Maximum number of decomposition levels ;in, Indicates the length of coarse-processed time-series data; The filter length represents the wavelet basis function; No. The formula for calculating the energy of the wavelet detail coefficients of a layer is: ;in, Indicates the first Energy of wavelet detail coefficients of the layer; Indicates the first The length of the wavelet detail coefficients of the layer; Indicates the first Wavelet detail coefficients of the layer; Indicates the first Wavelet approximation coefficients of the layer; Cumulative energy percentage , ;in, This represents the total energy of the wavelet detail coefficients across all layers; Indicates the first The energy of the wavelet approximation coefficients of the layer; Indicates the preceding Total energy of wavelet detail coefficients of the layer; If the cumulative energy percentage If the cumulative energy percentage is greater than or equal to the preset threshold, then the current layer number will be... The minimum value between the adaptive decomposition level and the maximum decomposition level is selected as the wavelet decomposition level. The method for thresholding the wavelet detail coefficients of each layer includes: An adaptive threshold is determined for the wavelet detail coefficients of each layer based on the noise standard deviation, and the wavelet detail coefficients of each layer are processed using a soft thresholding method based on the adaptive threshold to obtain the denoised detail coefficients. The formula for determining the adaptive threshold is: ;in, Indicates the first Adaptive threshold for the layer; Indicates the standard deviation of noise; Indicates the first Local variance of wavelet detail coefficients of the layer.

8. The IoT big data fusion processing platform according to claim 7, characterized in that, The methods for filtering compressed visualization data include: Initialize the filter; The compressed visualization data is sharpened using a sharpening function to obtain sharpened visualization data. An automatic grayscale adjustment framework is constructed, which includes initializing the framework parameters, including the gamma value. and dynamic adjustment coefficient The system dynamically adjusts the grayscale value of any pixel in the sharpened visualization data by adjusting the grayscale value of the automatic adjustment framework parameters; it also filters the sharpened visualization data based on the filter and the grayscale value automatic adjustment framework to obtain filtered visualization data. The formula for sharpening compressed and visualized data is as follows: Sharpening function ;in, This represents the pixel value of any single pixel in the compressed visualization data. This represents the contrast adjustment function; Indicates the sharpening intensity coefficient; Represents the Laplace operator; Contrast adjustment function ; Indicates the contrast control coefficient; Indicates the brightness control coefficient; This represents the pixel value after normalization of any pixel in the compressed visualization data. The mechanism for setting up the automatic grayscale adjustment framework is as follows: if the grayscale value of any pixel in the sharpened visualization data is lower or higher than the preset grayscale value range, the gamma value will be adjusted accordingly. and dynamic adjustment coefficient Perform a gamma transform on the sharpened visualization data to maintain the grayscale value of any pixel in the sharpened visualization data within a preset grayscale value range; The formula for calculating the gamma transform is: ;in, This represents the grayscale value of any pixel in the sharpened visualization data after gamma transformation. This represents the grayscale value of any single pixel in the original sharpened visualization data. The formula for filtering sharpened visualization data is as follows: ;in, This represents the pixel value of any pixel in the filtered visualization data. Indicates grayscale weight; Indicates spatial weights; This indicates that the data in the sharpening visualization is located in The pixel value of the pixel; Grayscale weight ;in, This represents the standard deviation of the Gaussian function. This indicates that the original sharpened visualization data is located in The grayscale value of the pixel; Spatial weight ;in, Represents the x-coordinate of a pixel. Represents the ordinate of a pixel.

9. The Internet of Things big data fusion processing platform according to claim 8, characterized in that, The methods for constructing an IoT database to store comprehensive IoT data include: Define primary keys, foreign keys, and indexes; The IoT database includes an IoT device ID data table, a time-series data table, and a visualization data table. Define the IoT device ID, the denoised temporal data number, and the filtered visualization data number; Based on the feature code, the IoT device ID is matched with the denoised temporal data number and the filtered visualization data number; Construct an IoT device ID data table, including IoT device ID and feature code; use IoT device ID as the primary key of the IoT device ID data table; use feature code as a foreign key of the IoT device ID data table, pointing to the time-series data table and the visualization data table; Construct a time-series data table, including IoT device ID, denoised time-series data number, time-series data type, time-series data measurement value, and time-series data timestamp; use the denoised time-series data number and time-series data timestamp as the primary key of the time-series data table; use the IoT device ID as the foreign key of the time-series data table, pointing to the IoT device ID data table; Construct a visualization data table, including IoT device ID, filtered visualization data number, visualization data resolution, and visualization data storage path; use the filtered visualization data number as the primary key of the visualization data table; use the IoT device ID as the foreign key of the visualization data table, pointing to the IoT device ID data table; The improved IoT data will be imported into an IoT database for storage, and an index will be added to the IoT database. The index includes IoT device ID, feature code, denoised time-series data number, and filtered visualization data number.

Images