Data mining method for Internet of Things based on neural network

Abstract

The present invention relates to the field of Internet of Things data mining, and in particular to an Internet of Things data mining method based on a neural network. The content includes: preprocessing the collected Internet of Things data through a multi-layer nested data cleaning algorithm; extracting features from the preprocessed data through high-dimensional mapping and adaptive embedding transformation to obtain features after adaptive embedding transformation; constructing a neural network, training the neural network using interactive weight adjustment and multi-level residual connection until the output features of the final layer of the neural network are obtained; based on the output features of the final layer of the neural network, using a self-explanatory neural network and a multidimensional data visualization method for interpretation and visualization. It solves the problem that the prior art does not fully consider the importance of neighboring data points and the natural trend of the data, cannot fully reveal the complex features and patterns in the data, lacks the ability to explain the model decision-making process, and makes it difficult for users to understand and trust the output results of the neural network model.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things data mining, and in particular to an Internet of Things data mining method based on a neural network. Background Art

[0002] With the rapid development of Internet of Things (IoT) technology, more and more devices and sensors are connected to the Internet, forming a huge IoT network. These devices and sensors are widely used in various industries, including smart homes, industrial automation, smart transportation, environmental monitoring, and medical health. With the rapid increase in the number of IoT devices, the amount of data generated has also exploded. These data contain rich information, which, if effectively mined and utilized, will have a significant impact on intelligent decision-making and automated control.

[0003] However, the diversity and complexity of IoT data poses great challenges to data mining and utilization. IoT data sources are diverse, data formats vary, and data quality varies, often containing a lot of noise and missing values. In addition, the spatiotemporal characteristics of IoT data make it highly dynamic and nonlinear, which poses a severe test to traditional data processing and analysis methods. Existing data processing technologies often seem powerless in dealing with these challenges and are unable to fully tap the potential value in the data.

[0004] There are at least the following technical problems in the existing technology: the existing technology does not fully consider the importance of neighboring data points and the natural trend of the data, the continuity and consistency of the filled data are poor, and most of them use linear methods, which cannot fully reveal the complex features and patterns in the data, resulting in insufficient ability to express data features; the neural networks in the existing technology are mostly black box models, which lack the ability to explain the model decision-making process, making it difficult for users to understand and trust the output results of the neural network model. Summary of the invention

[0005] The present invention provides an Internet of Things data mining method based on a neural network to solve the problem that the prior art does not fully consider the importance of neighboring data points and the natural trend of the data, the continuity and consistency of the filled data are poor, most of them use linear methods, which cannot fully reveal the complex features and patterns in the data, resulting in insufficient data feature expression capabilities; the neural networks in the prior art are mostly black box models, which lack the ability to explain the model decision-making process, making it difficult for users to understand and trust the output results of the neural network model.

[0006] The Internet of Things data mining method based on neural network of the present invention specifically includes the following technical solutions:

[0007] The IoT data mining method based on neural network includes the following steps:

[0008] S1. Collect IoT data, pre-process the collected IoT data through a multi-layer nested data cleaning algorithm to obtain pre-processed data; based on the pre-processed data, extract features through high-dimensional mapping and adaptive embedding transformation to obtain features after adaptive embedding transformation;

[0009] S2. Construct a neural network. Based on the features after adaptive embedding transformation, use interactive weight adjustment and multi-level residual connection to train the neural network until the output features of the final layer of the neural network are obtained. Based on the output features of the final layer of the neural network, use self-explanatory neural network and multidimensional data visualization method to interpret and visualize.

[0010] Preferably, the S1 specifically includes:

[0011] The multi-layer nested data cleaning algorithm calculates the mean and standard deviation of IoT data through an adaptive noise filter, and combines the time factor to obtain noise-filtered data.

[0012] Preferably, the S1 specifically includes:

[0013] Multivariate interpolation is performed on the noise-filtered data to fill in the missing values ​​in the noise-filtered data to obtain multivariate interpolated data.

[0014] Preferably, the S1 specifically includes:

[0015] Perform nonlinear transformation on the data after multivariate interpolation to obtain preprocessed data.

[0016] Preferably, the S1 specifically includes:

[0017] The preprocessed data is subjected to high-dimensional mapping to obtain features after the high-dimensional mapping; based on the features after the high-dimensional mapping, feature extraction is performed through adaptive embedding transformation to obtain features after adaptive embedding transformation.

[0018] Preferably, the S2 specifically includes:

[0019] The neural network adopts a multi-layer structure, and the layers are interactively connected through residuals; and nonlinear transformations and weight adjustments are introduced between the layers of the neural network to transform features.

[0020] Preferably, the S2 specifically includes:

[0021] In the feature transformation process, the features after adaptive embedding transformation are used as the initial input features, and the weight matrix and bias vector are introduced for linear transformation to obtain the features after linear transformation; the features after linear transformation are passed through a nonlinear activation function to introduce nonlinear features to obtain the features after nonlinear transformation; multi-level residual connections are performed based on the features after nonlinear transformation to obtain the output features of the current layer of the neural network, and the output features of the current layer of the neural network are used as the input features of the next layer to continue feature transformation until the output features of the final layer of the neural network are obtained.

[0022] Preferably, the S2 specifically includes:

[0023] The loss function is used to calculate the loss between the output features of the current layer of the neural network and the actual label. The back-propagation algorithm is used to calculate the gradient of the loss function relative to each weight and bias in the neural network. The weights and biases are updated based on the gradients of the weights and biases and the dynamically adjusted learning rate.

[0024] The beneficial effects of the technical solution of the present invention are:

[0025] 1. Design a multi-layer nested data cleaning algorithm, dynamically calculate the mean and standard deviation of the collected IoT data through an adaptive noise filter, and combine the time factor to effectively identify and remove noise data; in the multivariate interpolation process, use weighted average and smoothing adjustment to reasonably fill in the missing values ​​in the noise-filtered data, which can maintain the continuity and consistency of the data, reduce the impact of missing data on subsequent analysis, and ensure the integrity and reliability of the data;

[0026] 2. Through nonlinear transformation, the important features in IoT data are highlighted, irrelevant noise components are suppressed, the expressiveness of data features is enhanced, and higher quality data input is provided for data feature extraction; high-dimensional mapping and adaptive embedding transformation are adopted to map low-dimensional features to higher-dimensional space and further enhance the expressiveness of data features; through training, parameters are dynamically adjusted to make data feature extraction more accurate and stable, laying a solid foundation for the subsequent training of neural network models;

[0027] 3. The neural network training method based on interactive weight adjustment and multi-level residual connection can dynamically adjust the learning rate and improve the stability and convergence speed of neural network model training; residual connection effectively alleviates the gradient vanishing problem in deep neural networks, enhances the interaction and information flow between data features, and improves the expression ability and training depth of neural network models;

[0028] 4. Self-explanatory neural networks can reveal the contribution of neural network input features to output results, and realize the explanation of the model decision-making process; multidimensional data visualization methods can intuitively display the structure and relationship of IoT data by reducing high-dimensional feature data to low-dimensional space, making it easier to discover potential patterns and structures in IoT data, and enhancing the intuitiveness and interpretability of result analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of the Internet of Things data mining method based on neural network described in the present invention. DETAILED DESCRIPTION

[0030] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0032] The specific scheme of the Internet of Things data mining method based on neural network provided by the present invention is described in detail below with reference to the accompanying drawings.

[0033] Refer to the attached Figure 1 , which shows a flow chart of a method for Internet of Things data mining based on a neural network provided by an embodiment of the present invention, the method comprising the following steps:

[0034] S1. Collect IoT data, pre-process the collected IoT data through a multi-layer nested data cleaning algorithm to obtain pre-processed data; based on the pre-processed data, extract features through high-dimensional mapping and adaptive embedding transformation to obtain features after adaptive embedding transformation;

[0035] IoT data collection is performed using multi-threaded parallel processing and edge computing technology. The collected IoT data is standardized into a unified format for subsequent processing. Specifically, a distributed architecture is adopted, and each collection node is responsible for data collection of a part of IoT devices and sends the collected IoT data to the central processing unit. During the collection process, IoT data will be initially cleaned and filtered to remove abnormal data and noise.

[0036] After IoT data is collected, data preprocessing is performed. Facing the problems of diverse IoT data sources, complex data formats, and a large amount of noise and missing values, a multi-layer nested data cleaning algorithm is designed for data preprocessing. The data preprocessing steps are as follows:

[0037] First, an adaptive noise filter is used to remove noise from IoT data; its core formula is:

[0038]

[0039] in, is the noise-filtered data value at time t; Represents the mean of IoT data at time t, which is used to represent the central trend of IoT data; Indicates the standard deviation of IoT data at time t, which is used to indicate the degree of discreteness of IoT data; It is an adjustment coefficient used to control the strength of noise filtering; is a time frequency parameter used to dynamically adjust the noise threshold. The adaptive noise filter can effectively identify and remove noise data by dynamically calculating the mean and standard deviation of IoT data and combining the time factor. In actual operation, the adaptive noise filter can process a large amount of IoT data in parallel and use a rolling window to calculate the mean and standard deviation of IoT data in the current window, thereby achieving real-time noise filtering.

[0040] Secondly, multivariate interpolation is performed on the noise-filtered data;

[0041] In the multivariate interpolation process, the missing values ​​in the noise-filtered data are filled using the following formula:

[0042]

[0043] in, Indicates the location of missing values ​​in the noise-filtered data The data at , that is, the data after multivariate interpolation; is the weight of the kth neighboring data point, indicating the importance of the neighboring data points, and the missing value is estimated by weighted average; Indicates that the noise-filtered data is at position The data value at ; n is the number of neighboring data points; is the spatial position index; α is the smoothing coefficient, which controls the smoothness of multivariate interpolation. In the implementation of multivariate interpolation, the weighted average of the neighboring data points around the missing data points in the noise-filtered data is first calculated, and then the interpolation result is adjusted by the smoothing coefficient to make it more consistent with the natural trend of the data.

[0044] Furthermore, the data after multivariate interpolation is subjected to nonlinear transformation; nonlinear transformation is used to enhance the expressive power of data features, and its formula is:

[0045]

[0046] in, The data is the data after nonlinear transformation, that is, the preprocessed data. By performing nonlinear transformation on the data after multivariate interpolation, the important features in the IoT data can be highlighted and irrelevant noise components can be suppressed.

[0047] The data after nonlinear transformation is subjected to feature extraction through a feature extraction algorithm based on high-dimensional mapping and adaptive embedding transformation. The specific steps are as follows:

[0048] First, high-dimensional mapping is performed, and its core formula is:

[0049]

[0050] in, is the feature after high-dimensional mapping; T is the data after nonlinear transformation; is a randomly generated vector, represents its transpose; b is a randomly generated bias. High-dimensional mapping can map low-dimensional features to higher-dimensional space, thereby revealing richer features in the data. In the specific implementation, multiple random vectors and biases are generated and applied to each data point of the nonlinearly transformed data to calculate the features after high-dimensional mapping.

[0051] Furthermore, the features after high-dimensional mapping are adaptively embedded and transformed; the core formula of the adaptive embedding transformation is:

[0052]

[0053] in, is the u-th eigenvalue after adaptive embedding transformation; represents the weight of the u-th embedding function on the v-th feature; and are the frequency parameters and phase parameters of the vth feature after high-dimensional mapping, which can be dynamically adjusted through training, and V is the number of features after high-dimensional mapping. Adaptive embedding can further enhance the expressiveness of data features, making it more suitable for subsequent neural network model training.

[0054] S2. Construct a neural network. Based on the features after adaptive embedding transformation, use interactive weight adjustment and multi-level residual connection to train the neural network until the output features of the final layer of the neural network are obtained. Based on the output features of the final layer of the neural network, use self-explanatory neural network and multidimensional data visualization method to interpret and visualize.

[0055] The features after adaptive embedding transformation will be used as input for neural network model training. The neural network model training adopts a neural network training method based on interactive weight adjustment and multi-level residual connection, aiming to enhance the generalization ability of the neural network. The neural network adopts a multi-level structure, and each layer is interactively connected to the previous layers through residual connections. By introducing nonlinear transformation and weight adjustment between the layers of the neural network for feature transformation, the expression ability of the neural network and its ability to capture complex data features are enhanced.

[0056] The input features of the feature transformation are the output features of the previous layer of the neural network, and the initial input features are the features after the adaptive embedding transformation; the input features first pass through a linear transformation layer, which uses a weight matrix and a bias vector for transformation. The formula is as follows:

[0057]

[0058] in, It is The linear transformation output of the layer is represented by Features after layer linear transformation; Indicates the offset between the current layer and the previous s layers; and They are The weight matrix and bias vector of the layer.

[0059] The output of the linear transformation is passed through a nonlinear activation function, such as or , introducing nonlinear features to enhance the expressive power of the neural network model. The formula is as follows:

[0060]

[0061] in, For the The features after nonlinear transformation of the layer, i.e. The feature transformation of the layer, is the activation function.

[0062] No. Feature transformation of the layer As part of the multi-level residual connection, it is combined with the output features of the previous layers to form the first Output features of the layer ; Residual connection can effectively alleviate the gradient vanishing problem in deep networks and maintain the flow of information. The specific formula is:

[0063]

[0064] in, For the Layer output features; For the Output features of the layer; It is Output features of the layer; and is the residual weight, which is dynamically adjusted through training; is the current layer; Represents the ordinal numbers of the first few layers used for residual connections; is the number of the first few layers in the residual connection. Through multi-level residual connections, the neural network can better capture the complex relationship between data features and improve the expressiveness of the neural network model. In the implementation process, the residual connection uses a skip connection method to directly pass the output features of the first few layers to the subsequent layers to enhance the continuity of information flow.

[0065] The learning rate is a key parameter in neural network training, which is used to adjust the magnitude of weight updates at each iteration. In neural networks, the application of learning rate is as follows:

[0066] After each forward propagation, the loss between the output features of the current neural network and the actual label is calculated, which is achieved through various loss functions (such as mean square error, cross entropy, etc.), where the actual label comes from an existing database; using the back propagation algorithm, the gradient of the loss function relative to each weight and bias in the neural network is calculated. The gradient represents the rate of change of the loss function on each parameter of the neural network, which is used to indicate how to adjust the parameters to minimize the loss; based on the calculated gradient and learning rate, the weights and biases of all layers of the entire neural network are updated. The specific formula is:

[0067]

[0068] in, Indicates The first iteration The weight of the layer; is the loss function of the neural network; Indicates The first iteration Bias of the layer; Indicates The learning rate at the iteration; and They represent the gradients of the loss function with respect to weights and biases, respectively.

[0069] In each iteration, the learning rate is updated according to the dynamic adjustment formula, and the updated learning rate is used in the next iteration. The dynamic adjustment formula is:

[0070]

[0071] in, is the learning rate before updating, that is, the learning rate at the pth iteration; is the updated learning rate, i.e. The learning rate at the iteration; To adjust the coefficient, control the fluctuation range of the learning rate; To adjust the coefficient, control the exponential decay of the learning rate; The learning rate is controlled by the periodic fluctuation of the learning rate. By combining the cosine function and the exponential function to dynamically adjust the learning rate, the stability and convergence speed of the neural network training can be improved. The neural network adjusts the learning rate in each training iteration, so that the learning rate can maintain a high update speed and prevent drastic fluctuations during the training process.

[0072] The output features of the final layer of the neural network are used as the input for the result analysis. The method based on self-explanatory neural network and multidimensional data visualization is used to analyze the results and interpret and visualize the output features of the final layer of the neural network. The core formula of the self-explanatory neural network analysis is:

[0073]

[0074] in, The analysis results are used for specific prediction, classification, anomaly detection or clustering tasks, which specifically indicate the decision or analysis results made by the self-explanatory neural network on the input data, that is, the output features of the final layer of the neural network; is the activation function; is the number of neurons; For the The weights of neurons are dynamically adjusted through training; represents the final layer of a neural network; Represents the output features of the final layer of the neural network; is a bias parameter, which is dynamically adjusted through training and is used to adjust the input value of the activation function, thereby controlling the translation of the analysis results; It is a self-explanatory parameter that is dynamically adjusted through training. Self-explanatory neural networks can reveal the contribution of input data to output results through weights and cosine functions, and realize the explanation of analysis results. For example, neuron weights and important parameters and Indicates that the corresponding input data plays a key role in decision making. The purpose of the self-explanatory neural network is to reveal the contribution of input data to the output results, achieve a deep understanding of data features, help users identify key features in IoT data, and make decisions based on key features.

[0075] Multidimensional data visualization reduces high-dimensional feature data to a low-dimensional space (two-dimensional or three-dimensional) for visualization and pattern recognition. Through dimensionality reduction and visualization technology, the structure and relationship of high-dimensional feature data are more intuitive, easier to analyze and interpret, and help users discover potential patterns and structures in IoT data, such as identifying data points of different categories, discovering abnormal data points, and understanding the relationship between data points.

[0076] In summary, the IoT data mining method based on neural network is completed.

[0077] The order of the embodiments of the invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0078] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0079] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. The method of Internet of Things data mining based on neural network is characterized by: The following steps are involved: S1. Collect IoT data, and pre-process the collected IoT data through a multi-layer nested data cleaning algorithm to obtain pre-processed data; Based on the preprocessed data, feature extraction is performed through high-dimensional mapping and adaptive embedding transformation to obtain features after adaptive embedding transformation; S2. Construct a neural network, and train the neural network based on the features after adaptive embedding transformation by using interactive weight adjustment and multi-level residual connection until the output features of the final layer of the neural network are obtained; Based on the output features of the final layer of the neural network, self-explanatory neural networks and multidimensional data visualization methods are used for interpretation and visualization; The S1 specifically includes: The multi-layer nested data cleaning algorithm calculates the mean and standard deviation of IoT data through an adaptive noise filter, and combines the time factor to obtain noise-filtered data; Perform multivariate interpolation on the noise-filtered data to fill in the missing values ​​in the noise-filtered data to obtain multivariate interpolated data; Adaptive noise filter is used to remove noise from IoT data; its core formula is: ； in, yes The noise-filtered data value at the moment; express The mean of IoT data at a certain moment is used to indicate the central trend of IoT data. express The standard deviation of IoT data at a certain moment is used to indicate the degree of dispersion of IoT data; It is an adjustment coefficient used to control the strength of noise filtering; is a time frequency parameter used to dynamically adjust the noise threshold; In the multivariate interpolation process, the missing values ​​in the noise-filtered data are filled using the following formula: ； in, Indicates the location of missing values ​​in the noise-filtered data The data at , that is, the data after multivariate interpolation; For the The weights of neighboring data points indicate the importance of neighboring data points, and the missing values ​​are estimated by weighted average; Indicates that the noise-filtered data is at position The data value at is the number of neighboring data points; is the spatial position index; is the smoothing coefficient, which controls the smoothing degree of multivariate interpolation; The features after high-dimensional mapping are adaptively embedded and transformed; the core formula of the adaptive embedding transformation is: ； in, is the first eigenvalues; Indicates The embedding function The weight of each feature; and is the first The frequency parameters and phase parameters of each feature can be dynamically adjusted through training. is the number of features after high-dimensional mapping; The core formula of the self-explanatory neural network analysis is: ； in, The analysis results are used for specific prediction, classification, anomaly detection or clustering tasks, specifically indicating the decision or analysis results made by the self-explanatory neural network on the input data, that is, the output features of the final layer of the neural network; is the activation function; is the number of neurons; For the The weights of neurons are dynamically adjusted through training; represents the final layer of a neural network; Represents the output features of the final layer of the neural network; is a bias parameter, which is dynamically adjusted through training and is used to adjust the input value of the activation function, thereby controlling the translation of the analysis results; It is a self-explanatory parameter that is dynamically adjusted through training. The self-explanatory neural network can reveal the contribution of input data to output results through weights and cosine functions, and realize the explanation of analysis results. The purpose of the self-explanatory neural network is to reveal the contribution of input data to output results, realize in-depth understanding of data characteristics, help users identify key features in IoT data, and make decisions based on key features.

2. The method for Internet of Things data mining based on neural network according to claim 1, characterized in that: The S1 specifically includes: Perform nonlinear transformation on the data after multivariate interpolation to obtain preprocessed data.

3. The method for Internet of Things data mining based on neural network according to claim 2, characterized in that: The S1 specifically includes: The preprocessed data is subjected to high-dimensional mapping to obtain features after the high-dimensional mapping; based on the features after the high-dimensional mapping, feature extraction is performed through adaptive embedding transformation to obtain features after adaptive embedding transformation.

4. The method for Internet of Things data mining based on neural network according to claim 1, characterized in that: The S2 specifically includes: The neural network adopts a multi-layer structure, and the layers are interactively connected through residuals; and nonlinear transformations and weight adjustments are introduced between the layers of the neural network to transform features.

5. The method for Internet of Things data mining based on neural network according to claim 4 is characterized in that: The S2 specifically includes: In the feature transformation process, the features after adaptive embedding transformation are used as the initial input features, and the weight matrix and bias vector are introduced for linear transformation to obtain the features after linear transformation; the features after linear transformation are passed through a nonlinear activation function to introduce nonlinear features to obtain the features after nonlinear transformation; multi-level residual connections are performed based on the features after nonlinear transformation to obtain the output features of the current layer of the neural network, and the output features of the current layer of the neural network are used as the input features of the next layer to continue feature transformation until the output features of the final layer of the neural network are obtained.

6. The method for Internet of Things data mining based on neural network according to claim 5, characterized in that: The S2 specifically includes: The loss function is used to calculate the loss between the output features of the current layer of the neural network and the actual label. The back-propagation algorithm is used to calculate the gradient of the loss function relative to each weight and bias in the neural network. The weights and biases are updated based on the gradients of the weights and biases and the dynamically adjusted learning rate.

Images