Adaptive optimization method for machine learning model of edge node of Internet of Things
By collecting and preprocessing time series data in IoT devices, identifying important time windows and adjusting neural network models, the problem of edge node computing power limitation is solved, and a small and efficient machine learning model is realized, reducing manual intervention and suitable for resource utilization. Environment-limited applications.
Patent Information
- Application Number
- CN202411404182.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2024-12-27
AI Technical Summary
In IoT devices, due to the computing power and memory capacity limitations of edge nodes, traditional big data-based machine learning model training methods are not applicable, and manual intervention is required to adapt to diverse IoT data and scenarios. The existing technology It is difficult to reduce model size and adaptively optimize without reducing the capabilities of artificial intelligence models.
By collecting the timing data of target features of IoT devices, preprocessing and determining important time windows, building CNN and RNN neural network models, and adjusting convolution kernel and hidden units during training to optimize the model, reduce model size and improve the model efficiency, reduce manual intervention.
It realizes a compact and efficient machine learning model running on resource-constrained IoT edge devices, maintains good performance, reduces manual intervention, and is suitable for on-site application.
Smart Images
Figure CN119203772A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an adaptive optimization method for a machine learning model of an edge node of an Internet of Things, and belongs to the technical field of the Internet of Things and artificial intelligence. Background Art
[0002] Currently, the computing process in the edge nodes of the Internet of Things is combined with artificial intelligence agents, which plays an important role in multiple scenarios such as water management, agricultural management, and intelligent manufacturing. Taking water affairs as an example, edge devices can collect and process data such as water quality, flow, and pressure in real time, while artificial intelligence agents are responsible for more complex analysis and decision-making tasks. This combination can achieve real-time water quality monitoring and abnormal warning, accurately locate pipe network leaks, optimize water supply scheduling, and improve water pump energy efficiency. The combination of edge computing and artificial intelligence technology in the Internet of Things can effectively increase data processing speed, reduce latency, enhance data security and privacy protection, reduce network bandwidth and power consumption, improve system scalability, and achieve intelligent decision-making, thus bringing huge advantages and transformative changes.
[0003] When applying artificial intelligence models in IoT devices, due to the limitations of edge nodes in computing power and memory capacity, traditional big data-based model training methods have become inapplicable. Therefore, a method is needed to minimize the model size to adapt to the capacity limitations of IoT devices without significantly reducing the capabilities of the artificial intelligence model. At the same time, due to the diversity of IoT data and scenarios, this method needs to minimize manual intervention and be able to optimize adaptively. Summary of the invention
[0004] Technical problem to be solved by the invention
[0005] In view of the limitation of the application capability of artificial intelligence models in IoT devices, the present invention proposes an adaptive optimization method for machine learning models of IoT edge nodes.
[0006] Technical Solution
[0007] In order to achieve the above object, the technical solution provided by the present invention is:
[0008] A method for adaptively optimizing a machine learning model of an edge node of an Internet of Things comprises the following steps:
[0009] Step 1: Collect the target feature time series data of the IoT device. The target feature data includes the temperature, humidity, and device status collected by the sensor.
[0010] Step 2: pre-process the collected data.
[0011] Step 3, determining the important time window of the time series data processed in step 2, includes the following steps:
[0012] Step 3-1: Segment the preprocessed IoT time series data into fixed time windows.
[0013] Step 3-2, determining the score of each time window, and determining the important time window based on the score.
[0014] In step 3-3, windows with scores higher than a certain set threshold are regarded as important time windows, or all time windows are clustered into important time windows and non-important time windows by a clustering method.
[0015] Step 3-4, merging the adjacent important time windows determined in step 3-3.
[0016] Step 4: construct CNN and RNN neural network models and train them in the training set to obtain the initial CNN and RNN models.
[0017] Step 5: Search the position in the input time series corresponding to the center of the convolution kernel for the initial neural network model obtained in step 4 to determine whether the convolution kernel is located in an important time window.
[0018] Step 6: Based on the important time window determined in step 3 and the center position of the convolution kernel determined in step 5, the initial neural network model of step 4 is adjusted, including CNN model adjustment and RNN model adjustment.
[0019] Step 7: Retrain the adjusted neural network model based on the test set and test the performance of the trained model.
[0020] Furthermore, the process of preprocessing the data in step 2 includes:
[0021] Step 2-1, establishing a mapping for data related to target feature data in different data sources;
[0022] Step 2-2, clean the collected data, including processing missing data, detecting and correcting abnormal data, deleting duplicate data, unifying data formats and units, and converting data from different sources and types into a unified format;
[0023] Step 2-3, standardize data of different ranges to the same scale range to ensure that data from different sources have consistent formats and units for the same feature.
[0024] Furthermore, in step 3-2, the wavelet transform coefficient and information gain value of each window are calculated as a score, and the weighted sum of the wavelet transform score and the information gain score is taken as the comprehensive score of each time window.
[0025] Furthermore, the calculation method of the time window wavelet transform coefficient score is:
[0026] Step 3-3-1, perform wavelet transform on the time series to obtain wavelet coefficients of different scales;
[0027] Step 3-3-2, calculating the energy value or amplitude of the wavelet coefficients in each divided time window;
[0028] Step 3-3-3, normalize the calculated energy value or amplitude based on the extreme values corresponding to all windows, and use the value between [0, 1] as the score of the time window.
[0029] Furthermore, the calculation method of the time window information gain score is:
[0030] Calculate the information gain G(Y,X) of the target variable for each time window and normalize the information gain value to a score between 0 and 1:
[0031] G(Y,X)=H(Y)-H(Y|X)
[0032] Where H(Y) is the entropy of the target variable, H(Y|X) is the conditional entropy of the target variable under a given feature X, the target variable Y usually refers to the result data that you want to predict or classify, and feature X refers to the feature within each time window.
[0033] Furthermore, the calculation method of the above conditional entropy H(Y|X) is:
[0034] Step 1) Count the joint frequency of the target variable Y and the time window feature X;
[0035] Step 2) Calculate the joint probability distribution P(X=x) based on the statistical quantity i ,Y=y j );
[0036] Step 3) Calculate the conditional probability distribution, that is, the conditional probability P(Y = y) of the target variable given the time window characteristics j |X=x i ):
[0037]
[0038] Step 4) Calculate the conditional entropy H(Y|X) by:
[0039]
[0040] Furthermore, the method for determining whether the convolution kernel is located in the important time window in step 5 is:
[0041] Step 5-1, calculate the convolution kernel receptive field Sen by the convolution kernel size and step length, the method is:
[0042] Sen=k size +(ksize -1)*(k step -1)
[0043] where k size is the convolution kernel size, k step is the convolution kernel step size.
[0044] Step 5-2: For each position i of the neural network output sequence, calculate the center position i of its corresponding input sequence center , the method is:
[0045]
[0046] Step 5-3, based on the calculation of the starting time index start and the ending time index end corresponding to the receptive field Sen of the convolution kernel, the method is:
[0047]
[0048] (start, end) is the index range of the convolution kernel in the input sequence, and the index is used to determine whether the convolution kernel is within the important time window.
[0049] Furthermore, the adjustment method of the CNN network model in step 6 is: according to the comprehensive score of each time window obtained in step 3, the convolution kernel scale of the pooling layer corresponding to the important time window is reduced;
[0050] The method for adjusting the RNN network model includes the following steps:
[0051] Step 6-1, calculate the model loss function in the forward propagation phase of the training phase;
[0052] Step 6-2, calculate the contribution of the hidden state of each time step to the model loss function in the back propagation stage;
[0053] Step 6-3, for time steps whose contribution values are less than a certain threshold, reduce the number of hidden units corresponding to them, and the reduction is achieved by reducing the dimension of the hidden state weight matrix corresponding to the hidden unit.
[0054] Beneficial Effects
[0055] The method adopted in the present invention effectively optimizes the machine learning model. The model maintains good performance while being compact and efficient, and is suitable for running on resource-constrained IoT edge devices.
[0056] The method of the present invention reduces human intervention during implementation and can automatically optimize the artificial intelligence agent model on the edge device based on the input time series data, which is conducive to the implementation of artificial intelligence related methods in the edge devices of the Internet of Things. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 is a flowchart of the steps of the method of the present invention;
[0058] Figure 2 A flowchart of the method for determining an important time window of the present invention;
[0059] Figure 3 This is a schematic diagram of the initial CNN model established in an embodiment of the present invention;
[0060] Figure 4 Schematic diagram of the initial RNN model established in an embodiment of the present invention. DETAILED DESCRIPTION
[0061] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and specific embodiments. Figure 1 , Figure 2 As shown, the method for adaptive optimization of the machine learning model of the edge node of the Internet of Things of the present invention mainly includes the following steps:
[0062] 1. Collect IoT time series data
[0063] The method first collects the target feature time series data of IoT devices. The IoT data sources include temperature sensors, humidity sensors, water quality sensors, flow meters, pressure gauges, cameras, microphones and other device data, as well as databases and file systems that store IoT data.
[0064] The methods of collecting time series data include: collecting through the interface services provided by the device, collecting real-time data using IoT sensing technology, and synchronizing historical data from databases and file systems.
[0065] The target characteristic data includes temperature, humidity, equipment status and other data collected from the above data sources.
[0066] 2 Data preprocessing
[0067] The data preprocessing process includes the following steps:
[0068] 2-1 First, analyze the data from different sources, obtain the field names and data formats related to the target feature data in each data source, and establish a mapping. For example, sensor A may record the temperature as "temp_C" and sensor B may record the temperature as "temperature_fahrenheit". In this case, map the above two fields to a unified name.
[0069] 2-2 Clean the collected data, including the following process:
[0070] Handling missing data: fill in the data that can be filled, otherwise delete it;
[0071] Detect and correct abnormal data, such as checking whether the temperature data collected by the temperature sensor is within a reasonable range (such as -50°C to 50°C);
[0072] Remove duplicate data;
[0073] Standardize data formats and units, such as converting Fahrenheit to Celsius;
[0074] The data of different sources and types are processed uniformly. In this embodiment, data of different formats (such as CSV, JSON, XML, etc.) are converted into a unified format and then integrated into a unified database or data warehouse.
[0075] 2-3 Normalize data of different ranges to the same scale range to ensure that data from different sources have consistent formats and units for the same feature. In this embodiment, data of different value ranges are normalized to the range of [0, 1] through data mapping.
[0076] 3. Determine the important time window of time series data
[0077] 3-1 The pre-processed IoT time series data is divided into fixed time windows. In this embodiment, the time windows are set to 1 minute and 5 minutes;
[0078] 3-2 Determine the score of each time window, and determine the important time windows based on the score. In this embodiment, the wavelet transform coefficients and information gain values are used for time window scoring. The weighted sum of the wavelet transform score and the information gain score is used as the comprehensive score of each time window. In this example, the weights of the two scores are both 50%.
[0079] The calculation method of the time window wavelet transform coefficient score is:
[0080] 1) First, perform wavelet transform on the time series to obtain wavelet coefficients of different scales;
[0081] 2) Calculate the energy value or amplitude of the wavelet coefficients in each segmented time window;
[0082] 3) The calculated energy value or amplitude is normalized based on the extreme values corresponding to all windows, and the value between [0, 1] is used as the score of the time window.
[0083] The calculation method of the time window information gain score is:
[0084] Calculate the information gain of each time window for the target variable and normalize the information gain value to a score between 0 and 1. The calculation method of information gain G(Y,X) is:
[0085] G(Y,X)=H(Y)-H(Y|X)
[0086] Among them, H(Y) is the entropy of the target variable, and H(Y|X) is the conditional entropy of the target variable under a given feature X. In the IoT scenario, the target variable Y usually refers to the result data that you want to predict or classify, and the feature X refers to the features in each time window, such as mean, variance, frequency, etc. Taking the temperature data collected by the temperature sensor as an example, the target variable Y is the temperature category {low temperature, medium temperature, high temperature}, and the window feature X is the average temperature in the time window.
[0087] The calculation steps of conditional entropy H(Y|X) are:
[0088] a) Count the joint frequency of the target variable Y and the time window feature X;
[0089] b) Calculate the joint probability distribution P(X=x) based on the statistical quantity i ,Y=y j );
[0090] c) Calculate the conditional probability distribution, that is, the conditional probability P(Y = y) of the target variable given the time window characteristics j |X=x i ):
[0091]
[0092] d) Calculate the conditional entropy H(Y|X) as follows:
[0093]
[0094] 3-3 The windows with comprehensive scores higher than a certain set threshold are regarded as important time windows, or all time windows are clustered into important time windows and non-important time windows through clustering methods.
[0095] 3-4 Merge the adjacent important time windows determined in 3-3.
[0096] 4. Establish an initial neural network model, including a convolutional neural network (CNN) and a recurrent neural network (RNN). The preferred method is a network model including multi-layer convolution and pooling operation fully connected layers:
[0097] The convolutional neural network structure is as follows Figure 3 As shown, from input to data, it is convolution layer 1, pooling layer 1, convolution layer 2, pooling layer 2, flattening layer, and fully connected layer;
[0098] The structure of the recurrent neural network is as follows Figure 4 As shown in the figure, from input to output, there are embedding layer, RNN layer, and fully connected layer.
[0099] In this embodiment, a different neural network model is designed for each data type: a CNN model is used for image data; a one-dimensional CNN or RNN model is used for audio data; an RNN model is used for text data; and a one-dimensional CNN model is used for sensor data.
[0100] In the above model, the CNN neural network can include multiple layers of convolution and pooling operations, and the RNN layer in the RNN neural network can be implemented using LSTM or GRU units.
[0101] 5. Find the position in the input time series corresponding to the center of the convolution kernel for the initial neural network model to determine whether the convolution kernel is located in an important time window. The method is:
[0102] 5-1 Calculate the convolution kernel receptive field Sen by the convolution kernel size and step length. The method is:
[0103] Sen=k size +(k size -1)*(k step -1)
[0104] where k size is the convolution kernel size, k step is the convolution kernel step size.
[0105] 5-2 For each position i of the neural network output sequence, calculate the center position i of its corresponding input sequence center , the method is:
[0106]
[0107] 5-3 Based on the calculation of the starting time index start and the ending time index end corresponding to the convolution kernel receptive field Sen, the method is:
[0108]
[0109] (start, end) is the index range of the convolution kernel in the input sequence, and this index is used to determine whether the convolution kernel is within the important time window.
[0110] 6 Adjust the initial neural network based on important time windows, including CNN model adjustment and RNN model adjustment:
[0111] For the CNN network, according to the comprehensive score of each time window obtained in step 3, the scale of the convolution kernel of the neural network pooling layer corresponding to the time window is adjusted:
[0112] The convolution kernel scale of the pooling layer corresponding to the important time window is reduced to retain more detail information. In this embodiment, the convolution kernel scale of the pooling layer corresponding to the important window is uniformly reduced to 2*2.
[0113] For the RNN network, the model is adjusted according to the comprehensive score of each time window obtained in step 3. The RNN network input sequence consists of multiple time steps, and a time window may include one or more time steps. The length of the time step is related to the specific task, usually between 10 and 100. Each time step has an input vector representing the features at that moment. RNN will gradually update its internal state based on these inputs, called the hidden state. The hidden state is the RNN's memory of past information, which will affect the current output. The present invention uses back propagation to calculate the contribution of the hidden state of each time step in the time window to the output of the RNN network. It mainly includes the following steps:
[0114] 1) In the training set, the model loss value is calculated in the RNN forward propagation phase, starting from the first time step of the sequence and processing each input step by step. At each time step, the RNN calculates the current hidden state based on the current input and the hidden state of the previous time step:
[0115] h t =W t-1 h t-1 +W t X+b
[0116] Where X is the input sequence, W t is the weight matrix of the current time step, W t-1 is the weight matrix of the previous time step, b is the offset, h t-1 is the hidden state of the previous time step, h t is the hidden state of the current time step. This process is similar to adding new information to the continuously updated memory and finally generating the output Y. Calculate the final output Y and the true label Y true The loss function L(Y,Y true ), the loss function reflects the performance of the model on the current input. The smaller the loss, the more accurate the model's prediction.
[0117] 2) In the back-propagation phase, the contribution of the hidden state at each time step to the model loss function is calculated, and the gradient of the output is calculated starting from the last time step. Where L is the loss function, y tThe output of the current time step in the back propagation phase. The gradient indicates the degree of influence of the output change on the loss. It can be understood as a direction and size, reflecting how the loss will change if the output is slightly adjusted. In this embodiment, the partial derivative reflects the rate of change of the loss value when the output changes slightly, that is, the contribution of each element of the output to the loss. The contribution is calculated by back propagating the output of the RNN to the input to calculate the contribution of the hidden state of each time step to the loss, calculate how the hidden state of the current time step affects the output, and then pass this influence to the hidden state of the previous time step, and continue until the beginning of the sequence.
[0118] The contribution of the hidden state to the loss at this time step Multiply by the weight matrix W of the hidden state at this time step t That is, the contribution of the hidden state at time step t to the loss con t :
[0119]
[0120] 3) Contribution value t The time step less than a certain threshold reduces the number of corresponding hidden units, and reduces the weight matrix W corresponding to the hidden state of the hidden unit. t dimension to achieve.
[0121] 7 Retrain the adjusted neural network model based on the test set and test the performance of the trained model.
[0122] In this stage, the CNN and RNN networks obtained in step 6 can be fused as training object models. The fusion can be achieved by splicing the models and adjusting the weights of each model through machine learning methods or attention mechanisms.
[0123] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.
Claims
1. A method for adaptive optimization of machine learning models for edge nodes of the Internet of Things, characterized in that: The steps include: Step S1, collecting target characteristic time series data of IoT devices, where the target characteristic data includes temperature, humidity, and device status collected by sensors; Step S2, preprocessing the collected data; Step S3, determining the important time window of the time series data processed in step S2, includes the following steps: Step S3-1, segmenting the preprocessed IoT time series data into fixed time windows; Step S3-2, determining the score of each time window, and determining the important time window based on the score; Step S3-3, taking windows with scores higher than a certain set threshold as important time windows, or clustering all time windows into important time windows and non-important time windows by a clustering method; Step S3-4, merging the adjacent important time windows determined in step S3-3; Step S4, constructing CNN and RNN neural network models and training them in the training set to obtain initial CNN and RNN models; Step S5, searching the initial neural network model obtained in step S4 for the position in the input time series corresponding to the center of the convolution kernel, and determining whether the convolution kernel is located in an important time window; Step S6, based on the important time window determined in step S3 and the center position of the convolution kernel determined in step S5, adjusting the initial neural network model obtained in step S4, including adjusting the CNN model and adjusting the RNN model; Step S7, retraining the adjusted neural network model based on the test set, and testing the performance of the trained model.
2. The method for adaptive optimization of a machine learning model for an edge node of an Internet of Things as claimed in claim 1, characterized in that: The process of preprocessing the data in step S2 includes: Step S2-1, establishing a mapping for data related to target feature data in different data sources; Step S2-2, cleaning the collected data, including processing missing data, detecting and correcting abnormal data, deleting duplicate data, unifying data formats and units, and converting data from different sources and types into a unified format; Step S2-3, standardize the data of different ranges to the same scale range to ensure that the data from different sources have consistent formats and units on the same feature.
3. The method for adaptive optimization of a machine learning model for an edge node of an Internet of Things as claimed in claim 1, characterized in that: In step S3-2, the wavelet transform coefficient and information gain value of each window are calculated as a score, and the weighted sum of the wavelet transform score and the information gain score is taken as the comprehensive score of each time window.
4. The method for adaptive optimization of a machine learning model for an edge node of an Internet of Things as claimed in claim 3, characterized in that: The calculation method of the time window wavelet transform coefficient score is: Step 3-2-1, perform wavelet transform on the time series to obtain wavelet coefficients of different scales; Step 3-2-2, calculating the energy value or amplitude of the wavelet coefficients in each divided time window; Step 3-3-3, normalize the calculated energy value or amplitude based on the extreme values corresponding to all windows, and use the value between [0, 1] as the score of the time window.
5. The method for adaptive optimization of a machine learning model for an edge node of an Internet of Things as claimed in claim 3, characterized in that: The calculation method of the time window information gain score is: Calculate the information gain G(Y,X) of the target variable for each time window and normalize the information gain value to a score between 0 and 1: G(Y,X)=H(Y)-H(Y|X) Where H(Y) is the entropy of the target variable, H(Y|X) is the conditional entropy of the target variable under a given feature X, the target variable Y usually refers to the result data that you want to predict or classify, and feature X refers to the feature within each time window.
6. The method for adaptive optimization of a machine learning model for an edge node of an Internet of Things as claimed in claim 5, characterized in that: The calculation method of the conditional entropy H(Y|X) is: Step 1), counting the joint frequency of the target variable Y and the time window feature X; Step 2) Calculate the joint probability distribution P(X=x) based on the statistical quantity i ,Y=y j ); Step 3) Calculate the conditional probability distribution, that is, the conditional probability P(Y = y) of the target variable given the time window characteristics j |X=x i ): Step 4), calculate the conditional entropy H(Y|X) by:
7. The method for adaptive optimization of a machine learning model for an edge node of an Internet of Things as claimed in claim 1, characterized in that: The method for determining whether the convolution kernel is located in the important time window in step S5 is: Step S5-1, calculate the convolution kernel receptive field Sen by the convolution kernel size and step length, the method is: Sen=k size +(k size -1)*(k step -1), where k size is the convolution kernel size, k step is the convolution kernel step size; Step S5-2, for each position i of the neural network output sequence, calculate the center position i of its corresponding input sequence center , the method is: Step S5-3, based on the calculation of the starting time index start and the ending time index end corresponding to the receptive field Sen of the convolution kernel, the method is: (start, end) is the index range of the convolution kernel in the input sequence, and this index is used to determine whether the convolution kernel is within the important time window.
8. The method for adaptive optimization of a machine learning model for an edge node of an Internet of Things as claimed in claim 1, characterized in that: The adjustment method of the CNN network model in step S6 is: according to the comprehensive score of each time window obtained in step S3, the convolution kernel scale of the pooling layer corresponding to the important time window is reduced; The method for adjusting the RNN network model comprises the following steps: Step S6-1, calculating the model loss function in the forward propagation phase of the training phase; Step S6-2, calculating the contribution of the hidden state of each time step to the model loss function in the back propagation stage; Step S6-3, reducing the number of hidden units corresponding to the time step whose contribution value is less than a certain threshold, and the reduction is achieved by reducing the dimension of the hidden state weight matrix corresponding to the hidden unit.