Internet of Things time sequence analysis method based on adaptive hypergraph neural network

Abstract

The invention relates to the technical field of data analysis, in particular to an Internet of Things time sequence analysis method based on a self-adaptive hypergraph neural network, which comprises the following steps: acquiring a plurality of human activity identification data, determining a selected data volume according to data fluctuation, determining activity label similarity according to information association degree, and obtaining a time sequence analysis result; according to the activity label similarity, human activity identification data is selected as training data according to the overall similarity and the selection similarity or according to the amplitude change rate; according to the motion complexity of the training data, determining whether to increase and adjust the scale feature level to determine a target scale feature level; and the features of the points are connected through Tensor, and classification is carried out through Linear to obtain a prediction category result corresponding to the training data. The method can improve the recognition precision of the human activity recognition task.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to an Internet of Things (IoT) time series analysis method based on an adaptive hypergraph neural network. Background Technology

[0002] Current IoT time series analysis methods based on human activity recognition data have many shortcomings in key technical aspects. These shortcomings directly restrict the model's ability to capture complex activity patterns. When faced with multi-dimensional time series signals, dynamic variable correlations, and similar activity differentiation, they are unable to meet the requirements for high-precision and strong generalization of activity recognition, resulting in poor accuracy of human activity recognition tasks and even problems such as misjudgment of similar activities and missed detection of abnormal activities. Therefore, how to improve the recognition accuracy of human activity recognition tasks is a technical problem that urgently needs to be solved by those skilled in the art.

[0003] Chinese Patent Publication No. CN113792768A discloses a hypergraph neural network classification method and apparatus. The method includes: acquiring label data to be predicted; constructing a hypergraph correlation matrix and an original feature matrix of the label data to be predicted; constructing hypergraph neural networks for different modalities in the label data to be predicted, generating a first hypergraph neural network model; matching and merging a preset second hypergraph neural network model with the first hypergraph neural network model, replacing the untrained parameters of each layer in the first hypergraph neural network model with trained feature transformation parameters; and inputting the hypergraph correlation matrix and the original feature matrix into the first hypergraph neural network model to obtain the final classification prediction result. However, the above scheme has the following problems: it lacks a scientific basis for training data selection and does not dynamically adapt to the needs of multi-scale feature extraction, making it difficult to accurately capture the temporal characteristics of the data, thus resulting in poor accuracy of classification prediction. Summary of the Invention

[0004] To address this, the present invention provides an IoT time series analysis method based on an adaptive hypergraph neural network, which overcomes the problems in existing technologies such as the lack of scientific training data selection criteria and the failure to dynamically adapt to the needs of multi-scale feature extraction, making it difficult to accurately capture the temporal characteristics of the data, thus resulting in poor accuracy in classification and prediction.

[0005] To achieve the above objectives, this invention provides an IoT time series analysis method based on an adaptive hypergraph neural network, comprising:

[0006] Acquire a number of human activity recognition data, determine the amount of data to be selected based on the data fluctuation, determine the activity label similarity based on the information correlation, and select human activity recognition data as training data based on the overall similarity and the selected similarity or based on the amplitude change rate.

[0007] Determine whether to increase the scale feature level based on the action complexity of the training data to determine the target scale feature level.

[0008] Based on the target scale feature level, multi-scale feature extraction is performed on each training data to obtain a combination of feature data output at each scale. Edge hidden features are obtained based on the feature data combination, edge features are obtained based on the edge hidden features, and point features are obtained based on the edge features.

[0009] The features at each point are connected using Tensors, and then classified using Linear methods to obtain the predicted category results corresponding to the training data.

[0010] Furthermore, the amount of data to be selected is determined based on the data volatility;

[0011] The amount of data selected is positively correlated with the data volatility.

[0012] Furthermore, the similarity of activity tags is determined based on the relevance of information, including:

[0013] If the information relevance is greater than or equal to the preset information relevance, the activity tag similarity is determined based on the information relevance.

[0014] If the information relevance is less than the preset information relevance, the activity tag similarity is determined based on the motion smoothness similarity.

[0015] Furthermore, if the activity label similarity is greater than or equal to the preset activity label similarity, then human activity recognition data is selected as training data based on the overall similarity and the selected similarity.

[0016] When selecting human activity recognition data as training data based on overall similarity and selection similarity, the data selection order corresponding to each activity label is determined according to the similarity evaluation value. Training data corresponding to priority activity labels is selected based on overall similarity, and training data corresponding to non-priority activity labels is selected based on selection similarity.

[0017] The priority activity label is the activity label with the highest priority coefficient, and the non-priority activity label is any other activity label besides the priority activity label.

[0018] Furthermore, if the activity label similarity is less than the preset activity label similarity, then human activity recognition data is selected as training data based on the rate of change of amplitude.

[0019] When selecting human activity recognition data as training data based on the amplitude change rate, training data is selected for each activity label. When selecting training data for a single activity label, the human activity data corresponding to that activity label is sorted in descending order of amplitude change rate. The sorted human activity data is then evenly divided into a preset number of subsequences, and any human activity data is selected from each subsequence as training data.

[0020] The preset quantity is determined based on the ratio of the selected data volume to the types of activity tags.

[0021] Furthermore, if the action complexity of the training data is greater than or equal to the preset action complexity, then the scale feature level is increased.

[0022] The increase in the scale feature level is positively correlated with the comparison difference, which is determined based on the difference between the action complexity and the preset action complexity.

[0023] Furthermore, the complexity of the action is determined based on the mean of the inter-axis correlation;

[0024] The motion complexity and the inter-axis correlation are negatively correlated.

[0025] Furthermore, formulas (1), (2), and (3) are used to perform multi-scale feature extraction on each training data to obtain the feature data combination of each scale output. Formulas (1), (2), and (3) are as follows:

[0026] Layer0 = Conv(ε,9) (1),

[0027] LayerBlock n =Relu(Conv(ε,n)+MaxPool(Conv(ε,(n-1) / 2) (2),

[0028] Layer1=LayerBlock1(Layer0),Layer2=

[0029] LayerBlock2(Layer1),...,Layerk=LayerBlock k (Layer(k-1)) (3),

[0030] Where Layer0 represents the feature data combination of low-scale features, Conv represents the convolution operation, ε represents each training data input, 9 represents the convolution kernel size of 9, LayerBlock is the multi-scale feature data extraction module, n represents the convolution kernel size of n, Relu represents the Relu non-linear activation function, MaxPool represents the max pooling operation, Layer1, Layer2, ..., Layerk represent the feature data combination of higher-scale features in turn, and k is the target scale feature level.

[0031] Furthermore, when obtaining the hidden edge features based on the combination of feature data, formulas (4), (5), (6), (7), and (8) are used. Formulas (4), (5), (6), (7), and (8) are respectively:

[0032] Ftraj m =permute(Layer m (4),

[0033] Result m =MLP(Hdim,HdimExtend,(256),Ftraj m (5),

[0034] H = matmul(Ftraj) m ,permute(Ftraj m (6), AttentionWeight m =MLP(HdimExtend*2,1,(32,),matmul(H,Result m (7), EdgeHidden m =matmul(softmax(AttentionWeight m *H)*H,Result m (8),

[0035] Among them, Ftraj m Let m be the trajectory matrix, and permute denote the matrix transpose operation. m represents 0, 1, 2, ..., k. Result m For the trajectory matrix Ftraj m The feature representation obtained after inputting a multilayer perceptron, where AttentionWeight is the attention weight, MLP is the multilayer perceptron, and Hdim is the trajectory matrix Ftraj. mThe feature dimension is HdimExtend, which is a fixed value of 64. 256 is the number of neurons in the hidden layer of the network. 1 is the output dimension of the MLP. (32,) is the dimension of the hidden layer of the MLP. H is the feature matrix. softmax is the normalized exponential function. EdgeHidden is the edge hidden feature.

[0036] Edge features are obtained from hidden edge features using formula (9), and point features are obtained from edge features using formula (10). Formulas (9) and (10) are respectively:

[0037] EdgeFeat m =MLP(EdgeHidden) m (9),

[0038] NodeFeat m =MLP(matmul(EdgeFeat) m ,Ftraj m (10),

[0039] Among them, EdgeFeat represents edge features, and NodeFeat represents point features.

[0040] Furthermore, formula (11) is used to connect the features of each point using Tensors, and formula (12) is used to classify them using Linear. Formulas (11) and (12) are as follows:

[0041] X=cat(NodeFeat0,NodeFeat1,...,NodeFeat k (11),

[0042] Result=Linear(AvgPool(X)) (12),

[0043] Where cat is the tensor concatenation operation, X is the output result at each scale, AvgPool is the average pooling layer operation, Linear is the linear classification operation, and Result is the predicted class result of the training data.

[0044] Compared with the prior art, the beneficial effect of the present invention is that, in the technical solution of the present invention, the overall dispersion of the activity parameters of human activity recognition data is effectively reflected by the data volatility, and the amount of data to be selected is determined according to the data volatility. This enables dynamic matching between the amount of training data and the diversity of data features, which is beneficial to improving training efficiency while ensuring the model learning effect.

[0045] Furthermore, this invention effectively reflects the closeness of information matching between different activity tags at the mean parameter level through information correlation, and then determines the similarity of activity tags based on information correlation, making the confirmation of activity tag similarity more in line with actual application scenarios and providing a reliable basis for tag association for subsequent training data selection.

[0046] Furthermore, this invention effectively reflects the overall correlation between different activity labels through activity label similarity, and then adaptively determines different training data selection methods based on activity label similarity, providing a high-quality data foundation for subsequent model training, which in turn helps to improve the recognition accuracy of human activities.

[0047] Furthermore, this invention effectively reflects the complexity of human activities in the spatial dimension through action complexity, and then determines whether to increase the scale feature level based on the action complexity. This can avoid the recognition deviation of complex actions due to insufficient feature extraction, thereby improving the recognition accuracy of high-complexity actions. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the IoT time series analysis method based on adaptive hypergraph neural network of the present invention;

[0049] Figure 2 This is a flowchart illustrating how the similarity of activity tags is determined based on information relevance in this invention.

[0050] Figure 3 This is a flowchart illustrating how the present invention determines human activity recognition data as training data based on overall similarity or by selecting similarity or by amplitude change rate, according to the similarity of activity tags.

[0051] Figure 4 This is a flowchart illustrating the process of determining whether to increase the scale feature level based on the action complexity of the training data according to the present invention. Detailed Implementation

[0052] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0053] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0054] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0055] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0056] Please see Figures 1 to 4 As shown, this invention provides a method for IoT time series analysis based on an adaptive hypergraph neural network, comprising:

[0057] Acquire a number of human activity recognition data, determine the amount of data to be selected based on the data fluctuation, determine the activity label similarity based on the information correlation, and select human activity recognition data as training data based on the overall similarity and the selected similarity or based on the amplitude change rate.

[0058] Determine whether to increase the scale feature level based on the action complexity of the training data to determine the target scale feature level.

[0059] Based on the target scale feature level, multi-scale feature extraction is performed on each training data to obtain a combination of feature data output at each scale. Edge hidden features are obtained based on the feature data combination, edge features are obtained based on the edge hidden features, and point features are obtained based on the edge features.

[0060] The features at each point are connected using Tensors, and then classified using Linear methods to obtain the predicted category results corresponding to the training data.

[0061] The application scenario of this invention is multi-scale feature extraction and classification prediction of human daily activities in smart homes;

[0062] This invention includes several historical records. Each historical record records at least one instance of multi-scale feature extraction and classification prediction of human daily activities in a smart home, including the amount of selected data, data volatility, information correlation, and motion smoothness similarity. Each historical record also has a corresponding qualified marker, which records whether the process of multi-scale feature extraction and classification prediction of human daily activities in a smart home meets the user's needs. The qualified marker can be recorded manually. It is understood that the user can determine whether the process of multi-scale feature extraction and classification prediction of human daily activities in a smart home meets the needs based on self-defined indicators. Self-defined indicators can be, but are not limited to, prediction anomaly, which will not be elaborated here. Prediction anomaly = the number of training data whose prediction category results do not match the activity labels corresponding to the training data / the number of training data.

[0063] The single human activity identification data includes activity parameters corresponding to each time point within a single time period and activity tags corresponding to the human activity identification data. The duration of a single time period is 10 minutes. A method for setting time points is provided, taking the start time of a single time period as the starting point, setting an interval point every 1 second, and recording the starting point and each interval point as time points. Activity parameters include, but are not limited to, acceleration on the x, y, and z axes and rotational angular velocity on the x, y, and z axes. Acceleration and rotational angular velocity are monitored by accelerometer and gyroscope sensors installed on a smartphone, respectively. Activity tags include, but are not limited to, walking, running, and climbing stairs. This is content that is easily understood by those skilled in the art and will not be elaborated further.

[0064] Specifically, the amount of data to be selected is determined based on the data volatility;

[0065] The amount of data selected is positively correlated with the data volatility.

[0066] Among them, the data volatility is the average value of the volatility coefficients corresponding to each activity parameter, the volatility coefficient corresponding to a single activity parameter is the standard deviation of the data mean of the activity parameter corresponding to each human activity identification data, and the data mean of the activity parameter corresponding to a single human activity identification data is the average value of the activity parameter corresponding to each time point in the human activity identification data.

[0067] Selected data volume = Average of the selected data volume corresponding to the historical records that can meet the user's needs × (Data volatility / Average of the data volatility corresponding to the historical records that can meet the user's needs).

[0068] Specifically, the similarity of activity tags is determined based on the relevance of information, including:

[0069] If the information relevance is greater than or equal to the preset information relevance, the activity tag similarity is determined based on the information relevance.

[0070] If the information relevance is less than the preset information relevance, the activity tag similarity is determined based on the motion smoothness similarity.

[0071] Among them, the information relevance is the average of the sub-information relevance of each activity tag. For a single activity tag, the activity tag is recorded as the target activity tag. The sub-information relevance of the target activity tag is the average of the information reference values ​​of the target activity tag and other activity tags excluding the target activity tag.

[0072] For any two activity labels, the information reference value corresponding to the two activity labels is the average of the similar reference values ​​corresponding to each activity parameter. The similar reference value corresponding to a single activity parameter is the larger of 1-|u1-u2| / u1 and u2. The average of the data mean corresponding to the activity parameter in each human activity recognition data corresponding to one activity label is recorded as u1, and the average of the data mean corresponding to the activity parameter in each human activity recognition data corresponding to another activity label is recorded as u2.

[0073] The value of the preset information relevance can be determined by the user according to the actual application scenario. The smaller the value of the preset information relevance, the greater the user's need to determine the similarity of activity tags based on the information relevance. A method for determining the value of the preset information relevance is provided, which detects the historical records of users determining the similarity of activity tags based on motion smoothness similarity, and records the average value of the information relevance corresponding to the historical records that can meet the user's needs as the preset information relevance.

[0074] Motion smoothing similarity is the average of the sub-motion smoothing similarities corresponding to each activity label, and the sub-motion smoothing similarity corresponding to a single activity label is the average of the smoothing similarities between that activity label and all other activity labels except that activity label;

[0075] For any two activity labels, the smooth similarity between the two activity labels = (1 - acceleration smooth difference value / preset acceleration smooth difference value) + (1 - rotational angular velocity smooth difference value / preset rotational angular velocity smooth difference value);

[0076] The acceleration smoothing difference value corresponding to two active labels is the absolute value of the difference between the smoothing mean values ​​of acceleration corresponding to two active labels, and the rotational angular velocity smoothing difference value corresponding to two active labels is the absolute value of the difference between the smoothing mean values ​​of rotational angular velocity corresponding to two active labels.

[0077] The mean of acceleration smoothing and the mean of rotational angular velocity smoothing for a single activity tag correspond to the average of the acceleration smoothness and the average of the rotational angular velocity smoothness of each human activity recognition data corresponding to that activity tag, respectively.

[0078] The acceleration vector and rotational angular velocity vector at a single time point are respectively and Among them, a x a y and a z These represent the accelerations on the x, y, and z axes at that time point, respectively, v x v y and v z These are the rotational angular velocities along the x, y, and z axes at that time point, respectively;

[0079] For a single time point, this time point is designated as the target time point, and the acceleration vector and rotational angular velocity vector corresponding to the target time point are respectively designated as follows: and The acceleration vector and rotational angular velocity vector corresponding to the time points adjacent to but later than the target time point are respectively denoted as: and The formula for calculating the angle ε1 between the acceleration vectors at the target time point is: The formula for calculating the angle ε2 between the rotational angular velocity vectors is:

[0080] Acceleration smoothness = 1 / (standard deviation of the angle between acceleration vectors at each time point + 1); Rotational angular velocity smoothness = 1 / (standard deviation of the angle between rotational angular velocity vectors at each time point + 1); It should be noted that ε1 and ε2 corresponding to the last time point of each human activity recognition data are both recorded as 0°;

[0081] When determining the similarity of activity tags based on information relevance, the similarity of activity tags = information relevance / the average information relevance of each historical record that can meet the user's needs;

[0082] When determining the similarity of activity tags based on motion smoothness similarity, the similarity of activity tags = motion smoothness similarity / the average of the motion smoothness similarities corresponding to each historical record that can meet the user's needs;

[0083] It is understandable that information correlation can effectively quantify the closeness of information matching between different activity tags at the mean level of parameters. When the information correlation is greater than or equal to the preset information correlation, it indicates that the matching degree of each activity tag at the mean level of core parameters is relatively large. At this time, the similarity between tags can be accurately quantified by information correlation alone.

[0084] When the information correlation is less than the preset information correlation, it indicates that the matching degree of each activity label at the mean level of the core parameters is small. At this time, the information correlation is insufficient to support accurate judgment. It is necessary to supplement the dynamic features of the activity and use the smooth features of acceleration and rotation angular velocity during the activity to improve the quantitative dimension of label similarity. Therefore, the similarity of activity labels is determined based on motion smoothness similarity.

[0085] Specifically, if the activity label similarity is greater than or equal to the preset activity label similarity, then human activity recognition data is selected as training data based on the overall similarity and the selected similarity.

[0086] When selecting human activity recognition data as training data based on overall similarity and selection similarity, the data selection order corresponding to each activity label is determined according to the similarity evaluation value. Training data corresponding to priority activity labels is selected based on overall similarity, and training data corresponding to non-priority activity labels is selected based on selection similarity.

[0087] The priority activity label is the activity label with the highest priority coefficient, and the non-priority activity label is any other activity label besides the priority activity label.

[0088] The user can determine the value of the preset activity label similarity based on the actual application scenario. The greater the user's need for accuracy in widely filtering training data through the feature differences of the data itself, the smaller the value of the preset activity label similarity. A method for determining the value of the preset activity label similarity is provided, which detects the historical records of human activity recognition data selected by the user as training data based on the overall similarity and the selected similarity. The average value of the activity label similarity corresponding to the historical records that can meet the user's needs is recorded as the preset activity label similarity.

[0089] For a single activity tag, the similarity evaluation value corresponding to the activity tag = the sub-motion smoothness similarity corresponding to the activity tag / the average of the sub-motion smoothness similarities corresponding to all activity tags + the sub-information correlation / the average of the sub-information correlation of all activity tags;

[0090] The selection priority coefficient for a single activity tag is negatively correlated with the similarity evaluation value corresponding to that activity tag; the selection priority coefficient for a single activity tag = 1 - the similarity evaluation value corresponding to that activity tag / the average of the similarity evaluation values ​​corresponding to all activity tags;

[0091] The higher the selection priority coefficient for a single activity label, the higher the priority of the training data selection for that activity label;

[0092] When selecting training data for a priority activity label, human activity data corresponding to the priority activity label is selected in order of increasing overall similarity until a preset number is reached. When selecting training data for a single non-priority activity label, human activity data corresponding to the non-priority activity label is selected in order of increasing similarity until a preset number is reached.

[0093] The preset quantity is the smallest integer greater than or equal to n0, where n0 = the amount of data selected / the types of activity tags;

[0094] The overall similarity of a single human activity data point is the average of the similarity coefficients between that human activity data point and other human activity data points excluding that human activity data point;

[0095] The similarity of a single human activity data point is the average of the similarity coefficients between that human activity data point and the human activity data points selected by the activity labels in the training data selection process.

[0096] The similarity coefficient between two sets of human activity data = mean similarity between two sets of human activity data / preset mean similarity + smooth similarity between two sets of human activity data / preset smooth similarity;

[0097] Mean similarity is the average of the sub-similarity scores corresponding to each activity parameter in two sets of human activity data. For a single activity parameter, the larger of the mean values ​​of the data corresponding to that activity parameter in the two sets of human activity data is denoted as Umax, and the smaller of the mean values ​​is denoted as Umin. The sub-similarity score corresponding to that activity parameter is calculated as 1 - (Umax - Umin) / Umax.

[0098] The smoothed similarity between two human activity data points = 1 - the absolute value of the difference between the smoothed reference values ​​of the two human activity data points / the larger of the smoothed reference values ​​of the two human activity data points;

[0099] The smoothing reference value for a single human activity data point = the acceleration smoothness corresponding to that human activity data point / the average acceleration smoothness corresponding to all human activity data points + the rotational angular velocity smoothness corresponding to that human activity data point / the average rotational angular velocity smoothness corresponding to all human activity data points;

[0100] Users can determine the preset mean similarity and preset smooth similarity values ​​according to the actual application scenario. The greater the user's need to improve the sensitivity of similarity coefficient determination, the smaller the preset mean similarity and preset smooth similarity values ​​will be. One preset mean similarity value is provided: 80% and 70% preset smooth similarity.

[0101] It is understandable that the similarity of activity labels can effectively reflect the overall degree of correlation between different activity labels. When the similarity of activity labels is greater than or equal to the preset similarity of activity labels, it indicates that the overall correlation between the activity labels is high. If training data is simply selected randomly, it is easy to cause sample redundancy due to high feature overlap, which will lead to the model overfitting to repeated features. Therefore, it is necessary to conduct fine screening based on the overall similarity and the selected similarity, and prioritize retaining samples with significant differences to avoid the model learning repeated features.

[0102] When the similarity of activity labels is less than the preset similarity of activity labels, it indicates that the overall difference between the activity labels is significant. By using stratified sampling based on the rate of change of amplitude, the model avoids recognition bias caused by the sample being biased towards a certain intensity, and ultimately improves the generalization ability of different performance forms of the same type of activity.

[0103] Specifically, if the activity label similarity is less than the preset activity label similarity, then human activity recognition data is selected as training data based on the rate of change of amplitude.

[0104] When selecting human activity recognition data as training data based on the amplitude change rate, training data is selected for each activity label. When selecting training data for a single activity label, the human activity data corresponding to that activity label is sorted in descending order of amplitude change rate. The sorted human activity data is then evenly divided into a preset number of subsequences, and any human activity data is selected from each subsequence as training data.

[0105] The preset quantity is determined based on the ratio of the selected data volume to the types of activity tags.

[0106] Each subsequence contains the same amount of human activity data;

[0107] The rate of change of amplitude corresponding to a single human activity data point is the average of the sub-rates of change corresponding to each activity parameter. The sub-rate of change corresponding to a single activity parameter is calculated as follows: (maximum value of the activity parameter at each time point in the human activity data point - minimum value of the activity parameter at each time point in the human activity data point) / (average value of the activity parameter at each time point in the human activity data point + 1).

[0108] Specifically, if the action complexity of the training data is greater than or equal to the preset action complexity, then the scale feature level will be increased.

[0109] The increase in the scale feature level is positively correlated with the comparison difference, which is determined based on the difference between the action complexity and the preset action complexity.

[0110] If the action complexity of the training data is less than the preset action complexity, then there is no need to increase the scale feature level.

[0111] Comparison difference = Action complexity - Preset action complexity;

[0112] The value of the preset action complexity can be determined by the user according to the actual application scenario. The greater the user's need to improve the accuracy of classification prediction, the smaller the value of the preset action complexity. A method for determining the value of the preset action complexity is provided: detect the historical records of increasing the scale feature level, and record the average value of the action complexity corresponding to the historical records that can meet the user's needs as the preset action complexity.

[0113] The increment value of the scale feature level is the smallest integer greater than or equal to k0, where k0 = initial scale feature level × comparison difference / preset action complexity, and the initial scale feature level is 3;

[0114] The target scale feature level is k, where k = k0 + 3;

[0115] It is understandable that motion complexity effectively reflects the spatial complexity of human activities. When the motion complexity of the training data is greater than or equal to the preset motion complexity, it indicates that the current training data needs to capture more fine motion features in the spatial dimension. The existing initial scale feature level is no longer sufficient to extract these complex details. Therefore, it is necessary to increase the scale feature level and extract more multi-scale features to cover all-dimensional features from overall motion to local fine motion, ultimately improving the classification accuracy of complex motions.

[0116] Specifically, the complexity of the action is determined based on the mean of the inter-axis correlation;

[0117] The motion complexity and the inter-axis correlation are negatively correlated.

[0118] Wherein, motion complexity = 1 - (inter-axis correlation mean / average of inter-axis correlation mean corresponding to historical records that can meet user needs);

[0119] The mean of inter-axis correlation is the average of the inter-axis correlations for each training data point;

[0120] The inter-axis correlation of a single training data point = inter-axis correlation of acceleration × first weight coefficient + inter-axis correlation of rotational angular velocity × second weight coefficient; the inter-axis correlation of acceleration is the average of the acceleration matching coefficients corresponding to the x, y and z axes, and the inter-axis correlation of rotational angular velocity is the average of the rotational angular velocity matching coefficients corresponding to the x, y and z axes, with both the first and second weight coefficients being 0.5;

[0121] The acceleration matching coefficient for a single axis is the average of the sub-acceleration matching degree of that axis and the other two axes; the rotational angular velocity matching coefficient for a single axis is the average of the sub-rotational angular velocity matching degree of that axis and the other two axes.

[0122] The formula for calculating the sub-accelerometer matching degree r1 for any two axes is:

[0123]

[0124] Where j represents the number of time points in a single human activity dataset; x h and y h These are the accelerations at the h-th time point in the accelerations corresponding to the two axes, respectively. For x h The average value of the acceleration at each time point corresponding to the axis. For y h The average value of the acceleration at each time point in the acceleration corresponding to the axis, h = 1, 2, 3, ..., j;

[0125] The formula for calculating the sub-rotational angular velocity matching degree r2 for any two axes is:

[0126]

[0127] Where x1 h and y1 h These are the rotational angular velocities at the h-th time point, corresponding to the rotational angular velocities of the two axes, respectively. x1 h The average value of the rotational angular velocity at each time point corresponding to the axis. For y1 h The average value of the rotational angular velocity at each time point corresponding to the rotational angular velocity of the axis in question;

[0128] It is understandable that the mean of inter-axis correlation can effectively reflect the degree of spatial coordination and synchronization of acceleration and rotational angular velocity along the x / y / z axes during human activities. Since the complexity of human activities depends on the multi-directional independent motion dimensions required to complete the activity, the lower the coordination, the more complex the motion dimensions, meaning that the x / y / z axes need to carry out independent motion changes. Therefore, the complexity of the action can be inferred from the level of the mean of inter-axis correlation.

[0129] Specifically, formulas (1), (2), and (3) are used to extract features from each training data at multiple scales to obtain a combination of feature data output at each scale. Formulas (1), (2), and (3) are as follows:

[0130] Layer0 = Conv(ε,9) (1),

[0131] LayerBlock n =Relu(Conv(ε,n)+MaxPool(Conv(ε,(n-1) / 2) (2),

[0132] Layer1=LayerBlock1(Layer0),Layer2=

[0133] LayerBlock2(Layer1),...,Layerk=LayerBlock k (Layer(k-1)) (3),

[0134] Where Layer0 represents the feature data combination of low-scale features, Conv represents the convolution operation, ε represents each training data input, 9 represents the convolution kernel size of 9, LayerBlock is the multi-scale feature data extraction module, n represents the convolution kernel size of n, Relu represents the Relu non-linear activation function, MaxPool represents the max pooling operation, Layer1, Layer2, ..., Layerk represent the feature data combination of higher-scale features in turn, and k is the target scale feature level.

[0135] Where (n-1) / 2 represents the convolution kernel size as (n-1) / 2.

[0136] Specifically, when obtaining edge hidden features based on feature data combination, formulas (4), (5), (6), (7), and (8) are used. Formulas (4), (5), (6), (7), and (8) are respectively:

[0137] Ftraj m =permute(Layer m (4),

[0138] Result m =MLP(Hdim,HdimExtend,(256),Ftraj m (5),

[0139] H = matmul(Ftraj) m ,permute(Ftraj m (6), AttentionWeight m =MLP(HdimExtend*2,1,(32,),matmul(H,Result m(7), EdgeHidden m =matmul(softmax(AttentionWeight m *H)*H,Result m (8),

[0140] Among them, Ftraj m Let m be the trajectory matrix, and permute denote the matrix transpose operation. m represents 0, 1, 2, ..., k. Result m For the trajectory matrix Ftraj m The feature representation obtained after inputting a multilayer perceptron, where AttentionWeight is the attention weight, MLP is the multilayer perceptron, and Hdim is the trajectory matrix Ftraj. m The feature dimension is HdimExtend, which is a fixed value of 64. 256 is the number of neurons in the hidden layer of the network. 1 is the output dimension of the MLP. (32,) is the dimension of the hidden layer of the MLP. H is the feature matrix. softmax is the normalized exponential function. EdgeHidden is the edge hidden feature.

[0141] Edge features are obtained from hidden edge features using formula (9), and point features are obtained from edge features using formula (10). Formulas (9) and (10) are respectively:

[0142] EdgeFeat m =MLP(EdgeHidden) m (9),

[0143] NodeFeat m =MLP(matmul(EdgeFeat) m ,Ftraj m (10),

[0144] Among them, EdgeFeat represents edge features, and NodeFeat represents point features.

[0145] HdimExtend*2=64×2=128.

[0146] It is understandable that by using matrix operations, MLP transformations, and other operations, the scattered multi-scale features are transformed into edge features and point features that contain "global collaborative associations", thus realizing the complete process of multi-feature representation from scattered representation to association and fusion in hypergraph neural networks.

[0147] Specifically, formula (11) is used to connect the features of each point using Tensors, and formula (12) is used to classify using Linear. Formulas (11) and (12) are as follows:

[0148] X=cat(NodeFeat0,NodeFeat1,...,NodeFeat k (11),

[0149] Result=Linear(AvgPool(X)) (12),

[0150] Where cat is the tensor concatenation operation, X is the output result at each scale, AvgPool is the average pooling layer operation, Linear is the linear classification operation, and Result is the predicted class result of the training data.

[0151] Understandably, all point features are fused using a Tensor to form a unified representation that integrates all features and their associated information, avoiding information loss caused by simple splicing. The fused tensor is then input into a linear classifier, which learns the linear mapping relationship between high-dimensional features and activity categories to output the final predicted category result.

[0152] The predicted categories include, but are not limited to, walking, running, and climbing stairs.

[0153] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An adaptive hypergraph neural network-based Internet of Things time series analysis method, characterized in that, The method comprises the following steps: acquiring a plurality of human activity recognition data, determining a selected data amount according to data fluctuation degree, determining activity label similarity according to information correlation degree, and determining human activity recognition data as training data according to overall similarity and selected similarity or according to amplitude change rate based on the activity label similarity; determining whether to increase the scale feature level for adjustment according to the action complexity of the training data to determine the target scale feature level; extracting multi-scale features from each training data based on the target scale feature level to obtain a feature data combination of each scale output, obtaining edge hidden features based on the feature data combination, obtaining edge features based on the edge hidden features, and obtaining point features based on the edge features; connecting each point feature through Tensor and classifying through Linear to obtain the predicted category result corresponding to the training data.

2. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 1, characterized in that, determining the selected data amount according to the data fluctuation degree; the selected data amount and the data fluctuation degree are in a positive correlation relationship.

3. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 2, characterized in that, determining the activity label similarity according to the information correlation degree, comprising: if the information correlation degree is greater than or equal to a preset information correlation degree, determining the activity label similarity according to the information correlation degree; if the information correlation degree is less than the preset information correlation degree, determining the activity label similarity according to the motion smoothing similarity.

4. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 3, characterized in that, if the activity label similarity is greater than or equal to a preset activity label similarity, selecting human activity recognition data as training data according to the overall similarity and the selected similarity; when selecting human activity recognition data as training data according to the overall similarity and the selected similarity, determining the data selection order corresponding to each activity label according to the similarity evaluation value, selecting training data corresponding to the priority activity label based on the overall similarity, and selecting training data corresponding to the non-priority activity label based on the selected similarity; the priority activity label is the activity label with the largest priority selection coefficient, and the non-priority activity label is the activity label other than the priority activity label.

5. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 4, characterized in that, if the activity label similarity is less than the preset activity label similarity, selecting human activity recognition data as training data according to the amplitude change rate; when selecting human activity recognition data as training data according to the amplitude change rate, selecting training data for each activity label, and when selecting training data for a single activity label, sorting each human activity data corresponding to the activity label in descending order of amplitude change rate, and uniformly dividing the sorted human activity data into a preset number of subsequences, and selecting any human activity data in each subsequence as training data; the preset number is determined based on the ratio of the selected data amount to the type of activity label.

6. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 5, characterized in that, if the action complexity of the training data is greater than or equal to a preset action complexity, increasing the scale feature level for adjustment; the increase value of the scale feature level and the comparison difference value are in a positive correlation relationship, and the comparison difference value is determined based on the difference between the action complexity and the preset action complexity.

7. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 6, characterized in that, the action complexity is determined based on the inter-axis correlation mean value; the action complexity and the inter-axis correlation mean value are in a negative correlation relationship.

8. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 7, characterized in that, The training data is subjected to multi-scale feature extraction by using formula (1), formula (2) and formula (3) to obtain a feature data combination of each scale output, and formula (1), formula (2) and formula (3) are respectively: Layer0 = Conv(ε, 9) (1), LayerBlock n = Relu(Conv(ε, n) + MaxPool(Conv(ε, (n - 1) / 2) (2), Layer1 = LayerBlock1(Layer0), Layer2 = LayerBlock2(Layer1),..., Layerk = LayerBlock k (Layer(k - 1)) (3), Wherein, Layer0 represents the feature data combination of low scale features, Conv represents convolution operation, ε is the input of each training data, 9 represents that the convolution kernel size is 9, LayerBlock is a multi-scale feature data extraction module, n represents that the convolution kernel size is n, Relu represents a Relu nonlinear activation function, MaxPool represents a maximum pooling operation, Layer1, Layer2, …, Layerk represent the feature data combinations of higher scale features in turn, and k is the target scale feature level.

9. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 8, characterized in that, When the edge hidden features are obtained based on the feature data combination, formula (4), formula (5), formula (6), formula (7) and formula (8) are used for obtaining, and formula (4), formula (5), formula (6), formula (7) and formula (8) are respectively: Ftraj m = permute(Layer m ) (4), Result m = MLP(Hdim, HdimExtend, (256), Ftraj m ) (5), H = matmul(Ftraj m , permute(Ftraj m )) (6), AttentionWeight m = MLP(Hdim Extend * 2, 1, (32,), matmul(H, Result m )) (7), EdgeHidden m = matmul(softmax(AttentionWeight m *H)*H, Result m ) (8), where Ftraj m is the trajectory matrix, permute denotes the matrix transpose operation, m is 0, 1, 2, …, k, Result m is the trajectory matrix Ftraj m is the input feature representation obtained after a multi-layer perception, AttentionWeight is the attention weight, MLP is a multi-layer perception, Hd im is the feature dimension of the trajectory matrix Ftraj m is a fixed value of 64, 256 is the number of neurons in the network hidden layer, 1 is the output latitude of the MLP, (32,) is the dimension size of the MLP hidden layer, H is the feature matrix, softmax is a normalized exponential function, and EdgeHidden is the edge hidden feature. When the edge features are obtained according to the edge hidden features, formula (9) is used for obtaining, and when the point features are obtained based on the edge features, formula (10) is used for obtaining, and formula (9) and formula (10) are respectively: EdgeFeat m = MLP(EdgeHidden m ) (9), NodeFeat m = MLP(matmul(EdgeFeat m , Ftraj m ) (10), Wherein, EdgeFeat is the edge feature, and NodeFeat is the point feature.

10. The adaptive hypergraph neural network based Internet of Things time series analysis method according to claim 9, characterized in that, Formula (11) is used to connect each point feature through Tensor, and formula (12) is used for classification through Linear, and formula (11) and formula (12) are respectively: X = cat(NodeFeat0, NodeFeat1,..., NodeFeat k ) (11), Result=Linear(AvgPool(X)) (12), Wherein, cat is a tensor splicing operation, X is the result of each scale output, AvgPool is an average pooling layer operation, Linear is a linear classification operation, and Result is the prediction category result of the training data.

Images