Method for human behavior recognition oriented migration learning based on kolmogorov-arnold convolution filter
By employing the transfer learning method of Kolmogorov-Arnold convolutional filters, the problem of insufficient generalization ability of human behavior recognition models when device location and activity type change is solved, achieving high-precision and real-time human behavior recognition and reducing the conversion cost of recognition tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING NORMAL UNIVERSITY
- Filing Date
- 2026-03-04
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies lack the generalization ability of human behavior recognition models when faced with changes in device location, installation orientation, or activity type, making direct application difficult.
A transfer learning method based on Kolmogorov-Arnold convolutional filters is adopted. Temporal feature data is collected and preprocessed through wearable devices to build an activity recognition model. The model is trained using a preset prompt adjustment module and a classifier module, and then saved as a pt file for deployment on wearable devices.
It achieves high-precision human behavior recognition with a small number of labeled samples, reduces the conversion cost between different recognition tasks, and ensures the real-time and high-precision application of the model.
Smart Images
Figure CN121765607B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of computer technology, and in particular relates to a transfer learning method for human behavior recognition based on Kolmogorov-Arnold convolutional filters. Background Technology
[0002] Human behavior recognition based on time-series data (such as acceleration and angular velocity) from motion sensors has become a key technology in fields such as health monitoring and motion analysis due to its advantages of low power consumption and privacy protection. Current methods mainly rely on recognition models trained on specific datasets. However, current methods generally suffer from insufficient generalization ability and difficulty in direct application when faced with changes in device location, installation orientation, or activity type. Summary of the Invention
[0003] This application provides a transfer learning method for human behavior recognition based on Kolmogorov-Arnold convolutional filters, which can solve the problem that current methods generally have insufficient generalization ability and are difficult to directly transfer and apply when faced with changes in device location, installation direction or activity type.
[0004] In a first aspect, embodiments of this application provide a transfer learning method for human behavior recognition based on a Kolmogorov-Arnold convolutional filter, comprising: S1, collecting first temporal feature data through a wearable device, preprocessing the first temporal feature data to obtain second temporal feature data, and labeling the second temporal feature data with its corresponding activity category; S2, segmenting the second temporal feature data according to a sliding window to obtain multiple data segments of the same sequence length, and dividing the multiple data segments of the same sequence length into a training set, a validation set, and a test set according to a preset ratio; S3, integrating a preset prompt adjustment module and a preset classifier module... The blocks are respectively connected to the input and output of the preset temporal base model to construct an activity recognition model. The preset prompt adjustment module consists of three Kolmogorov-Arnold convolutional layers (KConv) with different receptive fields and an attention module. S4. The activity recognition model is trained according to the training set, and the backbone network parameters of the temporal base model are kept out of gradient update. The parameters of the activity recognition model are optimized using the Adam optimization algorithm until the optimal activity recognition model is obtained. S5. The optimal activity recognition model is saved as a pt file and the pt file is ported to the wearable device so that the wearable device can perform human behavior recognition.
[0005] In one possible implementation of the first aspect, the preprocessing of the first time feature data in S1 above to obtain the second time feature data specifically includes:
[0006] Perform data cleaning on the first-time feature data to remove outliers and null values;
[0007] The first time-specific feature data, after removing outliers and null values, is downsampled and normalized to obtain the second time-specific feature data.
[0008] Optionally, in another possible implementation of the first aspect, S4 above, training the activity recognition model based on the training set, while keeping the backbone network parameters of the temporal base model from participating in gradient updates, and optimizing the parameters of the activity recognition model using the Adam optimization algorithm until the optimal activity recognition model is obtained, specifically includes:
[0009] Each time-specific feature data in the training set Input is sent to the preset prompt adjustment module, and the sensor prompt is output. ;
[0010] Sensor prompt and feature data for each time period Input to the time series basic model, output sensor coded features ;
[0011] Encoding sensor features The input is fed into a preset classifier module, which then encodes sensor features through a fully connected layer. Mapped to the probability of each activity category;
[0012] The loss function is calculated based on the cross-entropy loss function, which determines the relationship between the probability of each activity category and the actual label. Through backpropagation and the Adam optimization algorithm, the parameters of the preset classifier module and the preset prompt adjustment module are updated using gradients until the loss function is reached. To reach the minimum;
[0013] Repeat the above process until the accuracy of the activity recognition model on the validation set reaches a preset threshold, then output the optimal activity recognition model.
[0014] Optionally, in another possible implementation of the first aspect, the above refers to the feature data of each time step in the training set. Input is sent to the preset prompt adjustment module, and the sensor prompt is output. Specifically, it includes:
[0015] Time feature data The first feature map, the second feature map, and the third feature map are respectively input into the first KConv, the second KConv, and the third KConv, and the corresponding first feature map, the second feature map, and the third feature map are output.
[0016] Time feature data Input to the attention module, and output the attention score corresponding to the first KConv, the attention score corresponding to the second KConv, and the attention score corresponding to the third KConv;
[0017] The first feature map, the second feature map, the third feature map, and the attention scores corresponding to the first KConv, the second KConv, and the third KConv are weighted and processed to output the sensor prompt. The details are as follows:
[0018]
[0019] in, The kernel size is Attention score Indicates the convolution size as The feeling of the wild, It is a learnable function that can be parameterized as a spline curve. For the location index is The input value.
[0020] Optionally, in another possible implementation of the first aspect, the attention module includes a global mean pooling layer, a linear layer, and a softmax function layer, and the process of obtaining the attention score specifically includes:
[0021] Using a global mean pooling layer to extract time feature data , Convert to ,in Indicates the length of the sequence after sliding window processing. Represented as the number of sequence channels, For the real number field;
[0022] Through linear layers Convert to a 3D vector ;
[0023] Using the Softmax function Convert to attention score.
[0024] Beneficial effects: First, sensor data from wearable devices is collected and preprocessed, divided into training, validation, and test sets. Then, a recognition model is constructed. A prompting module based on Kolmogorov-Arnold convolution is input to a pre-trained temporal baseline model with frozen backbone network parameters, and a classifier is input to its output. Next, the prompting module and classifier are fine-tuned using only training data. The optimal model is determined through the validation set. Finally, the trained model is deployed to wearable devices to achieve real-time, high-precision human behavior recognition. This application can be easily deployed to wearable devices to achieve high-precision behavior recognition with only a small number of labeled samples through transfer learning, greatly reducing the conversion cost between different recognition tasks. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is a flowchart illustrating a transfer learning method for human behavior recognition based on a Kolmogorov-Arnold convolutional filter, provided in an embodiment of this application.
[0027] Figure 2 This is a schematic diagram illustrating the principle of a transfer learning method for human behavior recognition based on a Kolmogorov-Arnold convolutional filter, provided in another embodiment of this application.
[0028] Figure 3 This is a schematic diagram of the training loss curve provided in one embodiment of this application;
[0029] Figure 4 This is a schematic diagram of the confusion matrix on the test set provided in an embodiment of this application; Detailed Implementation
[0030] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0031] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0032] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0033] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."
[0034] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0035] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0036] The following is a detailed description of a transfer learning method for human behavior recognition based on a Kolmogorov-Arnold convolutional filter, provided in this application, with reference to the accompanying drawings.
[0037] Figure 1 The illustration shows a flowchart of a transfer learning method for human behavior recognition based on a Kolmogorov-Arnold convolutional filter, as provided in an embodiment of this application.
[0038] like Figure 1As shown, this transfer learning method for human behavior recognition based on Kolmogorov-Arnold convolutional filters includes the following steps:
[0039] S1. Collect first time feature data through wearable devices, preprocess the first time feature data to obtain second time feature data, and label the second time feature data with its corresponding activity category;
[0040] Furthermore, in this embodiment of the application, the preprocessing of the first time feature data in S1 to obtain the second time feature data specifically includes:
[0041] Perform data cleaning on the first-time feature data to remove outliers and null values;
[0042] The first time-specific feature data, after removing outliers and null values, is downsampled and normalized to obtain the second time-specific feature data.
[0043] S2. Cut the second time feature data according to the sliding window to obtain multiple data segments with the same sequence length, and divide the multiple data segments with the same sequence length into training set, validation set and test set according to a preset ratio.
[0044] For example, the preset ratio can be 7:1:2.
[0045] S3. Connect the preset prompt adjustment module and the preset classifier module to the input and output of the preset temporal base model respectively to construct the activity recognition model. The preset prompt adjustment module consists of three Kolmogorov-Arnold convolutional layers KConv with different receptive fields and an attention module.
[0046] S4. Train the activity recognition model based on the training set, and keep the backbone network parameters of the time series basic model from participating in gradient updates. Use the Adam optimization algorithm to optimize the parameters of the activity recognition model until the optimal activity recognition model is obtained.
[0047] Furthermore, in this embodiment of the application, S4 includes:
[0048] Each time-specific feature data in the training set Input is sent to the preset prompt adjustment module, and the sensor prompt is output. ;
[0049] Sensor prompt and feature data for each time period Input to the time series basic model, output sensor coded features ;
[0050] Encoding sensor features The input is fed into a preset classifier module, which then encodes sensor features through a fully connected layer. Mapped to the probability of each activity category;
[0051] The loss function is calculated based on the cross-entropy loss function, which determines the relationship between the probability of each activity category and the actual label. Through backpropagation and the Adam optimization algorithm, the parameters of the preset classifier module and the preset prompt adjustment module are updated using gradients until the loss function is reached. To reach the minimum;
[0052] Repeat the above process until the accuracy of the activity recognition model on the validation set reaches a preset threshold, then output the optimal activity recognition model.
[0053] Furthermore, in the embodiments of this application, the above-mentioned feature data for each time period in the training set are... Input is sent to the preset prompt adjustment module, and the sensor prompt is output. Specifically, it includes:
[0054] Time feature data The first feature map, the second feature map, and the third feature map are respectively input into the first KConv, the second KConv, and the third KConv, and the corresponding first feature map, the second feature map, and the third feature map are output.
[0055] Time feature data Input to the attention module, and output the attention score corresponding to the first KConv, the attention score corresponding to the second KConv, and the attention score corresponding to the third KConv;
[0056] The first feature map, the second feature map, the third feature map, and the attention scores corresponding to the first KConv, the second KConv, and the third KConv are weighted and processed to output the sensor prompt. The details are as follows:
[0057]
[0058] in, The kernel size is Attention score Indicates the convolution size as The feeling of the wild, It is a learnable function that can be parameterized as a spline curve. For the location index is The input value.
[0059] It should be noted that the aforementioned preset prompt adjustment module consists of three KConv modules with different receptive fields. KConv is designed based on the Kolmogorov-Arnold representation theorem, which states that any multivariate continuous function can be represented as a finite combination of a univariate continuous function and a binary additive budget. Assuming... , A smooth function that takes n variables with values between 0 and 1 as input and outputs a real number result. For the real number field, the Kolmogorov-Arnold representation theorem can be mathematically expressed as:
[0060]
[0061] in, and It is a continuous function. Represents the input vector No. Each component. Extending this representation theorem to traditional convolutional layers can further improve the model's feature extraction capabilities. The position of the traditional convolutional kernel on the feature map. Calculated features It can be represented as:
[0062]
[0063] in This indicates that the traditional convolution kernel is at index 1. The weights on the surface, therefore, if the weights are... Replace with learnable spline functions This can further improve the model's nonlinear fitting ability, thus enabling it to better learn to extract effective features. At this point, the convolution kernel based on the Kolmogorov-Arnold representation theorem can be expressed as:
[0064]
[0065] At this point, the formula for calculating the convolution kernel based on the Kolmogorov-Arnold representation theorem can be expressed as:
[0066]
[0067] in It is a learnable function parameterized as a spline curve. To effectively fuse sensor features at different scales, this application specifically designs an attention module. Therefore, the complete sensor cues can be mathematically represented as:
[0068] .
[0069] Furthermore, in this embodiment, the attention module includes a global mean pooling layer, a linear layer, and a Softmax function layer; the process of obtaining the attention score specifically includes:
[0070] Using a global mean pooling layer to extract time feature data , Convert to ,in Indicates the length of the sequence after sliding window processing. Represented as the number of sequence channels, For the real number field;
[0071] Through linear layers Convert to a 3D vector ;
[0072] Using the Softmax function Convert to attention score.
[0073] S5. Save the optimal activity recognition model as a pt file and port the pt file to the wearable device so that the wearable device can perform human behavior recognition.
[0074] In one embodiment, the optimal activity recognition model is saved as a pt file and ported to an Android phone. Then, the model weight file is loaded using torch.load(), and the mode is changed to evaluation mode using model.eval(). This allows the sensor signals (i.e., acceleration signals and gyroscope signals) collected by the phone to be mapped to human behavior categories.
[0075] This application provides a transfer learning method for human behavior recognition based on Kolmogorov-Arnold convolutional filters. First, sensor data from wearable devices is collected and preprocessed, divided into training, validation, and test sets. Then, a recognition model is constructed. A prompting module based on Kolmogorov-Arnold convolutions is input to a pre-trained temporal baseline model with its backbone network parameters frozen, and a classifier is connected to its output. Next, the prompting module and classifier are fine-tuned using only training data. The optimal model is determined through the validation set. Finally, the trained model is deployed to wearable devices to achieve real-time, high-precision human behavior recognition. This application allows for easy deployment to wearable devices with high-precision behavior recognition using only a small number of labeled samples, significantly reducing the conversion cost between different recognition tasks.
[0076] The following specific embodiment illustrates the solution provided in this application.
[0077] Step 1: Acquire behavioral sensor data, clean and normalize it, and then divide it into training, validation, and test sets. In practice, users need to wear the corresponding wearable device, placing their smartphone in their left pocket and a smart bracelet on their left wrist to collect behavioral sensor data and record the behavioral label for each data segment. The data is saved in TXT file format. The collected data is then cleaned, including removing outliers and null values, and normalized, then divided into uniformly sized data windows. Finally, it is divided into training, validation, and test sets in a 7:1:2 ratio.
[0078] Step 2: Build the sensor prompt adjustment module and assemble the activity recognition model. Figure 2 This is a schematic diagram of the principle of this application, mainly including the internal architecture of the sensor cue adjustment module and the temporal base model, as well as the training process of the entire model. Specifically, the cue adjustment module consists of three KConvs of different sizes and a weighting module. The sizes of the three KConvs are (3, 1), (5, 1), and (7, 1), respectively. First, the sensor input is processed by the three different KConvs to generate feature maps of different scales. Then, the weighting module adaptively generates weighted scores for the three scales based on the input. Finally, the sensor cue features are calculated using the following formula. :
[0079]
[0080] At this time It is considered as input to the time series basic model, while This can then be used to adjust the input of the time-series basic model. The adjusted input is then fed into the backbone network of the time-series basic model for processing to obtain the sensor's encoded features. The classifier module will then... The probability is mapped to a specific activity category. The cross-entropy loss is calculated by plotting the probability against the actual label to obtain... Finally, the parameters of the sensor cue adjustment module and the classifier module are optimized based on the backpropagation and gradient update rules.
[0081] Step 3: Input the data from the training set into the built activity recognition model. Through training, update and optimize the parameters of the sensor cue adjustment module and the classifier module. In the actual implementation, the training epochs were set to 200, the batch size to 64, and the length of each data segment to 500. The Adam optimizer with an initial learning rate of 0.001 was used to optimize the model parameters. The model's performance on the validation set was used as the basis for judging whether the model was optimal. If the training accuracy deviates significantly from the expected value, the network hyperparameters can be adjusted appropriately, such as adding a larger-scale KConv branch or adjusting the KConv step size. If adjusting the hyperparameters has little effect, collecting more training samples can be considered. Figure 3 The graph shows that the model's loss on the training set changes with the number of training rounds. As can be seen from the graph, the training process is very stable and converges to near 0 after nearly 20 rounds of training, indicating that the training strategy is very efficient. Figure 4 This is the confusion matrix on the test set. As can be seen from the figure, the model has very few misclassified samples, and the predicted samples of each category can be recalled in large quantities.
[0082] Step 4: Port the trained activity recognition model to a mobile smart device to achieve real-time activity recognition. When the training activity recognition model achieves optimal prediction accuracy on the validation set, it is saved as a .pt file and transferred to the mobile terminal. In this implementation, an Android smartphone is used. The sensor data collected by the smartphone is processed into data fragments according to the data preprocessing method in Step 1, and then input into the trained activity recognition model to obtain the recognition results of human behavior. This achieves human behavior recognition without revealing the user's identity.
[0083] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0084] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A transfer learning method for human behavior recognition based on Kolmogorov-Arnold convolutional filters, characterized in that, include: S1. Collect first time feature data through wearable devices, preprocess the first time feature data to obtain second time feature data, and label the second time feature data with its corresponding activity category; S2. The second time feature data is segmented according to the sliding window to obtain multiple data segments with the same sequence length. The multiple data segments with the same sequence length are divided into a training set, a validation set and a test set according to a preset ratio. S3. Connect the preset prompt adjustment module and the preset classifier module to the input and output of the preset temporal base model respectively to construct the activity recognition model. The preset prompt adjustment module consists of three Kolmogorov-Arnold convolutional layers KConv with different receptive fields and an attention module. S4. Train the activity recognition model based on the training set, and keep the backbone network parameters of the time series basic model from participating in gradient updates. Optimize the parameters of the activity recognition model using the Adam optimization algorithm until the optimal activity recognition model is obtained. Step S4 involves training the activity recognition model based on the training set, while keeping the backbone network parameters of the time-series basic model from participating in gradient updates, and optimizing the parameters of the activity recognition model using the Adam optimization algorithm until the optimal activity recognition model is obtained. Specifically, this includes: Each time-feature data in the training set Input is sent to the preset prompt adjustment module, and sensor prompts are output. ; Prompt the sensor and feature data for each time period Input to the aforementioned time-series basic model, output sensor coded features ; Encode the sensor features The input is fed into the preset classifier module, so that the preset classifier module encodes the sensor features through a fully connected layer. Mapped to the probability of each activity category; The loss function is calculated based on the cross-entropy loss function, which determines the relationship between the probability of each activity category and the actual label. Through backpropagation and the Adam optimization algorithm, the parameters of the preset classifier module and the preset prompt adjustment module are updated using gradients until the loss function is reached. To reach the minimum; Repeat the above process until the accuracy of the activity recognition model on the validation set reaches a preset threshold, then output the optimal activity recognition model. The feature data at each time step in the training set Input is sent to the preset prompt adjustment module, and sensor prompts are output. Specifically, it includes: The time feature data The first feature map, the second feature map, and the third feature map are respectively input into the first KConv, the second KConv, and the third KConv, and the corresponding first feature map, the second feature map, and the third feature map are output. The time feature data The input is given to the attention module, which outputs the attention score corresponding to the first KConv, the attention score corresponding to the second KConv, and the attention score corresponding to the third KConv. The first feature map, the second feature map, the third feature map, the attention score corresponding to the first KConv, the attention score corresponding to the second KConv, and the attention score corresponding to the third KConv are weighted and processed to output the sensor prompt. The details are as follows: ； in, The kernel size is Attention score Indicates the convolution size as The feeling of the wild, It is a learnable function that can be parameterized as a spline curve. For the location index is The input value; S5. Save the optimal activity recognition model as a pt file and transfer the pt file to the wearable device so that the wearable device can perform human behavior recognition.
2. The method as described in claim 1, characterized in that, The preprocessing of the first time feature data in S1 to obtain the second time feature data specifically includes: The first time feature data is cleaned to remove outliers and null values. The first time feature data, after removing outliers and null values, is downsampled and normalized to obtain the second time feature data.
3. The method as described in claim 2, characterized in that, The attention module includes a global mean pooling layer, a linear layer, and a Softmax function layer; the process of obtaining the attention score specifically includes: The time feature data is processed using a global mean pooling layer. , Convert to ,in Indicates the length of the sequence after sliding window processing. Represented as the number of sequence channels, For the real number field; Through linear layers Convert to a 3D vector ; Using the Softmax function Converted into the attention score.