Online human activity identification method based on robust reinforcement learning
Through robust enhancement learning methods, using spatiotemporal encoding and data augmentation technology, the lack of performance of sensor human activity recognition model in complex activity and noise environments is solved, and higher recognition accuracy and efficiency are achieved, suitable for smart homes and intelligence Applications such as elderly care.
Patent Information
- Application Number
- CN202410963654.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-05
AI Technical Summary
Existing sensor-based human activity recognition models are difficult to maintain high performance and low resource consumption under real-time requirements and resource constraints, especially in real-time environments where complex activities and noise are mixed, lack of spatial and temporal correlation and no labeling. Data utilization capability.
Using the online human activity recognition method of robust enhancement learning, we generate high confidence-based enhancement samples through spatiotemporal coding, comparative self-supervised learning and data augmentation technology to improve the robustness and prediction performance of the model, including spatiotemporal joint coding and random noise Processing, insertion and replacement operations of graph transformation relationships and multitasking learning.
The generalization ability of the model and online recognition accuracy are improved, complex activity transformation and environmental changes can be better handled, and human activity recognition accuracy and efficiency in indoor scenarios are improved.
Smart Images

Figure CN118897984A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of human activity recognition, and in particular to an online human activity recognition method based on robust reinforcement learning. Background Art
[0002] Sensor-based human activity recognition (HAR) systems help identify activities performed by a single person or multiple people and provide detailed feedback interventions, which is the basis for reasoning about human intentions and actions. The natural complexity of user behavior, the noise problem of multimodal sensor signals, and the need for user privacy protection pose challenges to accurate and effective human activity recognition models. However, under the real-time requirements of real-world applications and resource constraints, human activity recognition based on non-contact environmental sensors in continuously monitored physical spaces faces many technical challenges to maintain high performance and low resource consumption.
[0003] For example, deep learning models dominate the field of sensor-based HAR. They are usually resource-intensive and cannot fully translate the semantics of the relationship between activities and scenes, and therefore lack sensitivity to complex activities, such as multi-user concurrent activities. Knowledge-driven methods combine information from different sources to build domain knowledge for complex activity recognition, such as interleaved and concurrent activities. However, knowledge-driven methods usually require complete domain knowledge and comprehensive preprocessing to build activity models.
[0004] In the existing technology, the activity recognition modeling of sensor data mainly focuses on feature extraction and activity category recognition. Based on multimodal information such as CSI, BV, WIFI, wearable sensors, and environmental sensor signals, wavelet transform, machine learning, and deep learning techniques are used to improve the performance of human activity recognition models through attention mechanisms, residual networks, online learning, and active learning. However, under the conditions of real-time requirements of real applications, limited labeled data, and noise interference, human activity recognition based on non-contact environmental sensors in continuously monitored physical spaces must maintain high performance and low resource consumption. Current inventions still face many technical challenges:
[0005] (1) Sliding window-based models dominate the field of sensor-based HAR and are naturally faced with challenges of user cross-activity and multi-user concurrent activities. Current inventions cannot fully translate the relationship semantics between activities and scenes, and therefore lack sensitivity and recognition capabilities for complex activities.
[0006] (2) The real environment faces problems such as noise and high data labeling costs. Existing inventions lack the ability to utilize unlabeled data and resist interference. Summary of the invention
[0007] In view of the above technical challenges faced by human activity recognition based on non-contact environmental sensors in a continuously monitored physical space to maintain high performance and low resource consumption under the real-time requirements of real applications and resource constraints, the present invention designs an online human activity recognition method based on robust reinforcement learning, aiming to alleviate the problems of neglecting spatiotemporal correlation (semantic relationship between activities and scenes), lack of utilization of unlabeled data, and poor real-time performance in traditional activity recognition models by utilizing spatiotemporal encoding of sensor events, contrastive self-supervised learning, and robust data enhancement that is more suitable for online scenarios, thereby improving the generalization ability of the model and online recognition accuracy. Compared with the existing recognition model based on self-supervised learning, the proposed method can better simulate data disturbances and uncertainties in online scenarios, generate high-confidence enhanced samples, and thus learn better data representation, improve model robustness and prediction performance.
[0008] The present invention adopts the following technical solution: an online human activity recognition method based on robust reinforcement learning, comprising the following steps:
[0009] S1: In a smart home environment, each piece of collected data is stored in the form of a tuple <date, time, sensor ID, value>, called an event, and each event will have a unique activity corresponding to it;
[0010] The sets of time, sensors, and activities are defined as T = [1, 2, ..., 24], S = [..., s, ...], and A = [..., a, ...], respectively, where 24 represents the time as 24 o'clock;
[0011] S2: Perform spatiotemporal joint encoding on the event data, count all the time-sensor pairs (t, s) that have appeared in the statistical data set, and form them into an event set E;
[0012] Among them, e = t||s represents the spatiotemporal encoding of the event, which is obtained by concatenating the time and sensor encoding;
[0013] S3: When performing online activity recognition, various types of encoders can be used to capture and transform the features of input data for effective recognition. That is, the encoder is first used to extract the feature representation of the window event, and then it is handed over to the downstream classifier for specific activity recognition;
[0014] S4: By artificially adding noise during the training phase, the model can learn to filter out irrelevant interference factors when facing noisy data, improving the robustness of the model when encountering noisy data in the real world. Two random data augmentation operations are proposed: random masking and random shuffling.
[0015] Among them, the random mask (M) uses the mask symbol [Mask] to replace some event codes in the window, and the mask ratio is controlled by the hyperparameter 0≤η≤1. The number of masks is |η·n|:
[0016]
[0017] Among them, when e j When selected, e′ j is a mask [Mask], otherwise e j is the original event symbol;
[0018] Among them, random order (R), a part of the subsequence in the window [e j , ..., e j+m-1 ] is randomly shuffled to [e′ j , ..., e′ j+m-1] , the enhancement ratio is controlled by the hyperparameter μ, and the disorder length is
[0019]
[0020] S5: The window in the online scenario has a unique continuity and slides on the complete event sequence. Two data enhancements suitable for online scenarios are proposed, namely insertion and replacement operations based on graph transformation relations.
[0021] An event transition graph g is constructed based on the temporal correlation of events in the window to help the model learn better. For any event pair (e 1 , e j ), with A 1 , j←A 1 , update the edge weights on the transition graph in the manner of j+1 / j. For each event node in the graph, keep its top-k weighted connecting edges, where k is a hyperparameter;
[0022] Among them, based on the insertion of graph transformation relations (GI), we first randomly select a window w i [α·n] indexes [idx 1 , ..., idx [α·n] ], where 0≤α≤0 is the insertion enhancement ratio. Then, for each event e corresponding to an index idx , get e from the transformation graph g idx The neighbor set N(e idx ) and the corresponding edge weight (correlation coefficient) P(e idx ). By adding the edge weight P(e idx ) as the probability value from the candidate set N(e idx ) is sampled, we can get the idxThe enhancement event e' inserted before idx , the whole process is defined as:
[0023]
[0024] Among them, the substitution based on graph transformation relationship (GS) first randomly samples [β·n] indexes [idx 1 , ..., idx [β·n] Then, get the event e idx The neighbor set N(e idx ) and the corresponding edge weight (correlation coefficient) P(e idx ). Finally, by idx ) is sampled to obtain the enhanced event e′ for replacement idx , the whole process can be defined as:
[0025]
[0026] S6: Integrate the enhanced operations of steps S4 and S5 into a set When using, sample from it and select a specific operation to enhance the window;
[0027] S7: For a batch of training window data From the collection Randomly sample two data enhancement operations to enhance the event window to generate 2 B An augmented sample is defined as:
[0028] {w′ 1 , w′ 2 , ..., w′ 2i-1 , w′ 2i , ..., w′ 2B-1 , w′ 2B}(Formula 9),
[0029] where i∈[1, 2, ..., B]. Through the in-batch negative sample strategy, from the same sample w i The window pair (w′) obtained by the enhancement 2i-1 , w′ 2i ) will be regarded as a pair of positive samples, and the contrast loss will maximize their consistency, while they and other samples in the batch are regarded as negative samples, and the distance will be pulled farther. According to Formula 2, the window pair (w′ 2i-1 , w′ 2i ) is encoded into a representation (h′ 2i-1 , h′ 2i ), in order to calculate the contrast loss:
[0030]
[0031] Among them, 1 j≠2i-1 represents the indicator function, which is 1 when the expression i≠2i-1 holds, otherwise it is 0;
[0032] S8: The model is jointly optimized using multi-task learning. The multi-task training loss is as follows:
[0033]
[0034] Where λ represents the contrastive loss weight.
[0035] Preferably, in S1, a series of sliding windows are moved on the sensor data stream according to the timestamp order to generate all the data W=[..., w i ,...],in represents the i-th sliding window, e = (t, s) represents single event data, and n represents the window size. Given the input window data w i , prediction model P θ , the online human activity recognition problem can be defined as:
[0036]
[0037] where w i Represents the window, a i The corresponding activity label.
[0038] Preferably, S3 specifically includes:
[0039] S31: Use Encoder to represent a specific encoder, with event window sequence As input, a window representation is obtained, the process of which is defined as follows:
[0040] h i =Encoder(w i ) (Formula 2)
[0041] where h i ∈R d Represents window w i The window representation after encoding, d represents the embedding dimension of the encoder;
[0042] S32: Use a downstream classifier to obtain the predicted score of the activity. The downstream classifier can be a simple linear transformation layer or MLP. The process is as follows:
[0043]
[0044] in represents the classification score of the model for different activities, and |A| is the number of activities;
[0045] S33: Using multi-classification cross entropy loss function, for a batch of training window data The loss for the activity classification task is as follows:
[0046]
[0047] in Indicates the batch size, a i Indicates w i The activity tag, Represents the model about activity a i The prediction score of .
[0048] Preferably, in step S3, the encoder types used include CNN, LSTM, Transformer, MLP, etc.
[0049] Preferably, in step S5, for insertion (GI) based on graph conversion relationships, events belonging to other activities are inserted into the window, which can increase the diversity of window content and simulate possible mixed behaviors or activity transition stages, so that the model can better cope with complex situations in actual application scenarios, such as rapid switching or performing multiple activities simultaneously.
[0050] Preferably, in step S5, for the substitution based on the graph transformation relationship (GS), some events in the window are replaced, which increases the diversity of window content and simulates user mixed behavior while also removing part of the original information, thereby reducing the possibility of the model over-relying on specific training sample features, so that the model can better cope with various changes and interference factors that may occur in reality.
[0051] The algorithm for online human activity recognition based on robust enhanced contrastive learning includes the following steps:
[0052] Input: data augmentation ratio η, μ, α, β, training batch size B, epoch size N, number of neighbors k, contrast loss weight λ;
[0053] Output: Optimal activity classification model
[0054] a. Randomly initialize model parameters θ;
[0055] b. Generate data W through online window segmentation;
[0056] c. Traverse the sliding window to construct the event transition graph g;
[0057] d.For each round epoch=0, 1, ..., N-1do;
[0058] e.For each batch B executes do in parallel;
[0059] f. Calculate the activity recognition loss according to formulas 2 to 4
[0060] g. From the collection Sample two data augmentation operations;
[0061] h. Generate enhanced samples according to formula 9;
[0062] i. Calculate the contrast loss according to formula 10
[0063] j. Calculate the joint training loss according to formula 12
[0064] k.end for
[0065] l. By minimizing Update the parameters θ;
[0066] m.end for.
[0067] Compared with the prior art, the advantages and positive effects of the present invention are:
[0068] In the present invention, in the data collection stage, sensor events are effectively mapped into data tuples with unique activity labels through time-space joint coding, ensuring efficient management and utilization of data. Secondly, in the feature extraction and representation stage, a variety of encoders such as CNN, LSTM, Transformer, etc. are used to capture the rich feature representation of window events, providing a solid foundation for subsequent activity classification. The key innovation lies in the data enhancement strategy, which simulates and handles complex situations in the real world by introducing random noise, event insertion and substitution operations, thereby improving the model's adaptability and robustness to noise and data changes. By enhancing sample generation and negative sample strategies within batches, the efficiency and effect of model training are optimized. This comprehensive training method not only improves the accuracy of activity classification, but also enables the model to better handle complex activity conversions and environmental changes. Improve the accuracy and efficiency of human activity recognition based on environmental, wireless and other sensors in indoor scenes. Provide support for smart homes, smart elderly care, smart security, etc. Improve the accuracy and efficiency of human activity recognition based on environmental, wireless and other sensors in indoor scenes. Provide support for smart homes, smart elderly care, smart security, etc. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 A flowchart of an online human activity recognition method based on robust reinforcement learning is proposed for the present invention;
[0070] Figure 2 An algorithm diagram of an online human activity recognition method based on robust reinforcement learning is proposed for the present invention. DETAILED DESCRIPTION
[0071] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0072] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.
[0073] Example
[0074] See also Figure 1-2 The present invention provides a technical solution: an online human activity recognition method based on robust reinforcement learning, comprising the following steps:
[0075] S1: In a smart home environment, each piece of collected data is stored in the form of a tuple <date, time, sensor ID, value>, called an event, and each event will have a unique activity corresponding to it;
[0076] The sets of time, sensors, and activities are defined as T = [1, 2, ..., 24], S = [..., s, ...], and A = [..., a, ...], respectively, where 24 represents the time as 24 o'clock;
[0077] S2: Perform spatiotemporal joint encoding on the event data, count all the time-sensor pairs (t, s) that have appeared in the statistical data set, and form them into an event set E;
[0078] Among them, e = t||s represents the spatiotemporal encoding of the event, which is obtained by concatenating the time and sensor encoding;
[0079] S3: When performing online activity recognition, various types of encoders can be used to capture and transform the features of input data for effective recognition. That is, the encoder is first used to extract the feature representation of the window event, and then it is handed over to the downstream classifier for specific activity recognition;
[0080] S4: By artificially adding noise during the training phase, the model can learn to filter out irrelevant interference factors when facing noisy data, improving the robustness of the model when encountering noisy data in the real world. Two random data augmentation operations are proposed: random masking and random shuffling.
[0081] Among them, the random mask (M) uses the mask symbol [Mask] to replace some event codes in the window, and the mask ratio is controlled by the hyperparameter 0≤η≤1. The number of masks is |η·n|:
[0082]
[0083] Among them, when e j When selected, e′ j is a mask [Mask], otherwise e j is the original event symbol;
[0084] Among them, random order (R), a part of the subsequence in the window [e j , ..., e j+m-1 ] is randomly shuffled to [e′ j , ..., e′ j+m-1 ], the enhancement ratio is controlled by the hyperparameter μ, and the disorder length is
[0085]
[0086] S5: The window in the online scenario has a unique continuity and slides on the complete event sequence. Two data enhancements suitable for online scenarios are proposed, namely insertion and replacement operations based on graph transformation relations.
[0087] An event transition graph g is constructed based on the temporal correlation of events in the window to help the model learn better. For any event pair (e 1 , e j ), with A 1,j ←A 1,j +1 / j updates the edge weights on the transition graph. For each event node in the graph, retains its top-k weighted connecting edges, where k is a hyperparameter.
[0088] Among them, based on the insertion of graph transformation relations (GI), we first randomly select a window w i [α·n] indexes [idx 1 , ..., idx [α·n] ], where 0≤α≤0 is the insertion enhancement ratio. Then, for each event e corresponding to an index idx , get e from the transformation graph g idx The neighbor set N(e idx ) and the corresponding edge weight (correlation coefficient) P(e idx ). By adding the edge weight P(e idx ) as the probability value from the candidate set N(e idx ) is sampled, we can get the idxThe enhancement event e' inserted before idx , the whole process is defined as:
[0089]
[0090] Among them, the substitution based on graph transformation relationship (GS) first randomly samples [β·n] indexes [idx 1 , ..., idx [β·n] Then, get the event e idx The neighbor set N(e idx ) and the corresponding edge weight (correlation coefficient) P(e idx ). Finally, by idx ) is sampled to obtain the enhanced event e′ for replacement idx , the whole process can be defined as:
[0091]
[0092] S6: Integrate the enhanced operations of steps S4 and S5 into a set When using, sample from it and select a specific operation to enhance the window;
[0093] S7: For a batch of training window data From the collection Randomly sample two data enhancement operations to enhance the event window to generate 2 B An augmented sample is defined as:
[0094] {w′ 1 , w′ 2 , ..., w′ 2i-1 , w′ 2i , ..., w′ 2B-1 , w′ 2B}(Formula 9),
[0095] where i∈[1, 2, ..., B]. Through the in-batch negative sample strategy, from the same sample w i The window pair (w′) obtained by the enhancement 2i-1 , w′ 2i ) will be regarded as a pair of positive samples, and the contrast loss will maximize their consistency, while they and other samples in the batch are regarded as negative samples, and the distance will be pulled farther. According to Formula 2, the window pair (w′ 2i-1 , w′ 2i ) is encoded into a representation (h′ 2i-1 , h′ 2i ), in order to calculate the contrast loss:
[0096]
[0097] Among them, 1 j≠2i-1 represents the indicator function, which is 1 when the expression i≠2i-1 holds, otherwise it is 0;
[0098] S8: The model is jointly optimized using multi-task learning. The multi-task training loss is as follows:
[0099]
[0100] Where λ represents the contrastive loss weight.
[0101] Furthermore, in S1, a series of sliding windows are moved on the sensor data stream according to the timestamp order to generate all the data W = [..., w i ,...],in represents the i-th sliding window, e = (t, s) represents single event data, and n represents the window size. Given the input window data w i , prediction model P θ , the online human activity recognition problem can be defined as:
[0102]
[0103] where w i Represents the window, a i The corresponding activity label.
[0104] Furthermore, S3 specifically includes:
[0105] S31: Use Encoder to represent a specific encoder, with event window sequence As input, a window representation is obtained, the process of which is defined as follows:
[0106] h i =Encoder(w i ) (Formula 2)
[0107] where h i ∈R d Represents window w i The window representation after encoding, d represents the embedding dimension of the encoder;
[0108] S32: Use a downstream classifier to obtain the predicted score of the activity. The downstream classifier can be a simple linear transformation layer or MLP. The process is as follows:
[0109]
[0110] in represents the classification score of the model for different activities, and |A| is the number of activities;
[0111] S33: Using multi-classification cross entropy loss function, for a batch of training window data The loss for the activity classification task is as follows:
[0112]
[0113] in Indicates the batch size, a i Indicates w i The activity tag, Represents the model about activity a i The prediction score of .
[0114] Furthermore, in step S3, the encoder types used include CNN, LSTM, Transformer, MLP, etc.
[0115] Furthermore, in step S5, for insertion (GI) based on graph conversion relationships, events belonging to other activities are inserted into the window, which can increase the diversity of the window content and simulate possible mixed behaviors or activity transition stages, so that the model can better cope with complex situations in actual application scenarios, such as rapid switching or performing multiple activities at the same time.
[0116] Furthermore, in step S5, for the substitution based on graph transformation relationship (GS), some events in the window are replaced, which increases the diversity of window content and simulates user mixed behavior, while also removing some original information, reducing the possibility of the model over-relying on specific training sample features, so that the model can better cope with various changes and interference factors that may occur in reality.
[0117] The algorithm for online human activity recognition based on robust enhanced contrastive learning includes the following steps:
[0118] Input: data augmentation ratio η, μ, α, β, training batch size B, epoch size N, number of neighbors k, contrast loss weight λ;
[0119] Output: Optimal activity classification model
[0120] a. Randomly initialize model parameters θ;
[0121] b. Generate data W through online window segmentation;
[0122] c. Traverse the sliding window to construct the event transition graph g;
[0123] d.For each round epoch=0, 1, ..., N-1do;
[0124] e.For each batch B executes do in parallel;
[0125] f. Calculate the activity recognition loss according to formulas 2 to 4
[0126] g. From the collection Sample two data augmentation operations;
[0127] h. Generate enhanced samples according to formula 9;
[0128] i. Calculate the contrast loss according to formula 10
[0129] j. Calculate the joint training loss according to formula 12
[0130] k.end for
[0131] l. By minimizing Update the parameters θ;
[0132] m.end for.
[0133] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.
Claims
1. Online human activity recognition method based on robust reinforcement learning, It is characterized in that The following steps are involved: S1: In a smart home environment, each piece of collected data is stored in the form of a tuple <date, time, sensor ID, value>, called an event, and each event will have a unique activity corresponding to it; The sets of time, sensors, and activities are defined as T = [1, 2, ..., 24], S = [..., s, ...], and A = [..., a, ...], respectively, where 24 represents the time as 24 o'clock; S2: Perform spatiotemporal joint encoding on event data, count all time-sensor pairs (t, s) that have appeared in the statistical data set, and form them into an event set E; Among them, e = t||s represents the spatiotemporal encoding of the event, which is obtained by concatenating the time and sensor encoding; S3: When performing online activity recognition, various types of encoders can be used to capture and transform the features of input data for effective recognition. That is, the encoder is first used to extract the feature representation of the window event, and then it is handed over to the downstream classifier for specific activity recognition; S4: By artificially adding noise during the training phase, the model can learn to filter out irrelevant interference factors when facing noisy data, improving the robustness of the model when encountering noisy data in the real world. Two random data augmentation operations are proposed: random masking and random shuffling. Among them, the random mask (M) uses the mask symbol [Mask] to replace some event codes in the window, and the mask ratio is controlled by the hyperparameter 0≤η≤1. The number of masks is |η·n|: Among them, when e j When selected, e j is a mask [Mask], otherwise e j is the original event symbol; Among them, random order (R), a part of the subsequence in the window [e j ,…,e j+m-1 ] is randomly shuffled to [e′ j ,…,e′ j+m-1 ], the enhancement ratio is controlled by the hyperparameter μ, and the disorder length is S5: The window in the online scenario has a unique continuity and slides on the complete event sequence. Two data enhancements suitable for online scenarios are proposed, namely insertion and replacement operations based on graph transformation relations. An event transition graph g is constructed based on the temporal correlation of events in the window to help the model learn better. For any event pair (e 1 , e j ),by Update the edge weights on the transition graph in a way that, for each event node in the graph, retains its top-k weighted connecting edges, where k is a hyperparameter; Among them, based on the insertion of graph transformation relations (GI), we first randomly select a window w i [α·n] indexes [idx 1 , ..., idx |α·n] ], where 0≤α≤0 is the insertion enhancement ratio. Then, for each event e corresponding to an index idx , get e from the transformation graph g idx The neighbor set And the corresponding edge weight (correlation coefficient) By setting the edge weight As the probability value from the candidate set Sampling can be performed to obtain the idx The enhancement event e' inserted before idx , the whole process is defined as: Among them, the substitution based on graph transformation relationship (GS) first randomly samples [β·n] indexes [idx 1 , ..., idx [β·n] Then, get the event e idx The neighbor set And the corresponding edge weight (correlation coefficient) Finally, by The enhanced event e′ is obtained by sampling idx , the whole process can be defined as: S6: Integrate the enhanced operations of steps S4 and S5 into a set When using, sample from it and select a specific operation to enhance the window; S7: For a batch of training window data From the collection Randomly sample two data enhancement operations to enhance the event window to generate 2B enhanced samples, which are defined as: {w 1 , w 2 ,..., w 2i-1 , w 2i , …, w 2B-1 , w 2B}(Formula 9), where i∈[1, 2, ..., B]. Through the in-batch negative sample strategy, from the same sample w i The window pair (w′) obtained by the enhancement 2i-1 , w′ 2i ) will be regarded as a pair of positive samples, and the contrast loss will maximize their consistency, while they and other samples in the batch are regarded as negative samples, and the distance will be pulled farther. According to Formula 2, the window pair (w′ 2i-1 , w′ 2i ) is encoded into a representation (h′ 2i-1 , h′ 2i ), in order to calculate the contrast loss: Among them, I j≠2i-1 represents the indicator function, which is 1 when the expression j≠2i-I holds, otherwise it is 0; S8: The model is jointly optimized using multi-task learning. The multi-task training loss is as follows: Where λ represents the contrastive loss weight.
2. The online human activity recognition method based on robust reinforcement learning according to claim 1, Features: In S1, a series of sliding windows are moved on the sensor data stream according to the timestamp order to generate all the data W = […, w i ,…],in represents the i-th sliding window, e = (t, s) represents single event data, and n represents the window size. Given the input window data w i , prediction model P θ , the online human activity recognition problem can be defined as: where w i Represents the window, a i The corresponding activity label.
3. The online human activity recognition method based on robust reinforcement learning according to claim 1, Features: The S3 specifically includes: S31: Use Encoder to represent a specific encoder, with event window sequence As input, a window representation is obtained, the process of which is defined as follows: h i = Encoder(w i ) (Formula 2) where h i ∈R d Represents window w i The window representation after encoding, d represents the embedding dimension of the encoder; S32: Use a downstream classifier to obtain the predicted score of the activity. The downstream classifier can be a simple linear transformation layer or MLP. The process is as follows: in represents the classification score of the model for different activities, and |A| is the number of activities; S33: Using multi-classification cross entropy loss function, for a batch of training window data The loss for the activity classification task is as follows: in Indicates the batch size, a i Indicates w i The activity tag, Represents the model about activity a i The prediction score of .
4. The online human activity recognition method based on robust reinforcement learning according to claim 1, Features: In step S3, the encoder types used include CNN, LSTM, Transformer, MLP, etc.
5. The online human activity recognition method based on robust reinforcement learning according to claim 1, Features: In step S5, for insertion (GI) based on graph conversion relationships, events belonging to other activities are inserted into the window, which can increase the diversity of window content and simulate possible mixed behaviors or activity transition stages, so that the model can better cope with complex situations in actual application scenarios, such as fast switching or performing multiple activities at the same time.
6. The online human activity recognition method based on robust reinforcement learning according to claim 1, Features: In step S5, for the substitution based on graph transformation relationship (GS), some events in the window are replaced, which increases the diversity of window content and simulates user mixed behavior, while also removing some original information, reducing the possibility of the model over-relying on specific training sample features, so that the model can better cope with various changes and interference factors that may occur in reality.
7. The online human activity recognition method based on robust reinforcement learning according to any one of claims 1 to 6, It is characterized in that The following steps are involved: enter: Data augmentation ratio η, μ, α, β, training batch size B, epoch size N, number of neighbors k, contrast loss weight λ; Output: Optimal activity classification model a. Randomly initialize model parameters θ; b. Generate data W through online window segmentation; c. Traverse the sliding window to construct the event transition graph g; d. For each round epoch = 0, 1, ..., N-1 do ; e.For each batch B executes do in parallel; f. Calculate the activity recognition loss according to formulas 2 to 4 g. From the collection Sample two data augmentation operations; h. Generate enhanced samples according to formula 9; i. Calculate the contrast loss according to formula 10 j. Calculate the joint training loss according to formula 12 k.end for l. By minimizing Update the parameters θ; m.end for.
Citation Information
Cited By
Online forum-oriented low-resource topic key topic extraction method
CN120146046A
A Low-Resource Topic Key Theme Extraction Method for Online Forums
CN120146046B
