Weakly supervised multi-class activity recognition method based on meta-attention mechanism

Abstract

The application belongs to the field of intelligent classification and recognition, and provides a weakly supervised activity recognition method based on a meta-attention mechanism. The method is based on an end-to-end self-relational attention weakly supervised prototype network model for human activity recognition. The prototype network model comprises a feature extraction module, a meta-learning attention module and a classification module. First, the original sensor data sequence is subjected to feature extraction to obtain a set of example feature vectors. The contribution weight of each example feature vector is obtained through the meta-learning attention module. The prototype of the target activity is purified and optimized through an enhancement operation. Classification is performed through the classification module to complete the training and testing of the model. The method provided by the application can better solve the problem of recognizing new human activity categories from weakly supervised and small training sample sensor data, and effectively avoids the negative impact of noise data.

Description

Technical Field

[0001] The present invention relates to the field of intelligent classification and recognition, and in particular to a weakly supervised incremental activity recognition method based on a meta-attention mechanism. Background Art

[0002] Knowledge mining and analysis of wearable time-series data is an essential research issue in "human-machine-object" converged computing. By modeling wearable data, extracting data features, and mining hidden knowledge, we can achieve human activity recognition and analysis (HAR), including health analysis, behavior analysis, and posture analysis. This provides users with reference suggestions based on specific application contexts to assist in decision-making. Wearable devices are widely used in healthcare, smart homes, and sports monitoring. Accurately identifying human activity is a fundamental function in many health monitoring, smart factory, and sports scenarios, and is crucial for advancing the field of artificial intelligence.

[0003] Existing HAR services all use general models to identify and analyze data. A general model is defined as a model trained offline on a composite dataset and can be deployed in the cloud or on a terminal. The biggest drawback of the general model is that it ignores individual differences and differences in user needs, resulting in inaccurate activity recognition. For example, in a healthy lifestyle monitoring application scenario, a general model identifies activities based on data generated by a user's wearable device. However, the general model can only identify activities that are already in the model, such as routine activities like walking and running. In addition to routine activities in the general model, users will engage in activities that vary from person to person, such as square dancing, playing badminton, or playing table tennis. At this time, the general model cannot recognize these new categories of activities, and users prefer the model to have the ability to identify personally customized activities to achieve accurate monitoring of daily healthy life.

[0004] This shows that general-purpose models cannot provide application services tailored to the specific needs of different groups and individual users. Therefore, user-centric personalized models should address the need for personalized category addition. Personalized model learning requires large-scale, high-quality labeled data, but in reality, users lack the ability to label large amounts of data, especially for newly added custom categories. The data characteristics are as follows:

[0005] (1) Small number of user data labels: Wearable data modeling and analysis based on deep learning technology requires high quality and quantity of data labels. However, with limited resources and manpower, it is difficult to obtain a large amount of completely accurate labeled data. Therefore, for users with personalized needs, it is impossible to obtain sufficient labeled data to update or rebuild the user model.

[0006] (2) Poor quality of user data labels: When user data labels are missing, although user self-feedback labeling or other auxiliary labeling technologies can be used to obtain labels, most labeling technologies usually use sliding windows to perform fixed-length segmentation on time series data, resulting in low label quality and high noise data in data samples. Therefore, low-quality label data is widely present in wearable data, and its amount far exceeds that of high-quality data.

[0007] Therefore, how to use low-quality limited label data to realize dynamic classification and recognition of models is a problem with great research significance and application value.

[0008] To meet users' needs for customizing new activities and provide more accurate activity records, users must be able to add custom activity categories. Furthermore, considering that the number of activities and annotations required for customizing new activities must be minimal, the algorithm must also be able to accurately identify the target activity in this situation to provide a better user experience. Weakly supervised techniques can address the issues of data quantity and label quality. Weakly supervised learning includes incomplete learning, imprecise learning, and inaccurate learning. Incomplete learning addresses the label quantity issue, such as using semi-supervised transfer learning to perform large-scale labeling across datasets; imprecise and inaccurate learning address label quality issues, such as weak label classification techniques. Meta-learning techniques are currently being used to address the problem of identifying new categories and scenarios driven by personalized customization needs. Existing related technologies are widely used in the field of vision. However, due to the different structures of wearable data and image data, these techniques cannot be directly applied to wearable sensor data. Considering the characteristics and data structure of wearable data, existing technologies have problems such as simple data distribution measurement methods across data sets, weak ability to express the characteristics of weakly labeled data, and high requirements for training data for new category recognition models, which brings new challenges to the current stage of knowledge mining and analysis based on wearable data.

[0009] (1) HAR based on inaccurate label classification

[0010] Multiple Instance Learning (MIL) is a common approach for solving the problem of ambiguous label classification. MIL defines two different data granularities: bag and instance. Training set samples consist of bag-level data samples and bag-level category labels, with bags containing multiple unlabeled examples. When a bag contains an example with a target label, the bag is labeled with that label, and bag-level label classification is the ultimate classification goal. Currently, multi-instance learning is widely used in the field of video imaging, such as in athlete detection in "A Method for Athlete Detection Based on Multiple Instance Learning" and in lung cancer case images in "A Multi-instance Convolutional Neural Network Based on Prototype Learning."

[0011] "Weakly supervised recognition of daily life activities with wearable sensors" first applied MIL to wearable data classification and proposed a label annotation strategy based on MIL support vector machines to reduce the empirical sampling frequency in human activity recognition. "Efficient multi-instance learning for activity recognition from time series data using an auto-regressive hidden Markov model" proposed a multi-instance learning model for human activity recognition based on graph generation using autoregressive hidden Markov, and annotated bag labels and instance labels based on a dynamic programming inference algorithm. In addition, "Multiple instance learning for efficient sequential data classification on resource-constrained devices" deployed the model based on MIL's LSTM on tiny, resource-constrained devices to achieve early recognition and monitoring of activities. "Sequential Weakly Labeled Multi-Activity Localization and Recognition on Wearable Sensors using Recurrent Attention Networks" proposes a recurrent attention network to achieve weakly labeled multi-activity recognition and localization tasks, and in "Attention-based convolutional neural network for weakly labeled human activities'recognition with wearable sensors" a soft attention CNN model is proposed to amplify significant activity information and suppress irrelevant confusing information.

[0012] (2) HAR based on small sample learning

[0013] Common solutions to few-shot problems include data augmentation, fine-tuning, and regularization techniques. There are also metric-based few-shot methods, such as Siamese Networks (in "Siamese Neural Networks for One-Shot Image Recognition"), Matching Networks (in "Matching Networks for One-Shot Learning"), and Prototypical Networks. These methods are widely used for few-shot problems based on images and videos, achieving promising results in, for example, "A Generative Approach to Zero-Shot and Few-Shot Action Recognition." In the field of few-shot action recognition, recent research has demonstrated that deep metric-based meta-learners perform well without retraining, where network parameters can be directly transferred to classification models for novel categories based on a distance metric between the support set and the query set. However, research on few-shot learning with wearable sensor data is limited, and much of this work relies on supervised models trained on high-quality data. However, in existing annotation techniques, labels are often inaccurate and ambiguous, which affects the accuracy of the algorithm model when finally classifying the target activity.

[0014] (3) HAR based on attention mechanism

[0015] To extract and select discriminative activity features from long time series and multimodal signals, attention-based deep learning networks have been widely adopted in recent years and have achieved success in sensor-based HAR. In most existing work, CNNs, RNNs, and LSTMs are typically used to extract features from raw data, with self-attention layers selecting these features by learning relevant weights. For example, "Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition" adds an attention layer after a DeepConvLSTM to explore the relevance of temporal context. Furthermore, multi-head attention enhances feature representations over bidirectional LSTMs and multi-path CNNs. Furthermore, some research focuses on labeling activities in long, weakly labeled sequences. "Weakly Supervised Human Activity Recognition from Wearable Sensors by Recurrent Attention Learning" proposes a weakly supervised HAR model based on recurrent attention, trained via reinforcement learning. Similarly, CNN attention models can localize target activities amidst noisy background signals, and attention mechanisms also play a key role in multimodal HAR. The AttnSense model in "AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition" captures the spatiotemporal dependencies of multimodal signals on CNNs and GRUs. In "DanHAR: Dual Attention Network for Multimodal Human Activity Recognition Using Wearable Sensors," DanHAR introduces a dual approach for multimodal HAR that fuses channel attention and temporal attention. In "Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention," sensor attention incorporates weights of sensor modalities in addition to the signal modality. Unlike previous work, we focus on extracting accurate features from small-sample, long-sequence data.

[0016] While many existing models and methods mitigate overfitting in small-sample learning, much of this work relies on supervised models trained on high-quality data. Existing annotation techniques and data segmentation methods often result in weakly labeled time series samples, meaning that labeled examples may contain other activities. Clearly, weakly labeled samples can obfuscate discriminative features, and the impact on accuracy is even more severe in FSL due to the small number of samples. Furthermore, most existing research focuses on computer vision, with little research on wearable sensor data. Summary of the Invention

[0017] The purpose of this invention is to design a weakly supervised incremental activity recognition method based on a meta-attention mechanism. This model, designed for human incremental activity recognition using weakly labeled wearable sensor data, relies on only a minimal number of weakly labeled human activity samples to identify and analyze new activity categories based on wearable sensor data. This method is adaptable to human activity recognition under dynamically changing categories, promoting the development of personalized application services. This model constructs a small-sample learning scheme based on a prototype network to simulate small-sample learning tasks and utilizes multi-instance learning techniques based on an attention mechanism to address the weak labeling problem.

[0018] The technical solution of the present invention: The present invention proposes an end-to-end self-relational attention weakly supervised prototype network model. The network mainly includes a feature extraction module, a meta-attention module and a classification module. The overall structure of the model is as follows: Figure 1 The specific implementation scheme of this model for human activity recognition is as follows:

[0019] Step 1: Extract features from the raw sensor data sequence. Divide the raw sensor data sequence into a support set S and a query set Q. After the raw sensor data sequence A is processed through a sliding time window of size T, the sensor data subsequence is obtained. The feature extraction module uses a convolutional neural network (CNN) to extract the weakly labeled sensor data subsequence A n The spatiotemporal features are extracted from the dataset. The feature extraction module includes a convolution layer and a maximum pooling layer. The convolution layer has a two-dimensional convolution kernel, which extracts the temporal correlation along the time axis and the spatial correlation along the sensor axis. The spatiotemporal features of the sensor raw data sequence A are defined as in Including multiple CNN modules, is a learnable parameter; this method focuses on the relationship and reliability between the fine-grained features of the support set S and the query set Q. However, due to Only K×N samples can be extracted in each training and testing batch, and it is difficult to directly extract the features of the example in each sample, which is a subsequence of the sample. Sensor data subsequence A n The spatiotemporal characteristics of Hn , H n The sliding time window l is divided into m example features, each example feature is recorded as h i , i∈[1,m]; each example feature h i After being expanded into a one-dimensional vector through the fully connected layer, a one-dimensional example feature vector z is obtained. i ; One-dimensional example feature vector set Z = {z1,z2,…,z m} facilitates distance metric, which is fed into the meta-learning attention module.

[0020] Step 2: Obtain the contribution weight of each example feature vector through the meta-learning attention module. In the case of weak supervision, the characteristics of noise activity will confuse the target features. Although the feature extraction module has obtained the example features, it is very difficult to manually identify the noise features and target activity features. Different from the existing work on obtaining feature weights through learning, we propose a meta-attention mechanism based on the relationship between the subsequence example feature vectors and learn a nonlinear function g Ψ Automatically weight the target activity feature vector. In this module, the goal is to calculate the contribution of each example feature vector.

[0021] First, for each sample in the support set S and the query set Q, a self-relation matrix R is defined on the one-dimensional example feature vector set Z to represent the similarity between all example feature vectors; the self-relation matrix R is a symmetric matrix, R i，j represents the example feature vector z i With the example feature vector z j The similarity between i，j =sim(z i , z j );R i is the i-th row of the self-relation matrix R, which represents the example eigenvector z i The similarity between all other example feature vectors; R i，j The larger the value, the more likely it is that the two example feature vectors are the same type of activity in the feature space; for example, in Figure 3 In R, there are six examples, and the main diagonal elements are equal to 1, indicating self-similarity. 1，2 and R 3，5 is equal to 0.95, which is the highest similarity after removing the diagonal elements, but we cannot tell whether they are positive examples belonging to the target activity or negative examples belonging to the noise activity. In existing work, the weight function usually adopts the maximum function and the average function. i The mean vector of z iThe average similarity with other example feature vectors. However, we believe that the non-trainable weights of example feature vectors based on similarity have limited effect on improving the performance of small-sample learning tasks. After generating the self-relation matrix R, the self-relation matrix R is fed into the meta-learning attention module; the meta-learning attention module outputs each example feature vector z i The meta-learning attention module contains two convolutional layers, a pooling layer and a softmax layer. The structure of this part is as follows Figure 3 shown.

[0022] The meta-learning attention module will output the contribution weight of each example feature vector to the total feature vector. The corresponding contribution score is calculated by formula (2):

[0023] sc=g ψ (R)=C2(AVG(σ(C1(R)))), (2)

[0024] in, is a one-dimensional convolutional layer used to extract feature vectors, c is the number of channels, σ is the ReLU activation function, AVG is the average pooling layer, Refers to a convolutional layer with one channel. Considering the continuity of time series, there may be multiple consecutive high similarities. Therefore, the size of the convolution kernel r < m, focusing on a small receptive field is more suitable for feature extraction of partial continuous relationships. Then, an average pooling layer is used to expand the receptive field to all example feature vectors. Finally, the extracted example feature vectors are converted into score vectors sc i represents the contribution of the i-th example to the target activity feature. After nonlinear transformation, the noise sub-feature vector with lower similarity has lower score. The softmax layer is used to map the unnormalized score to the probability distribution in formula (3), where α i is the eigenvector z of the i-th example i The contribution weight of a feature to its data subsequence. A high weight means that the example feature is more likely to be the target active feature.

[0025]

[0026] We refer to the MIL pooling operation solution in this module. We use the weighted average of the sub-feature vectors as the feature vector of the sample. Flexible and adaptive weights can draw the attention of all examples to the target activity, which will achieve better results when adapting to new tasks.

[0027] Step 3: Refine and optimize the prototype of the target activity through enhancement operations. Following the setting of the prototype network, the prototype k of the class is the average value of the extracted feature vectors, as shown in Equation (4).

[0028]

[0029] Although the target activity feature H of each sample * Improvements have been made to address the noisy data problem, but H * It is a weighted aggregation of the feature vectors of each example, which may lose potentially useful information and lead to biased prototypes. Therefore, an effective weakly supervised small-shot learning model should be able to both ignore noise information and extract valuable information to obtain a more discriminative prototype.

[0030] Step 3-1: Enhance each sub-feature vector by performing an enhancement operation, such as Figure 4 As shown in Figure 1, the enhancement operation is performed to better generalize low-quality new class data. Inspired by the prototype network, the example feature vector after the enhancement operation is the average of the example feature vectors that meet the threshold constraint in each example. The purified feature vector It is weighted according to the weight in formula (3)

[0031]

[0032]

[0033] It is calculated in Equation (5) and Equation (6), where ω is an adjacency matrix, ω i,j Indicates z i Is it related to z j adjacent in the metric space with a threshold of θ. The threshold θ determines the degree of information confusion. An appropriate value of θ will ensure the difference between examples and maintain the ability to purify the features of the examples. How to find a suitable threshold is an important issue in our work. Intuitively, a global θ can be set for all categories and samples. g But setting the global θ g It is not suitable because a uniform threshold is not adaptive for each class in few-shot learning. Moreover, the similarity scores of examples from different classes have different distributions.

[0034] Step 3-2: For each sample in each class, a local threshold θ is introduced l In time series data, the target activity example has multiple similar activity examples. In contrast, the non-target example has almost no adjacent examples. In most cases, the similarity score of the adjacent examples of the non-target example is lower than that of the target example. We believe At least one adjacent example information is involved. Therefore, consider setting a local threshold θ l =min(max(R′ i)), where i∈[1,m], R′ i is the vector of row i in R with diagonal elements of 0. It finds the minimum similarity from the maximum similarity set of each example, ensuring that all examples can confuse the nearest neighbor information. Some examples may have multiple adjacent examples, corresponding will contain more potential information. However, due to the existence of non-target examples, θ l may be very small, which leads to the target example It incorporates more worthless information. Therefore, we l A constraint τ is set, Final weighted The adjacency matrix ω is calculated by formula (7):

[0035]

[0036] Based on the above operations, we will get the feature representation feature vector of the purified data subsequence For the k-th class sample, the enhanced discriminative prototype is calculated by formula (8), where It's A n The purified feature vector set.

[0037]

[0038] Step 4: Perform classification through the classification module to complete the training and testing of the model. The classification module includes training and testing. The neural network is trained by minimizing the classification loss of the query set Q in the training dataset. The loss function is the negative log probability of the target class, that is, Equation (9):

[0039]

[0040] By calculating the cross entropy loss between the probability of each category output by the classification module and the true activity label, the classification loss of the end-to-end self-relational attention weakly supervised prototype network model is obtained, and the parameters in the end-to-end self-relational attention weakly supervised prototype network model are updated by backpropagation.

[0041] Beneficial effects of the present invention: The present invention proposes an end-to-end self-relational attention weakly supervised prototype network model to solve the problem of small sample learning with noisy labels in personalized human action recognition. By designing a meta-learning attention module based on the similarity between multiple example feature vectors to assign contribution weights to the sample subsequence example feature vectors, the target activity characteristics are highlighted. The designed enhancement operation is used to purify the prototype of the target activity, which retains the potentially valuable information of the small sample. The improved prototype also helps the feature extraction module to learn more discriminative feature representations. The above operations help the feature extraction module to learn more discriminative feature representations, enhance the distinguishability between the prototype feature representations of the prototype network, so as to overcome the problem of few training samples and noisy labels in the task of human activity recognition based on wearable sensor data, and achieve good generalization ability for new categories. The method proposed in the present invention can better solve the problem of identifying new categories of human activities on sensor data with weak supervision and few training samples, and effectively avoid the negative impact of noisy data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is the framework diagram of the end-to-end self-relation attention weakly supervised prototype network model;

[0043] Figure 2 Schematic diagram of feature extraction;

[0044] Figure 3 Schematic diagram of the meta-attention module;

[0045] Figure 4 This is a schematic diagram of enhanced operation. DETAILED DESCRIPTION

[0046] The process of weakly supervised augmented activity recognition method is as follows.

[0047] Table 1. Training methods for weakly supervised augmented activity recognition methods

[0048]

[0049]

[0050] The training pseudocode of the weakly supervised augmented activity recognition method is described in Table 1. In each batch of data, we select the support set and query set D from the input time series data. tr (Corresponding to rows 5-6 in Table 1). The parameters are The feature extraction module extracts the original data S k and Q kThe algorithm extracts features from the example feature vector (corresponding to rows 7 and 14 in Table 1). The algorithm highlights the target activity features in the example feature vector by constructing the self-relation matrix R and learning the training parameters Ψ (corresponding to rows 8-9 and 15-16 in Table 1). In each training batch, the algorithm reduces the total loss is reduced to the lowest value (corresponding to row 17 in Table 1). and Ψ are updated and optimized using the gradient descent algorithm (corresponding to row 19 in Table 1).

[0051] During testing, the number of samples in the support set is the same as that in training. For example, 5-shot means that the number of samples in the support set is 5 in both training and testing. We perform enhancement operations on the feature vectors extracted from the query set and refine them according to the learned parameters. and Ψ-computation-enhanced prototypes.

[0052] Predicted labels for samples in the query set is the most likely category, as shown in formula (10).

[0053]

[0054] The network model obtained after training and testing according to the above steps can complete the task of human activity recognition for wearable sensor data, especially in the case of weak supervision and small samples, and can still achieve good results.

[0055] Experiments were conducted on three benchmark datasets: the PAMAP2 dataset, the UCI-HAR dataset, and the SKODA dataset, and compared with supervised and weakly supervised few-shot learning methods. With five samples per class, compared to existing supervised representative methods, FSHAR and PN, the proposed method achieved improvements of 24.27% and 8.69% on the PAMAP2 dataset, 6.61% and 8.45% on the UCI-HAR dataset, and 8.91% on the SKODA dataset. Comparative experiments with weakly supervised methods, including weakly supervised prototype networks, relational networks, and Proto-IATT, achieved average improvements of 4.31% on the SKODA dataset and 3.72% on the PAMAP2 dataset, while only achieving an average improvement of 1.88% on the UCI-HAR dataset. This is due to the relatively low noise content and higher label quality in the UCI-HAR dataset. Experimental results demonstrate that the proposed method outperforms other models in terms of recognition accuracy and feature representation.

Claims

1. A weakly supervised incremental activity recognition method based on meta-attention mechanism, characterized by: Human activity recognition is performed based on an end-to-end self-relational attention weakly supervised prototype network model; the end-to-end self-relational attention weakly supervised prototype network model includes a feature extraction module, a meta-learning attention module, and a classification module; The feature extraction module includes a convolution layer and a maximum pooling layer. The convolution layer has a two-dimensional convolution kernel, which extracts the temporal correlation along the time axis and the spatial correlation along the sensor axis. The meta-learning attention module consists of two convolutional layers, an average pooling layer, and a softmax layer. The meta-learning attention module refers to the pooling operation solution in multi-instance learning, and the sensor data subsequence A n After segmentation through the sliding time window l, the sample feature vector is obtained; the classification module includes training and testing; The specific steps are as follows: Step 1: Extract features from the sensor raw data sequence to obtain a sample feature vector set; Step 2: Obtain the contribution weight of each example feature vector through the meta-learning attention module; Step 2 is specifically as follows: First, for each sample in the support set S and the query set Q, a self-relation matrix R is defined on the one-dimensional example feature vector set Z to represent the similarity between all example feature vectors; the self-relation matrix R is a symmetric matrix, R i,j represents the example feature vector z i With the example feature vector z j The similarity between i,j =sim(z i ,z j );R i is the i-th row of the self-relation matrix R, which represents the example eigenvector z i The similarity between the feature vectors of all other examples; R i,j The larger the value, the greater the possibility that the two example feature vectors are the same type of activity in the feature space; After generating the self-relation matrix R, the self-relation matrix R is fed into the meta-learning attention module; The meta-learning attention module outputs a feature vector z for each example i Contribution weight; Based on the example feature vector z i The meta-attention mechanism of the mutual relationship between them learns a nonlinear function g Ψ For each example feature vector z i assigning weights to contributions; Step 3: Refine and optimize the prototype of the target activity through enhanced operations; Step 4: Perform classification through the classification module to complete the training and testing of the model.

2. The weakly supervised incremental activity recognition method based on the meta-attention mechanism according to claim 1 is characterized in that: The step 1 is specifically as follows: the sensor raw data sequence is divided into a support set S and a query set Q; the sensor raw data sequence A is processed through a sliding time window of size T to obtain a sensor data subsequence The feature extraction module uses a convolutional neural network to extract the weakly labeled sensor data subsequence A n Extract spatiotemporal features from the original data sequence A of the sensor; the spatiotemporal features of the original data sequence A of the sensor are defined as in Including multiple CNN modules, is a learnable parameter; sensor data subsequence A n The spatiotemporal characteristics of H n , H n The sliding time window l is divided into m example features, each example feature is recorded as h i , i∈[1,m]; each example feature h i After being expanded into a one-dimensional vector through the fully connected layer, a one-dimensional example feature vector z is obtained. i ; One-dimensional example feature vector set Z = {z1,z2,…,z m } is input into the meta-learning attention module.

3. The weakly supervised incremental activity recognition method based on meta-attention mechanism according to claim 1 is characterized in that: The contribution weight is first calculated as the contribution score, which is calculated by formula (1): Among them, is a one-dimensional convolutional layer, the convolution kernel size r < m, c is the number of channels; σ is the ReLU activation function, AVG is the average pooling layer, is a convolutional layer with one channel; The obtained contribution score is input into the softmax layer to map it to the weight distribution in formula (2), where α i represents the i-th example feature vector z i Contribution weight to its data subsequence characteristics; 4. The weakly supervised incremental activity recognition method based on the meta-attention mechanism according to claim 1 or 3, characterized in that: The step 3 is specifically as follows: Through the enhancement operation, each example feature vector z is refined and optimized i , the example feature vector after the enhancement operation is purified and optimized is recorded as is the example eigenvector z i The average of the example feature vectors that satisfy the threshold constraint θ; Calculated in formula (4) and formula (5), ω is an adjacency matrix, ω i,j Indicates z i Is it related to z j adjacent in the metric space with a threshold constraint of θ; the threshold constraint θ determines the degree of confusion of the information; Feature representation of the data subsequence after purification and optimization by enhancement operation Calculated by formula (4): For the k-th class sample, the enhanced prototype is calculated by formula (5) in It is the feature vector set of the k-th class sample after purification and optimization through enhancement operation; 5. The weakly supervised incremental activity recognition method based on the meta-attention mechanism according to claim 4 is characterized in that: The adjacency matrix ω satisfies the constraint condition of formula (6); where θ l A local threshold is introduced for each sample in each category; the local threshold θ is set l =min(max(R i ′ )), where i∈[1,m], R ′ is the matrix obtained by setting the diagonal element values ​​of the self-relation matrix R to 0; the local threshold θ l The method of finding the minimum similarity from the maximum similarity set of each example feature vector ensures that all example feature vectors can confuse the nearest neighbor information; τ is θ l Set constraints.

6. The weakly supervised incremental activity recognition method based on meta-attention mechanism according to claim 5 is characterized in that: The step 4 is specifically as follows: The end-to-end self-relational attention weakly supervised prototype network model is trained through the classification loss of the query set Q. The loss function is the negative log probability of the target class, that is, formula (9): By calculating the cross entropy loss between the probability of each category output by the classification module and the true activity label, the classification loss of the end-to-end self-relational attention weakly supervised prototype network model is obtained, and the parameters in the end-to-end self-relational attention weakly supervised prototype network model are updated by backpropagation.

Images