Memory-enhanced action recognition method and system on Riemannian manifold, and storage medium

Abstract

The present invention discloses a memory-enhanced action recognition method and system on a Riemannian manifold, and a storage medium. The method comprises the following steps: step 1: collecting human motion data and expressing it as a third-order tensor; step 2: expanding along three modes to obtain three corresponding matrices; step 3: calculating a memory-enhanced weight matrix using the human short-term memory mechanism; step 4: decomposing the weight matrix using the principal component analysis method to obtain weighted basis vectors; step 5: reorganizing and parallelizing the normalized mapping to a unit hypersphere, preserving the angular relationship; step 6: learning the modal weight parameters using the Monte Carlo Markov algorithm and calculating the geometric differences between points on the hypersphere; and step 7: using a K-nearest neighbor classifier to classify human motions. The present invention effectively solves the problem of temporal information loss in complex scenarios and is suitable for a variety of application scenarios such as medical health, virtual reality, and sports training.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a memory-enhanced action recognition method and system on a Riemannian manifold. Background Art

[0002] Human action recognition (HAR) is a key research area in computer vision. Its primary goal is to accurately and rapidly identify human behavioral patterns from video or sensor data. In recent years, with the rapid development of deep learning, HAR technology has been widely applied in healthcare monitoring, intelligent security, sports training, virtual reality, and other fields. However, existing methods still face numerous challenges in handling complex spatiotemporal relationships, multi-view data fusion, keyframe capture, and long time series modeling.

[0003] Traditional methods rely primarily on manual feature extraction, such as optical flow and spatial-temporal points of interest. However, these methods have poor adaptability in complex scenarios and limited performance for highly dynamic actions. In recent years, action recognition methods based on deep learning have made significant progress. Convolutional neural networks (CNNs) can effectively extract spatial features but cannot fully model temporal information. Recurrent neural networks (RNNs) and their variants, such as long short-term memory (LSTM) gated recurrent units (GRUs), can capture short-term dependencies but suffer from the vanishing gradient problem when modeling long-term temporal information. Spatiotemporal graph convolutional networks (STGCNs) utilize graph neural networks to model the human skeleton, but their reliance on skeletal data limits their generalization across various application scenarios. In recent years, Transformer-based methods such as ST-TransNet have captured long-term dependencies through a self-attention mechanism, but their high computational complexity makes them difficult to apply to real-time scenarios.

[0004] Existing methods still have limitations in modeling fine-grained dynamic information of human motion, optimizing spatiotemporal feature fusion, and improving cross-category recognition capabilities. For long time series, existing models have difficulty maintaining long-term temporal dependencies, which affects recognition accuracy. Furthermore, most methods adopt a separate strategy for modeling spatial and temporal features, resulting in insufficient feature fusion and reduced expressiveness of motion patterns. Specifically, existing methods have the following three major technical issues:

[0005] First, traditional covariance matrix methods suffer from a loss of temporal information when modeling spatiotemporal features. When quantifying inter-modal correlations, the averaging effect of the covariance matrix can dilute or mask important temporal information, including motion rhythm and acceleration changes. This limitation, especially in long sequences, reduces the model's ability to detect and respond to key temporal features.

[0006] Second, existing methods perform poorly when dealing with fast actions (such as throwing or running) and long continuous action sequences. These methods often model space and time as independent processes, which limits their ability to effectively integrate spatiotemporal features and leads to insufficient representation of fine-grained dynamics and complex spatiotemporal fusion patterns.

[0007] Third, existing dynamic weighting strategies primarily focus on the temporal dimension, while dynamic modeling of spatially significant features remains insufficient. Most methods employ static or random weight distribution strategies, failing to balance the importance of spatiotemporal features. This is particularly true in high-dimensional data such as human motion sequences, where modeling of nonlinear dynamic features remains insufficient.

[0008] Therefore, proposing a method that can simultaneously optimize the fusion of spatiotemporal features, improve short-term dynamic sensitivity, and enhance model robustness is of great significance for the accuracy and stability of human action recognition. Summary of the Invention

[0009] The technical problem to be solved by the embodiments of the present invention is to address the limitations of existing human action recognition methods in handling complex dynamic changes, long-term temporal dependencies, and spatial-temporal feature interactions, and to provide a memory-enhanced action recognition method and system on a Riemannian manifold to improve recognition accuracy. To address the above technical problems, the embodiments of the present invention propose a memory-enhanced action recognition method on a Riemannian manifold, comprising:

[0010] Step 1: Collect human motion data, process the data according to the first-order difference information of the data, and express the data as a third-order tensor X i ;

[0011] Step 2: Convert the third-order tensor X i Expand to get three matrices, each corresponding to a different subspace;

[0012] Step 3: Use the human short-term memory mechanism to calculate the corresponding memory enhancement weight matrix;

[0013] Step 4: Decompose the weight matrix by principal component analysis to obtain basis vectors with weights;

[0014] Step 5: Reorganize the basis vectors into a matrix and normalize each row vector of the matrix so that it is mapped onto the unit hypersphere, where each row vector of the matrix corresponds to a point on the sphere;

[0015] Step 6: Use the Monte Carlo Markov chain algorithm to learn the optimal modal weight parameters, calculate the angular distance between points on the hypersphere, and obtain the geometric difference between the points;

[0016] Step 7: Use the K nearest neighbor classifier to classify human actions based on geometric differences and output the classification results.

[0017] Accordingly, an embodiment of the present invention further provides a memory-enhanced action recognition system on a Riemannian manifold, comprising:

[0018] Data acquisition and processing module: collects human motion data, processes the data according to the first-order difference information of the data, and represents the data as a third-order tensor; expands the third-order tensor to obtain three matrices, each matrix corresponding to a different subspace;

[0019] Dynamic weighting module: uses the human short-term memory mechanism to calculate the corresponding memory-enhanced weight matrix; decomposes the weight matrix through the principal component analysis method to obtain the basis vector with weights;

[0020] Riemannian metric module: reorganizes the basis vectors into a matrix and normalizes each row vector of the matrix so that it is mapped onto the unit hypersphere, where each row vector of the matrix corresponds to a point on the sphere; calculates the angular distance between points on the hypersphere to obtain the geometric difference between the points;

[0021] Parameter optimization module: Use Monte Carlo Markov chain algorithm to optimize the weight parameter w of each mode j , adaptively balancing the contributions of different modes to the overall geometric differences;

[0022] Action recognition module: uses K-nearest neighbor classifier to classify human actions according to geometric differences and outputs the classification results.

[0023] Accordingly, an embodiment of the present invention further provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the memory enhancement action recognition method on a Riemannian manifold.

[0024] The beneficial effects of the present invention are:

[0025] 1. Improved recognition accuracy: By mapping tensors to a hypersphere and utilizing angular relationships to quantify inter-modal variations, this method effectively preserves the geometric structure of spatial and temporal information, reduces information loss, and achieves state-of-the-art recognition results on datasets such as CMB and NW.

[0026] 2. Enhanced modeling capabilities for complex motions: The dynamic weighting mechanism of the present invention can adaptively capture rapid motion changes and long-term temporal dependencies, especially the sensitivity to subtle motion changes in complex scenarios.

[0027] 3. Reduced computational complexity: Compared with deep learning methods, the tensor geometry-based method of the present invention is more efficient, especially when processing long time series, the computation time is greatly reduced.

[0028] 4. Improved model robustness: Through the normalization processing of hyperspherical mapping, the present invention has stronger resistance to external factors such as illumination changes, perspective changes, and background noise, reducing the impact of environmental interference on recognition results.

[0029] 5. Wide range of application scenarios: This invention achieves accurate motion recognition, laying the foundation for multiple application fields, including healthcare monitoring, virtual reality interaction, sports training analysis, and intelligent security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 3 is a structural diagram of a memory-enhanced action recognition method on a Riemannian manifold according to an embodiment of the present invention.

[0031] Figure 2 3 is a schematic diagram of tensor expansion according to an embodiment of the present invention, illustrating the process of expanding the tensor along different modes.

[0032] Figure 3 3 is a mapping diagram of an embodiment of the present invention, showing the distribution of normalized points on a unit hypersphere. DETAILED DESCRIPTION

[0033] It should be noted that, unless there is a conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The present invention is further described in detail below with reference to the drawings and specific embodiments.

[0034] In the embodiments of the present invention, if there are directional indications (such as up, down, left, right, front, back, etc.), they are only used to explain the relative position relationship and movement status of the various components under a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0035] In addition, the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Therefore, features specified as "first" or "second" may explicitly or implicitly include at least one of these features.

[0036] Please refer to Figure 1 The memory-enhanced action recognition method on a Riemannian manifold according to an embodiment of the present invention includes steps 1 to 7.

[0037] Step 1: Collect human motion data (such as Figure 1As shown in (a), the original data of human body movement includes original information and differential information. The data is processed according to the first-order differential information of the data and represented as a third-order tensor X i The human motion data contains n samples, expressed as Each sample A video sequence representing an action process, is the corresponding label, where is the total number of action categories. is organized as a third-order tensor, where and Represent the spatial dimensions of the video frame, represents the time dimension, i.e., the number of frames in the sequence. This tensor representation effectively captures the spatiotemporal structure of the action data.

[0038] Human motion data is inherently complex and often affected by external factors such as illumination variations, noise, and individual differences, which can obscure key spatiotemporal patterns and pose challenges to accurate classification. To mitigate these effects, this paper introduces first-order difference information to emphasize the dynamics of motion along the time dimension, thereby reducing the influence of external static features and noise. Indicates along The discrete time variation of the temporal pattern of is defined as:

[0039] ;

[0040] in Representing a tensor The discrete difference along the time pattern is defined as:

[0041] ;

[0042] Loop padding is applied at the boundaries to ensure that the first-order difference tensor The dimensions of the original tensor identical, thus ensuring compatibility with subsequent processing steps.

[0043] First-order difference tensor It extracts temporal features such as action onset, intensity changes, and speed fluctuations. It suppresses redundant information such as static background and illumination changes, thereby highlighting key spatiotemporal features and improving the accuracy and robustness of the model in dynamic environments.

[0044] Step 2: Convert the third-order tensor X i Expand to get three matrices (such as Figure 1 As shown in (b), each matrix corresponds to a different subspace.

[0045] Spatiotemporal tensor representations provide a structured geometric framework for video data by expanding the tensor into subspaces for analysis. These representations have been widely used in action recognition because they are able to model the intrinsic relationship between spatial and temporal modalities. For a given video tensor , which contains the width ,high and time Dimensions, the tensor is expanded into three mode-specific matrices:

[0046] ;

[0047] 、 、 Denote matrices expanded along the width dimension, along the height dimension, and along the time dimension, respectively. Each expanded matrix corresponds to a different subspace, providing a deeper understanding of the spatial and temporal characteristics of the motion data. This decomposition enables a more comprehensive geometric interpretation of the tensor by focusing on its inherent multimodal structure.

[0048] A common method to generate a subspace representation is to calculate the covariance matrix of the unfolded tensor. The covariance matrix is usually defined as:

[0049] ;

[0050] Among them, the tensor Indicates that along The tensor of the mode expansion. Covariance-based methods are often used to generate subspace representations. They can effectively capture global correlations, but they also have significant limitations. These methods lose key spatial or temporal information by averaging variance across modalities. Specifically, for temporal patterns ,The covariance operation suppresses rapid changes and masks dynamic features such as speed changes and ,action boundaries.

[0051] In order to overcome the limitations of the covariance-based method, the present invention represents the video tensor as a spatiotemporal subspace. , which captures the width, height, and time dimensions, and the tensor is unfolded along three modes to extract structured subspaces (see Figure 2, Figure 2 Modes 1 and 2: Visual attention mechanisms in the spatial dimension, from left to right / right to left; from top to bottom / bottom to top, respectively; Mode 3: In the temporal dimension, different observation orders correspond to different dynamic memory modes, emphasizing the beginning / end phase of the sequence. This approach leads to the loss of critical spatial or temporal information, especially for tensors containing temporal dependencies (such as videos).

[0052] In time mode, tensors can be expanded to:

[0053] ;

[0054] The pattern matrix Represents frames along the time dimension. Each column vector Corresponds to the time frame in the sequence The time information is essentially passed through the matrix The order of the columns in the byte array is encoded, reflecting the order of the frames.

[0055] However, constructing the time modal subspace requires the autocorrelation matrix Perform singular value decomposition (SVD). SVD is insensitive to the order of the matrix columns, which means that no matter how the frame sequence is arranged, the eigenvectors and eigenvalues it produces are the same. Therefore, the original matrix The temporal information encoded by the column order is not preserved in the subspace representation.

[0056] The present invention adopts a multi-layer feature fusion strategy, combining temporal information and spatial structure, so that the model can capture motion features at different time and spatial scales, and improve the modeling ability of high-order spatiotemporal relationships.

[0057] Step 3: Use the human short-term memory mechanism to calculate the corresponding memory enhancement weight matrix (such as Figure 1 (As shown in (c), the corresponding forward and backward matrices are obtained based on the initial matrix). This invention adopts a memory-enhanced subspace modeling strategy. By combining motion features with memory weights, it calculates the covariance matrix and extracts the main motion patterns. This avoids the loss of temporal information during the calculation process and enhances the ability to model time-dependent dependencies. Since temporal information is reduced (eliminated) during matrix calculation during covariance matrix decomposition, this invention addresses this issue through STM weighting.

[0058] Step 4: Decompose the weight matrix using principal component analysis to obtain basis vectors with weights.

[0059] The human short-term memory (STM) mechanism is able to retain historical information for a short period of time, allowing the encoding of sequential information. Various computational models have simulated STM. For example, the biologically inspired neuron model simulates the dynamic changes of membrane potential through delays and encodes temporal information by calculating the difference between the input sequence and the neuron state. The response vector of the neuron response The calculation formula is:

[0060] ;

[0061] in, is the current input, It is The reference point of each neuron, is the forgetting parameter. This parameter controls the balance between past information and current information. Specifically, when When , the system will retain all past information equally; and when When updating, it only keeps the latest information.

[0062] In order to better process sequence data, a time-adaptive self-organizing map (TASOM) is proposed. This model captures the temporal characteristics of sequence signals through a dynamic update mechanism. In this model, the input pattern is defined as a weighted combination of the current input and the previous input:

[0063] ;

[0064] in, Indicates the current input mode. Is the memory depth parameter that dynamically controls the weighting between historical information and current information. Emphasizes long-term historical information, while larger Changes to the current input are given priority.

[0065] The above formula can be re-expressed as:

[0066] ;

[0067] It can be expressed as matrix multiplication:

[0068] ;

[0069] It is the sum of the columns of the weight matrix AX under the memory mechanism.

[0070] The traditional covariance matrix is calculated as , put it into memory mode to get ,get ;

[0071] Since A is a diagonal matrix, , and finally get:

[0072] ;

[0073] At this point, the structure of the subspace is determined only by It is determined by the covariance matrix of , which has nothing to do with the memory matrix A, resulting in the loss of timing information.

[0074] To solve this problem, the present invention explicitly integrates the temporal structure and memory mechanism into the subspace representation of the data, and centralizes the data and applies PCA (Principal Component Analysis) projection (e.g. Figure 1 As shown in (d), the new subspace representation is obtained , the PCA projection is: .

[0075] For mode with memory , its PCA projection expression is:

[0076] ;

[0077] In this way, the projection result not only includes the influence of standard PCA, but also considers the offset of the data structure caused by the memory mechanism, thus avoiding the loss of temporal information.

[0078] Step 5: Reorganize the basis vectors into a matrix and normalize each row vector of the matrix so that it is mapped onto the unit hypersphere (e.g. Figure 1 As shown in (e), each row vector of the matrix corresponds to a point on the sphere. The present invention reduces mean drift and improves the stability of data distribution through normalization.

[0079] Step 6: Use the Monte Carlo Markov chain algorithm to learn the optimal modal weight parameters, calculate the angular distance between points on the hypersphere, and obtain the geometric differences between samples (such as Figure 1 (shown in (e)).

[0080] In order to compare subspaces from different samples, the row normalization operation converts the mode matrix Mapped onto the hypersphere. Normalization is performed row by row, and the formula is as follows:

[0081] ;

[0082] in, Indicates the Euclidean norm of the row. Normalized matrix Represents a set of points on the hypersphere, each point corresponding to a row vector of the original matrix (see Figure 3 ). This mapping preserves the geometric structure of the data while facilitating the computation of distances between subspaces.

[0083] Because the hypersphere is a Riemannian manifold with constant positive curvature, this paper uses the Riemannian metric to measure the geometric differences between points on the manifold. Metrics on a hypersphere must account for its curvature. To this end, this paper introduces the geodesic distance, which is the shortest distance between two points on the manifold. On a hypersphere, the geodesic distance is calculated as the angular distance between two points. This angle measures the angle between them on the hypersphere and effectively reflects their geometric differences.

[0084] The calculation of geometric differences includes:

[0085] First calculate two normalized mode matrices and The standard angular distance between:

[0086] ;

[0087] The standard angle θ l pass It can be calculated that, Representation matrix The singular values of , and then use the Monte Carlo Markov chain algorithm to learn the optimal modal weight parameter w j , summarizing the geometric relationships of all tensor modes:

[0088] .

[0089] in, is a modality-specific weight parameter that adjusts the contribution of each modality to the overall geometric difference metric. This weighting mechanism ensures that key modalities (such as temporal or spatial dimensions) are prioritized. As a result, the model can more effectively focus on features that are highly relevant to the task.

[0090] The Monte Carlo Markov chain algorithm includes the following steps:

[0091] (1) Initialize the weight parameter w j ;

[0092] (2) For each iteration t = 1 to T:

[0093] a) Generate candidate weights from the proposal distribution: ;

[0094] b) Calculate the probability of acceptance

[0095] ;

[0096] in represents the posterior probability distribution;

[0097] c) Generate uniform random numbers ;

[0098] d) If the conditions are met Accept the proposal and update ; otherwise keep the original value: ;

[0099] (3) Return the optimized weight parameters .

[0100] Step 7: Use K nearest neighbor classifier (such as Figure 1 As shown in (e) in the figure, human motion classification is performed based on geometric differences, and the classification results are output. To identify human motion data, the present invention calculates the Riemann distance between all data points and uses the KNN method (k-nearest neighbor algorithm, a commonly used supervised learning method) to select the K closest points. The labels of these K points are then voted to determine the label for the human motion data. To optimize the KNN classification results, the present invention optimizes the neighborhood relationship on the hypersphere, adapting the data distribution to the geometric structure of different categories and improving cross-category recognition capabilities.

[0101] The present invention is applicable to different categories of human motion, including basic movements (walking, running, jumping) as well as complex activities (throwing, dancing, martial arts, etc.), enhancing the model's adaptability. It supports the processing of video data at varying resolutions, ensuring recognition performance in both high- and low-resolution environments. It employs an adaptive weight adjustment strategy to optimize the model's sensitivity to different motion categories, improving cross-category generalization and adapting it to a variety of application scenarios. The present invention can be integrated into online learning frameworks, enabling the model to dynamically update as new data is input, enhancing its adaptability to environmental changes and individual differences, and ultimately improving recognition accuracy in long-term use scenarios.

[0102] The memory-enhanced action recognition system on a Riemannian manifold of the present invention comprises:

[0103] Data acquisition and processing module: collects human motion data and processes the data according to the data X i First-order difference information Process the data and transform the third-order tensor X i and Expand them separately to get three matrices, each corresponding to a different subspace;

[0104] Dynamic weighting module: uses the human short-term memory mechanism to calculate the corresponding memory-enhanced weight matrix; decomposes the weight matrix through the principal component analysis method to obtain the basis vector with weights;

[0105] Riemannian metric module: reorganizes the basis vectors into a matrix and normalizes each row vector of the matrix so that it is mapped onto the unit hypersphere, where each row vector of the matrix corresponds to a point on the sphere; calculates the angular distance between points on the hypersphere to obtain the geometric difference between the points;

[0106] Parameter optimization module: Use MCMC algorithm to optimize the weight parameter w of each mode j , adaptively balancing the contributions of different modes to the overall geometric differences;

[0107] Action recognition module: uses K-nearest neighbor classifier to classify human actions according to geometric differences and outputs the classification results.

[0108] As an implementation method, the data acquisition and processing module includes:

[0109] Image acquisition unit: used to collect human motion sequence data from the video source array;

[0110] First-order difference calculation unit: used to calculate the first-order difference in the time dimension of the collected video data, emphasizing the dynamic characteristics of motion;

[0111] Tensor building unit: used to organize the differential data into a third-order tensor structure ;

[0112] Tensor expansion unit: used to expand the third-order tensor along different modes into corresponding matrix representations 、 、 .

[0113] As an embodiment, the dynamic weighting module includes:

[0114] Memory parameter calculation unit: used to calculate the memory depth parameter a∈(0,1) based on the human short-term memory mechanism;

[0115] Weight matrix generation unit: used to generate the corresponding weight matrix A according to the memory depth parameter;

[0116] Principal component analysis unit: used to perform principal component analysis on weighted data and extract the main eigenvector V with weights;

[0117] The memory parameter calculation unit realizes adaptive fusion of information of different time scales by dynamically adjusting the value of parameter a.

[0118] As an implementation method, the Riemannian metric module includes:

[0119] Hypersphere mapping unit: used to normalize the matrix row vectors and map them onto the unit hypersphere;

[0120] Angular distance calculation unit: used to calculate the geodesic distance between points on the hypersphere and quantify the geometric differences between subspaces; wherein, the angular distance calculation unit calculates the standard angle θ by the singular value decomposition method l , ensuring geometric preservation of spatial and temporal features.

[0121] As an implementation method, the parameter optimization module includes:

[0122] Objective function construction unit: used to construct an optimization objective function based on recognition accuracy;

[0123] Monte Carlo Markov chain sampling unit: used to execute the Monte Carlo Markov chain algorithm and generate the Markov chain of weight parameters;

[0124] Parameter convergence judgment unit: used to evaluate the convergence of weight parameters and determine the final optimization result; wherein, the Monte Carlo Markov chain sampling unit guides the parameter search process through the posterior probability distribution P(·) to ensure that the weight distribution matches the action category discrimination ability.

[0125] In addition, the present invention also proposes a storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the memory enhancement action recognition method on a Riemannian manifold are implemented.

[0126] Inspired by human cognitive mechanisms, the present invention introduces a dynamic weighted allocation strategy that can adapt to changes in time and space. Specifically, the present invention maps the unfolded tensor patterns onto a unit hypersphere to ensure the preservation of global and local geometric relationships in spatiotemporal features. By utilizing the angular relationship on the hypersphere, the present invention can quantify the relative changes between tensor patterns, thereby alleviating the problem of temporal information loss common in traditional covariance-based methods. In addition, the present invention prioritizes key temporal moments and spatial regions through dynamic weighting, effectively addressing the challenges posed by rapid motion and subtle spatial changes. This dynamic weighting mechanism enhances the fusion and modeling of spatiotemporal features, thereby improving the model's ability to capture the temporal dynamics and spatial dependencies inherent in human actions. The method of the present invention was evaluated on the CMB, NW, UTK, KTH, and MHAD datasets, and achieved state-of-the-art results on the CMB and NW datasets. These results demonstrate the effectiveness and robustness of the method of the present invention in action recognition scenarios.

[0127] The main innovations of the embodiments of the present invention include:

[0128] 1. An adaptive dynamic weighted allocation strategy inspired by human memory patterns is proposed. This strategy dynamically adjusts the weight distribution according to temporal changes and spatial observation order, enabling the model to efficiently focus on key time points and important spatial regions.

[0129] 2. A dynamic feature modeling method based on hypersphere embedding is designed, which maps the matrix rows of the expanded tensor onto the unit hypersphere, alleviating the temporal information loss problem common in traditional covariance matrix methods.

[0130] 3. The constant curvature property of the hypersphere and manifold geometry tools are used to jointly analyze spatiotemporal features, adjust the spatial weight distribution, enhance the model's sensitivity to key spatial features, and effectively integrate spatiotemporal information.

[0131] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A memory-enhanced action recognition method on a Riemannian manifold, characterized in that: include: Step 1: Collect human motion data, process the data according to the first-order difference information of the data, and express the data as a third-order tensor X i ; Step 2: Convert the third-order tensor X i Expand to get three matrices, each corresponding to a different subspace; Step 3: Use the human short-term memory mechanism to calculate the corresponding memory enhancement weight matrix; Step 4: Decompose the weight matrix by principal component analysis to obtain basis vectors with weights; Step 5: Reorganize the basis vectors into a matrix and normalize each row vector of the matrix so that it is mapped onto the unit hypersphere, where each row vector of the matrix corresponds to a point on the sphere; Step 6: Use the Monte Carlo Markov chain algorithm to learn the optimal modal weight parameters, calculate the angular distance between points on the hypersphere, and obtain the geometric difference between samples; Step 7: Use the K-nearest neighbor classifier to classify human actions based on geometric differences and output the classification results; In step 6, the calculation of geometric differences includes: First calculate two normalized mode matrices and The standard angular distance between: ； where θ l pass It can be calculated that, Representation matrix The singular values of , and then use the Monte Carlo Markov chain algorithm to learn the optimal modal weight parameter w j , summarizing the geometric relationships of all tensor modes: ； The Monte Carlo Markov chain algorithm includes the following steps: (1) Initialize the weight parameter w j ; (2) For each iteration t = 1 to T: a) Generate candidate weights from the proposal distribution: ; b) Calculate the probability of acceptance ； in represents the posterior probability distribution; c) Generate uniform random numbers ; d) If the conditions are met Accept the proposal and update ; otherwise keep the original value: ; (3) Return the optimized weight parameters .

2. The memory-enhanced action recognition method on a Riemannian manifold according to claim 1, wherein: In step 1, the human motion data is represented as a third-order tensor ,in, and Represent the width and height in the spatial dimension respectively; Represents the time dimension, containing the number of frames in the video sequence.

3. The memory-enhanced action recognition method on a Riemannian manifold according to claim 1, wherein: In step 1, through the first-order difference tensor Indicates along The discrete time variation of the temporal pattern of is defined as: ; in Representing a tensor The discrete difference along the time pattern is defined as: ； Among them, circular filling is used at the boundary of the time dimension to maintain the consistency of the data dimension.

4. The memory-enhanced action recognition method on a Riemannian manifold according to claim 2, wherein: In step 2, the three matrices are expressed as: ； in, 、 、 Represents the matrices expanded along the width dimension, height dimension, and time dimension respectively.

5. The memory-enhanced action recognition method on a Riemannian manifold according to claim 1, wherein: In step 3, temporal information is encoded by computing the difference between the input sequence and the neuron state: ； in, is the current input, It is Reference point of each neuron is the memory depth parameter, Indicates the The response vector of each neuron's response; And through the dynamic update mechanism to capture the time characteristics of the sequence signal, the input pattern It is defined as a weighted combination of the current input and the previous input: 。 6. The memory-enhanced action recognition method on a Riemannian manifold according to claim 2, wherein: In step 5, row normalization is performed according to the following formula: ； in, Indicates the Euclidean norm of the rows, normalized matrix Represents a set of points on the hypersphere, each point corresponding to a row of vectors in the matrix.

7. A memory-enhanced action recognition system on a Riemannian manifold, characterized in that: include: Data acquisition and processing module: collects human motion data, processes the data according to the first-order difference information of the data, and represents the data as a third-order tensor; Expand the third-order tensor to obtain three matrices, each matrix corresponding to a different subspace; Dynamic weighting module: uses the human short-term memory mechanism to calculate the corresponding memory-enhanced weight matrix; decomposes the weight matrix through the principal component analysis method to obtain the basis vector with weights; Riemannian metric module: reorganizes the basis vectors into a matrix and normalizes each row vector of the matrix so that it is mapped onto the unit hypersphere, where each row vector of the matrix corresponds to a point on the sphere; calculates the angular distance between points on the hypersphere to obtain the geometric difference between the points; Parameter optimization module: Use Monte Carlo Markov chain algorithm to optimize the weight parameter w of each mode j , adaptively balancing the contributions of different modes to the overall geometric differences; Action recognition module: uses the K-nearest neighbor classifier to classify human actions based on geometric differences and outputs the classification results; The Riemannian metric module obtains geometric differences according to the following steps: First calculate two normalized mode matrices and The standard angular distance between: ； where θ l pass It can be calculated that, Representation matrix The singular values of , and then use the Monte Carlo Markov chain algorithm to learn the optimal modal weight parameter w j , summarizing the geometric relationships of all tensor modes: ； The Monte Carlo Markov chain algorithm includes the following steps: (1) Initialize the weight parameter w j ; (2) For each iteration t = 1 to T: a) Generate candidate weights from the proposal distribution: ; b) Calculate the probability of acceptance ； in represents the posterior probability distribution; c) Generate uniform random numbers ; d) If the conditions are met Accept the proposal and update ; otherwise keep the original value: ; (3) Return the optimized weight parameters .

8. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the memory-enhanced action recognition method on a Riemannian manifold according to any one of claims 1 to 6 are implemented.

Images