An AI-driven behavior feature recognition algorithm and device

Abstract

This invention discloses an AI-driven behavior feature recognition algorithm and apparatus, comprising the following steps: Step 1: Acquiring a video sequence of the target object's behavior and extracting a confidence sequence; Step 2: Generating a skeleton graph observation sequence based on a multi-source temporal observation set and constructing continuous-time interpolation; Step 3: Continuously evolving and updating the hidden state by combining adjacent time intervals and mapping it to continuous latent variables to obtain continuous latent trajectories; Step 4: Inputting the continuous latent trajectories into an improved Koopman model to obtain Koopman latent space trajectories; Step 5: Performing stage decomposition on the Koopman latent space trajectories; Step 6: Constructing a point cloud representation of the Koopman latent space trajectories and calculating persistent cohomology; Step 7: Constructing joint features to obtain the behavior feature recognition result. This invention integrates continuous temporal modeling with an improved Koopman model to achieve stable and interpretable behavior recognition.

Description

Technical Field

[0001] This invention relates to the field of behavior understanding and human posture analysis technology, and in particular to an AI-driven behavior feature recognition algorithm and device. Background Technology

[0002] In applications such as smart security, smart elderly care, and compliance monitoring of industrial operations, video-based behavior recognition typically uses image frames or short clips as input, utilizing human pose key points or skeleton sequences for action classification and detection. A common approach in existing technologies is to assemble key point coordinates into discrete time series at a fixed frame rate, then extract features and output behavior categories using recursive models or temporal convolution. However, in real-world scenarios, video acquisition is often accompanied by frame rate fluctuations, frame drops, occlusion, and unstable pose estimation confidence, resulting in unequal sampling intervals and local missing values ​​in the key point sequence. This makes discrete sequence modeling sensitive to changes in time intervals, leading to jitter in behavior recognition results, unstable stage boundaries, and insufficient cross-scene generalization ability. Furthermore, for long-term or complex behavioral processes, existing technologies primarily rely on end-to-end feature extraction followed by direct category determination, lacking explicit characterization and constraints on the internal stage structure of the behavior. This makes it difficult to establish stable distinctions between similar actions and to provide information on stage transitions and boundaries. Even when using dynamics to model the latent space, common linear evolutionary modeling is prone to divergence or instability during multi-step prediction, causing the latent trajectory to drift over time, thus affecting stage decomposition and subsequent discrimination. Meanwhile, existing methods generally focus on local temporal features, lacking characterization of the overall structural invariants of the trajectory, and are prone to losing behavioral structural information under conditions of viewpoint changes, action speed changes, or local missing features. Based on this, this invention addresses the shortcomings of unstable modeling of keypoint sequences with unequal interval sampling and missing keypoint sequences, unreliable stage decomposition due to unstable latent space dynamic evolution, and insufficient discrimination robustness due to the lack of structural invariant features. It proposes a behavioral feature recognition scheme oriented towards continuous-time control signals, an improved Koopman latent space trajectory, and a joint discrimination scheme using a set of stage evolution operators and persistent homology topological signatures, to improve recognition stability and structured output capabilities in complex scenarios.

[0003] Therefore, how to provide an AI-driven behavior feature recognition algorithm and device is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose an AI-driven behavior feature recognition algorithm and device. This invention constructs a continuous-time control signal using human key points and confidence levels, generates potential trajectories through continuous evolution of hidden states, obtains stable latent space trajectories using an improved Koopman model with spectral stability constraints, and performs stage decomposition and persistent homology topological signature extraction. By combining stage operators and topological features, stable and interpretable behavior recognition output is achieved.

[0005] According to an embodiment of the present invention, an AI-driven behavior feature recognition algorithm includes the following steps: Step 1: Collect video sequences of the target object's behavior, extract the coordinate sequence of human body key points and the confidence sequence corresponding to each human body key point, and form a multi-source time series observation set; Step 2: Based on the multi-source time-series observation set, establish nodes using human joints, establish edges using skeletal topological connections, generate a skeleton graph observation sequence, and construct continuous-time interpolation to obtain continuous-time control signals; Step 3: Read the continuous time control signals in the order of timestamps, combine the adjacent time intervals to continuously evolve and update the hidden state, and map the hidden state to continuous latent variables to obtain the continuous potential trajectory; Step 4: Input the continuous latent trajectory into the improved Koopman model, and obtain the Koopman latent space trajectory through the Koopman encoder, evolution operator solver, spectral stability constraint module, linear evolution module, Koopman decoder and consistency verification module; Step 5: Perform stage decomposition on the Koopman latent space trajectory to obtain the final stage switching sequence and stage boundaries, and generate a set of final stage evolution operators corresponding to each stage. Step 6: Construct a point cloud representation of the Koopman latent space trajectory, and calculate persistent cohomology on the point cloud representation to obtain the topological signature set for each stage; Step 7: Construct joint features based on the final stage evolution operator set, final stage switching sequence, stage boundary and topological signature set to obtain behavioral feature recognition results.

[0006] Optionally, step one specifically includes: The behavior video sequence of the target object is acquired, and the behavior video sequence is decoded frame by frame to obtain the original image frame sequence arranged in frame order; The original image frame sequence is subjected to pose estimation processing, which involves outputting the coordinates of human key points for each frame and concatenating the coordinates of human key points in time order to form a sequence of human key point coordinates. While outputting the coordinates of human body key points, the system outputs the confidence score corresponding to each human body key point for each frame of the image, and splices the confidence scores of each frame in chronological order to form a confidence score sequence. The sequence of human body key point coordinates and the confidence sequence are written into the acquisition timestamp, and the acquisition timestamp corresponds one-to-one with the corresponding image frame; Based on the acquisition timestamp, the coordinate sequence of human key points and the confidence sequence are time-aligned to obtain the observation sequence sorted by timestamp, and the time interval between adjacent observations is recorded as unequal interval sampling information. The validity screening process is performed on the human body key point coordinates based on the confidence score sequence. The validity screening process is to write missing markers on the human body key point coordinates with confidence scores below a preset threshold, and retain the key point coordinates with no missing points and their corresponding confidence scores. The coordinates, confidence scores, and unequal interval sampling information of human key points that are completed and have missing markers are collected to form a multi-source time-series observation set with unequal interval sampling.

[0007] Optionally, the two steps specifically involve: Extract the human body key point coordinate sequence, confidence sequence and missing marker information sorted by timestamp from a multi-source time-series observation set sampled at unequal intervals; Using each human joint in the human keypoint set as a node, edges are established for any two adjacent joints according to the preset human skeleton topology connection relationship. At each time stamp, a corresponding skeleton graph observation is generated, resulting in a skeleton graph observation sequence composed of skeleton graph observations at each time stamp. For each node at each timestamp, a node feature vector is constructed. The node feature vector includes the coordinate components of the joint corresponding to the node, the joint velocity components calculated from the joint coordinate differences of adjacent timestamps, the confidence level corresponding to each node, and the corresponding missing marker. In the case of missing markers, write a missing identifier to the feature vector of the node marked as missing and retain its timestamp position as a missing event; Based on the timestamps corresponding to the feature vectors of each node, piecewise linear interpolation over continuous time is constructed for each dimension of the feature vectors of each node, and the corresponding interpolated features are output at any continuous time point. The continuous time interpolation output of each node is combined according to the node dimension and feature dimension to form a continuous time control signal. The continuous time control signal contains node feature information, confidence level and missing marker information corresponding to the skeleton graph observation at any continuous time point.

[0008] Optionally, step three specifically includes: Based on continuous time control signals, the start and end times of the corresponding time period of the behavioral video sequence are determined, and the control signal values ​​at each timestamp are read sequentially according to the order of the timestamps. At the same time, the time interval between any two adjacent reads is recorded. Between any two adjacent timestamps, the hidden state is updated by time evolution according to the time interval until the hidden state continuously evolves from the state corresponding to the previous timestamp to the intermediate state corresponding to the next timestamp within the current time interval. The step of performing time evolution update on the hidden state based on the time interval specifically includes: At each timestamp, the control signal value at the current timestamp is input into the gated recursive update unit, and the update gate and reset gate are generated with the control signal value and the intermediate state as input. Based on the update gate and reset gate, the hidden state is updated using gating to obtain the hidden state at the current timestamp; For each timestamp, a read mapping is performed on the hidden state. The read mapping is to map the hidden state to the corresponding continuous latent variable, and the read mapping is kept consistent within the corresponding time period of the behavioral video sequence. The continuous latent variables obtained at each timestamp are arranged into a latent variable sequence in chronological order, and a continuous time interpolation is constructed on the latent variable sequence based on the time interval between adjacent timestamps, until the corresponding continuous latent variable can be obtained at any time between the start time and the end time. By arranging continuous latent variables consecutively within the corresponding time period of the behavioral video sequence, a continuous latent trajectory covering the corresponding time period of the behavioral video sequence is formed.

[0009] Optionally, the improved Koopman model is specifically as follows: Obtain the continuous latent variable sequence in the continuous latent trajectory, and assemble the continuous latent variable sequence into a sample sequence in chronological order; The sample sequence is mapped time-by-time using a Koopman encoder. The mapping process involves inputting the continuous latent variables at each time step into the Koopman encoder and outputting a Koopman latent space vector of the same dimension, forming a Koopman embedding sequence arranged in chronological order. The evolution operator solver constructs paired samples of adjacent time steps for the Koopman embedding sequence, forming a set of sample pairs of Koopman latent space vectors of the previous time step and the Koopman latent space vectors of the next time step. The linear mapping matrix is ​​solved with the objective of minimizing the mapping residual in the sample pair set to obtain the initial evolution operator, which is used to linearly map the Koopman latent space vector of the previous time step to the Koopman latent space vector of the next time step. The initial evolution operator is stabilized by the spectral stability constraint module. The stabilization process involves eigenvalue decomposition of the initial evolution operator to obtain a set of eigenvalues ​​and a set of eigenvectors. The modulus of the eigenvalue set that exceeds a preset upper bound is truncated to obtain a restricted eigenvalue set. A stable evolution operator is reconstructed based on the restricted eigenvalue set and eigenvector set; The linear evolution module performs linear evolution on the stable evolution operator. The linear evolution is to perform a linear mapping on the Koopman latent space vector at each time step in the Koopman embedding sequence to obtain a one-step predicted embedding sequence, and to perform a recursive linear mapping on the one-step predicted embedding sequence within a preset number of steps to obtain a multi-step predicted embedding sequence, thereby generating the evolved Koopman embedding sequence. The evolved Koopman embedding sequence is reconstructed by a Koopman decoder. The reconstruction process involves inputting the evolved Koopman embedding sequence into the Koopman decoder time-by-time and outputting reconstructed latent variables to form a reconstructed latent variable sequence. The reconstructed latent variable sequence has the same dimension and representation as the continuous latent variable sequence. The consistency verification module performs consistency calculations on the reconstructed latent variable sequence and the continuous latent variable sequence. The consistency calculation involves calculating the reconstruction deviation index between the reconstructed latent variable sequence and the corresponding continuous latent variable, and calculating the multi-step deviation index between the multi-step predicted latent variable after the multi-step predicted embedding sequence is reconstructed by the Koopman decoder and the corresponding continuous latent variable, thus forming the Koopman latent space trajectory.

[0010] Optionally, step five specifically includes: Arrange the Koopman latent space trajectories in chronological order to form a trajectory sample sequence; Set the number of stages and assign a stage identifier to each sample in the trajectory sample sequence to form an initial stage switching sequence; Based on the initial stage switching sequence, trajectory samples with the same stage identifier are divided into a subset of trajectory samples of the same stage, and a set of paired trajectory samples at adjacent time points is constructed in each stage's subset of trajectory samples. For each stage's set of paired trajectory samples, solve for the linear mapping matrix that can linearly map the trajectory sample from the previous moment to the trajectory sample from the next moment, and obtain the stage evolution operator that corresponds one-to-one with the current stage, forming a set of stage evolution operators. The linear evolution residuals of the trajectory sample sequence at each stage are calculated based on the set of stage evolution operators, and the stage identifier sequence is updated with the linear evolution residuals to obtain the updated stage switching sequence. Repeatedly perform the stage evolution operator solution and stage switching sequence update until the linear evolution residual meets the preset threshold or the stage switching sequence remains consistent in two adjacent iterations, and output the final stage switching sequence and the final stage evolution operator set. The stage boundaries are determined based on the final stage switching sequence, whereby the stage boundaries are the dividing positions between adjacent moments when the stage identifier changes.

[0011] Optionally, step six specifically includes: Based on the stage boundaries, the Koopman latent space trajectory is divided into multiple stage trajectory segments. For each stage trajectory segment, extract the Koopman latent space vector in the current stage trajectory segment in chronological order, use each Koopman latent space vector as a point in the point cloud, and collect all points in the same stage to construct the point cloud representation corresponding to the current stage. A metric space is constructed for the point cloud representation. Euclidean distance is used as the distance metric between points. The Euclidean distance between any two points in the point cloud at the current stage is calculated to obtain the distance matrix. Performing persistent cohomology calculations specifically includes: A filtered complex is constructed based on a distance matrix. The filtered complex is a simple complex that monotonically expands with a distance threshold. Points, edges, and higher-order simplexes are added to the simple complex in order of increasing distance threshold to form a hierarchical complex sequence. Calculate the birth and extinction information of homology groups on the complex sequence to obtain a persistent graph of the point cloud representation at the current stage. The persistent graph contains the birth and extinction times of topological features under different homology dimensions. The persistent graph is converted into a numerical topological signature, and the numerical topological signature is a persistent image; The point cloud construction and persistent cohomology calculation are performed separately for each stage, and the numerical topological signatures of each stage are collected to form a topological signature set for each stage. The topological signature set is used to characterize the topological invariant features of the trajectory structure of the corresponding stage.

[0012] Optionally, step seven specifically includes: For each stage, the Koopman latent space trajectory segment corresponding to the current stage is determined based on the stage boundary, and the final stage evolution operator corresponding to the current stage is parametrically expanded to generate stage operator features. At the same time, the topological signature features are read from the topological signature of the current stage to generate stage feature pairs. The stage features of each stage are concatenated in the order of the stages to form stage sequence features, and the final stage switching sequence is encoded as a switching sequence feature in the form of a discrete identifier sequence. The stage sequence features and the switching sequence features are combined to generate a joint feature vector, which simultaneously contains stage operator features, topological signature features and stage switching information. The joint feature vector is input into the sequence consistency discriminator, which uses a conditional random field to perform global consistency decoding on the stage sequence, outputs the category score of each stage, and determines the behavior category based on the decoding path with the highest category score, thus obtaining the behavior feature recognition result.

[0013] According to an embodiment of the present invention, an AI-driven behavior feature recognition device includes the following modules: The observation set construction module is used to collect behavioral video sequences of target objects, extract the human body key point coordinate sequence and the confidence sequence corresponding to each human body key point, and form a multi-source time series observation set. The skeleton control generation module is used to establish nodes based on human joints and establish edges based on the skeleton topological connection relationship based on the multi-source time series observation set, generate a skeleton graph observation sequence, and construct continuous time interpolation to obtain a continuous time control signal. The latent trajectory generation module is used to read the continuous time control signal in the order of timestamps, combine the adjacent time intervals to continuously evolve and update the hidden state, and map the hidden state to continuous latent variables to obtain continuous latent trajectories; The Koopman trajectory module is used to input the continuous latent trajectory into the improved Koopman model, and obtain the Koopman latent space trajectory through the Koopman encoder, evolution operator solver, spectral stability constraint module, linear evolution module, Koopman decoder and consistency verification module. The stage decomposition module is used to decompose the Koopman latent space trajectory into stages, obtain the final stage switching sequence and stage boundaries, and generate a set of final stage evolution operators that correspond one-to-one with each stage. The topology signature module is used to construct a point cloud representation of the Koopman latent space trajectory and calculate persistent cohomology on the point cloud representation to obtain a set of topology signatures for each stage. The joint discrimination module is used to construct joint features based on the final stage evolution operator set, the final stage switching sequence, the stage boundary and the topological signature set, and obtain the behavioral feature recognition result.

[0014] The beneficial effects of this invention are: This invention constructs a multi-source temporal observation set by extracting human keypoint coordinates and confidence scores from behavioral video sequences. It further generates a skeleton graph observation sequence and a continuous-time interpolation control signal, ensuring that keypoints maintain a continuous and consistent input representation even with unequal sampling intervals, occlusion, and confidence fluctuations. This avoids the sensitivity of traditional discrete frame sequence modeling to frame rate changes and local missing values, thus improving the stability of temporal representation from the source. In the latent trajectory generation stage, a mechanism for continuously evolving and updating hidden states by reading in time-stamp order and combining adjacent time intervals ensures that the hidden state updates are consistent with the actual time intervals. Continuous latent trajectories can cover the entire behavior period and maintain temporal continuity, thereby reducing jitter in behavior recognition output and enhancing the ability to express long-term behavior processes. In the Koopman latent space modeling stage, continuous latent trajectories are mapped to the Koopman latent space by improving the Koopman model and learning evolutionary operators. Combined with spectral stability constraints, the eigenvalue magnitudes of the evolutionary operators are truncated and reconstructed to obtain stable evolutionary operators, ensuring that the linear evolution of the latent space remains bounded in multi-step recursion. This, along with decoding reconstruction and consistency verification, forms a closed-loop constraint, improving the predictability and repeatability of Koopman latent space trajectories. The structure provides a stable and reliable dynamic foundation for subsequent stage decomposition. In the stage decomposition stage, a set of stage evolution operators is constructed from the Koopman latent space trajectory, and dynamic programming is performed using linear evolution residuals, switching penalties, and minimum stage length constraints as criteria to obtain stage switching sequences and stage boundaries. This structures the behavioral process into several stage segments with consistent linear evolution laws, thereby improving the ability to characterize the internal processes of complex behaviors and outputting stage information that can be interpreted and traced. In the topological signature stage, the stage trajectory segments are constructed as point clouds, and persistent cohomology is calculated to obtain a set of topological signatures in the form of persistent images. This allows the recognition process to not only rely on local temporal changes but also utilize the topological invariant features of the trajectory structure, enhancing the robustness of discrimination under conditions of action speed changes, viewpoint differences, and local missing values. Finally, the stage evolution operator features, topological signature features, and stage switching information are jointly constructed, and global consistency decoding is performed through conditional random fields. This ensures that the behavior category discrimination and stage sequence consistency simultaneously satisfy the constraints, outputting behavior feature recognition results that include behavior category, stage switching sequence, and stage boundaries. This achieves a more stable, interpretable, and applicable behavior recognition effect for complex real-world scenarios. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of an AI-driven behavior feature recognition algorithm proposed in this invention; Figure 2This is a schematic diagram of the structure of an AI-driven behavior feature recognition device proposed in this invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0017] refer to Figure 1 An AI-driven behavior feature recognition algorithm includes: Step 1: Collect video sequences of the target object's behavior, extract the coordinate sequence of human body key points and the confidence sequence corresponding to each human body key point, and form a multi-source time series observation set; Step 2: Based on the multi-source time-series observation set, establish nodes using human joints, establish edges using skeletal topological connections, generate a skeleton graph observation sequence, and construct continuous-time interpolation to obtain continuous-time control signals; Step 3: Read the continuous time control signals in the order of timestamps, combine the adjacent time intervals to continuously evolve and update the hidden state, and map the hidden state to continuous latent variables to obtain the continuous potential trajectory; Step 4: Input the continuous latent trajectory into the improved Koopman model, and obtain the Koopman latent space trajectory through the Koopman encoder, evolution operator solver, spectral stability constraint module, linear evolution module, Koopman decoder and consistency verification module; Step 5: Perform stage decomposition on the Koopman latent space trajectory to obtain the final stage switching sequence and stage boundaries, and generate a set of final stage evolution operators corresponding to each stage. Step 6: Construct a point cloud representation of the Koopman latent space trajectory, and calculate persistent cohomology on the point cloud representation to obtain the topological signature set for each stage; Step 7: Construct joint features based on the final stage evolution operator set, final stage switching sequence, stage boundary and topological signature set to obtain behavioral feature recognition results.

[0018] In this embodiment, step one specifically includes: The behavior video sequence of the target object is acquired, and the behavior video sequence is decoded frame by frame to obtain the original image frame sequence arranged in frame order; The original image frame sequence is subjected to pose estimation processing, which involves outputting the coordinates of human key points for each frame and concatenating the coordinates of human key points in time order to form a sequence of human key point coordinates. While outputting the coordinates of human body key points, the system outputs the confidence score corresponding to each human body key point for each frame of the image, and splices the confidence scores of each frame in chronological order to form a confidence score sequence. The sequence of human body key point coordinates and the confidence sequence are written into the acquisition timestamp, and the acquisition timestamp corresponds one-to-one with the corresponding image frame; Based on the acquisition timestamp, the coordinate sequence of human key points and the confidence sequence are time-aligned to obtain the observation sequence sorted by timestamp, and the time interval between adjacent observations is recorded as unequal interval sampling information. The validity screening process is performed on the human body key point coordinates based on the confidence score sequence. The validity screening process is to write missing markers on the human body key point coordinates with confidence scores below a preset threshold, and retain the key point coordinates with no missing points and their corresponding confidence scores. The coordinates, confidence scores, and unequal interval sampling information of human key points that are completed and have missing markers are collected to form a multi-source time-series observation set with unequal interval sampling.

[0019] In this embodiment, the two steps specifically involve: Extract the human body key point coordinate sequence, confidence sequence and missing marker information sorted by timestamp from a multi-source time-series observation set sampled at unequal intervals; Using each human joint in the human keypoint set as a node, edges are established for any two adjacent joints according to the preset human skeleton topology connection relationship. At each time stamp, a corresponding skeleton graph observation is generated, resulting in a skeleton graph observation sequence composed of skeleton graph observations at each time stamp. For each node at each timestamp, a node feature vector is constructed. The node feature vector includes the coordinate components of the joint corresponding to the node, the joint velocity components calculated from the joint coordinate differences of adjacent timestamps, the confidence level corresponding to each node, and the corresponding missing marker. In the case of missing markers, write a missing identifier to the feature vector of the node marked as missing and retain its timestamp position as a missing event; Based on the timestamps corresponding to the feature vectors of each node, piecewise linear interpolation over continuous time is constructed for each dimension of the feature vectors of each node, and the corresponding interpolated features are output at any continuous time point. The continuous time interpolation output of each node is combined according to the node dimension and feature dimension to form a continuous time control signal. The continuous time control signal contains node feature information, confidence level and missing marker information corresponding to the skeleton graph observation at any continuous time point.

[0020] In this embodiment, step three specifically includes: Based on continuous time control signals, the start and end times of the corresponding time period of the behavioral video sequence are determined, and the control signal values ​​at each timestamp are read sequentially according to the order of the timestamps. At the same time, the time interval between any two adjacent reads is recorded. Between any two adjacent timestamps, the hidden state is updated by time evolution according to the time interval until the hidden state continuously evolves from the state corresponding to the previous timestamp to the intermediate state corresponding to the next timestamp within the current time interval. The step of performing time evolution update on the hidden state based on the time interval specifically includes: At each timestamp, the control signal value at the current timestamp is input into the gated recursive update unit, and the update gate and reset gate are generated with the control signal value and the intermediate state as input. Based on the update gate and reset gate, the hidden state is updated using gating to obtain the hidden state at the current timestamp; For each timestamp, a read mapping is performed on the hidden state. The read mapping is to map the hidden state to the corresponding continuous latent variable, and the read mapping is kept consistent within the corresponding time period of the behavioral video sequence. The continuous latent variables obtained at each timestamp are arranged into a latent variable sequence in chronological order, and a continuous time interpolation is constructed on the latent variable sequence based on the time interval between adjacent timestamps, until the corresponding continuous latent variable can be obtained at any time between the start time and the end time. By arranging continuous latent variables consecutively within the corresponding time period of the behavioral video sequence, a continuous latent trajectory covering the corresponding time period of the behavioral video sequence is formed.

[0021] In this embodiment, the improved Koopman model is specifically as follows: Obtain the continuous latent variable sequence in the continuous latent trajectory, and assemble the continuous latent variable sequence into a sample sequence in chronological order; The sample sequence is mapped time-by-time using a Koopman encoder. The mapping process involves inputting the continuous latent variables at each time step into the Koopman encoder and outputting a Koopman latent space vector of the same dimension, forming a Koopman embedding sequence arranged in chronological order. The evolution operator solver constructs paired samples of adjacent time steps for the Koopman embedding sequence, forming a set of sample pairs of Koopman latent space vectors of the previous time step and the Koopman latent space vectors of the next time step. The linear mapping matrix is ​​solved with the objective of minimizing the mapping residual in the sample pair set to obtain the initial evolution operator, which is used to linearly map the Koopman latent space vector of the previous time step to the Koopman latent space vector of the next time step. The initial evolution operator is stabilized by the spectral stability constraint module. The stabilization process involves eigenvalue decomposition of the initial evolution operator to obtain a set of eigenvalues ​​and a set of eigenvectors. The modulus of the eigenvalue set that exceeds a preset upper bound is truncated to obtain a restricted eigenvalue set. A stable evolution operator is reconstructed based on the restricted eigenvalue set and eigenvector set; The linear evolution module performs linear evolution on the stable evolution operator. The linear evolution is to perform a linear mapping on the Koopman latent space vector at each time step in the Koopman embedding sequence to obtain a one-step predicted embedding sequence, and to perform a recursive linear mapping on the one-step predicted embedding sequence within a preset number of steps to obtain a multi-step predicted embedding sequence, thereby generating the evolved Koopman embedding sequence. The evolved Koopman embedding sequence is reconstructed by a Koopman decoder. The reconstruction process involves inputting the evolved Koopman embedding sequence into the Koopman decoder time-by-time and outputting reconstructed latent variables to form a reconstructed latent variable sequence. The reconstructed latent variable sequence has the same dimension and representation as the continuous latent variable sequence. The consistency verification module performs consistency calculations on the reconstructed latent variable sequence and the continuous latent variable sequence. The consistency calculation involves calculating the reconstruction deviation index between the reconstructed latent variable sequence and the corresponding continuous latent variable, and calculating the multi-step deviation index between the multi-step predicted latent variable after the multi-step predicted embedding sequence is reconstructed by the Koopman decoder and the corresponding continuous latent variable, thus forming the Koopman latent space trajectory.

[0022] The improved Koopman model proposed in this step is similar to the traditional Koopman model in that both follow the basic idea that nonlinear systems can be described by linear operators in a suitable observation space. Both encode and map the original latent variable sequence to the Koopman latent space to form a Koopman embedding sequence. By learning linear evolution operators, the Koopman latent space vector of the previous time step can be linearly mapped to the Koopman latent space vector of the next time step. At the same time, both contain the basic closed loop of "linear evolution of latent space - reconstruction back to the original latent variable space", that is, the sequence is predicted to evolve in the latent space, and the corresponding latent variable representation is obtained by decoding and reconstruction, which is used to characterize the dynamic changes of the sequence over time and serve the subsequent recognition task.

[0023] The difference lies in that the improved Koopman model, based on the traditional Koopman model's learning of linear evolution operators, adds a spectral stability constraint module, a multi-step recursive mechanism for linear evolution, and a consistency verification mechanism. This explicitly limits the stability of the linear evolution operators and the reliability of multi-step predictions. Specifically, the traditional Koopman model typically uses the obtained evolution operators directly for single-step or finite-step linear progression, lacking stabilization processing for the eigenvalue magnitudes of the evolution operators. This makes it prone to latent space vector divergence or drift during multi-step evolution. In contrast, this improved scheme performs eigenvalue decomposition on the initial evolution operators and truncates eigenvalues ​​whose magnitudes exceed a preset upper bound. Then, based on the constrained eigenvalues ​​and eigenvectors, a stable evolution operator is reconstructed to keep the evolution process bounded. At the same time, this improved scheme explicitly generates one-step prediction embedding sequences and multi-step prediction embedding sequences to form the evolved Koopman embedding sequence. The reconstructed latent variable sequence is then obtained through the Koopman decoder, and the reconstruction deviation index and multi-step deviation index are calculated by the consistency verification module. The reconstruction consistency and multi-step prediction consistency are incorporated into the constraints and training objectives, forming a closed-loop process of "encoding - operator stabilization - multi-step evolution - decoding - consistency verification". This is different from the traditional Koopman model, which often uses training methods that mainly focus on mapping residuals or reconstruction errors.

[0024] The beneficial effects of the improvements are as follows: By using spectral stability constraints to upper-bound the eigenvalue magnitude of the evolution operator and reconstructing a stable evolution operator, the linear evolution of the Koopman latent space remains bounded during multi-step recursion, thereby suppressing the problems of latent space trajectory divergence, drift, or numerical instability over time, and improving the predictability and repeatability of the Koopman latent space trajectory; by explicitly constructing the embedding sequences of one-step prediction and multi-step prediction and performing recursive linear evolution, the model not only fits the local dynamic relationship between adjacent time points, but also constrains the evolution consistency over a longer time span, thereby enhancing the ability to dynamically characterize long-term behavioral processes; by using a consistency verification module to simultaneously calculate the reconstruction bias index and the multi-step bias index, and taking "decoding and reconstruction close to the original latent variable" and "multi-step prediction close to the evolution of the real latent variable" as joint constraints, the encoding mapping, evolution operator, and decoding and reconstruction mutually verify each other and converge together during training, reducing overfitting and error accumulation caused by relying solely on single-step fitting, and ultimately providing a more stable and reliable foundation for the Koopman latent space trajectory for subsequent stage decomposition and behavior discrimination.

[0025] In this embodiment, step five specifically includes: Arrange the Koopman latent space trajectories in chronological order to form a trajectory sample sequence; Set the number of stages and assign a stage identifier to each sample in the trajectory sample sequence to form an initial stage switching sequence; Based on the initial stage switching sequence, trajectory samples with the same stage identifier are divided into a subset of trajectory samples of the same stage, and a set of paired trajectory samples at adjacent time points is constructed in each stage's subset of trajectory samples. For each stage's set of paired trajectory samples, solve for the linear mapping matrix that can linearly map the trajectory sample from the previous moment to the trajectory sample from the next moment, and obtain the stage evolution operator that corresponds one-to-one with the current stage, forming a set of stage evolution operators. The linear evolution residuals of the trajectory sample sequence at each stage are calculated based on the set of stage evolution operators, and the stage identifier sequence is updated using the linear evolution residuals to obtain the updated stage switching sequence, which specifically includes: For each pair of adjacent trajectory samples in the trajectory sample sequence, substitute each stage evolution operator and calculate the linear evolution residual of the current adjacent trajectory sample pair under the current stage evolution operator; The linear evolution residual is used as the cost of assigning adjacent trajectory sample pairs to the current stage, forming a cost table for adjacent sample pairs to a stage. For any two adjacent stage identifiers, if the two stage identifiers are the same, the transition cost is zero. If the two phase identifiers are different, the transfer cost is the preset switching penalty value; The total transfer cost, including the switching penalty, is obtained by adding the switching penalty value to the cost of the corresponding adjacent trajectory sample pair. Starting from the initial sample, dynamic programming is used to calculate the minimum cumulative cost when reaching the current trajectory sample and being in any stage, and the previous stage identifier that minimizes the cumulative cost is recorded. After completing the full sequence traversal, the stage identifier with the minimum cumulative cost at the terminating sample is selected as the endpoint, and a new stage identifier sequence is obtained by backtracking according to the previous stage identifier. The new stage identifier sequence is the updated stage switching sequence. In the dynamic programming process, if the duration of a stage does not reach the preset minimum length, stage switching is not allowed, so that the updated stage switching sequence satisfies the stage continuity constraint. Repeatedly perform the stage evolution operator solution and stage switching sequence update until the linear evolution residual meets the preset threshold or the stage switching sequence remains consistent in two adjacent iterations, and output the final stage switching sequence and the final stage evolution operator set. The stage boundaries are determined based on the final stage switching sequence, whereby the stage boundaries are the dividing positions between adjacent moments when the stage identifier changes.

[0026] In this embodiment, step six specifically includes: Based on the stage boundaries, the Koopman latent space trajectory is divided into multiple stage trajectory segments. For each stage trajectory segment, extract the Koopman latent space vector in the current stage trajectory segment in chronological order, use each Koopman latent space vector as a point in the point cloud, and collect all points in the same stage to construct the point cloud representation corresponding to the current stage. A metric space is constructed for the point cloud representation. Euclidean distance is used as the distance metric between points. The Euclidean distance between any two points in the point cloud at the current stage is calculated to obtain the distance matrix. Perform persistent cohomology calculations, which specifically include: A filtered complex is constructed based on a distance matrix. The filtered complex is a simple complex that monotonically expands with a distance threshold. Points, edges, and higher-order simplexes are added to the simple complex in order of increasing distance threshold to form a hierarchical complex sequence. Calculate the birth and extinction information of homology groups on the complex sequence to obtain a persistent graph of the point cloud representation at the current stage. The persistent graph contains the birth and extinction times of topological features under different homology dimensions. The persistent graph is converted into a numerical topological signature, and the numerical topological signature is a persistent image; The point cloud construction and persistent cohomology calculation are performed separately for each stage, and the numerical topological signatures of each stage are collected to form a topological signature set for each stage. The topological signature set is used to characterize the topological invariant features of the trajectory structure of the corresponding stage.

[0027] In this embodiment, step seven specifically includes: For each stage, the Koopman latent space trajectory segment corresponding to the current stage is determined based on the stage boundary, and the final stage evolution operator corresponding to the current stage is parametrically expanded to generate stage operator features. At the same time, the topological signature features are read from the topological signature of the current stage to generate stage feature pairs. The stage features of each stage are concatenated in the order of the stages to form stage sequence features, and the final stage switching sequence is encoded as a switching sequence feature in the form of a discrete identifier sequence. The stage sequence features and the switching sequence features are combined to generate a joint feature vector, which simultaneously contains stage operator features, topological signature features and stage switching information. The joint feature vector is input into the sequence consistency discriminator, which uses a conditional random field to perform global consistency decoding on the stage sequence, outputs the category score of each stage, and determines the behavior category based on the decoding path with the highest category score, thus obtaining the behavior feature recognition result.

[0028] refer to Figure 2 An AI-driven behavior feature recognition device includes the following modules: The observation set construction module is used to collect behavioral video sequences of target objects, extract the human body key point coordinate sequence and the confidence sequence corresponding to each human body key point, and form a multi-source time series observation set. The skeleton control generation module is used to establish nodes based on human joints and establish edges based on the skeleton topological connection relationship based on the multi-source time series observation set, generate a skeleton graph observation sequence, and construct continuous time interpolation to obtain a continuous time control signal. The latent trajectory generation module is used to read the continuous time control signal in the order of timestamps, combine the adjacent time intervals to continuously evolve and update the hidden state, and map the hidden state to continuous latent variables to obtain continuous latent trajectories; The Koopman trajectory module is used to input the continuous latent trajectory into the improved Koopman model, and obtain the Koopman latent space trajectory through the Koopman encoder, evolution operator solver, spectral stability constraint module, linear evolution module, Koopman decoder and consistency verification module. The stage decomposition module is used to decompose the Koopman latent space trajectory into stages, obtain the final stage switching sequence and stage boundaries, and generate a set of final stage evolution operators that correspond one-to-one with each stage. The topology signature module is used to construct a point cloud representation of the Koopman latent space trajectory and calculate persistent cohomology on the point cloud representation to obtain a set of topology signatures for each stage. The joint discrimination module is used to construct joint features based on the final stage evolution operator set, the final stage switching sequence, the stage boundary and the topological signature set, and obtain the behavioral feature recognition result.

[0029] Example 1: To verify the feasibility of this invention in practice, it was applied to the packaging and loading / unloading area of ​​a continuous chemical production unit. Personnel need to perform actions such as bag removal, sealing, labeling, palletizing, and transfer between conveyor belts, palletizing stations, and forklift lanes. The lighting in this area varies significantly between day and night shifts, and personnel are frequently partially obstructed by pillars, stacked materials, or vehicles. The cameras are installed at high angles with fixed viewing angles, and there are also frame rate fluctuations and frame drops due to network jitter. Management aims to reliably identify behaviors such as "standard handling," "illegally crossing warning lines," "dangerously bending and lifting," "transferring without proper sealing procedures," and "tipping risks due to palletizing and pushing / pulling," and to provide behavioral stage boundaries in the identification results for easy review and traceability. Existing solutions mostly rely on skeleton sequence classification with a fixed frame rate. When encountering unequal sampling intervals, fluctuations in key point confidence, and occlusion / missing data, the identification results are prone to jitter. Long-term behaviors are often fragmented into multiple short segments, and it is difficult to output usable stage boundaries, leading to increased false alarms, unexplained missed alarms, and high review costs.

[0030] In this scenario, after the camera video stream is received, pose estimation is performed on each frame, outputting the coordinates of human keypoints and simultaneously outputting the confidence level corresponding to each keypoint. All frames are written with acquisition timestamps and aligned according to the timestamps, forming a multi-source temporal observation set with unequal sampling intervals. Due to frequent occlusion in the scene, the confidence level of some keypoints drops significantly in certain frames. The system writes missing keypoints with confidence levels below a threshold as missing markers and retains their timestamp positions, allowing missing events to be explicitly perceived in subsequent modeling. Subsequently, based on the keypoint set, nodes are established using human joints, and edges are established using skeletal topological connections. A skeleton graph observation sequence is generated time-stamped, and the node features simultaneously include joint coordinates, joint velocities obtained from the difference between adjacent timestamps, confidence levels, and missing markers. To cope with frame rate fluctuations, the system constructs piecewise linear interpolation of continuous time for each node feature according to the timestamp, thereby obtaining the skeleton at any continuous time point. Figure 1The interpolation characteristics are used to form a continuous time control signal. This continuous time control signal is read in time stamp order. The system records the time interval between any two adjacent reads. Between two adjacent timestamps, the hidden state is continuously evolved and updated according to the time interval. After reaching the current timestamp, the control signal value is input into the gated recursive update unit to generate update gate and reset gate and update the hidden state. Then, the hidden state is mapped to obtain continuous latent variables. The continuous latent variables are arranged in time order to form a latent variable sequence, and combined with adjacent time intervals to construct continuous time interpolation, thereby obtaining a continuous potential trajectory covering the entire operation period. The continuous latent trajectory is input into the improved Koopman model, and the Koopman encoder maps it to obtain the Koopman embedding sequence. The evolution operator solver solves the linear mapping matrix based on adjacent time sample pairs to obtain the initial evolution operator. The spectral stability constraint performs eigenvalue decomposition on the initial evolution operator and truncates eigenvalues ​​whose modulus exceeds the upper bound, reconstructing to obtain the stable evolution operator. The linear evolution module generates a one-step predicted embedding sequence under the stable evolution operator and recursively obtains a multi-step predicted embedding sequence, thus forming the evolved Koopman embedding sequence. The Koopman decoder reconstructs the evolved Koopman embedding sequence into a reconstructed latent variable sequence. The consistency check module calculates the reconstruction deviation index between the reconstructed latent variable sequence and the corresponding continuous latent variable, and calculates the multi-step deviation index between the multi-step predicted latent variable after decoding and reconstruction of the multi-step predicted embedding sequence and the corresponding continuous latent variable, finally obtaining the Koopman latent space trajectory. Based on the Koopman latent space trajectory, the system sets the number of stages and initializes stage identifiers to form an initial stage switching sequence. Linear evolution residuals are calculated using a set of stage evolution operators. Combining switching penalties and minimum stage length constraints, dynamic programming is used to update the stage identifier sequence and iterate until convergence. The final stage switching sequence and stage boundaries are output, along with a set of final stage evolution operators corresponding to each stage. To enhance robustness to changes in motion speed, occlusion loss, and viewpoint differences, the system divides the Koopman latent space trajectory into stage trajectory segments based on stage boundaries. The Koopman latent space vectors in each stage segment are extracted in chronological order and aggregated into a point cloud. The Euclidean distance between any two points in the point cloud is calculated to obtain a distance matrix. Based on the distance matrix, a filtered complex that monotonically expands with a distance threshold is constructed, and the birth and death information of homology groups are calculated to obtain a persistent graph. The persistent graph is then converted into a persistent image as the topological signature for that stage, and the resulting set of topological signatures is aggregated.During joint discrimination, the system performs parametric expansion of the final stage evolution operator for each stage to generate stage operator features, reads the topological signature features from the topological signature and forms stage feature pairs, concatenates them in stage order to obtain stage sequence features, and encodes the final stage switching sequence as switching sequence features. Then, the stage sequence features and switching sequence features are combined to form a joint feature vector, which is input into the sequence consistency discriminator for global consistency decoding, outputting the behavior category. At the same time, the stage switching sequence and stage boundaries are retained as structured results, allowing the on-site verification system to directly locate "when the action occurred, which stages it went through, and where the boundaries are."

[0031] To verify the beneficial effects, video was continuously collected from multiple shifts in the same packaging and loading / unloading area, and operation segments containing typical occlusions, different individual body types, and different handling speeds were extracted. The segments covered categories such as standardized handling, dangerous bending and lifting, crossing warning lines, transfer with missing seals, and unstable stacking due to palletizing and pushing. In actual operation, the frame rate of the video was not constant, and the observed effective frame rate fluctuated within a certain range. Network jitter caused short-term frame drops, and the confidence of the pose estimation key points fluctuated, especially when personnel and forklifts met, materials were obscured, or personnel turned away, the proportion of missing upper limb and torso key points increased significantly. Three commonly used baselines were selected for comparison: one is a discrete temporal method that resamples the key point sequence to a fixed frame rate and then inputs it into a gated recursive unit for direct classification; another is a discrete temporal method that inputs the key point sequence of a fixed-length window into a temporal convolutional structure for classification; and the third is a skeleton graph method that inputs the skeleton graph sequence into a spatial-temporal convolutional structure to output categories. For comparison, the same pose estimation output, training set size, and category definition were used uniformly. Evaluation metrics included overall accuracy, macro-average F1 score, F1 score for occluded segments, average stage boundary deviation, stage switching jitter rate, end-to-end average latency, and false positive and false negative rates. Stage boundary deviation was obtained by statistically analyzing the deviation between the stage start and end points obtained through manual verification and the algorithm output boundary. Stage switching jitter rate was obtained by statistically analyzing the frequency of changes in the output stage identifier within the same real stage. Occluded segments were defined as segments with a proportion of missing keypoint markers exceeding a threshold.

[0032] Table 1. Comparison of Behavior Recognition Performance under Unequal Interval Sampling and Occlusion Conditions As shown in Table 1, the method of this invention achieves higher recognition levels in terms of overall accuracy and macro-average F1 score under the same data sources and category definitions, especially with a significant improvement in F1 score for occluded segments. This indicates that by forming a multi-source temporal observation set using keypoint confidence and missing markers and constructing a continuous time control signal, the input representation can remain consistent even when keypoints are unstable. The continuous evolution and update of the hidden state combined with adjacent time intervals prevents the temporal distortion caused by simple resampling from masking unequal interval sampling, thereby reducing misjudgments by discrete methods under frame rate fluctuations. The decrease in the average deviation of stage boundaries and the stage switching jitter rate is consistent with the ability of this invention to output stage switching sequences and stage boundaries. Stage decomposition utilizes a set of stage evolution operators and dynamic programming to update the stage identifier sequence, making stage division more stable. The reduced fragmentation rate reflects that long-term behavior is less prone to fragmentation under the support of continuous latent trajectories and Koopman latent space trajectories. The reduction in multi-step prediction and reconstruction bias aligns with the stabilization of evolution operators by spectral stability constraints. The stable evolution operator remains bounded during multi-step recursion, allowing the evolved Koopman embedding sequence to be decoded and reconstructed, and constrained to the original latent variable distribution through consistency checks. This provides a more reliable trajectory basis for stage decomposition and joint discrimination. For categories requiring understanding of behavioral processes rather than instantaneous postures, such as "missing seals leading to transfer" and "unstable stacking push-pull," the F1 score improvement of this invention is more significant, reflecting the characterization effect of stage boundaries and topological signatures on behavioral structures. The topological signature, derived from persistent cohomology calculations of stage trajectory point clouds, maintains a stable representation of the trajectory structure even under changes in action speed or local missing information. After combining stage operator features and stage switching information, a sequence consistency discriminator performs global consistency decoding, making the output category more consistent with the stage sequence overall, thus reducing false positive and false negative rates. The end-to-end average latency is on the same order of magnitude as the comparison method, indicating that while meeting the requirements of real-time on-site inspection and post-event verification, the method of this invention has achieved comprehensive improvements in terms of stability, interpretable structured output capability, and robustness in complex scenarios, which are more in line with engineering needs.

[0033] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An AI-driven behavior feature recognition algorithm, characterized in that, include: Step 1: Collect video sequences of the target object's behavior, extract the coordinate sequence of human body key points and the confidence sequence corresponding to each human body key point, and form a multi-source time series observation set; Step 2: Based on the multi-source time-series observation set, establish nodes using human joints, establish edges using skeletal topological connections, generate a skeleton graph observation sequence, and construct continuous-time interpolation to obtain continuous-time control signals; Step 3: Read the continuous time control signals in the order of timestamps, combine the adjacent time intervals to continuously evolve and update the hidden state, and map the hidden state to continuous latent variables to obtain the continuous potential trajectory; Step 4: Input the continuous latent trajectory into the improved Koopman model, and obtain the Koopman latent space trajectory through the Koopman encoder, evolution operator solver, spectral stability constraint module, linear evolution module, Koopman decoder and consistency verification module; Step 5: Perform stage decomposition on the Koopman latent space trajectory to obtain the final stage switching sequence and stage boundaries, and generate a set of final stage evolution operators corresponding to each stage. Step 6: Construct a point cloud representation of the Koopman latent space trajectory, and calculate persistent cohomology on the point cloud representation to obtain the topological signature set for each stage; Step 7: Construct joint features based on the final stage evolution operator set, final stage switching sequence, stage boundary and topological signature set to obtain behavioral feature recognition results.

2. The AI-driven behavior feature recognition algorithm according to claim 1, characterized in that, Step one specifically involves: The behavior video sequence of the target object is acquired, and the behavior video sequence is decoded frame by frame to obtain the original image frame sequence arranged in frame order; The original image frame sequence is subjected to pose estimation processing, which involves outputting the coordinates of human key points for each frame and concatenating the coordinates of human key points in time order to form a sequence of human key point coordinates. While outputting the coordinates of human body key points, the system outputs the confidence score corresponding to each human body key point for each frame of the image, and splices the confidence scores of each frame in chronological order to form a confidence score sequence. The sequence of human body key point coordinates and the confidence sequence are written into the acquisition timestamp, and the acquisition timestamp corresponds one-to-one with the corresponding image frame; Based on the acquisition timestamp, the coordinate sequence of human key points and the confidence sequence are time-aligned to obtain the observation sequence sorted by timestamp, and the time interval between adjacent observations is recorded as unequal interval sampling information. The validity screening process is performed on the human body key point coordinates based on the confidence score sequence. The validity screening process is to write missing markers on the human body key point coordinates with confidence scores below a preset threshold, and retain the key point coordinates with no missing points and their corresponding confidence scores. The coordinates, confidence scores, and unequal interval sampling information of human key points that are completed and have missing markers are collected to form a multi-source time-series observation set with unequal interval sampling.

3. The AI-driven behavior feature recognition algorithm according to claim 1, characterized in that, The specific steps for the two individuals are as follows: Extract the human body key point coordinate sequence, confidence sequence and missing marker information sorted by timestamp from a multi-source time-series observation set sampled at unequal intervals; Using each human joint in the human keypoint set as a node, edges are established for any two adjacent joints according to the preset human skeleton topology connection relationship. At each time stamp, a corresponding skeleton graph observation is generated, resulting in a skeleton graph observation sequence composed of skeleton graph observations at each time stamp. For each node at each timestamp, a node feature vector is constructed. The node feature vector includes the coordinate components of the joint corresponding to the node, the joint velocity components calculated from the joint coordinate differences of adjacent timestamps, the confidence level corresponding to each node, and the corresponding missing marker. In the case of missing markers, write a missing identifier to the feature vector of the node marked as missing and retain its timestamp position as a missing event; Based on the timestamps corresponding to the feature vectors of each node, piecewise linear interpolation over continuous time is constructed for each dimension of the feature vectors of each node, and the corresponding interpolated features are output at any continuous time point. The continuous time interpolation output of each node is combined according to the node dimension and feature dimension to form a continuous time control signal. The continuous time control signal contains node feature information, confidence level and missing marker information corresponding to the skeleton graph observation at any continuous time point.

4. The AI-driven behavior feature recognition algorithm according to claim 1, characterized in that, Step three specifically involves: Based on continuous time control signals, the start and end times of the corresponding time period of the behavioral video sequence are determined, and the control signal values ​​at each timestamp are read sequentially according to the order of the timestamps. At the same time, the time interval between any two adjacent reads is recorded. Between any two adjacent timestamps, the hidden state is updated by time evolution according to the time interval until the hidden state continuously evolves from the state corresponding to the previous timestamp to the intermediate state corresponding to the next timestamp within the current time interval. The step of performing time evolution update on the hidden state based on the time interval specifically includes: At each timestamp, the control signal value at the current timestamp is input into the gated recursive update unit, and the update gate and reset gate are generated with the control signal value and the intermediate state as input. Based on the update gate and reset gate, the hidden state is updated using gating to obtain the hidden state at the current timestamp; For each timestamp, a read mapping is performed on the hidden state. The read mapping is to map the hidden state to the corresponding continuous latent variable, and the read mapping is kept consistent within the corresponding time period of the behavioral video sequence. The continuous latent variables obtained at each timestamp are arranged into a latent variable sequence in chronological order, and a continuous time interpolation is constructed on the latent variable sequence based on the time interval between adjacent timestamps, until the corresponding continuous latent variable can be obtained at any time between the start time and the end time. By arranging continuous latent variables consecutively within the corresponding time period of the behavioral video sequence, a continuous latent trajectory covering the corresponding time period of the behavioral video sequence is formed.

5. The AI-driven behavior feature recognition algorithm according to claim 1, characterized in that, The improved Koopman model is specifically as follows: Obtain the continuous latent variable sequence in the continuous latent trajectory, and assemble the continuous latent variable sequence into a sample sequence in chronological order; The sample sequence is mapped time-by-time using a Koopman encoder. The mapping process involves inputting the continuous latent variables at each time step into the Koopman encoder and outputting a Koopman latent space vector of the same dimension, forming a Koopman embedding sequence arranged in chronological order. The evolution operator solver constructs paired samples of adjacent time steps for the Koopman embedding sequence, forming a set of sample pairs of Koopman latent space vectors of the previous time step and the Koopman latent space vectors of the next time step. The linear mapping matrix is ​​solved with the objective of minimizing the mapping residual in the sample pair set to obtain the initial evolution operator, which is used to linearly map the Koopman latent space vector of the previous time step to the Koopman latent space vector of the next time step. The initial evolution operator is stabilized by the spectral stability constraint module. The stabilization process involves eigenvalue decomposition of the initial evolution operator to obtain a set of eigenvalues ​​and a set of eigenvectors. The modulus of the eigenvalue set that exceeds a preset upper bound is truncated to obtain a restricted eigenvalue set. A stable evolution operator is reconstructed based on the restricted eigenvalue set and eigenvector set; The linear evolution module performs linear evolution on the stable evolution operator. The linear evolution is to perform a linear mapping on the Koopman latent space vector at each time step in the Koopman embedding sequence to obtain a one-step predicted embedding sequence, and to perform a recursive linear mapping on the one-step predicted embedding sequence within a preset number of steps to obtain a multi-step predicted embedding sequence, thereby generating the evolved Koopman embedding sequence. The evolved Koopman embedding sequence is reconstructed by a Koopman decoder. The reconstruction process involves inputting the evolved Koopman embedding sequence into the Koopman decoder time-by-time and outputting reconstructed latent variables to form a reconstructed latent variable sequence. The reconstructed latent variable sequence has the same dimension and representation as the continuous latent variable sequence. The consistency verification module performs consistency calculations on the reconstructed latent variable sequence and the continuous latent variable sequence. The consistency calculation involves calculating the reconstruction deviation index between the reconstructed latent variable sequence and the corresponding continuous latent variable, and calculating the multi-step deviation index between the multi-step predicted latent variable after the multi-step predicted embedding sequence is reconstructed by the Koopman decoder and the corresponding continuous latent variable, thus forming the Koopman latent space trajectory.

6. The AI-driven behavior feature recognition algorithm according to claim 1, characterized in that, Step five specifically involves: Arrange the Koopman latent space trajectories in chronological order to form a trajectory sample sequence; Set the number of stages and assign a stage identifier to each sample in the trajectory sample sequence to form an initial stage switching sequence; Based on the initial stage switching sequence, trajectory samples with the same stage identifier are divided into a subset of trajectory samples of the same stage, and a set of paired trajectory samples at adjacent time points is constructed in each stage's subset of trajectory samples. For each stage's set of paired trajectory samples, solve for the linear mapping matrix that can linearly map the trajectory sample from the previous moment to the trajectory sample from the next moment, and obtain the stage evolution operator that corresponds one-to-one with the current stage, forming a set of stage evolution operators. The linear evolution residuals of the trajectory sample sequence at each stage are calculated based on the set of stage evolution operators, and the stage identifier sequence is updated with the linear evolution residuals to obtain the updated stage switching sequence. Repeatedly perform the stage evolution operator solution and stage switching sequence update until the linear evolution residual meets the preset threshold or the stage switching sequence remains consistent in two adjacent iterations, and output the final stage switching sequence and the final stage evolution operator set; The stage boundaries are determined based on the final stage switching sequence, whereby the stage boundaries are the dividing positions between adjacent moments when the stage identifier changes.

7. The AI-driven behavior feature recognition algorithm according to claim 1, characterized in that, Step six specifically involves: Based on the stage boundaries, the Koopman latent space trajectory is divided into multiple stage trajectory segments. For each stage trajectory segment, extract the Koopman latent space vector in the current stage trajectory segment in chronological order, use each Koopman latent space vector as a point in the point cloud, and collect all points in the same stage to construct the point cloud representation corresponding to the current stage. A metric space is constructed for the point cloud representation. Euclidean distance is used as the distance metric between points. The Euclidean distance between any two points in the point cloud at the current stage is calculated to obtain the distance matrix. Performing persistent cohomology calculations specifically includes: A filtered complex is constructed based on a distance matrix. The filtered complex is a simple complex that monotonically expands with a distance threshold. Points, edges, and higher-order simplexes are added to the simple complex in order of increasing distance threshold to form a hierarchical complex sequence. Calculate the birth and extinction information of homology groups on the complex sequence to obtain a persistent graph of the point cloud representation at the current stage. The persistent graph contains the birth and extinction times of topological features under different homology dimensions. The persistent graph is converted into a numerical topological signature, and the numerical topological signature is a persistent image; The point cloud construction and persistent cohomology calculation are performed on all stages respectively, and the numerical topological signatures of each stage are collected to form a topological signature set for each stage. The topological signature set is used to characterize the topological invariant features of the trajectory structure of the corresponding stage.

8. The AI-driven behavior feature recognition algorithm according to claim 1, characterized in that, Step seven specifically involves: For each stage, the Koopman latent space trajectory segment corresponding to the current stage is determined based on the stage boundary, and the final stage evolution operator corresponding to the current stage is parametrically expanded to generate stage operator features. At the same time, the topological signature features are read from the topological signature of the current stage to generate stage feature pairs. The stage features of each stage are concatenated in the order of the stages to form stage sequence features, and the final stage switching sequence is encoded as a switching sequence feature in the form of a discrete identifier sequence. The stage sequence features and the switching sequence features are combined to generate a joint feature vector, which simultaneously contains stage operator features, topological signature features and stage switching information. The joint feature vector is input into the sequence consistency discriminator, which uses a conditional random field to perform global consistency decoding on the stage sequence, outputs the category score of each stage, and determines the behavior category based on the decoding path with the highest category score, thus obtaining the behavior feature recognition result.

9. An AI-driven behavior feature recognition device, executing the AI-driven behavior feature recognition algorithm according to any one of claims 1 to 8, characterized in that, Includes the following modules: The observation set construction module is used to collect behavioral video sequences of target objects, extract the human body key point coordinate sequence and the confidence sequence corresponding to each human body key point, and form a multi-source time series observation set. The skeleton control generation module is used to establish nodes based on human joints and establish edges based on the skeleton topological connection relationship based on the multi-source time series observation set, generate a skeleton graph observation sequence, and construct continuous time interpolation to obtain a continuous time control signal. The latent trajectory generation module is used to read the continuous time control signal in the order of timestamps, combine the adjacent time intervals to continuously evolve and update the hidden state, and map the hidden state to continuous latent variables to obtain continuous latent trajectories; The Koopman trajectory module is used to input the continuous latent trajectory into the improved Koopman model, and obtain the Koopman latent space trajectory through the Koopman encoder, evolution operator solver, spectral stability constraint module, linear evolution module, Koopman decoder and consistency verification module. The stage decomposition module is used to decompose the Koopman latent space trajectory into stages, obtain the final stage switching sequence and stage boundaries, and generate a set of final stage evolution operators that correspond one-to-one with each stage. The topology signature module is used to construct a point cloud representation of the Koopman latent space trajectory and calculate persistent cohomology on the point cloud representation to obtain a set of topology signatures for each stage. The joint discrimination module is used to construct joint features based on the final stage evolution operator set, the final stage switching sequence, the stage boundary and the topological signature set, and obtain the behavioral feature recognition result.

Images