Behavior risk perception detection method based on stream matching, electronic equipment and medium

Abstract

The invention provides a behavior risk perception detection method based on stream matching, electronic equipment and a medium, and belongs to the technical field of human body action detection. The method comprises the following steps: acquiring a to-be-detected human body action sequence; an abnormal score of the to-be-detected human body action sequence is calculated through a pre-trained vector field network, and the pre-trained vector field network is used for simulating a continuous time probability flow from prior distribution to normal action data distribution; and according to a comparison result of the abnormal score and a preset safety threshold value, determining whether an abnormal human body action exists or not. According to the invention, on the premise of ensuring the real-time processing capability, accurate sensing detection of the abnormal behavior risk of the human body action can be efficiently and robustly realized.

Description

Technical Field

[0001] This application relates to the field of human motion detection technology, and in particular to a behavior risk perception detection method, electronic device and medium based on flow matching. Background Technology

[0002] With the widespread application of video surveillance technology in the field of security protection, real-time detection of abnormal human movements and postures has become one of the core functions of intelligent security protection systems. However, human movements are highly diverse and complex, and the boundary between normal and abnormal behavior is often blurred. In addition, practical application scenarios place strict requirements on the real-time processing capabilities of the system, making the achievement of high-precision detection under limited computing resources a technical challenge that urgently needs to be solved.

[0003] To address the aforementioned issues, existing technical solutions mainly include statistical methods based on traditional machine learning, discriminative models based on deep neural networks, and probabilistic modeling methods based on generative models. Early statistical methods extracted manually designed action features and used support vector machines or hidden Markov models for classification. However, these methods rely on manual feature engineering, making it difficult to capture the high-dimensional semantic information of complex actions and exhibiting weak generalization ability. With the development of deep learning, discriminative models based on convolutional neural networks and recurrent neural networks have gradually become mainstream. These models extract features from videos or skeleton sequences for classification through end-to-end learning. However, these methods typically employ a supervised learning paradigm, requiring a large number of labeled anomalous samples for training. In practical applications, anomalous behavior samples are scarce and diverse, leading to a high risk of model overfitting. To address the issue of insufficient labeled data, researchers have begun exploring unsupervised or semi-supervised anomaly detection methods based on generative models. Among these, approaches based on Normalizing Flows (a powerful probabilistic modeling method) have received widespread attention. This approach maps complex data distributions to simpler basic distributions through invertible transformations, thereby achieving explicit modeling of the normal data distribution. During the inference phase, anomalies are identified by calculating the log-likelihood of test samples. The advantage of this method is that it only requires training on normal samples, eliminating the need for labeled anomaly samples, and theoretically offers better generalization ability.

[0004] However, existing human motion anomaly detection schemes based on Normalizing Flows have revealed several shortcomings in practical applications: First, there is the issue of computational complexity. Both training and inference processes require calculating the Jacobian determinant. For high-dimensional motion sequence data, the dimension of the Jacobian matrix increases quadratically with the input dimension, making determinant calculation a severe performance bottleneck. Although computation can be simplified by designing special network structures, such structural constraints limit the model's expressive power and make it difficult to fully capture the complex spatiotemporal dependencies of human motion. Second, optimization is difficult. Normalizing Flows requires the construction of accurate bijective mappings, which severely restricts network design. During training, parameters need to be carefully adjusted to maintain invertibility, making optimization difficult and prone to numerical instability. Third, temporal modeling capabilities are insufficient. Human motion is a dynamic process; motion sequences not only contain spatial structural information for each frame but also contain temporal evolution patterns between frames. Existing schemes typically flatten the entire sequence into a high-dimensional vector or use simple recurrent structures to process temporal information, failing to effectively model long-term dependencies and complex dynamic patterns. Fourthly, there's the issue of inference efficiency. In real-time video surveillance scenarios, the system needs to complete anomaly detection every frame or every few frames. Normalizing Flows, due to the need for complete forward propagation and Jacobian determinant calculations, suffers from high latency per inference, making it difficult to meet real-time requirements. Even with model compression or hardware acceleration, a compromise between accuracy and speed is often necessary. Fifthly, it exhibits poor robustness to occlusion and noise. Video surveillance environments often contain interference factors such as target occlusion, lighting changes, and camera shake, leading to noise or missing data in the extracted motion data. Normalizing Flows is sensitive to the quality of input data; when motion estimation has significant errors, the likelihood value calculated by the model may lose its discriminative power, resulting in a high false positive or false negative rate.

[0005] Therefore, how to efficiently and robustly achieve accurate perception and detection of risks associated with abnormal human movements while ensuring real-time processing capabilities is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this application is to provide a behavior risk perception and detection method, electronic device, and medium based on flow matching to solve the above-mentioned problems.

[0007] To achieve the above objectives, firstly, this application proposes a behavior risk perception detection method based on flow matching, the method comprising: Obtain the sequence of human actions to be detected; The anomaly score of the human motion sequence to be detected is calculated by a pre-trained vector field network, wherein the pre-trained vector field network is used to simulate the continuous time probability flow from the prior distribution to the normal motion data distribution. Based on the comparison between the anomaly score and the preset safety threshold, it is determined whether there is any abnormal human movement.

[0008] In some implementations, prior to calculating the anomaly score of the human action sequence to be detected via a pre-trained vector field network, the following steps are included: Obtain a normal action sequence sample set, and sample normal action sequence samples from the normal action sequence sample set; and Noise samples are sampled from the prior distribution, and continuous-time variables are sampled over a continuous time interval. Based on the normal action sequence sample, the noise sample, and the continuous time variable, a conditional probability path is constructed, and an interpolated noisy sequence and a corresponding conditional vector field are generated along the conditional probability path. The continuous-time variables and the noisy sequence are input into the vector field network to be trained to obtain the predicted vector field; The pre-trained vector field network is obtained by minimizing the expected squared error between the predicted vector field and the conditional vector field.

[0009] In some implementations, the vector field network to be trained includes a spatial modeling module, a temporal modeling module, and a fusion module. The step of inputting the continuous-time variables and the noisy sequence into the vector field network to be trained to obtain the predicted vector field includes: The spatial modeling module processes the noisy sequence to extract spatial features. The spatial features are processed by the Transformer network of the temporal modeling module to extract temporal features; The fusion module uses a cross-attention mechanism to fuse the spatial features and the temporal features, and outputs a prediction vector field.

[0010] In some embodiments, the noisy sequence includes coordinates of multiple human motion joints, and the processing of the noisy sequence by the spatial modeling module to extract spatial features includes: Linear projection is performed on the noisy sequence to map the coordinates of human action key points in each frame to the feature space, thus obtaining the initial feature tensor. Based on the initial feature tensor, the adaptive adjacency matrix is ​​calculated using the linear projection matrix. Based on the adaptive adjacency matrix and degree matrix, the initial feature tensor is subjected to graph convolution iteration to obtain spatial features.

[0011] In some implementations, the step of processing the spatial features and extracting temporal features through the Transformer network of the temporal modeling module includes: The spatial features are reshaped into a frame sequence format, and the feature vectors of all relevant nodes in each frame are flattened and spliced ​​to generate the token of the corresponding frame. The continuous time variables are mapped into time embedding vectors using sinusoidal position encoding. The token and the time embedding vector are input into the Transformer network, and the temporal dependencies are captured through a multi-head causal self-attention mechanism to obtain the temporal features.

[0012] In some implementations, the step of inputting the token and the temporal embedding vector into a Transformer network to capture temporal dependencies through a multi-head causal self-attention mechanism to obtain the temporal features includes: The token is linearly projected to obtain a query matrix, a key matrix, and a value matrix; The attention score matrix is ​​obtained by calculating the product of the query matrix and the transpose of the key matrix; A causal mask matrix is ​​applied to the attention score matrix, and the attention score corresponding to the future time step is set to negative infinity; The attention score matrix after applying the causal mask is scaled and the attention weights are calculated using a normalized exponential function. Multiplying the attention weights by the value matrix yields the output features of each attention head; The output features of multiple attention heads are concatenated and linearly transformed to obtain temporal features.

[0013] In some implementations, calculating the anomaly score of the human action sequence to be detected using a pre-trained vector field network includes: Based on the ordinary differential equation corresponding to the pre-trained vector field network, noise samples are sampled from the prior distribution, and the noise samples are integrated along the ordinary differential equation to obtain the reconstructed sequence. The reconstruction error between the human action sequence to be detected and the reconstructed sequence is calculated as an anomaly score; or Based on the ordinary differential equation corresponding to the pre-trained vector field network, the human action sequence to be detected is mapped to the prior distribution space by inverse integration along the ordinary differential equation, and the negative log-likelihood of the mapping result under the prior distribution is calculated as the anomaly score.

[0014] In some implementations, after determining whether abnormal human movement exists based on the comparison result between the anomaly score and a preset safety threshold, the method further includes: If no abnormal human movement is found, the abnormality score is used to update the exponential moving average of the preset safety threshold. When the updated exponential moving average exceeds the drift boundary of the initial threshold, the preset safety threshold is updated.

[0015] Secondly, to achieve the above objectives, this application also proposes an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the behavior risk perception detection method based on flow matching as described above.

[0016] Thirdly, to achieve the above objectives, this application also proposes a computer storage medium storing executable instructions, which, when executed by a processor, cause the processor to perform the behavior risk perception detection method based on flow matching as described above.

[0017] Compared with existing technologies, the beneficial effects of this application include: This application transforms the anomaly detection problem into learning a continuous-time probability flow from a simple prior distribution to a normal action data distribution. By matching the continuous-time probability flow rather than the exact density transformation to train the model, the Jacobian determinant calculation is bypassed, which greatly improves the computational efficiency and model expressive power. It can efficiently and robustly achieve accurate perception and detection of abnormal human action behavior risks while ensuring real-time processing capabilities. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0019] Figure 1 This is a flowchart illustrating a behavior risk perception and detection method based on flow matching in one embodiment; Figure 2 This is a flowchart illustrating a behavior risk perception and detection method based on flow matching in another embodiment; Figure 3 This is a schematic diagram illustrating the process of refining the prediction vector field by inputting the continuous time variable and the noisy sequence into the vector field network to be trained in another embodiment. Figure 4 This is a schematic diagram of the electronic device involved in the behavior risk perception and detection method based on flow matching in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0022] For example, the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For instance, without departing from the scope of this application, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. Both the first element and the second element are elements, but they are not the same element.

[0023] For example, the terms "comprising" or "including" used in this application indicate the presence of features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0024] As mentioned earlier, existing human motion anomaly detection schemes based on Normalizing Flows have revealed several shortcomings in practical applications: First, both training and inference processes require calculating the Jacobian determinant. For high-dimensional action sequence data, the dimension of the Jacobian matrix increases quadratically with the input dimension, making determinant calculation a significant performance bottleneck. Although the computation can be simplified by designing special network structures, such structural constraints limit the model's expressive power and make it difficult to fully capture the complex spatiotemporal dependencies of human motion. Second, Normalizing Flows requires the construction of accurate bijective mappings, which severely restricts network design. Careful parameter adjustment is necessary during training to maintain invertibility, making optimization difficult and prone to numerical instability. Furthermore, this scheme is insufficient in temporal modeling capabilities, struggling to effectively handle long-term dependencies and complex dynamic patterns. Its inference efficiency also falls short of the requirements for real-time video surveillance. Additionally, it exhibits poor robustness to interference factors such as occlusion and lighting changes; when motion estimation errors occur, the detection accuracy drops significantly.

[0025] Therefore, how to efficiently and robustly achieve accurate perception and detection of abnormal human behavior risks while ensuring real-time processing capabilities is a pressing technical problem in this field. To this end, this application proposes a behavior risk perception and detection method, electronic device, and medium based on flow matching, which can efficiently and robustly achieve accurate perception and detection of abnormal human behavior risks while ensuring real-time processing capabilities.

[0026] like Figure 1 As shown in the figure, this application provides a behavior risk perception and detection method based on flow matching, the method including the following steps: Step S10: Obtain the sequence of human actions to be detected.

[0027] In this embodiment, the human motion sequence to be detected refers to a set of several frames of human skeleton posture data continuously collected from the video stream and arranged in chronological order.

[0028] Specifically, a human motion sequence consists of a series of motion frames. Each frame contains coordinates of multiple key joints extracted by human pose estimation algorithms (such as HRNet and OpenPose), including two-dimensional image coordinates or three-dimensional spatial coordinates of joints such as the head, shoulders, elbows, wrists, hips, knees, and ankles. These coordinates are organized into a temporal data structure according to the frame order to obtain the human motion sequence to be detected.

[0029] To eliminate the influence of different individual body shapes and shooting angles, the coordinates can be standardized: the origin of the coordinate system of each frame is translated to the center of the pelvis or the center of mass of the torso, and then normalized and scaled according to shoulder width or height, thereby providing structured and comparable input data for subsequent anomaly detection, while reducing the interference of irrelevant variables (such as human body size and camera angle) on the detection results.

[0030] In some implementations, confidence scores for each joint in the human motion sequence can be obtained. Joints with confidence scores below a preset threshold are then repaired using temporal interpolation or skeleton constraints to obtain a repaired human motion sequence. This repaired sequence is then used as input data for a pre-trained vector field network. Robustness experiments show that even with 30% joint occlusion, the embodiments of this application maintain a detection accuracy of over 80%, while the accuracy of the Normalizing Flows method drops below 60%.

[0031] Furthermore, since real-time anomaly detection requires the system to continuously process newly arriving frames in the video stream, rather than waiting for complete video segments, this application embodiment also sets up a sliding window mechanism: maintaining a fixed-length action sequence buffer, whenever a new frame arrives, the oldest frame is removed from the buffer, and the new frame is added to form an updated human action sequence, which is then used as input data for a pre-trained vector field network. In some implementations, the causal attention mask of the Transformer can also be utilized, requiring only attention calculation for the new frame, while features of historical frames can be cached and reused, avoiding redundant calculations.

[0032] Step S20: Calculate the anomaly score of the human action sequence to be detected using a pre-trained vector field network.

[0033] In this embodiment, the vector field network is a deep neural network used to simulate the continuous-time probability flow from the prior distribution to the normal action data distribution. The prior distribution is a known, easily sampled, simple probability distribution, typically the standard normal distribution N(0, I). The normal action data distribution is a complex, high-dimensional probability distribution formed by all normal human action sequences in mathematical space. The training goal is to teach the model a time-dependent vector field such that the probability flow evolving from the prior distribution along this vector field converges precisely to the normal action data distribution. The continuous-time probability flow refers to the process of data continuously and smoothly evolving from the prior distribution at t=0 to the normal action data distribution at t=1, completely bypassing the bottleneck of calculating the Jacobian determinant in traditional flow models.

[0034] As an alternative implementation for calculating anomaly scores, it can be based on the ordinary differential equations corresponding to a pre-trained vector field network. Noise samples are sampled from the prior distribution. ~N(0, I), and the noise samples Integrating along the ordinary differential equation from t=0 to t=1 yields the reconstructed sequence. 1. Calculate the sequence of human actions to be detected and the reconstructed sequence. The reconstruction error (mean squared error or mean absolute error) between 1 and 2 is used as an anomaly score.

[0035] in, Indicates the learnable parameters are Vector field network; This represents a data sample at continuous time t. The range of t is typically [0, 1]. At t=0, These are noise samples sampled from a prior distribution (such as the standard normal distribution); at t=1, It is the final distribution of normal motion data.

[0036] As another alternative implementation for calculating anomaly scores, the human action sequence to be detected can be mapped to the prior distribution space by inverse integration along the ordinary differential equation corresponding to the pre-trained vector field network, and the negative log-likelihood of the mapping result under the prior distribution can be calculated as the anomaly score.

[0037] In some implementations, it is considered that abnormal human movements may occur at different time scales: some abnormalities manifest as instantaneous abrupt changes (such as sudden falls), while others manifest as long-term deviations (such as persistent abnormal postures). To simultaneously capture abnormal patterns at different scales, the human movement sequence to be detected can be downsampled at different time steps to generate multiple subsequences with different temporal resolutions. Among them, coarse-grained subsequences can capture long-term movement trends, while fine-grained subsequences can retain short-term motion details. An abnormality score is calculated for each subsequence using a pre-trained vector field network, and the maximum abnormality score is taken as the abnormality rating.

[0038] For example, the human action sequence to be detected, which is the original input, is processed by temporal average pooling to generate three subsequences with different temporal resolutions: x^(1), x^(2), and x^(4). Where x^(1) ∈ ^{B×2×24×18} represents the original frame rate of 24 frames, with a step size of 1. It is fine-grained, with complete details in each frame, and is used to detect sudden changes (falling down, suddenly throwing a punch); x^(2)∈ ^{B×2×12×18} represents an average of 2 frames per frame, with a step size of 2, medium granularity, used to detect mid-range action rhythm, abnormal running rhythm, and gait incoordination; x^(4)∈ ^{B×2×6×18} represents an average of 4 frames per frame, with a step size of 4. This coarse-grained approach is used to detect overall motion trends, including long-term, slow deviations (wandering, persistent abnormal postures). Let B be a set of real numbers used to describe the mathematical representation of the action data, and let B represent the batch size. Anomaly scores for each subsequence are calculated using a pre-trained vector field network, and the maximum anomaly score is taken as the anomaly rating.

[0039] It should be noted that the vector field network used to calculate the anomaly scores of multiple subsequences needs to learn multi-scale weighted loss merging. After calculating the flow matching loss at each of the different scales, it is then processed... Learning weight fusion, in which, The flow matching loss at the s-th scale This represents the weight of the s-th scale. , Let represent the learnable logarithmic uncertainty scalar at the s-th scale. Index representing all scales, This represents the normalization factor, ensuring that the sum of all weights is 1.

[0040] Step S30: Based on the comparison result between the abnormality score and the preset safety threshold, determine whether there is any abnormal human movement.

[0041] In this embodiment, the preset safety threshold is a predetermined numerical boundary used to distinguish between normal and abnormal actions. The preset safety threshold can be obtained before model deployment by running the model on a validation set and setting an acceptable false positive rate. For example, the preset safety threshold can be set as the mean of the abnormal scores corresponding to normal action samples plus k times the standard deviation. If the abnormal score is greater than the preset safety threshold, it is determined that there is an abnormal action (such as a fall) in the current human action sequence. If the abnormal score is less than or equal to the preset safety threshold, it is determined that there are no abnormal actions in the current human action sequence, and all actions are normal.

[0042] In some implementations, considering that the actual environment may change, such as changes in light from day to night, variations in keypoint recognition quality due to different clothing in winter and summer, and camera angle shifts caused by wind, these slow changes can cause the overall abnormal scores corresponding to normal actions to drift slowly, requiring updates to the preset safety threshold. Therefore, if no abnormal human actions exist, the abnormal scores can be used to update the exponential moving average of the preset safety threshold; when the updated exponential moving average exceeds the drift boundary of the initial threshold, the preset safety threshold is updated. The exponential moving average (EMA) is a continuously updated statistic that tracks the historical average of abnormal scores corresponding to normal actions. It is calculated using an exponential moving average method, giving higher weight to recent data to reflect the recent trend of abnormal scores corresponding to normal actions. The initial threshold is the average of abnormal scores corresponding to normal action samples calculated using an offline validation set before model deployment, serving as the baseline threshold. The drift boundary is a preset tolerance range used to determine whether the change in EMA is significant; it can be defined as a multiple of the initial threshold or a multiple of the initial standard deviation. The updated safety threshold... .in, Let be the exponential moving average at time t, and k be a preset multiple of the standard deviation. Let be the exponentially moving standard deviation of the abnormal score corresponding to the normal action at time t. It should be noted that, assuming that the abnormal score of the normal action still approximately follows a normal distribution (or at least has a similar relative shape) after drifting, then using the method of "mean + k times standard deviation" for both the initialization and updating of the preset safety threshold can maintain the same false alarm rate as in the initial stage.

[0043] The behavior risk perception detection method based on flow matching proposed in this application transforms the anomaly detection problem into directly learning a continuous-time probability flow by applying flow matching to anomaly action detection. The network is trained by minimizing the expected squared error between the prediction vector field and the conditional vector field, completely bypassing the Jacobian determinant calculation and freeing the network design from the strict constraints of invertible transformation. Furthermore, the calculation of anomaly scores in this application only requires solving ordinary differential equations, avoiding tedious density calculations, significantly reducing single-detection latency, and meeting the stringent processing speed requirements of real-time video surveillance scenarios. In addition, since the vector field network only requires normal action samples during the training phase and does not rely on scarce and variable anomaly sample annotations, it has good generalization ability. Simultaneously, the modeling method of the vector field network for continuous-time probability flows is inherently more robust, effectively coping with common interference factors in real-world monitoring environments such as occlusion and lighting changes. In summary, this application achieves a balance between computational efficiency, model expressive power, and environmental robustness while ensuring detection accuracy.

[0044] In one embodiment, such as Figure 2 As shown, prior to step S20, the following steps are included: Step A10: Obtain a normal action sequence sample set, and sample normal action sequence samples from the normal action sequence sample set.

[0045] In this embodiment, the normal action sequence sample set refers to the training dataset formed by processing a large number of video clips containing only normal human actions (such as walking, standing, sitting, etc.).

[0046] In some implementations, a confidence score for each joint can be calculated using a pose estimator and retained as a confidence weight in subsequent loss calculations, allowing the vector field network to focus more on joints with high confidence.

[0047] Step A20: Sample noise samples from the prior distribution and sample continuous-time variables within the continuous-time interval.

[0048] In this embodiment, the prior distribution is a known and easily sampled simple probability distribution, typically a standard normal distribution N(0, I), where I is the identity matrix with the same dimensions as the normal action sequence samples. Noise samples are vectors randomly sampled from the prior distribution, denoted as... , representing the state at time t=0. A continuous time interval refers to t∈[0,1], representing the time range of the entire evolution process from noise to normal action. A continuous time variable refers to a value of t randomly sampled during training to construct the conditional probability path at that time point.

[0049] Step A30: Based on the normal action sequence sample, the noise sample, and the continuous time variable, construct a conditional probability path, and generate an interpolated noisy sequence and a corresponding conditional vector field along the conditional probability path.

[0050] In this embodiment, the conditional probability path is based on known normal action sequence samples. Under the given conditions, a deterministic or random path is defined from the prior distribution to the sample. This can be achieved using a linear interpolation formula: =t +(1-t) The interpolated noisy sequence was calculated. The noisy sequence is composed of t, and The intermediate state generated by interpolation, which mixes normal action sequence samples and noise samples, can be viewed as a noisy sequence. The conditional vector field refers to the velocity field corresponding to this conditional probability path. For a linear interpolation path, its conditional vector field is... = That is, a constant velocity vector pointing from noise to a normal action sequence sample.

[0051] Step A40: Input the continuous time variable and the noisy sequence into the vector field network to be trained to obtain the predicted vector field.

[0052] In this embodiment, the vector field network to be trained is a deep neural network with parameters as follows: , recorded as It receives noisy sequences. Given a continuous-time variable t, the network outputs a prediction vector field, where the prediction vector field refers to the prediction velocity output by the network, denoted as . .

[0053] Step A50: The pre-trained vector field network is obtained by minimizing the expected squared error between the predicted vector field and the conditional vector field.

[0054] In this embodiment, the training objective of the vector field network is to minimize the expected squared error between the predicted vector field and the conditional vector field, which is to minimize the following loss function L:

[0055] in, Represents the predicted vector field, Let L represent the conditional vector field. The loss function L is designed to make the evolution speed of the network's predictions approximate the ideal speed from noise to normal action sequence samples as closely as possible. Since the loss function does not involve Jacobian determinant calculations, the training process only requires standard gradient descent optimization, significantly improving training efficiency.

[0056] The behavior risk perception detection method based on flow matching proposed in this application utilizes normal action sequence samples and noise to generate intermediate states and their ideal evolution speeds through linear interpolation. A deep neural network is then trained to match these ideal speeds, thereby learning a continuous probability flow from noise to normal action sequence samples. After training, this network can be used in the inference phase to assess the anomalousness of the input sequence by calculating the reconstruction error or negative log-likelihood. Compared to traditional Normalizing Flows, this training method bypasses Jacobian determinant calculations, significantly improving training and inference efficiency, while allowing for more flexible network structures to better model the complex spatiotemporal patterns of human movement.

[0057] In one embodiment, the vector field network to be trained includes a spatial modeling module, a temporal modeling module, and a fusion module. The spatial modeling module captures the joint geometry relationships within each frame of a human motion sequence. The human skeleton naturally forms a graph structure, with joints as nodes and skeletal connections as edges. The goal of the spatial modeling module is to learn the spatial dependencies between joints, including direct anatomical connections (e.g., elbow and wrist) and indirect connections in motor coordination (e.g., hand-foot coordination during walking). The temporal modeling module captures the dynamic evolution patterns between frames in the human motion sequence. The fusion module interactively fuses the single-frame spatial features extracted by the spatial modeling module and the cross-frame temporal features extracted by the temporal modeling module to generate the final predicted vector field. Figure 3 As shown, step A40 includes: Step A41: The noisy sequence is processed by the spatial modeling module to extract spatial features.

[0058] In this embodiment, spatial features refer to the high-dimensional feature vector that each joint point obtains in each frame after being processed by the spatial modeling module, which incorporates information from other joints.

[0059] In some implementations, the noisy sequence Linear projection is performed to map the coordinates of the human motion joints in each frame to the feature space, resulting in the initial feature tensor. .in, This represents a linear projection layer, where B is the batch size, 2 is the joint pixel coordinate dimension, T is the number of frames, V is the number of joints, and d is the feature dimension (e.g., d=64). Based on the initial feature tensor, an adaptive adjacency matrix is ​​calculated using the linear projection matrix. = + , Represents the fixed topological structure of the human skeleton. This indicates non-anatomically connected motor coordination relationships. Among them, =Softmax , From The feature matrix of all joints in a given frame is extracted from the data. and It is a learnable linear projection matrix. This refers to the projected dimension. Based on the adaptive adjacency matrix and degree matrix, the initial feature tensor is subjected to graph convolution iteration to obtain spatial features.

[0060] Specifically, the graph convolution iteration process refers to the formula: .in, for The degree matrix is ​​used for normalization; It is an activation function (such as ReLU); It is a learnable weight matrix; , , is the The feature tensor of the layer.

[0061] Spatial features Each joint has a d-dimensional feature vector in each frame, which has been fused with spatial information from other joints.

[0062] Step A42: The spatial features are processed by the Transformer network of the temporal modeling module to extract temporal features.

[0063] In this embodiment, the Transformer network is a deep learning architecture based on a self-attention mechanism, which can efficiently capture long-range dependencies in a sequence. Temporal features refer to the feature vector of each frame after processing by the temporal modeling module, which incorporates information from historical frames (due to the use of causal masking, only past and current frames can be seen).

[0064] In some implementations, the spatial features are reshaped into a frame sequence format, and the feature vectors of all relevant nodes in each frame are flattened and spliced ​​to generate a token for the corresponding frame. ,in, The spatial features of the input are a four-dimensional tensor. Indicates in In the four-dimensional tensor, we take all samples (first colon), all feature dimensions (second colon), frame t (fixed value), and all key points (fourth colon), which is equivalent to removing the time dimension, to obtain the three-dimensional tensor. Indicates the flattening operation. Indicates the output shape. The token length is given (e.g., d=64, V=18, then the token length is 1152). The continuous time variable is mapped to a temporal embedding vector through sinusoidal positional encoding. The temporal embedding vector is the result of mapping t to a high-dimensional vector space, containing rich information about time t, and can be used by different layers of the network in various ways (e.g., addition, multiplication, as conditional input, etc.). The token and the temporal embedding vector are input into a Transformer network, and the temporal dependencies are captured through a multi-head causal self-attention mechanism to obtain the temporal features.

[0065] In some implementations, the token is linearly projected to obtain a query matrix Q, a key matrix K, and a value matrix. V .in, , , , , , It is a learnable projection matrix. The attention score matrix is ​​obtained by calculating the product of the query matrix and the transpose of the key matrix. Apply a causal mask matrix to the attention score matrix. The attention scores for future time steps are set to negative infinity; the attention score matrix after applying the causal mask is scaled, and the attention weights are calculated using a normalized exponential function. Multiply the attention D weights by the value matrix to obtain the output features of each attention head. The output features of multiple attention heads are concatenated and linearly transformed to obtain temporal features. .

[0066] Step A43: The spatial features and temporal features are fused using the cross-attention mechanism through the fusion module to output a prediction vector field.

[0067] In this embodiment, the cross-attention mechanism is an attention mechanism where the query (temporal feature) comes from one modality, and the key and sum (spatial feature) come from another modality. It allows the model to "query" relevant information from one representation to another, achieving deep fusion of heterogeneous features. The output of the cross-attention mechanism is passed through residual connections, layer normalization, and a feedforward network (MLP) to obtain the fused features, with the shape [formula missing]. The fused features are then passed through a linear projection layer and mapped back to the coordinate dimensions of the input data to obtain the final prediction vector field. .

[0068] The behavioral risk perception and detection method based on flow matching proposed in this application solves the performance bottleneck and limited expressiveness problem of NormalizingFlows in the prior art due to the calculation of Jacobian determinant by constructing a spatially and temporally decoupled vector field network, which works synergistically from three levels.

[0069] First, at the spatial modeling level, by introducing an adaptive graph convolutional network, the traditional method's reliance on predefined skeleton topology is overcome. This module not only utilizes the physical connections of the human skeleton for local feature aggregation, but more importantly, it can automatically mine non-anatomical joint dependencies with kinematic coordination relationships (such as hand-foot coordination during running) from the data through a learnable adaptive adjacency matrix. This mechanism enables the model to have a stronger representational ability for the spatial geometric constraints of complex human postures, and can accurately capture unconventional joint relationships (such as hand-ground contact) that often appear in abnormal movements such as "falls".

[0070] Secondly, at the temporal modeling level, a causal Transformer architecture is introduced, completely changing the traditional approach that relies solely on simple loop structures or flattened sequences. Through a multi-head causal self-attention mechanism, the model can not only perform parallel computation and efficiently capture long-distance temporal dependencies (such as the slow process from standing to imbalance), but also ensure that the inference process strictly follows the unidirectional flow of time. This allows the model to output anomaly scores in real time during online video analysis without waiting for complete future frames, providing technical support for low-latency real-time early warning.

[0071] Finally, spatial features provide the precise joint structure of the current frame, while temporal features provide dynamic evolution information. A cross-attention mechanism fuses these two features, enabling the vector field network to accurately determine whether the current action conforms to normal spatiotemporal evolution patterns. For example, when detecting a sudden fall, the temporal module captures abnormal changes in the center of gravity acceleration, while the spatial module confirms that the geometric distribution of the joints has deviated from the normal gait. The error between the fused predicted vector field and the ideal conditional vector field increases dramatically, thus achieving highly sensitive anomaly detection.

[0072] In one embodiment, a computer storage medium is provided that stores executable instructions that, when executed by a processor, cause the processor to perform the steps in the above method embodiments.

[0073] In one embodiment, an electronic device is also provided, including one or more processors; and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the steps in the above method embodiments.

[0074] In one embodiment, such as Figure 4 The diagram illustrates the structure of an electronic device used to implement an embodiment of this application. The electronic device includes a central processing unit (CPU) 101, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 102 or a program loaded from a storage portion 108 into a random access memory (RAM) 103. The RAM 103 also stores various programs and data required for the operation of the electronic device 100. The CPU 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0075] The following components are connected to I / O interface 105: an input section 106 including a keyboard, mouse, etc.; an output section 107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN card, modem, etc. The communication section 109 performs communication processing via a network such as the Internet. A drive 110 is also connected to I / O interface 105 as needed. A removable medium 111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 110 as needed so that computer programs read from it can be installed into storage section 108 as needed.

[0076] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer-readable medium carrying instructions that, in such embodiments, can be downloaded and installed from a network via communication section 109, and / or installed from removable medium 111. When the instructions are executed by central processing unit (CPU) 101, the various method steps described in this application are performed.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0078] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, any of the embodiments or implementations claimed above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A behavior risk perception and detection method based on flow matching, characterized in that, The method includes: Obtain a normal action sequence sample set, sample normal action sequence samples from the normal action sequence sample set, sample noise samples from the prior distribution, and sample continuous time variables within a continuous time interval; Based on the normal action sequence sample, the noise sample, and the continuous time variable, a conditional probability path is constructed, and an interpolated noisy sequence and a corresponding conditional vector field are generated along the conditional probability path. The continuous time variable and the noisy sequence are input into the vector field network to be trained to obtain the predicted vector field. The pre-trained vector field network is obtained by minimizing the expected squared error between the predicted vector field and the conditional vector field. Obtain the sequence of human actions to be detected; The anomaly score of the human motion sequence to be detected is calculated by a pre-trained vector field network, wherein the pre-trained vector field network is used to simulate the continuous time probability flow from the prior distribution to the normal motion data distribution. Based on the comparison between the anomaly score and the preset safety threshold, it is determined whether there is any abnormal human movement.

2. The behavior risk perception and detection method based on flow matching according to claim 1, characterized in that, The vector field network to be trained includes a spatial modeling module, a temporal modeling module, and a fusion module. The step of inputting the continuous time variables and the noisy sequence into the vector field network to be trained to obtain the predicted vector field includes: The spatial modeling module processes the noisy sequence to extract spatial features. The spatial features are processed by the Transformer network of the temporal modeling module to extract temporal features; The fusion module uses a cross-attention mechanism to fuse the spatial features and the temporal features, and outputs a prediction vector field.

3. The behavior risk perception and detection method based on flow matching according to claim 2, characterized in that, The noisy sequence includes coordinates of multiple human motion joints. The spatial modeling module processes the noisy sequence to extract spatial features, including: Linear projection is performed on the noisy sequence to map the coordinates of human action key points in each frame to the feature space, thus obtaining the initial feature tensor. Based on the initial feature tensor, the adaptive adjacency matrix is ​​calculated using the linear projection matrix. Based on the adaptive adjacency matrix and degree matrix, the initial feature tensor is subjected to graph convolution iteration to obtain spatial features.

4. The behavior risk perception and detection method based on flow matching according to claim 2, characterized in that, The step of processing the spatial features and extracting temporal features through the Transformer network of the temporal modeling module includes: The spatial features are reshaped into a frame sequence format, and the feature vectors of all relevant nodes in each frame are flattened and spliced ​​to generate the token of the corresponding frame. The continuous time variables are mapped into time embedding vectors using sinusoidal position encoding. The token and the time embedding vector are input into the Transformer network, and the temporal dependencies are captured through a multi-head causal self-attention mechanism to obtain the temporal features.

5. The behavior risk perception and detection method based on flow matching according to claim 4, characterized in that, The step involves inputting the token and the temporal embedding vector into a Transformer network, capturing temporal dependencies through a multi-head causal self-attention mechanism, and obtaining the temporal features, including: The token is linearly projected to obtain a query matrix, a key matrix, and a value matrix; The attention score matrix is ​​obtained by calculating the product of the query matrix and the transpose of the key matrix; A causal mask matrix is ​​applied to the attention score matrix, and the attention score corresponding to the future time step is set to negative infinity; The attention score matrix after applying the causal mask is scaled and the attention weights are calculated using a normalized exponential function. Multiplying the attention weights by the value matrix yields the output features of each attention head; The output features of multiple attention heads are concatenated and linearly transformed to obtain temporal features.

6. The behavior risk perception and detection method based on flow matching according to claim 1, characterized in that, The calculation of anomaly scores for the human motion sequence to be detected using a pre-trained vector field network includes: Based on the ordinary differential equation corresponding to the pre-trained vector field network, noise samples are sampled from the prior distribution, and the noise samples are integrated along the ordinary differential equation to obtain the reconstructed sequence. The reconstruction error between the human action sequence to be detected and the reconstructed sequence is calculated as an anomaly score; or Based on the ordinary differential equation corresponding to the pre-trained vector field network, the human action sequence to be detected is mapped to the prior distribution space by inverse integration along the ordinary differential equation, and the negative log-likelihood of the mapping result under the prior distribution is calculated as the anomaly score.

7. The behavior risk perception and detection method based on flow matching according to claim 1, characterized in that, After determining whether abnormal human movement exists based on the comparison result between the anomaly score and the preset safety threshold, the method further includes: If no abnormal human movement is found, the abnormality score is used to update the exponential moving average of the preset safety threshold. When the updated exponential moving average exceeds the drift boundary of the initial threshold, the preset safety threshold is updated.

8. The behavior risk perception and detection method based on flow matching according to claim 1, characterized in that, The step of calculating the anomaly score of the human action sequence to be detected through a pre-trained vector field network further includes: The human motion sequence to be detected is downsampled at different time steps to generate multiple sub-sequences with different time resolutions; Anomaly scores for each subsequence are calculated using a pre-trained vector field network, and the maximum anomaly score is taken as the anomaly rating.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the behavior risk perception detection method based on flow matching as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that, The storage medium stores executable instructions, which, when executed by a processor, cause the processor to perform the behavior risk perception detection method based on flow matching as described in any one of claims 1 to 8.

Images