Human abnormal behavior video generation method based on mechanical constraint and timing optimization

Abstract

The application discloses a human abnormal behavior video generation method based on mechanical constraint and time sequence optimization, obtains an initial video sequence generated by a video generation model according to input abnormal behavior static pictures; performs pretreatment on the initial video sequence and then performs space-time feature extraction; a biomechanical constraint unit and a time sequence optimization unit are constructed; the optimized motion parameters are coded into latent space features and input into a video generation system, and finally, an optimized video is output. While guaranteeing generation efficiency, the application effectively avoids distortion phenomena such as joint dislocation, limb deformation and incoherent action, and improves the physical authenticity of the abnormal behavior video.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and video data generation technology, and relates to a method for generating videos of abnormal human behavior, specifically an optimized method for generating videos of abnormal human behavior based on biomechanical constraints and temporal optimization. Background Technology

[0002] Currently, with the widespread application of computer vision technology in fields such as intelligent security, abnormal behavior detection has become an important technical means to ensure public safety. In scenarios such as security monitoring, accurately identifying abnormal behaviors such as falls and attacks is crucial for timely early warning and intervention.

[0003] Currently, abnormal behavior video data is mainly acquired in two ways: one is by deploying surveillance cameras for long-term recording, which suffers from high data redundancy and a scarcity of effective samples; the other is by using manual simulation recording, but this is limited by labor costs and security risks, making it difficult to cover all types of anomalies. Furthermore, video data obtained through traditional acquisition methods often suffers from problems such as blurred behavior and occlusion of actions, severely impacting the training effect of subsequent algorithms.

[0004] In recent years, with the development of deep learning technologies such as generative adversarial networks, research on anomaly detection based on synthetic data has made some progress. Compared with traditional data acquisition methods, synthetic data methods have advantages such as strong controllability and low cost, and are gradually becoming a new approach to solving the problem of data scarcity.

[0005] However, existing video generation methods have significant limitations: while generative adversarial networks (GANs) can generate diverse motion samples, they often lack physical plausibility, leading to distortions such as joint misalignment and limb penetration; and while physics engine-based methods can ensure motion realism, their complex modeling and high computational costs make them difficult to implement on a large scale. Especially in generating high-risk abnormal behaviors (such as falls and violent attacks), existing technologies cannot simultaneously meet the requirements of motion diversity, physical realism, and visual coherence.

[0006] Therefore, how to efficiently generate physically realistic and visually coherent videos of abnormal behavior, and solve problems such as motion distortion and computational complexity in existing technologies, is a technical challenge that urgently needs to be addressed by those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to provide an optimized method for generating abnormal behavior videos based on biomechanical constraints and temporal optimization. While ensuring generation efficiency, it effectively avoids distortion phenomena such as joint misalignment, limb deformation, and discontinuous movements, thereby improving the physical realism of abnormal behavior videos.

[0008] To achieve the above objectives, the technical solution adopted by this invention is as follows: a method for generating videos of abnormal human behavior based on biomechanical constraints and temporal optimization, comprising the following steps: Step 1: Generate an initial video sequence based on the input static images of abnormal behavior; Step 2: Preprocess the initial video sequence, and then extract and fuse spatiotemporal features, including spatial and motion features; Step 3: First, verify the rationality of the motion based on the preset joint angle threshold. When an abnormal joint angle is detected, generate motion parameters that conform to the physical laws through inverse dynamics calculation. Then, perform deformation correction on the flexible parts. Finally, optimize the continuity of the motion through optical flow smoothing constraints. Step 4: Encode the optimized motion parameters into latent space features and input them into the video generation system to finally output the optimized video.

[0009] Preferably, in step 2, the preprocessing includes spatial normalization and block segmentation. The spatial normalization process uses bilinear interpolation to uniformly scale each frame of the image to a preset size, obtains the standard input size through center cropping, and performs pixel value normalization processing. The segmentation operation divides the video evenly into K segments. Each segment randomly selects a starting position to extract N consecutive frames. When there are fewer than N frames in a segment, a last-frame cyclic filling strategy is adopted. Here, N is a preset value.

[0010] Preferably, in step 2, the preprocessed video tensor sequence X is... i First, the TV-L1 algorithm is used to minimize the energy function that combines the L1 norm data term and the total variational regularization term. Solve the dense optical flow field between adjacent frames; then input the deep feature learning into a 3D convolutional neural network to extract spatiotemporal feature information. ; in, Ω represents the displacement components of the optical flow field in the x and y directions of the image plane, respectively, in pixels; Ω represents the domain of the image, i.e., the set of coordinates of all pixels; λ is the smoothing coefficient, used to balance the weights of the data fidelity term and the regularization term; I1(p) represents the pixel intensity of the reference frame at position p, and I2(p + w) represents the pixel intensity of the target frame at position p + w after deformation, where p = (x, y) x and y represent the horizontal and vertical coordinate components of pixel p, respectively; w represents the displacement vector. ; u and v represents the spatial gradient operators for displacement components u and v, respectively; The 3D convolutional neural network comprises an input layer, a primary feature extraction layer, a multi-level Inception module group, and an output layer connected in sequence. The primary feature extraction layer contains a cascaded combination of 7×7×7 convolutional kernels, 1×1×1 convolutional kernels, and 3×3×3 convolutional kernels, connected to a max pooling layer. The Inception module group consists of three cascaded Inception modules, each containing four parallel convolutional branches, employing 1×1×1 convolution, 3×3×3 convolution, 5×5×5 convolution, and max pooling operations respectively. The modules are connected through max pooling layers. The output layer outputs a 512-dimensional spatiotemporal feature vector through a global average pooling operation. Finally, the spatiotemporal feature vectors are mapped into nodal space coordinate matrices using a motion dynamics decoder. , where R 21×3 The matrix represents the set of coordinates of 21 joints of the human body in three-dimensional space.

[0011] Preferably, in step 2, the TV-L1 algorithm is solved efficiently using a multi-scale pyramid structure and a first-order primal-dual algorithm, and the final output dense optical flow field serves as the input feature of the motion dynamics decoder; wherein, the joint angular velocity is calculated through second-order central difference. Contact flag C∈{0,1} 21 and overall motion acceleration Among them, R 21×3 The matrix represents the set of coordinates of the 21 joints of the human body in three-dimensional space; {0,1} 21 It is a 21-dimensional binary vector used to represent the contact state between the 21 joints of the human body and the environment: This represents a three-dimensional real vector space, specifically referring to the overall motion acceleration vector. The value space of is such that its three components correspond to the linear acceleration of the human body's center of mass along the X, Y, and Z axes in the three-dimensional coordinate system, with units of m / s².

[0012] Preferably, in step 3, the angles of each joint are calculated. ;in, This represents the angle between the bone segments connecting joints i and j; This represents the bone vector pointing from the parent joint to the current joint i. The length of the mold, x i y i z i For the skeletal vector v i Coordinate components in three-dimensional space; when a joint angle θ exists. ijWhen the behavior exceeds the biomechanical limits, a constraint violation alarm is triggered, and joint angle correction is performed; the joint angle correction amount is calculated using the pseudo-inverse of the Jacobian matrix. ,in To maintain the required displacement of the end effector position, and to limit the corrected joint angle to... Within the range; where the parameter This represents the raw joint angle vector directly output by the 3D pose estimation network, with dimension R. 21 ; and These represent the lower and upper limits of the physiological range of motion of each joint, determined based on clinical biomechanical studies; the clip() function is a numerical clipping function. .

[0013] Preferably, in step 3, the deformation correction of the flexible part is based on the MediaPipe algorithm to extract sparse 2D key points from the video frame, denoted as... , indicating the first extracted from the video frame using the MediaPipe algorithm. Two-dimensional key point coordinates, where the subscript Keypoint index numbers are assigned; combined with 3D joint coordinates P, a 2D-3D correspondence is established through projection geometry; then, biomechanical constraint correction is performed, imposing angle restrictions on finger keypoints and constraining the displacement range of facial keypoints, generating the corrected target keypoints. ; A smooth deformation field is constructed using thin-plate spline interpolation, and its energy function is: in, To smooth out the weights, and These represent the horizontal and vertical coordinates of the key points in the image coordinate system, respectively. Dependent on deformation function At the key point The mapping result at the point is the new coordinate after deformation, where N represents the total number of key points; The smooth deformation field, constructed through thin-plate spline interpolation, plays a crucial role in enabling key points to transition from coordinates... arrive To maintain spatial continuity and avoid local distortion during deformation; the optimal deformation mapping is obtained by solving a system of linear equations. Finally, the deformation field is fused with the extracted 3D joint coordinates P, and the geometric correction of the video frame is achieved through bilinear interpolation. The linear equations are derived from the discretization solution process of the thin-plate spline energy function ETPS, by transforming the variational problem into a linear system. Obtain the analytical solution of the optimal deformation mapping f, where K is the radial basis function matrix, λ is the smoothing weight, and w is the deformation parameter.

[0014] Preferably, step 3, which optimizes the motion coherence through optical flow smoothing constraints, specifically includes the following sub-steps: Step 3.1: Based on joint angular velocity characteristics and overall motion acceleration characteristics Establish constraints on the continuity of actions, and optimize the weight matrix by dynamically adjusting these constraints. To ensure a smooth transition between adjacent frames, the calculation formula is as follows: ; in This indicates the change in joint angle between adjacent frames, in rad. The sampling time interval is expressed in seconds (s). To match the weighting coefficients for acceleration, The overall motion Jacobian matrix; dynamically adjust and optimize the weights. ,in Indicates the first In the past, the joint angular velocity The sample variance of the frame. θ˙ is the numerical stability constant. i This represents the angular velocity of the i-th joint, in rad / s, calculated using the second-order central difference of the joint angles. The formula is as follows: ,in Seconds represent the time sampling interval; Step 3.2: Combine the contact flag bit vector C∈{0,1} 21 Optimize the motion trajectory of the contact area, when When indicates that the j-th joint comes into contact with the environment, when This indicates that no contact has occurred; contact constraints are introduced. By constraining the squared displacements of all contact points through summation operations, non-physical sliding phenomena in the contact area are effectively suppressed; among which... Indicates the first Each contact point in time The three-dimensional coordinates The maximum permissible sliding distance; || || denotes the Euclidean distance norm; Step 3.3: Unify the smoothing constraint and the contact constraint into a solvable quadratic programming problem. The final generated optimization objective function is:

[0015] in To maintain the weighting coefficient for contact; Step 3.4: Output the optimal joint angle change that satisfies all constraints. Directly used Generate the motion state for the next frame.

[0016] Preferably, in step 4, the optimized parameters from step 3 are mapped into latent vectors by an MLP encoder and then input into the video generation system, finally outputting the optimized video sequence.

[0017] This invention also provides a video generation system for abnormal human behavior based on mechanical constraints and temporal optimization; comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the video method for abnormal human behavior based on mechanical constraints and timing optimization.

[0018] The present invention also provides a video generation product for abnormal human behavior based on mechanical constraints and temporal optimization, including computer program instructions, which, when the computer program instructions are run on a computer, cause the computer to execute the video generation method for abnormal human behavior based on mechanical constraints and temporal optimization.

[0019] Compared with the prior art, the beneficial effects of the present invention include: (1) The present invention adopts a dual-stream feature extraction architecture (parallel processing of RGB stream and optical flow stream) to obtain efficient and complementary extraction capabilities of spatial posture features and motion dynamics features. Specifically, the RGB stream accurately captures static joint positions through a 3D convolutional neural network, while the optical flow stream analyzes dynamic information such as joint velocity and acceleration through the TV-L1 algorithm and motion dynamics decoder, overcoming the shortcomings of insufficient representation capabilities of a single data source and laying a solid data foundation for generating high-fidelity motion sequences.

[0020] (2) The present invention adopts a joint angle correction mechanism based on the principle of inverse dynamics and biomechanical constraints, which achieves high reliability to ensure that the generated action strictly conforms to the laws of human physiology. By constructing the Jacobian matrix to solve the joint angle correction amount, and using the preset joint range of motion thresholds (such as elbow joint 20°-150°, knee joint 0°-135°) to hard constrain the results, the physical distortion phenomena such as abnormal joint angle and limb penetration commonly found in the prior art are fundamentally avoided, and the naturalness and rationality of the generated action are significantly improved. (3) The present invention adopts a multi-objective temporal optimization function (integrating angular velocity smoothing, overall motion coordination and contact constraints) to achieve excellent performance in maintaining motion continuity and physical realism in complex motion scenarios. The optimization model balances the smoothness requirements of each joint through an adaptive weight matrix and dynamically constrains the sliding of the contact point through the contact flag vector, so that the generated video sequence can still maintain smooth, stable and physical laws in challenging scenarios such as violent movement and rapid turning. Attached Figure Description

[0021] The technical solutions of the present invention will be further illustrated below using embodiments and specific implementation methods. In addition, some accompanying drawings are used in the description of the technical solutions. Those skilled in the art can obtain other drawings and the intent of the present invention from these drawings without any creative effort.

[0022] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the dual-stream feature extraction process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the biomechanical constraints and timing optimization process in an embodiment of the present invention. Detailed Implementation

[0023] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] Please see Figure 1 This embodiment provides a method for generating videos of abnormal human behavior based on biomechanical constraints and temporal optimization, including the following steps: Step 1: The video generation model generates an initial video sequence based on the input static images of abnormal behavior; In one implementation, the input static image of abnormal behavior is processed by a video generation model to generate an initial video sequence. The initial video sequence has a resolution of 1280×720 pixels, a frame rate of 25 frames per second, and a total number of frames of T.

[0025] Step 2: Preprocess the initial video sequence, and then extract and fuse spatiotemporal features, including spatial and motion features; In one implementation, the preprocessing and block segmentation of the initial video sequence for spatiotemporal feature extraction specifically includes: (1) Spatial normalization processing: Each frame of the image is uniformly scaled to 256×256 pixels using bilinear interpolation, and a standard input size of 224×224 pixels is obtained by center cropping. Then, pixel value normalization processing is performed (mean [0.485, 0.456, 0.406], variance [0.229, 0.224, 0.225]). (2) Temporal block strategy: Divide the video into even blocks. Each segment is randomly selected from the starting position to extract 16 consecutive frames (snippet). When a segment has less than 16 frames, a last frame loop filling strategy is used. In one implementation, please see Figure 2 The processed fragments are encapsulated into a 16×224×224×3 tensor sequence. As an RGB stream input, the TV-L1 algorithm is first employed to minimize the energy function that combines the L1 norm data terms and the total variational regularization term. Solve the dense optical flow field between adjacent frames; then input the deep feature learning into a 3D convolutional neural network to extract spatiotemporal feature information. The 3D convolutional neural network comprises an input layer, a primary feature extraction layer, a multi-level Inception module group, and an output layer connected in sequence. The input layer receives a tensor sequence of 16×224×224×3 dimensions. The primary feature extraction layer contains a cascaded combination of 7×7×7 convolutional kernels, 1×1×1 convolutional kernels, and 3×3×3 convolutional kernels, connected to a max-pooling layer. Initial convolution is performed using 7×7×7 kernels and a stride of 2×2×2 to reduce the spatial and temporal dimensions of the data and increase the number of feature channels. Then, a 1×1×1 convolutional layer is applied to adjust the number of feature channels while maintaining the spatial and temporal dimensions, further optimizing the feature representation. Next, a 3×3×3 convolutional layer is used to further extract features with a stride of 1×1×1 to capture deeper features. After the initial convolution operation, a max-pooling layer with a stride of 2×2×2 is applied to reduce the spatial dimension of the feature map. The Inception module group consists of three cascaded Inception modules. Each module contains four parallel convolutional branches, which respectively employ 1×1×1 convolution, 3×3×3 convolution, 5×5×5 convolution, and max pooling operations. The modules are connected through max pooling layers. The output layer outputs a 512-dimensional spatial feature vector through global average pooling operations.

[0026] After the last Inception module, a 2×7×7 average pooling method is used (2 frames in the temporal dimension and 7 pixels × 7 pixels in the spatial dimension) to output a 512-dimensional spatial feature vector. It can be further mapped to a joint space coordinate matrix through a decoding network. Equal characteristic variables, the R21×3 This matrix represents the coordinates of 21 major joints of the human body in three-dimensional space. The 21 rows correspond to the number of key joints in the human kinematic model, including anatomical landmarks such as the head, spine, shoulder, elbow, wrist, hip, knee, and ankle. The 3 columns represent the X, Y, and Z axis coordinates of each joint in the three-dimensional coordinate system. This matrix is ​​obtained by decoding RGB features through a 3D pose estimation network, providing precise spatial location data of the joints for subsequent biomechanical constraint verification, and forming the basis for motion rationality detection. Its dimensional design conforms to the human joint annotation standards defined by the International Society for Biomechanics, and can comprehensively represent the posture information during human movement.

[0027] Finally, the spatiotemporal feature vectors are mapped into nodal space coordinate matrices using a motion dynamics decoder. , where R 21×3 The matrix represents the set of coordinates of 21 joints of the human body in three-dimensional space.

[0028] In one implementation, the TV-L1 algorithm employs a coarse-to-fine multi-scale pyramid optimization strategy, iteratively solving the problem within a first-order primal-dual algorithm framework. The smoothing coefficient λ = 0.15 ± 0.01, the convergence threshold ε = 0.01 ± 0.001, and the maximum number of iterations is 100 ± 5. After 3×3 median filtering and normalization, a 224×224×2 optical flow tensor is output. This provides a dense optical flow field input that preserves the motion edges for motion feature extraction.

[0029] In one implementation, the TV-L1 algorithm efficiently solves the problem using a multi-scale pyramid structure and a first-order primal-dual algorithm, and the final output dense optical flow field serves as the input feature for the motion dynamics decoder; the TV-L1 optical flow algorithm calculates the dense optical flow field between adjacent frames. Its algorithm parameters are: smoothing coefficient Convergence threshold The maximum number of iterations is 100±5. The TV-L1 optical flow algorithm is used to calculate the dense optical flow field between adjacent frames, and its output tensor has a dimension of R. 224×224×2×15 Where 224 represents the spatial height and width resolution, 2 represents the horizontal and vertical displacement components of the optical flow field, and 15 represents the number of continuous optical flow fields generated from 16 frames of input video. The algorithm parameters were optimized and determined as follows: smoothing coefficient λ = 0.15 ± 0.01 (used to balance the weights of data fidelity and regularization terms), convergence threshold ε = 0.01 ± 0.001 (as the iteration termination condition), and maximum number of iterations 100 ± 5 (to ensure a balance between computational efficiency and accuracy). This algorithm calculates optical flow by minimizing the energy function combining the L1 norm data term and the total variational regularization term, specifically expressed as: ; in, Ω represents the displacement components of the optical flow field in the x and y directions of the image plane, respectively, in pixels; Ω represents the domain of the image, i.e., the set of coordinates of all pixels; λ is the smoothing coefficient, used to balance the weights of the data fidelity term and the regularization term; I1(p) represents the pixel intensity of the reference frame at position p, and I2(p + w) represents the pixel intensity of the target frame at position p + w after deformation, where p = (x, y) x and y represent the horizontal and vertical coordinate components of pixel p, respectively; w represents the displacement vector. ; u and v represents the spatial gradient operators of displacement components u and v, respectively; this energy function achieves robust calculation of the optical flow field by combining the L1 norm data fidelity term and the total variational regularization term.

[0030] This embodiment employs an optical flow feature extraction network based on a 3D convolutional neural network architecture. The input is a dense optical flow field sequence calculated by the TV-L1 algorithm. Spatiotemporal features are extracted through multi-layer 3D convolution and pooling operations. Finally, motion feature vectors (such as joint angular velocities and contact markers) are output through a fully connected layer or a specific decoding head. In this embodiment, the optical flow feature extraction network based on a 3D convolutional neural network architecture forms a complementary hierarchical processing architecture. Both networks undertake different levels of computational tasks in the motion feature extraction process, achieving a complete transformation from low-level visual signals to high-level motion parameters through division of labor and cooperation. The TV-L1 algorithm, as the bottom-level motion estimation module, directly processes pixel-level data based on the principle of variational energy minimization. Its core value lies in solving the dense optical flow field between adjacent frames through mathematical optimization methods (minimizing the energy function combining L1 norm data terms and total variational regularization terms). This process ensures the mathematical rigor of motion estimation and the ability to preserve motion boundaries. The optical flow field output by this algorithm contains accurate displacement vector information, but it has not yet formed a structured and semantic motion representation. The optical flow feature extraction network undertakes high-level semantic abstraction. It receives the low-level optical flow field generated by the TV-L1 algorithm as input and performs deep feature learning through a 3D convolutional neural network based on the Inception architecture. The core functions of this network are reflected in three aspects: First, it achieves dimensionality reduction and structuring of data, transforming the pixel-level optical flow field into joint-level motion parameters (such as joint angular velocity ω, contact marker C, etc.); second, it extracts motion dynamics characteristics through spatiotemporal feature fusion, generating feature vectors that meet the requirements of biomechanical analysis; and finally, it provides directly usable standardized input for the subsequent constraint verification module, bridging the gap between low-level motion estimation and high-level biomechanical constraints.

[0031] This division of labor design addresses the limitations of traditional methods: while using only the TV-L1 algorithm can obtain an accurate displacement field, it lacks semantic understanding of motion; directly using an end-to-end network may compromise the rigor of physical constraints. This patent ensures the physical accuracy of motion estimation through the TV-L1 algorithm, while simultaneously utilizing a feature extraction network to intelligently decode motion parameters, ultimately forming a technical solution that combines mathematical rigor with semantic understanding. Figure 2 In the dual-stream architecture shown, the optical flow feature extraction network and the RGB feature extraction network process in parallel, jointly providing deeply abstracted feature inputs for the subsequent temporal optimization module. Therefore, the two modules form a complete technical chain through a strict division of responsibilities: the TV-L1 algorithm is responsible for generating the basic motion field, while the feature extraction network specializes in motion parameter abstraction. This collaborative design retains the stability advantages of traditional optical flow methods while incorporating the environmental adaptability of deep learning, jointly ensuring the realization of motion realism in the generation of abnormal behavior videos.

[0032] Step 3: Biomechanical constraints and timing optimization; First, the rationality of the motion is verified based on the preset joint angle threshold. When an abnormal joint angle is detected, motion parameters that conform to physical laws are generated through inverse dynamics calculation. In addition, deformation correction is performed on flexible parts. Finally, the continuity of the motion is optimized through optical flow smoothing constraints. In one implementation, please see Figure 3 The biomechanical constraints are first calculated by determining the angles of each joint. The calculation formula is as follows: ; in This represents the angle between the bone segments connecting joints i and j. and It represents the unit direction vector of adjacent bone segments. This represents the bone vector pointing from the parent joint to the current joint i. The length of the mold, x i y i z i For the skeletal vector v i Coordinate components in three-dimensional space; Then, a biomechanical constraint verification mechanism was established, with its judgment condition strictly limited to: when a joint angle exists. Beyond biomechanical limits (hip flexion) Outreach Knee flexion When this occurs, a constraint violation alarm is triggered; Next, joint angle correction is performed, and the joint angle correction amount is calculated using the pseudo-inverse of the Jacobian matrix. ,in To maintain the required displacement of the end effector position, and to limit the corrected joint angle to... Within the range. Parameters This represents the original joint angle vector directly output by the 3D pose estimation network (the 3D pose estimation network used in this embodiment belongs to the scope of existing technology, and its specific implementation can be based on the publicly available VideoPose3D or a similar architecture), with dimension R. 21 ; This indicates the pseudo-inverse of the Jacobian matrix. The calculated joint angle correction amount; and These represent the lower and upper limits of physiological range of motion for each joint, determined based on clinical biomechanical studies (including elbow flexion 20°-150°, knee flexion 0°-135°, hip flexion 0°-120° and abduction 0°-45°, and interphalangeal joint flexion 0°-90°). Ultimately, through... The function constrains the corrected joint angles to within the physiologically permissible range, ensuring the biomechanical rationality of the generated movements.

[0033] For deformation correction of flexible parts such as fingers and face, sparse 2D keypoints (21 points for hands / 68 points for face) are extracted from video frames based on the MediaPipe algorithm, denoted as... Combining 3D joint coordinates P, a 2D-3D correspondence is established through projection geometry (the MediaPipe algorithm used in this embodiment belongs to the existing technology category, and its specific implementation is based on Google's publicly available cross-platform machine learning framework; "extracting from video frames" refers to using pre-trained models provided by MediaPipe (such as hand_landmarker, face_landmarker) to detect and output the two-dimensional coordinate data of 21 key points of the human hand or 68 key points of the face in real time from each frame of the input video sequence. These two technical elements together constitute the basic data acquisition link for flexible part deformation correction). Subsequently, biomechanical constraint corrections were performed, imposing angle restrictions on finger keypoints (e.g., proximal interphalangeal joint flexion θ≤90°) and constraining the displacement range of facial keypoints (e.g., mouth corner movement distance ≤15 pixels) to generate corrected target keypoints. A smooth deformation field is constructed using thin-plate spline interpolation (TPS), and its energy function is:

[0034] in To smooth the weights (determined through cross-validation), the optimal deformation mapping is obtained by solving a system of linear equations. Finally, the deformation field is fused with the extracted 3D joint coordinates P, and the geometric correction of the video frame is achieved through bilinear interpolation. This indicates the first [frame] extracted from the video frame using the MediaPipe algorithm. Two-dimensional key point coordinates, where the subscript For key point index numbers, hand region (Corresponding to each joint of the five fingers and the palm), facial area (Corresponding to feature points such as around the eyes, bridge of the nose, and corners of the mouth); Indicates the first after biomechanical constraint correction The coordinates of each target key point, through Calculation obtained ( (A damping coefficient of 0.8 is used to correct the vector and prevent overshoot). and These represent the horizontal and vertical coordinates of the keypoints in the image coordinate system, respectively; optimal deformation mapping. The nonlinear transformation from the original two-dimensional coordinate space to the deformed coordinate space is obtained by minimizing the interpolation energy function of the thin plate spline, where the smoothing weight coefficients are... This was confirmed through cross-validation. The mapping ensures biomechanical rationality while maintaining the smoothness of the deformation field.

[0035] In this embodiment, the smooth deformation field is constructed through thin-plate spline interpolation. Its core function is to realize the transformation of key points from coordinates. arrive Maintain spatial continuity during deformation and avoid local distortion; Dependent on deformation function At the key point The mapping result at the point is the new coordinate after deformation, where N represents the total number of key points (21 points for the hand / 68 points for the face); the linear equations are derived from the discretization solution process of the thin plate spline energy function ETPS, by transforming the variational problem into a linear system. (where K is the radial basis function matrix, λ is the smoothing weight, and w is the deformation parameter) to obtain the analytical solution of the optimal deformation mapping f.

[0036] In one implementation, please see Figure 3 The timing optimization is specifically implemented by including the following steps: (1) Based on the joint angular velocity characteristics provided in step S2 and overall motion acceleration characteristics Establish constraints on the continuity of actions, and optimize the weight matrix by dynamically adjusting these constraints. The core calculation formula for ensuring smooth motion transitions between adjacent frames is as follows: ; in This indicates the change in joint angle between adjacent frames, in rad. The sampling time interval is expressed in seconds (s). To match the weighting coefficients for acceleration, The overall motion Jacobian matrix; dynamically adjust and optimize the weights. subscript For joint index ( ), Indicates the first In the past, the joint angular velocity The sample variance of the frame. θ˙ is the numerical stability constant. i This represents the angular velocity of the i-th joint, in rad / s, calculated using the second-order central difference of the joint angles. The formula is as follows: ,in Seconds represent the time sampling interval; Represents a three-dimensional real vector space, specifically referring to the overall motion acceleration vector. The value space of , whose three components correspond to the linear accelerations of the human body's center of mass along the X, Y, and Z axes in the three-dimensional coordinate system (unit: m / s²).

[0037] (2) Use contact markers to suppress slippage and ensure the physical rationality of the motion trajectory in the contact area, combined with the contact marker vector C∈{0,1} 21 Optimize the motion trajectory of the contact area by introducing contact constraint terms: in Indicates the first Each contact point in time The three-dimensional coordinates This is the maximum permissible sliding distance; Contact flag vector C∈{0,1} 21 It is a 21-dimensional binary vector used to represent the contact state between the 21 joints of the human body and the environment: when When the j-th joint comes into contact with the environment (e.g., foot in contact with the ground, hand in contact with an object), when The time indicates that no contact has occurred. Contact constraint formula. middle: Indicates the first Each contact point in time 3D coordinates (unit: meters); || || denotes the Euclidean distance norm; The maximum permissible sliding distance threshold is defined in meters, a value determined based on clinical biomechanical studies to ensure that the displacement at the contact point conforms to physical constraints. This formula constrains the squared displacements of all contact points through summation, effectively suppressing non-physical sliding phenomena in the contact area.

[0038] (3) By unifying the smoothing constraint and the contact constraint into a solvable quadratic programming problem, the final optimization objective function is: in To maintain the weighting coefficient for contact; Indicates the first The change in angle between adjacent frames of each joint (in radians). Seconds represent the time sampling interval; Indicates the first The angular velocity of each joint (unit: radians / second) is obtained by second-order central difference calculation. Indicates the first The optimization weights of each joint are calculated using the following formula: ,in For the past Frame angular velocity sample variance It is the numerical stability constant; Weighting coefficients are matched for acceleration. The Jacobian matrix represents the overall motion. The acceleration vector of the human body's center of mass in three-dimensional space (unit: m / s²). To maintain the weighting coefficient for contact; This is the contact marker for the j-th joint; This represents the three-dimensional coordinates (in meters) of the j-th contact point at time t. This function ensures the physical plausibility and temporal consistency of the generated motion through multi-objective constraint optimization.

[0039] (4) The output is the optimal joint angle change that satisfies all constraints. The result The sliding threshold and acceleration threshold are satisfied in one go through quadratic programming with hard constraints, therefore it can be directly used. The motion state of the next frame is generated without additional iterative backtracking.

[0040] Step 4: Encode the optimized motion parameters into latent space features and input them into the video generation system to finally output the optimized video.

[0041] In one implementation, the initial video sequence is kinematically constrained and optimized. The optimized parameters in step 3 are mapped into latent vectors by an MLP encoder and then input into the video generation system. Finally, the optimized video sequence with a resolution of 1280×720 and a frame rate of 24fps is output.

[0042] The invention will be further illustrated below through specific experiments.

[0043] In this experiment, the video generation optimization of the two-person fighting behavior is carried out according to the above process, including: (1) extracting the key spatiotemporal features of the attacker and the attacked in the video sequence, then verifying the rationality of the action through biomechanical constraints, and using joint angle thresholds and deformation correction to improve abnormal situations such as joint angle exceeding limits and partial flexible part distortion; (2) first detecting the overall motion pattern, then determining the motion parameters of each limb, and finally ensuring the continuity of the action through temporal optimization; In this embodiment, the above two processes are used at the same time to meet the requirements of biomechanical rationality and temporal continuity when the video is generated.

[0044] The initial video sequence is preprocessed and spatiotemporal features are extracted. A dual-stream I3D network architecture is used to extract spatiotemporal features and perform feature fusion. The RGB feature extraction stream decodes spatial feature vectors through a 3D pose estimation network, and the optical flow feature extraction stream extracts motion feature vectors through a motion dynamics decoder. The preprocessing process of the dual-stream I3D network architecture includes the following steps: First, the initial video sequence is spatially normalized. Bilinear interpolation is used to uniformly scale each frame to 256×256 pixels, and then center-cropping is performed to obtain a standard input size of 224×224 pixels. Pixel value normalization is then performed, with the normalization parameters set to mean [0.485, 0.456, 0.406] and variance [0.229, 0.224, 0.225]. In the temporal segmentation strategy, the video is uniformly divided into K=[T / 16] segments. Each segment randomly selects a starting position and extracts 16 consecutive frames. When a segment has fewer than 16 frames, a last-frame cyclic filling strategy is used to ensure temporal integrity.

[0045] After preprocessing, the processed fragments are encapsulated into a 16×224×224×3 tensor sequence as the input of the RGB stream. At the same time, the TV-L1 algorithm is used to calculate the horizontal and vertical optical flow fields, generating a 16×224×224×2 optical flow tensor sequence as the input of the optical flow stream.

[0046] The RGB feature extraction process specifically includes: inputting a 16×224×224×3 tensor sequence into a 3D convolutional neural network based on the Inception-V1 architecture; firstly, initial convolution is performed using a 7×7×7 convolutional kernel and a 2×2×2 stride to reduce the spatial and temporal dimensions of the data; then, a 1×1×1 convolutional layer is applied to adjust the number of feature channels while maintaining the spatial and temporal dimensions; next, a 3×3×3 convolutional layer with a 1×1×1 stride captures deeper features. After the initial convolution operation, a max pooling layer with a 2×2×2 stride is used to reduce the spatial dimension of the feature map. The feature extraction stage uses three cascaded Inception modules for multi-scale feature capture. Each module contains multiple convolutional branches: the first Inception module is followed by a 3×3×3 max pooling layer, the second by a 2×2×2 max pooling layer, and the third by a 2×7×7 average pooling layer. The final output is a 512-dimensional spatial feature vector, which can be further mapped to feature variables such as joint space coordinate matrices through a decoding network.

[0047] The optical flow feature extraction process includes: calculating the dense optical flow field between adjacent frames using the TV-L1 optical flow algorithm, with smoothing coefficient λ=0.15±0.01, convergence threshold ε=0.01±0.001, and a maximum number of iterations of 100±5. In the initial convolution stage, a 7×7×7 convolutional kernel and a 2×2×2 stride are used to reduce the spatial and temporal dimensions. Then, a 1×1×1 convolutional layer is used to adjust the number of feature channels, followed by a 3×3×3 convolutional layer with a 1×1×1 stride to extract deep features. Feature extraction is also completed through three cascaded Inception modules, each followed by a max-pooling layer of different sizes. Finally, a 2×7×7 average pooling is used to output a 512-dimensional motion feature vector, which can be further decoded into feature variables such as joint angular velocity, contact markers, and overall motion acceleration.

[0048] The biomechanical constraint process specifically includes the following key steps: First, the angles of each joint are calculated based on the extracted joint coordinates. For calculating the attacker's elbow angle, the vector dot product formula is used. ,in This represents the upper arm skeletal vector (from the shoulder joint to the elbow joint). This represents the forearm skeletal vector (from elbow to wrist joint), at this angle. Precisely quantify the degree of elbow flexion. Establish a rigorous biomechanical constraint verification mechanism for typical punching movements: when the elbow joint angle is detected to exceed a preset safe range... Upon this, the system immediately triggers a three-level alarm mechanism: Level 1 is a visual warning (video frames are marked with red bounding boxes), Level 2 is data recording (abnormal frames and parameters are saved), and Level 3 is the initiation of an automatic correction process. When performing joint angle correction, it first uses the pseudo-inverse of the Jacobian matrix. Calculate joint angle correction amount ,in To maintain the displacement required for the actuator position at the end of the fist, the corrected joint angle is then strictly limited to... Within the specified range, ensure that elbow joint movement conforms to human anatomical limitations. The same applies to the knee joint during kicking movements. Within the safe range, when knee hyperextension is detected ( or excessive bending At the same time, the same inverse dynamic correction process is adopted, in which the Jacobian matrix is ​​calculated in real time based on the lower limb skeletal chain, and finally outputs biomechanically reasonable joint motion parameters. For the deformation correction of the attacker's hand striking action and the attacked facial impact area, the system implements the following refined processing procedure: First, the MediaPipe algorithm is used to extract 21 key points of the hand (including the joints of the five fingers and the palm) and 68 feature points of the face (including areas around the eyes, bridge of the nose, and corners of the mouth), denoted as... Simultaneously, combined with the 3D joint coordinates generated in step S2 Establish a 2D-3D correspondence matrix using perspective projection model. This achieves precise mapping between 2D keypoints and 3D skeletons. For the attacker's punching motion, the focus is on constraining the flexion angle of the proximal interphalangeal joints. (Biomechanical safety threshold) When the index finger is detected during a fist-clenching motion... At that time, the system automatically calculates the correction vector. Adjust the target key points to (Damping coefficient 0.8 to prevent overshoot); For facial deformation of the attacked party, strictly limit the displacement of the corner of the mouth. The pixel (corresponding to an actual distance of approximately 1.2cm @ 1080p resolution) triggers a deformation correction mechanism when a displacement of 18 pixels at the corner of the mouth caused by an impact is detected. The deformation field construction stage employs the Thin Plate Spline Interpolation (TPS) algorithm, whose energy function optimization objective includes a data fitting term and a bending energy term: ; Smoothing weight The deformation mapping was determined using 10-fold cross-validation on the CASIA-FACE dataset and solved. This ensures coordinated movement of the facial muscle groups. Finally, the optimized deformation field is fused with the original 3D joint coordinates via bilinear interpolation, achieving sub-pixel-level (0.3px error) geometric correction at 1280×720 resolution. Especially when dealing with facial distortion caused by high-speed impacts, the system dynamically adjusts the smoothing weights. (1.5 times the default value) to ensure visual naturalness under drastic deformation.

[0049] Timing optimization ensures motion continuity through a multi-level constraint mechanism. The specific implementation process is as follows: First, based on the joint angular velocity features extracted in step S2... (Including the elbow joint angular velocity of the attacker's right hook motion) The angular velocity of the cervical spine when the attacker dodges the head. (including 21 joint parameters) and overall motion acceleration characteristics a∈R³ (including trunk center of mass acceleration) A constraint condition for action continuity is constructed. This constraint condition is determined by a dynamically adjusted optimization weight matrix. To achieve fine-grained control, including diagonal elements Adaptive adjustments based on the motion stability of each joint (e.g., the variance of the elbow joint angular velocity during a rapid punch). Assign a low weight of 0.85 to the relatively stable trunk joints. (Assigning a higher weight 4.2). The core optimization objective function contains two key constraints: the first term Ensure a smooth transition in joint angular velocity (e.g., changing the sudden change in elbow joint angular velocity of an attacker's right hook from...). Down to ), the second item Through the overall motion Jacobian matrix Coordinate the consistency of whole-body movements (e.g., maintain synchronization between the acceleration of the supporting leg's knee joint and the forward lean acceleration of the torso during a kick). For typical striking actions, the system performs [this] frame by frame. Solving this optimization problem on RTX 3060 hardware takes less than 1ms per run, meeting real-time requirements. Specifically, for high-speed striking actions (such as a straight punch), the system automatically increases λ to 0.4 (twice the default value of 0.2) to enhance the explosive power of the action. Simultaneously, it suppresses joint jitter through variance monitoring using a 5-frame sliding window (N=5), ultimately outputting a result that conforms to biomechanical principles. Used to generate the pose of the next frame. Contact constraints ensure the physical rationality of limb contact through a multi-dimensional control mechanism. First, based on the 3D joint coordinates extracted in step S2, a contact marker vector is constructed. (e.g., when a fist touches the face) Other joints The system updates the contact state in real time using a collision detection algorithm. For typical fist-face contact scenarios, the system applies strict sliding constraints. (in Represents the three-dimensional coordinates of the j-th contact point. (The maximum permissible sliding distance recommended for clinical biomechanical studies). When an attacker's fist (joint) is detected... ) and the attacked face (joints) Upon contact, the system activates a triple constraint mechanism: the first layer of spatial constraint uses projective geometry to limit the relative displacement of the contact point to within 0.01m (equivalent to approximately 1.2 pixels at 1080p resolution); the second layer of mechanical constraint calculates the normal force based on the Hertzian contact model. (Stiffness coefficient) To ensure that the impact force does not exceed the safety threshold of 500N, the third layer of friction constraint applies a Coulomb friction force of μ=0.3 to limit tangential sliding.

[0050] The optimized motion parameters are encoded into latent space features and input into the video generation system, ultimately outputting an optimized video. Experimental results are shown in Tables 1 and 2, demonstrating that the present invention significantly outperforms the baseline method in terms of both violation rate and average latency.

[0051] Table 1. Biomechanical constraint compliance test results

[0052] Table 2 Comparison of System Timing Performance Indicators (Unit: milliseconds, ms)

[0053] It should be understood that the embodiments described above are only some, not all, of the embodiments of the present invention. Furthermore, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0054] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization, characterized in that, Includes the following steps: Step 1: Generate an initial video sequence based on the input static images of abnormal behavior; Step 2: Preprocess the initial video sequence, and then extract and fuse spatiotemporal features, including spatial and motion features; Step 3: First, verify the rationality of the motion based on the preset joint angle threshold. When an abnormal joint angle is detected, generate motion parameters that conform to physical laws through inverse dynamics calculation. Then, perform deformation correction on the flexible parts. Finally, optimize the continuity of the motion through optical flow smoothing constraints. The optimization of motion coherence through optical flow smoothing constraints specifically includes the following sub-steps: Step 3.1: Based on joint angular velocity characteristics and overall motion acceleration characteristics Establish constraints on the continuity of actions, and optimize the weight matrix by dynamically adjusting these constraints. To ensure a smooth transition between adjacent frames, the calculation formula is as follows: ； in This indicates the change in joint angle between adjacent frames, in rad. The sampling time interval is expressed in seconds (s). To match the weighting coefficients for acceleration, The overall motion Jacobian matrix; dynamically adjust and optimize the weights. ,in Indicates the first In the past, the joint angular velocity The sample variance of the frame. θ˙ is the numerical stability constant. i This represents the angular velocity of the i-th joint, in rad / s, calculated using the second-order central difference of the joint angles. The formula is as follows: ,in Seconds represent the time sampling interval; Step 3.2: Combine the contact flag bit vector C∈{0,1} 21 Optimize the motion trajectory of the contact area, when When indicates that the j-th joint comes into contact with the environment, when This indicates that no contact has occurred; contact constraints are introduced. By constraining the squared displacements of all contact points through summation operations, non-physical sliding phenomena in the contact area are effectively suppressed; among which... Indicates the first Each contact point in time The three-dimensional coordinates The maximum permissible sliding distance; || || denotes the Euclidean distance norm; Step 3.3: Unify the smoothing constraint and the contact constraint into a solvable quadratic programming problem. The final optimization objective function is: in To maintain the weighting coefficient for contact; Step 3.4: Output the optimal joint angle change that satisfies all constraints. Directly used Generate the motion state for the next frame; Step 4: Encode the optimized motion parameters into latent space features and input them into the video generation system to finally output the optimized video.

2. The method for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization according to claim 1, characterized in that: In step 2, the preprocessing includes spatial normalization and block division. The spatial normalization process uses bilinear interpolation to uniformly scale each frame of the image to a preset size, obtains the standard input size through center cropping, and performs pixel value normalization processing. The segmentation operation divides the video evenly into K segments. Each segment randomly selects a starting position to extract M consecutive frames. When there are fewer than M frames, a last-frame cyclic filling strategy is adopted. Here, M is a preset value.

3. The method for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization according to claim 1, characterized in that: In step 2, the preprocessed video tensor sequence X is... i First, the TV-L1 algorithm is used to minimize the energy function that combines the L1 norm data term and the total variational regularization term. Solve the dense optical flow field between adjacent frames; then input the deep feature learning into a 3D convolutional neural network to extract spatiotemporal feature information. ； in, Ω represents the displacement components of the optical flow field in the x and y directions of the image plane, respectively, in pixels; Ω represents the domain of the image, i.e., the set of coordinates of all pixels; λ is the smoothing coefficient, used to balance the weights of data fidelity and regularization terms; I1(p) represents the pixel intensity of the reference frame at position p, and I2(p + w) represents the pixel intensity of the target frame at position p + w after deformation, where p = (x, y) x and y represent the horizontal and vertical coordinate components of pixel p, respectively; w represents the displacement vector. ; u and v represents the spatial gradient operators for displacement components u and v, respectively; The 3D convolutional neural network comprises an input layer, a primary feature extraction layer, a multi-level Inception module group, and an output layer connected in sequence. The primary feature extraction layer contains a cascaded combination of 7×7×7 convolutional kernels, 1×1×1 convolutional kernels, and 3×3×3 convolutional kernels, connected to a max pooling layer. The Inception module group consists of three cascaded Inception modules, each containing four parallel convolutional branches, employing 1×1×1 convolution, 3×3×3 convolution, 5×5×5 convolution, and max pooling operations respectively. The modules are connected through max pooling layers. The output layer outputs a 512-dimensional spatiotemporal feature vector through a global average pooling operation. Finally, the spatiotemporal feature vectors are mapped to nodal space coordinate matrices using a motion dynamics decoder. , where R 21×3 The matrix represents the set of coordinates of 21 joints of the human body in three-dimensional space.

4. The method for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization according to claim 3, characterized in that: In step 2, the TV-L1 algorithm efficiently solves the problem using a multi-scale pyramid structure and a first-order primal-dual algorithm. The resulting dense optical flow field serves as the input feature for the motion dynamics decoder, used to analyze and output specific kinematic parameters, including joint angular velocities calculated via second-order central difference. Contact flag C∈{0,1} 21 and overall motion acceleration Among them, R 21×3 The matrix represents the set of coordinates of the 21 joints of the human body in three-dimensional space; {0,1} 21 It is a 21-dimensional binary vector used to represent the contact state between the 21 joints of the human body and the environment: This represents a three-dimensional real vector space, specifically referring to the overall motion acceleration vector. The value space of is such that its three components correspond to the linear acceleration of the human body's center of mass along the X, Y, and Z axes in the three-dimensional coordinate system, with units of m / s².

5. The method for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization according to claim 1, characterized in that: In step 3, calculate the angles of each joint. ;in, This represents the angle between the bone segments connecting joints i and j; This represents the bone vector pointing from the parent joint to the current joint i. The length of the mold, x i y i z i For the skeletal vector v i Coordinate components in three-dimensional space; when a joint angle θ exists. ij When the behavior exceeds the biomechanical limits, a constraint violation alarm is triggered, and joint angle correction is performed; the joint angle correction amount is calculated using the pseudo-inverse of the Jacobian matrix. ,in To maintain the required displacement of the end effector position, and to limit the corrected joint angle to... Within the range; where the parameter This represents the raw joint angle vector directly output by the 3D pose estimation network, with dimension R. 21 ; and These represent the lower and upper limits of the physiological range of motion of each joint, determined based on clinical biomechanical studies; the clip() function is a numerical clipping function. .

6. The method for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization according to claim 1, characterized in that: In step 3, the deformation correction of the flexible part is based on the MediaPipe algorithm, which extracts sparse 2D key points from the video frame, denoted as... , indicating the first extracted from the video frame using the MediaPipe algorithm. Two-dimensional key point coordinates, where the subscript Keypoint index numbers are assigned; combined with 3D joint coordinates P, a 2D-3D correspondence is established through projection geometry; then, biomechanical constraint correction is performed, imposing angle restrictions on finger keypoints and constraining the displacement range of facial keypoints, generating the corrected target keypoints. ; A smooth deformation field is constructed using thin-plate spline interpolation, and its energy function is: in, To smooth out the weights, and These represent the horizontal and vertical coordinates of the key points in the image coordinate system, respectively. Dependent on deformation function At the key point The mapping result at the point is the new coordinate after deformation, where N represents the total number of key points; The smooth deformation field, constructed through thin-plate spline interpolation, plays a crucial role in enabling key points to transition from coordinates... arrive To maintain spatial continuity and avoid local distortion during deformation; the optimal deformation mapping is obtained by solving a system of linear equations. Finally, the deformation field is fused with the extracted 3D joint coordinates P, and the geometric correction of the video frame is achieved through bilinear interpolation. The linear equations are derived from the discretization solution process of the thin plate spline energy function ETPS, by transforming the variational problem into a linear system. Obtain the analytical solution of the optimal deformation mapping f, where K is the radial basis function matrix, λ is the smoothing weight, and w is the deformation parameter.

7. The method for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization according to any one of claims 1-6, characterized in that: In step 4, the optimized parameters from step 3 are mapped into latent vectors by an MLP encoder and then input into the video generation system, finally outputting the optimized video sequence.

8. A system for generating videos of abnormal human behavior based on mechanical constraints and temporal optimization; characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the video method for abnormal human behavior based on mechanical constraints and timing optimization as described in any one of claims 1 to 7.

9. A computer program instruction, characterized in that: When the computer program instructions are executed on the computer, the computer performs the video method for abnormal human behavior based on mechanical constraints and timing optimization as described in any one of claims 1 to 7.

Images