Human body abnormal behavior video generation method based on mechanical constraint and time sequence optimization

Abstract

The invention discloses a human body abnormal behavior video generation method based on mechanical constraint and time sequence optimization. The method comprises the following steps: acquiring an initial video sequence generated by a video generation model according to an input abnormal behavior static picture; the initial video sequence is preprocessed and then spatio-temporal feature extraction is carried out; constructing a biomechanical constraint unit and a time sequence optimization unit; and encoding the optimized motion parameter into a submerged space feature, inputting the submerged space feature into a video generation system, and finally outputting an optimized video. According to the method, distortion phenomena such as joint dislocation, limb deformation and incoherent actions are effectively avoided while the generation efficiency is ensured, and the physical authenticity of the abnormal behavior video is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision and video data generation, and relates to a human abnormal behavior video generation method, in particular to an optimization method for human abnormal behavior video generation based on biomechanical constraints and time sequence optimization. BACKGROUND

[0002] At present, with the wide application of computer vision technology in intelligent security and other fields, abnormal behavior detection has become an important technical means to ensure public safety. In security monitoring and other scenarios, accurately identifying abnormal behaviors such as falling and attacking is crucial for timely warning and intervention.

[0003] At present, abnormal behavior video data is mainly obtained in two ways: one is to deploy monitoring cameras for long-term shooting, which has the problems of high data redundancy and few effective samples; the other is to use artificial simulation shooting, but it is difficult to cover all abnormal types due to labor cost and safety risks. In addition, the video data obtained by traditional collection methods often has problems such as blurred behavior and motion occlusion, which seriously affects the subsequent algorithm training effect.

[0004] In recent years, with the development of deep learning technologies such as generative adversarial networks, the research on abnormal behavior detection based on synthetic data has made some progress. Compared with traditional data collection methods, synthetic data methods have the advantages of strong controllability and low cost, and have gradually become a new idea to solve the problem of data scarcity.

[0005] However, the existing video generation methods have obvious limitations: although the method based on generative adversarial network can generate diversified motion samples, it often lacks physical rationality, resulting in distortion phenomena such as joint misplacement and limb penetration; while the method based on physical engine can ensure the reality of motion, but the modeling is complex and the calculation cost is high, making it difficult to realize large-scale application. Especially in the generation of high-risk abnormal behaviors (such as falling and violent attack), the existing technology cannot meet the requirements of motion diversity, physical reality and visual coherence at the same time.

[0006] Therefore, how to efficiently generate physically realistic and visually coherent abnormal behavior videos and solve the problems of motion distortion and complex calculation in existing technologies is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0007] The purpose of the present application is to provide an optimization method for abnormal behavior video generation based on biomechanical constraints and time sequence optimization, which can effectively avoid distortion phenomena such as joint misplacement, limb deformation and motion incoherence while ensuring generation efficiency, and improve the physical reality of abnormal behavior videos.

[0008] In order to achieve the above object, the technical scheme adopted by the present application is as follows: a human abnormal behavior video generation method based on biomechanical constraint and time sequence optimization, comprising the following steps: Step 1: generating an initial video sequence according to the input abnormal behavior static picture; Step 2: preprocessing the initial video sequence, and then performing space-time feature extraction and fusion including spatial features and motion features; Step 3: first, verifying the motion rationality based on a preset joint angle threshold, when detecting an abnormal joint angle, generating motion parameters conforming to the physical law through inverse dynamics calculation, then performing deformation correction on the flexible part, and finally optimizing the action continuity through optical flow smoothing constraint; Step 4: encoding the optimized motion parameters into latent space features and inputting them into a video generation system, and finally outputting the optimized video.

[0009] As a preferred, in step 2, the preprocessing includes spatial standardization processing and block operation; The spatial standardization processing uniformly scales each frame of image to a preset size by bilinear interpolation, obtains a standard input size through center cropping, and performs pixel value normalization processing; The block operation uniformly divides the video into K segments, randomly selects a starting position for each segment to cut N consecutive segments, and adopts a last frame loop filling strategy when the segment is less than N frames; wherein N is a preset value.

[0010] As a preferred, in step 2, the video tensor sequence X i obtained after preprocessing is first solved by TV-L1 algorithm, and the energy function combining L1 norm data item and total variation regularization term is minimized to solve the dense optical flow field between adjacent frames; then the deep feature learning is input into a 3D convolutional neural network for deep feature learning to extract space-time feature information; ; Wherein, u and v represent the displacement components of the optical flow field in the image plane x and y direction, unit: pixel; Ω represents the definition domain of the image, that is, the coordinate set of all pixel points; λ is a smoothing coefficient, used to balance the weight of data fidelity term and regularization term; I1(p) represents the pixel intensity of the reference frame at position p, I2(p + w) represents the pixel intensity of the target frame at the deformed position p + w, p = (x, y) , x and y represent the horizontal and vertical coordinate components of the pixel point p; w represents the displacement vector ; u and v represent the spatial gradient operators of displacement components u and v respectively; The 3D convolutional neural network comprises an input layer, a primary feature extraction layer, a multi-level Inception module group and an output layer connected in sequence; the primary feature extraction layer comprises a series combination of 7*7*7 convolution kernels, 1*1*1 convolution kernels and 3*3*3 convolution kernels, and is connected with a maximum pooling layer; the Inception module group is composed of three cascaded Inception modules, each module comprising four parallel convolution branches, respectively adopting 1*1*1 convolution, 3*3*3 convolution, 5*5*5 convolution and maximum pooling operation, and the modules are connected through a maximum pooling layer; the output layer outputs a 512-dimensional space-time feature vector through a global average pooling operation; Finally, the space-time feature vector is mapped into a joint space coordinate matrix through a motion dynamics decoder , wherein R 21×3 is a matrix representing a coordinate set of 21 human joint nodes in a three-dimensional space.

[0011] As preferred, in step 2, the TV-L1 algorithm is efficiently solved through a multi-scale pyramid structure and a first-order primal-dual algorithm, and a finally output dense optical flow field is used as an input feature of the motion dynamics decoder; wherein the joint angular velocity , a contact flag C∈{0,1} 21 , and the overall motion acceleration ; wherein R 21×3 is a matrix representing a coordinate set of 21 human joint nodes in a three-dimensional space; {0,1} 21 is a 21-dimensional binary vector used to represent the contact state of 21 human joint nodes with the environment: represents a three-dimensional real vector space, specifically the value space of the overall motion acceleration vector , whose three components respectively correspond to the linear acceleration of the body center in the X, Y and Z axes of the three-dimensional coordinate system, with the unit of m / s².

[0012] As preferred, in step 3, the joint angle is calculated; wherein represents the included angle of the bone segment connecting joints i and j; represents the modulus of the bone vector pointing from the parent joint to the current joint i, , x i , y i , z i are the coordinate components of the bone vector v i in the three-dimensional space; when there is a joint angle ijWhen the behavior exceeds the biomechanical limits, a constraint violation alarm is triggered, and joint angle correction is performed; the joint angle correction amount is calculated using the pseudo-inverse of the Jacobian matrix. ,in To maintain the required displacement of the end effector position, and to limit the corrected joint angle to... Within the range; where the parameter This represents the raw joint angle vector directly output by the 3D pose estimation network, with dimension R. 21 ; and These represent the lower and upper limits of the physiological range of motion of each joint, determined based on clinical biomechanical studies; the clip() function is a numerical clipping function. .

[0013] Preferably, in step 3, the deformation correction of the flexible part is based on the MediaPipe algorithm to extract sparse 2D key points from the video frame, denoted as... , indicating the first extracted from the video frame using the MediaPipe algorithm. Two-dimensional key point coordinates, where the subscript Keypoint index numbers are assigned; combined with 3D joint coordinates P, a 2D-3D correspondence is established through projection geometry; then, biomechanical constraint correction is performed, imposing angle restrictions on finger keypoints and constraining the displacement range of facial keypoints, generating the corrected target keypoints. ; A smooth deformation field is constructed using thin-plate spline interpolation, and its energy function is: in, To smooth out the weights, and These represent the horizontal and vertical coordinates of the key points in the image coordinate system, respectively. Dependent on deformation function At the key point The mapping result at the point is the new coordinate after deformation, where N represents the total number of key points; The smooth deformation field, constructed through thin-plate spline interpolation, plays a crucial role in enabling key points to transition from coordinates... arrive To maintain spatial continuity and avoid local distortion during deformation; the optimal deformation mapping is obtained by solving a system of linear equations. Finally, the deformation field is fused with the extracted 3D joint coordinates P, and the geometric correction of the video frame is achieved through bilinear interpolation. The linear equations are derived from the discretization solution process of the thin-plate spline energy function ETPS, by transforming the variational problem into a linear system. Obtain the analytical solution of the optimal deformation mapping f, where K is the radial basis function matrix, λ is the smoothing weight, and w is the deformation parameter.

[0014] Preferably, step 3, which optimizes the motion coherence through optical flow smoothing constraints, specifically includes the following sub-steps: Step 3.1: Based on joint angular velocity characteristics and overall motion acceleration characteristics Establish constraints on the continuity of actions, and optimize the weight matrix by dynamically adjusting these constraints. To ensure a smooth transition between adjacent frames, the calculation formula is as follows: ; in This indicates the change in joint angle between adjacent frames, in rad. The sampling time interval is expressed in seconds (s). To match the weighting coefficients for acceleration, The overall motion Jacobian matrix; dynamically adjust and optimize the weights. ,in Indicates the first In the past, the joint angular velocity The sample variance of the frame. θ˙ is the numerical stability constant. i This represents the angular velocity of the i-th joint, in rad / s, calculated using the second-order central difference of the joint angles. The formula is as follows: ,in Seconds represent the time sampling interval; Step 3.2: Combine the contact flag bit vector C∈{0,1} 21 Optimize the motion trajectory of the contact area, when When indicates that the j-th joint comes into contact with the environment, when This indicates that no contact has occurred; contact constraints are introduced. By constraining the squared displacements of all contact points through summation operations, non-physical sliding phenomena in the contact area are effectively suppressed; among which... Indicates the first Each contact point in time The three-dimensional coordinates The maximum permissible sliding distance; || || denotes the Euclidean distance norm; Step 3.3: Unify the smoothing constraint and the contact constraint into a solvable quadratic programming problem. The final generated optimization objective function is:

[0015] in To maintain the weighting coefficient for contact; Step 3.4: output the optimal joint angle change amount that satisfies all constraint conditions , directly used for generating the next frame of motion state.

[0016] As a preferred, in step 4, the optimized parameters in step 3 are input into the video generation system after being mapped into latent vectors by the MLP encoder, and finally the optimized video sequence is output.

[0017] The application also provides a human abnormal behavior video generation system based on mechanical constraint and timing optimization; comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the human abnormal behavior video generation method based on mechanical constraint and timing optimization.

[0018] The application also provides a human abnormal behavior video generation product based on mechanical constraint and timing optimization, comprising computer program instructions, which, when running on a computer, cause the computer to execute the human abnormal behavior video generation method based on mechanical constraint and timing optimization.

[0019] Compared with the prior art, the application has the following advantages: (1) The application adopts a dual-stream feature extraction architecture (RGB stream and optical flow stream are processed in parallel), which has efficient and complementary extraction capability for spatial pose features and motion dynamics features; specifically, the RGB stream accurately captures the static joint position through the 3D convolutional neural network, while the optical flow stream analyzes dynamic information such as joint speed and acceleration through the TV-L1 algorithm and motion dynamics decoder, overcoming the deficiency of insufficient representation ability of a single data source, and laying a solid data foundation for generating high-fidelity motion sequences.

[0020] (2) The application adopts a joint angle correction mechanism based on inverse dynamics principle and biomechanical constraint, which has high reliability to ensure that the generated motion strictly conforms to the laws of human physiology; by constructing the Jacobian matrix to solve the joint angle correction amount, and using the preset joint range of motion threshold (such as elbow joint 20°-150°, knee joint 0°-135°) to hard constrain the result, the physical distortion phenomena such as abnormal joint angle and limb penetration in the prior art are fundamentally avoided, and the naturalness and rationality of the generated motion are significantly improved. (3) The application adopts a multi-target timing optimization function (fusing angular velocity smoothing, overall motion coordination and contact constraint) to obtain excellent performance in maintaining motion coherence and physical authenticity in complex motion scenarios; the optimization model balances the smoothness requirements of each joint through an adaptive weight matrix, and dynamically constrains the sliding of the contact point through a contact flag vector, so that the generated video sequence can still maintain smooth and stable motion and comply with physical laws in challenging scenarios such as violent motion and rapid turning. BRIEF DESCRIPTION OF DRAWINGS

[0021] The technical solutions of the application are further described below using examples and specific embodiments. In addition, some drawings are also used in the description of the technical solutions. For those skilled in the art, other drawings and the intent of the application can also be obtained from these drawings without creative labor.

[0022] Figure 1 The method flowchart of the embodiment of the application is shown in Figure 1. Figure 2 The double-flow feature extraction process schematic diagram of the embodiment of the application is shown in Figure 2. Figure 3 The biomechanical constraint and timing optimization processing process schematic diagram of the embodiment of the application is shown in Figure 3. DETAILED DESCRIPTION

[0023] In order to facilitate those skilled in the art to understand and implement the application, the application is further described in detail below in combination with the drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0024] See Figure 1 The abnormal behavior video generation method based on biomechanical constraint and timing optimization provided by the embodiment includes the following steps: Step 1: An initial video sequence is generated from an input abnormal behavior static picture by a video generation model; In one embodiment, the input abnormal behavior static picture is processed by the video generation model to generate an initial video sequence, the resolution of the initial video sequence is 1280x720 pixels, the frame rate is 25 frames / second, and the total number of frames is T.

[0025] Step 2: The initial video sequence is preprocessed, and then spatial and motion features are extracted and fused; In one embodiment, the initial video sequence is preprocessed and blocked for spatial and motion feature extraction, specifically including: (1) Spatial standardization: Each frame of image is uniformly scaled to 256x256 pixels by bilinear interpolation, and a standard input size of 224x224 pixels is obtained by center cropping, and pixel value normalization processing (mean [0.485, 0.456, 0.406], variance [0.229, 0.224, 0.225]) is performed; (2) Time sequence block strategy: the video is evenly divided into segments, and 16 consecutive snippets are randomly selected for each segment (when the number of snippets is less than 16, the last frame is used for cyclic padding); In an embodiment, see Figure 2 , the processed snippets are packaged into a tensor sequence of 16x224x224x3 as an RGB stream input, first, the TV-L1 algorithm is used to solve the dense optical flow field between adjacent frames by minimizing the energy function combining the L1 norm data item and the total variation regularization term ; then, the 3D convolutional neural network is input for depth feature learning to extract spatio-temporal feature information; The 3D convolutional neural network comprises an input layer, a primary feature extraction layer, a multi-level Inception module group and an output layer connected in sequence. The input layer receives a tensor sequence of 16x224x224x3 dimensions; the primary feature extraction layer comprises a series combination of 7x7x7 convolution kernels, 1x1x1 convolution kernels and 3x3x3 convolution kernels, and is connected to a maximum pooling layer; a 7x7x7 convolution kernel and a 2x2x2 step are used for preliminary convolution to reduce the spatial and temporal dimensions of the data and increase the number of feature channels; then, a 1x1x1 convolution layer is applied to adjust the number of feature channels while keeping the spatial and temporal dimensions unchanged, further optimizing the feature representation; then, a 3x3x3 convolution layer is used to further extract features with a step of 1x1x1 to capture deeper features; after the preliminary convolution operation, a maximum pooling layer with a step of 2x2x2 is used to reduce the spatial dimension of the feature map. The Inception module group is composed of three cascaded Inception modules, each module containing four parallel convolution branches using 1x1x1 convolution, 3x3x3 convolution, 5x5x5 convolution and maximum pooling operation, and the modules are connected by a maximum pooling layer; the output layer outputs a 512-dimensional spatial feature vector through global average pooling operation.

[0026] After the last Inception module, a 2x7x7 average pooling (2 frames in time dimension and 7 pixelsx7 pixels in space dimension) is used to output a 512-dimensional spatial feature vector . Further mapping to feature variables such as joint space coordinate matrix , etc. can be performed by a decoding network, and the R21×3 The matrix represents a set of coordinates of 21 main joints of the human body in three-dimensional space, wherein the row dimension 21 corresponds to the number of key joints in the human body kinematics model, including head, spine, shoulder, elbow, wrist, hip, knee, ankle and other anatomical landmark points; the column dimension 3 represents the X, Y and Z axis coordinate values of each joint in the three-dimensional coordinate system. The matrix is obtained by decoding from the RGB features through the 3D pose estimation network, and provides accurate spatial position data of the joint for subsequent biomechanical constraint verification, and is the basis for realizing the motion rationality detection. The dimension design conforms to the human joint labeling standard defined by the International Society of Biomechanics, and can completely represent the posture information in the human motion process.

[0027] Finally, the spatiotemporal feature vector is mapped into a joint space coordinate matrix by a motion dynamics decoder , wherein R 21×3 The matrix represents a set of coordinates of 21 joints of the human body in three-dimensional space.

[0028] In an embodiment, the TV-L1 algorithm adopts a multi-scale pyramid optimization strategy from coarse to fine, and iteratively solves in a first-order primal-dual algorithm framework, wherein the smoothing coefficient λ = 0.15 ± 0.01, the convergence threshold ε = 0.01 ± 0.001, and the maximum number of iterations is 100 ± 5 times. After 3x3 median filtering and normalization, the output optical flow tensor is 224x224x2 , which provides a dense optical flow field input that maintains the motion edge for motion feature extraction.

[0029] In an embodiment, the TV-L1 algorithm is efficiently solved by a multi-scale pyramid structure and a first-order primal-dual algorithm, and the finally output dense optical flow field is used as the input feature of the motion dynamics decoder; the TV-L1 optical flow algorithm calculates the dense optical flow field between adjacent frames , and the algorithm parameters are: smoothing coefficient , convergence threshold , and maximum number of iterations 100 ± 5 times; The TV-L1 optical flow algorithm is used to calculate the dense optical flow field between adjacent frames, and the output tensor dimension is R 224×224×2×15 , wherein 224 represents the spatial height and width resolution, 2 represents the horizontal and vertical displacement components of the optical flow field, and 15 represents the number of continuous optical flow fields generated by 16 input videos. The algorithm parameters are optimized and determined as follows: smoothing coefficient λ = 0.15 ± 0.01 (used to balance the weight of the data fidelity term and the regularization term), convergence threshold ε = 0.01 ± 0.001 (as the iteration termination condition), and maximum number of iterations 100 ± 5 times (to ensure the balance between calculation efficiency and accuracy). The algorithm realizes optical flow calculation by minimizing the energy function combining the L1 norm data term and the total variation regularization term, and the specific expression is: ; wherein, denote the displacement components of the optical flow field in the x and y directions of the image plane, unit: pixel; Ω denotes the definition domain of the image, i.e. the coordinate set of all pixel points; λ is a smoothing coefficient, used to balance the weight of the data fidelity term and the regularization term; I1(p) denotes the pixel intensity of the reference frame at position p, I2(p + w) denotes the pixel intensity of the target frame at the deformed position p + w, p = (x, y) , x and y respectively denote the horizontal and vertical coordinate components of the pixel point p; w denotes the displacement vector ; u and v respectively denote the spatial gradient operators of the displacement components u and v; the energy function realizes robust calculation of the optical flow field by combining the L1 norm data fidelity term and the total variation regularization term.

[0030] The embodiment adopts an optical flow feature extraction network based on a 3D convolutional neural network architecture, the input is a dense optical flow field sequence calculated by a TV-L1 algorithm, spatiotemporal features are extracted through multi-layer 3D convolution and pooling operations, and finally a motion feature vector (such as joint angular velocity, contact flag, etc.) is output through a full connection layer or a specific decoding head; the optical flow feature extraction network based on the 3D convolutional neural network architecture constitutes a hierarchical complementary architecture in the embodiment, and the two respectively undertake different levels of calculation tasks in the motion feature extraction process, and realize complete conversion from low-level visual signals to high-level motion parameters through division of labor. The TV-L1 algorithm as a bottom motion estimation module directly processes pixel-level data based on the variational energy minimization principle. Its core value lies in solving the dense optical flow field between adjacent frames through a mathematical optimization method (minimizing an energy function combining an L1 norm data term and a total variation regularization term), which guarantees the mathematical rigor of motion estimation and the ability to maintain motion boundaries. The optical flow field output by the algorithm contains accurate displacement vector information, but has not yet formed a structured and semantic motion representation. The optical flow feature extraction network undertakes high-level semantic abstraction, which receives the low-level optical flow field generated by the TV-L1 algorithm as input, and performs deep feature learning through a 3D convolutional neural network based on an Inception architecture. The core role of the network is reflected in three aspects: first, dimension reduction and structuring of data are realized, and the pixel-level optical flow field is converted into joint-level motion parameters (such as joint angular velocity ω, contact flag C, etc.); second, the motion dynamics characteristics are extracted through spatiotemporal feature fusion, and a feature vector conforming to the requirements of biomechanical analysis is generated; finally, standardized input directly usable for the subsequent constraint verification module is provided, and a bridge between low-level motion estimation and high-level biomechanical constraints is built.

[0031] This division of labor design addresses the limitations of traditional methods: while using only the TV-L1 algorithm can obtain an accurate displacement field, it lacks semantic understanding of motion; directly using an end-to-end network may compromise the rigor of physical constraints. This patent ensures the physical accuracy of motion estimation through the TV-L1 algorithm, while simultaneously utilizing a feature extraction network to intelligently decode motion parameters, ultimately forming a technical solution that combines mathematical rigor with semantic understanding. Figure 2 In the dual-stream architecture shown, the optical flow feature extraction network and the RGB feature extraction network process in parallel, jointly providing deeply abstracted feature inputs for the subsequent temporal optimization module. Therefore, the two modules form a complete technical chain through a strict division of responsibilities: the TV-L1 algorithm is responsible for generating the basic motion field, while the feature extraction network specializes in motion parameter abstraction. This collaborative design retains the stability advantages of traditional optical flow methods while incorporating the environmental adaptability of deep learning, jointly ensuring the realization of motion realism in the generation of abnormal behavior videos.

[0032] Step 3: Biomechanical constraints and timing optimization; First, the rationality of the motion is verified based on the preset joint angle threshold. When an abnormal joint angle is detected, motion parameters that conform to physical laws are generated through inverse dynamics calculation. In addition, deformation correction is performed on flexible parts. Finally, the continuity of the motion is optimized through optical flow smoothing constraints. In one implementation, please see Figure 3 The biomechanical constraints are first calculated by determining the angles of each joint. The calculation formula is as follows: ; in This represents the angle between the bone segments connecting joints i and j. and It represents the unit direction vector of adjacent bone segments. This represents the bone vector pointing from the parent joint to the current joint i. The length of the mold, x i y i z i For the skeletal vector v i Coordinate components in three-dimensional space; Then, a biomechanical constraint verification mechanism was established, with its judgment condition strictly limited to: when a joint angle exists. Beyond biomechanical limits (hip flexion) Outreach Knee flexion When this occurs, a constraint violation alarm is triggered; Next, joint angle correction is performed, and the joint angle correction amount is calculated using the pseudo-inverse of the Jacobian matrix. ,in The amount of displacement required to maintain the position of the end effector, and the corrected joint angle is limited to The parameters represent the original joint angle vector directly output by the 3D pose estimation network (the 3D pose estimation network used in this embodiment belongs to the prior art category, and its specific implementation can be based on the publicly disclosed VideoPose3D or similar architecture), with a dimension of R 21 ; represents the joint angle correction amount calculated by the pseudo-inverse of the Jacobian matrix ; and respectively represent the lower and upper limits of the physiological range of motion of each joint determined based on clinical biomechanical research (including elbow flexion 20°-150°, knee flexion 0°-135°, hip flexion 0°-120° and abduction 0°-45°, interphalangeal joint flexion 0°-90°). Finally, the corrected joint angle is constrained within the physiologically allowed range by the function, ensuring the biomechanical reasonableness of the generated motion.

[0033] For deformation correction of flexible parts such as fingers and faces, sparse 2D key points (21 points for hands / 68 points for faces) are extracted from video frames based on the MediaPipe algorithm, denoted as , combined with 3D joint coordinates P, a 2D-3D correspondence is established through projection geometry (the MediaPipe algorithm cited in this embodiment belongs to the prior art category, and its specific implementation is based on the cross-platform machine learning framework publicly disclosed by Google; "extracting from video frames" means that through the pre-trained models provided by MediaPipe (such as hand_landmarker, face_landmarker), the 2D coordinate data of 21 key points of the human hand or 68 key points of the face is detected and output in real time from each frame of the input video sequence. These two technical elements together constitute the basic data acquisition link for deformation correction of flexible parts. Subsequently, biomechanical constraint correction is performed, angle constraints are imposed on finger key points (such as proximal interphalangeal joint flexion θ≤90°), and displacement range constraints are imposed on face key points (such as mouth corner movement distance ≤15 pixels), to generate corrected target key points ; a thin plate spline interpolation (TPS) is used to construct a smooth deformation field, and its energy function is:

[0034] where is the smoothing weight (determined by cross-validation), and the optimal deformation mapping Finally, the deformation field is fused with the extracted 3D joint coordinates P, and the geometric correction of the video frame is achieved through bilinear interpolation. This indicates the first [frame] extracted from the video frame using the MediaPipe algorithm. Two-dimensional key point coordinates, where the subscript For key point index numbers, hand region (Corresponding to each joint of the five fingers and the palm), facial area (Corresponding to feature points such as around the eyes, bridge of the nose, and corners of the mouth); Indicates the first after biomechanical constraint correction The coordinates of each target key point, through Calculation obtained ( (A damping coefficient of 0.8 is used to correct the vector and prevent overshoot). and These represent the horizontal and vertical coordinates of the keypoints in the image coordinate system, respectively; optimal deformation mapping. The nonlinear transformation from the original two-dimensional coordinate space to the deformed coordinate space is obtained by minimizing the interpolation energy function of the thin plate spline, where the smoothing weight coefficients are... This was confirmed through cross-validation. The mapping ensures biomechanical rationality while maintaining the smoothness of the deformation field.

[0035] In this embodiment, the smooth deformation field is constructed through thin-plate spline interpolation. Its core function is to realize the transformation of key points from coordinates. arrive Maintain spatial continuity during deformation and avoid local distortion; Dependent on deformation function At the key point The mapping result at the point is the new coordinate after deformation, where N represents the total number of key points (21 points for the hand / 68 points for the face); the linear equations are derived from the discretization solution process of the thin plate spline energy function ETPS, by transforming the variational problem into a linear system. (where K is the radial basis function matrix, λ is the smoothing weight, and w is the deformation parameter) to obtain the analytical solution of the optimal deformation mapping f.

[0036] In one implementation, please see Figure 3 The timing optimization is specifically implemented by including the following steps: (1) Based on the joint angular velocity characteristics provided in step S2 and overall motion acceleration characteristics Establish constraints on the continuity of actions, and optimize the weight matrix by dynamically adjusting these constraints. The core calculation formula for ensuring smooth motion transitions between adjacent frames is as follows: ; in This indicates the change in joint angle between adjacent frames, in rad. The sampling time interval is expressed in seconds (s). To match the weighting coefficients for acceleration, The overall motion Jacobian matrix; dynamically adjust and optimize the weights. subscript For joint index ( ), Indicates the first In the past, the joint angular velocity The sample variance of the frame. θ˙ is the numerical stability constant. i This represents the angular velocity of the i-th joint, in rad / s, calculated using the second-order central difference of the joint angles. The formula is as follows: ,in Seconds represent the time sampling interval; Represents a three-dimensional real vector space, specifically referring to the overall motion acceleration vector. The value space of , whose three components correspond to the linear accelerations of the human body's center of mass along the X, Y, and Z axes in the three-dimensional coordinate system (unit: m / s²).

[0037] (2) Use contact markers to suppress slippage and ensure the physical rationality of the motion trajectory in the contact area, combined with the contact marker vector C∈{0,1} 21 Optimize the motion trajectory of the contact area by introducing contact constraint terms: in Indicates the first Each contact point in time The three-dimensional coordinates This is the maximum permissible sliding distance; Contact flag vector C∈{0,1} 21 It is a 21-dimensional binary vector used to represent the contact state between the 21 joints of the human body and the environment: when When the j-th joint comes into contact with the environment (e.g., foot in contact with the ground, hand in contact with an object), when The time indicates that no contact has occurred. Contact constraint formula. middle: Indicates the first Each contact point in time 3D coordinates (unit: meters); || || denotes the Euclidean distance norm; The maximum permissible sliding distance threshold is defined in meters, a value determined based on clinical biomechanical studies to ensure that the displacement at the contact point conforms to physical constraints. This formula constrains the squared displacements of all contact points through summation, effectively suppressing non-physical sliding phenomena in the contact area.

[0038] (3) The smooth constraint and the contact constraint are unified as a solvable quadratic programming problem, and the final generated optimization objective function is: wherein is a contact maintenance weight coefficient; represents the adjacent inter-frame angle change amount (unit: radian) of the jth joint; represents the time sampling interval; represents the angular velocity (unit: radian / second) of the jth joint, which is obtained by second-order central difference calculation; represents the optimization weight of the jth joint, and the calculation formula is wherein is the angular velocity sample variance of the past frames, is a numerical stability constant; is an acceleration matching weight coefficient; is the overall motion Jacobian matrix; represents the acceleration vector (unit: m / s2) of the human body center of mass in the three-dimensional space; is a contact maintenance weight coefficient; is the contact flag of the jth joint; represents the three-dimensional coordinates (unit: meters) of the jth contact point at time t. This function ensures the physical rationality and time sequence continuity of the generated motion through multi-objective constraint optimization. (4) The output is the optimal joint angle change amount that satisfies all constraint conditions, and the obtained satisfies the sliding threshold and the acceleration threshold at one time through quadratic programming hard constraint, so it can be directly used to generate the next frame of motion state without additional iteration and backtracking.

[0039] Step 4: Encode the optimized motion parameters into latent space features and input them into the video generation system, and finally output the optimized video. In one embodiment, the initial video sequence is subjected to kinematic constraint optimization, and the optimized parameters in step 3 are input into the video generation system after being mapped into latent vectors by the MLP encoder, and finally an optimized video sequence with a resolution of 1280x720 and a frame rate of 24fps is output.

[0040] The following further illustrates the present application through specific experiments.

[0041] In one embodiment, the initial video sequence is subjected to kinematic constraint optimization, and the optimized parameters in step 3 are input into the video generation system after being mapped into latent vectors by the MLP encoder, and finally an optimized video sequence with a resolution of 1280x720 and a frame rate of 24fps is output.

[0042] The following further illustrates the present application through specific experiments.

[0043] ​​In the experiment, the video generation optimization of the two people fighting behavior is performed according to the above process, including: (1) extracting the key spatiotemporal features of the attacker and the attacked person in the video sequence, and then verifying the action rationality through biomechanical constraints, using joint angle threshold and deformation correction to improve the joint angle over-limit, and some abnormal conditions such as partial flexible part distortion; (2) detecting the overall motion mode first, then determining the motion parameters of each limb part, and finally ensuring the action continuity through timing optimization; in this embodiment, the above two processes are used simultaneously when generating the video to meet the requirements of biomechanical rationality and timing continuity.

[0044] The initial video sequence is preprocessed and spatiotemporal features are extracted, and a dual-flow I3D network architecture is used for spatiotemporal feature extraction and feature fusion, wherein the RGB feature extraction flow decodes the spatial feature vector through the 3D pose estimation network, and the optical flow feature extraction flow extracts the motion feature vector through the motion dynamics decoder; The preprocessing process of the dual-flow I3D network architecture includes the following steps: first, the initial video sequence is subjected to spatial standardization processing, and after uniform scaling of each frame image to 256x256 pixels, 224x224 pixel standard input size is obtained through center cropping, and pixel value normalization processing is performed, and the normalization parameters are set to mean [0.485, 0.456, 0.406] and variance [0.229, 0.224, 0.225]. In the timing block strategy, the video is evenly divided into K=[T / 16] segments, and 16 frames of continuous segments are randomly selected for each segment, and when the segment is less than 16 frames, the end frame is filled with a cycle strategy to ensure the timing integrity.

[0045] After preprocessing, the processed segments are packaged as a tensor sequence of 16x224x224x3 as an RGB flow input, and the TV-L1 algorithm is used to calculate the horizontal and vertical optical flow fields to generate a light flow tensor sequence of 16x224x224x2 as an optical flow flow input.

[0046] The processing flow of the RGB feature extraction stream specifically includes: inputting a 16x224x224x3 tensor sequence into a 3D convolutional neural network based on an Inception-V1 architecture, first performing preliminary convolution by using a 7x7x7 convolution kernel and a 2x2x2 step to reduce the spatial and temporal dimensions of the data, then applying a 1x1x1 convolution layer to adjust the feature channel number and keep the spatial and temporal dimensions unchanged, and then capturing deeper features by a 3x3x3 convolution layer with a 1x1x1 step. After completing the preliminary convolution operation, the spatial dimension of the feature map is reduced by a maximum pooling layer with a 2x2x2 step. The feature extraction stage captures multi-scale features through three cascaded Inception modules, each module includes multiple convolution branches, the first Inception module is followed by a 3x3x3 maximum pooling layer, the second is followed by a 2x2x2 maximum pooling layer, and the third is followed by a 2x7x7 average pooling layer. Finally, a 512-dimensional spatial feature vector is output, which can be further mapped to feature variables such as joint space coordinate matrix by a decoding network.

[0047] The processing process of the optical flow feature extraction stream includes: using a TV-L1 optical flow algorithm to calculate the dense optical flow field between adjacent frames, and setting the algorithm parameters as a smoothing coefficient λ = 0.15 ± 0.01, a convergence threshold ε = 0.01 ± 0.001, and a maximum iteration number 100 ± 5 times. In the preliminary convolution stage, a 7x7x7 convolution kernel and a 2x2x2 step are used to reduce the spatial and temporal dimensions, then a 1x1x1 convolution layer is used to adjust the feature channel number, and then a 3x3x3 convolution layer is used to extract deep features with a 1x1x1 step. Feature extraction is also completed through three cascaded Inception modules, each module is followed by a maximum pooling layer of different sizes, and finally a 2x7x7 average pooling is used to output a 512-dimensional motion feature vector. This vector can be further decoded into feature variables such as joint angular velocity, contact flag bit, and overall motion acceleration.

[0048] The process of biomechanical constraints specifically includes the following key steps: first, based on the extracted joint coordinates, the joint angles of each joint are calculated For the elbow joint angle of the attacker, the vector dot product formula is used wherein represents the upper arm skeleton vector (shoulder joint to elbow joint), represents the forearm skeleton vector (elbow joint to wrist joint), and the angle accurately quantifies the degree of elbow bending. For a typical punching action, a strict biomechanical constraint verification mechanism is established: when the elbow joint angle is detected to be outside the preset safe range At this time, the system immediately triggers a three-level alarm mechanism - the first level visual warning (video frame marked with a red bounding box), the second level data recording (save abnormal frames and parameters), and the third level automatic correction process starts. When executing joint angle correction, first, the pseudo-inverse of the Jacobian matrix is used to calculate the correction amount of the joint angle . wherein is the displacement amount required to maintain the end effector position of the fist, and then the corrected joint angle is strictly limited within , ensuring that the elbow joint movement conforms to the human anatomy limit. For the knee joint in the kicking action, a safety range of is also set, and when overextension or excessive flexion of the knee joint is detected, the same inverse dynamics correction process is used, where the Jacobian matrix is calculated in real time according to the lower limb skeletal chain, and finally the biomechanically reasonable joint movement parameters are output. For the attacker's hand hitting action and the deformation correction of the attacked face hitting area, the system implements the following refinement processing process: first, the MediaPipe algorithm is used to extract the 21 key points of the hand (including the joints of the five fingers and the palm) and the 68 feature points of the face (including the eye area, nose bridge, and mouth corner area) in real time, denoted as , and combined with the 3D joint coordinates generated in step S2, the 2D-3D correspondence matrix is established using the perspective projection model to realize the accurate mapping of two-dimensional key points and three-dimensional skeletons. For the attacker's fist hitting action, the bending angle of the proximal interphalangeal joint (biomechanical safety threshold) is mainly constrained, and when the index finger is detected in the fist action, the system automatically calculates the correction vector to adjust the target key point to (damping coefficient 0.8 to prevent overshoot); for the deformation of the attacked face, the displacement amplitude of the mouth corner is strictly limited to pixels (corresponding to an actual distance of about 1.2 cm @ 1080p resolution), and when the mouth corner displacement caused by the hitting reaches 18 pixels, the deformation correction mechanism is triggered. The thin plate spline interpolation (TPS) algorithm is used in the deformation field construction stage, and the energy function optimization target contains the data fitting term and the bending energy term: ; wherein the smoothing weight is determined by 10-fold cross-validation on the CASIA-FACE dataset, and the deformation mapping The overall coordinated movement of the facial muscle groups can be ensured. The optimized morph field and the original 3D joint coordinates are finally fused through bilinear interpolation, and sub-pixel level (0.3px error) geometric correction is realized under a resolution of 1280x720. Especially when dealing with facial distortion caused by high-speed punches, the system will dynamically adjust the smoothing weight to (1.5 times the default value) to ensure visual naturalness under severe deformation.

[0049] The timing optimization ensures the coherence of the motion through a multi-level constraint mechanism. The specific implementation process is as follows: first, based on the joint angular velocity features (including the elbow joint angular velocity of the attacker's right hook punch and the cervical spine angular velocity of the attacked person's head dodge , etc. 21 joint parameters) and the overall motion acceleration feature a∈R³ (including the trunk centroid acceleration , the motion coherence constraint condition is constructed. The constraint condition is realized through a dynamically adjusted optimization weight matrix , in which the diagonal elements are adaptively adjusted according to the stability of each joint motion (such as the elbow joint angular velocity variance when punching quickly, which is given a lower weight of 0.85; the relatively stable trunk joints , which are given a higher weight of 4.2). The core optimization objective function includes two key constraints: the first constraint ensures smooth transition of joint angular velocity (such as reducing the elbow joint angular velocity mutation of the attacker's right hook punch from to ), and the second constraint coordinates the consistency of the whole body motion through the overall motion Jacobian matrix (such as maintaining the synchronization of the supporting leg knee joint acceleration and the trunk forward acceleration when kicking). For typical punching actions, the system solves this optimization problem every frame , and the single solving time is <1ms on RTX 3060 hardware, meeting the real-time requirements. Especially for high-speed punching actions (such as straight punches), the system will automatically increase λ to 0.4 (2 times the default value of 0.2) to enhance the motion burst performance, while suppressing joint jitter through a 5-frame sliding window (N=5) variance monitoring, and finally outputting for generating the next frame pose . The contact constraint ensures the physical reasonableness of the limb contact through a multi-dimensional control mechanism. First, based on the 3D joint coordinates extracted in step S2, the contact flag bit vector is constructed (such as when the fist contacts the face , other joints ), and the contact status is updated in real time by a collision detection algorithm. For typical fist-face contact scenarios, the system imposes strict sliding constraints (where represents the three-dimensional coordinates of the jth contact point, is the maximum allowed sliding distance recommended for clinical biomechanical studies). When the attacker's fist (joint ) is detected to be in contact with the defender's face (joint ), the system initiates a triple constraint mechanism: the first layer of spatial constraints limits the relative displacement of the contact points within a range of 0.01m (equivalent to about 1.2 pixels at 1080p resolution) through projection geometry; the second layer of mechanical constraints calculates the normal force (stiffness coefficient ) based on the Hertz contact model, ensuring that the impact force does not exceed the safety threshold of 500N; the third layer of friction constraints imposes Coulomb friction with μ=0.3 to limit tangential sliding.

[0050] The optimized motion parameters are encoded as latent space features and input into the video generation system, finally outputting the optimized video. The experimental results in Table 1 and Table 2 prove that the present application is far superior to the baseline method in terms of violation rate and average latency.

[0051] Table 1 Biomechanical constraint compliance test results

[0052] Table 2 Comparison of system timing performance indicators (unit: milliseconds, ms)

[0053] It should be understood that the above-described embodiments are part of the embodiments of the present application, rather than all the embodiments. In addition, the technical features of each embodiment or single embodiment provided by the present application can be combined with each other arbitrarily to form a feasible technical solution, and such combination is not restricted by the order of steps and / or structure composition mode, but must be based on the realization by ordinary skilled in the art, when the combination of technical solutions appears contradictory or unfeasible, it should be considered that such combination of technical solutions does not exist, nor within the protection scope claimed by the present application.

[0054] It should be understood that the above description of the preferred embodiments is more detailed and therefore should not be considered as a limitation on the scope of patent protection of the present application. Ordinary skilled in the art can make substitutions or modifications without departing from the scope of the claims, which fall within the protection scope of the present application. The scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for generating human abnormal behavior video based on mechanical constraint and timing optimization, characterized in that, The method comprises the following steps: Step 1: generating an initial video sequence according to an input abnormal behavior static picture; Step 2: pre-processing the initial video sequence, and then performing spatio-temporal feature extraction and fusion including spatial features and motion features; Step 3: first, verifying motion rationality based on a preset joint angle threshold value, and when an abnormal joint angle is detected, generating motion parameters conforming to physical laws through inverse dynamics calculation; then, performing deformation correction on flexible parts, and finally optimizing action continuity through optical flow smoothing constraint; Step 4: encoding the optimized motion parameters into latent space features and inputting the video generation system to finally output an optimized video.

2. The method of claim 1, wherein the method is based on mechanical constraints and timing optimization for human abnormal behavior video generation. In step 2, the pre-processing comprises spatial normalization processing and blocking operation; The spatial normalization processing uniformly scales each image to a preset size through bilinear interpolation, obtains a standard input size through center cropping, and performs pixel value normalization processing; The blocking operation uniformly divides the video into K segments, randomly selects a starting position for each segment to intercept M consecutive segments, and adopts a last frame loop filling strategy when the segment is less than M frames; wherein M is a preset value. 3.The method of claim 1, wherein: In step 2, the video tensor sequence X obtained after preprocessing i First, the TV-L1 algorithm is adopted to minimize the energy function combining the L1 norm data item and the total variation regular term Solve the dense optical flow field between adjacent frames; then input the deep feature learning into the 3D convolutional neural network for deep feature learning to extract spatiotemporal feature information; ； wherein, denote the displacement components of the optical flow field in the x and y directions of the image plane, respectively, in units of pixels; Ω denotes the definition domain of the image, i.e., the coordinate set of all pixel points; λ is a smoothing coefficient for balancing the weight of the data fidelity term and the regularization term; I1(p) denotes the pixel intensity at position p in the reference frame, and I2(p + w) denotes the pixel intensity at the deformed position p + w in the target frame, p = (x, y) , x and y denote the horizontal and vertical coordinate components of the pixel point p, respectively; w denotes the displacement vector ; u and v denote the spatial gradient operators of the displacement components u and v, respectively; The 3D convolutional neural network comprises an input layer, a primary feature extraction layer, a multi-level Inception module group and an output layer connected in sequence; the primary feature extraction layer comprises a series combination of 7×7×7 convolution kernels, 1×1×1 convolution kernels and 3×3×3 convolution kernels, and is connected with a maximum pooling layer; the Inception module group is composed of three cascaded Inception modules, each module comprising four parallel convolution branches, respectively adopting 1×1×1 convolution, 3×3×3 convolution, 5×5×5 convolution and maximum pooling operation, and the modules are connected through a maximum pooling layer; and the output layer outputs a 512-dimensional spatio-temporal feature vector through a global average pooling operation; Finally, the spatio-temporal feature vectors are mapped to joint space coordinate matrices by a motion dynamics decoder where R 21×3 The matrix represents the coordinate set of 21 human body joints in three-dimensional space.

4. The method of claim 3, wherein the method is characterized by: In step 2, the TV-L1 algorithm is solved efficiently by a multi-scale pyramid structure and a first-order primal-dual algorithm, and the final output of the dense optical flow field is used as the input feature of the motion dynamics decoder to analyze and output specific kinematic parameters, including joint angular velocity calculated by second-order central difference , contact flag C∈{0,1} 21 , and overall motion acceleration ; wherein R 21×3 The matrix represents the coordinate set of the 21 joint nodes of the human body in the three-dimensional space; {0,1} 21 is a 21-dimensional binary vector, which is used to represent the contact state of the 21 joint nodes of the human body with the environment: , which represents a three-dimensional real vector space, specifically the value space of the overall motion acceleration vector , whose three components respectively correspond to the linear acceleration of the center of mass of the human body in the X, Y, and Z axes of the three-dimensional coordinate system, with a unit of m / s².

5. The method of claim 1, wherein the method is characterized by: In step 3, the joint angles are calculated ; wherein, represents the included angle of the bone segment connecting joints i and j; represents the bone vector pointing from the parent joint to the current joint i , , x i , y i , z i are the coordinate components of the bone vector v i in the three-dimensional space; when the joint angle θ ij exceeds the biomechanical range, a constraint violation alarm is triggered and joint angle correction is performed; the joint angle correction amount is calculated by pseudo-inverse of the Jacobian matrix , wherein is the displacement amount required to maintain the position of the end effector, and the corrected joint angle is limited within the range of ; wherein, the parameter represents the original joint angle vector directly output by the 3D pose estimation network, with a dimension of R 21 ; and respectively represent the lower limit and upper limit of the physiological range of motion of each joint determined based on clinical biomechanical research; the clip() function is a numerical clipping function, .

6. The method of claim 1, wherein the method is based on mechanical constraints and timing optimization for human abnormal behavior video generation. In step 3, the flexible part is deformed to correct, based on the MediaPipe algorithm, sparse 2D key points extracted from the video frame, denoted as , represents the first two-dimensional key point coordinates extracted from the video frame by the MediaPipe algorithm, where the subscript is the key point index number; combined with the 3D joint coordinates P, the 2D-3D correspondence is established by projection geometry; then the biomechanical constraint correction is performed, the angle limit is applied to the finger key points, and the displacement range is constrained to the face key points, to generate the corrected target key points ; A smooth deformation field is constructed by using thin plate spline interpolation, and an energy function thereof is: wherein, is a smoothing weight, and denote the horizontal and vertical coordinates of the keypoint in the image coordinate system, respectively; denotes a warping function the mapping result at the keypoint , i.e. the new coordinates after warping, N denotes the total number of keypoints. The smooth deformation field is constructed by thin plate spline interpolation, which plays a core role in maintaining spatial continuity in the deformation process of key points from coordinates to , and avoiding local distortion; the optimal deformation mapping is obtained by solving a linear equation system ; finally, the deformation field is fused with the extracted 3D joint coordinates P, and the geometric correction of the video frame is realized through bilinear interpolation; the linear equation system is derived from the discretization solving process of the thin plate spline energy function ETPS, and the analytic solution of the optimal deformation mapping f is obtained by converting the variational problem into a linear system , wherein K is a radial basis function matrix, λ is a smoothing weight, and w is a deformation parameter.

7. The method of claim 1, wherein the method is based on mechanical constraints and timing optimization for human abnormal behavior video generation. In step 3, the action continuity is optimized by the optical flow smoothing constraint, and specifically comprises the following sub-steps: Step 3.1: joint angular velocity feature based and overall motion acceleration feature , the action coherence constraint condition is established, and the optimization weight matrix is dynamically adjusted to ensure smooth transition between adjacent frames, and the calculation formula is: ； in This indicates the change in joint angle between adjacent frames, in rad. The sampling time interval is expressed in seconds (s). To match the weighting coefficients for acceleration, The overall motion Jacobian matrix; dynamically adjust and optimize the weights. ,in Indicates the first In the past, the joint angular velocity The sample variance of the frame. θ˙ is the numerical stability constant. i This represents the angular velocity of the i-th joint, in rad / s, calculated using the second-order central difference of the joint angles. The formula is as follows: ,in Seconds represent the time sampling interval; Step 3.2: Combine the contact flag vector C∈{0,1} 21 Optimize the motion trajectory of the contact region, when indicates that the jth joint node is in contact with the environment, and when indicates that there is no contact; by introducing a contact constraint term The displacement square of all contact points is constrained by summation operation, effectively inhibiting the non-physical sliding phenomenon of the contact region; wherein represents the th contact point in the three-dimensional coordinate at time is the maximum allowed sliding distance; and || represents the Euclidean distance norm. Step 3.3: unifying the smoothing constraint and the contact constraint into a solvable quadratic programming problem, and the final generated optimization objective function is: wherein is a contact retention weight factor; Step 3.4: output the optimal joint angle change amount that satisfies all constraint conditions , directly for generating the next frame of motion state.

8. The method of claim 1-7, wherein the method is characterized in that: In step 4, the optimized parameters in step 3 are input into the video generation system after being mapped into latent vectors by the MLP encoder, and finally an optimized video sequence is output. 9.A system for generating human abnormal behavior video based on mechanical constraint and timing optimization, characterized in that, Comprise: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the human abnormal behavior video method based on mechanical constraint and time sequence optimization according to any one of claims 1-7. 10.A product for generating human abnormal behavior video based on mechanical constraint and timing optimization, comprising computer program instructions, characterized in that: When the computer program instructions run on the computer, the computer executes the human abnormal behavior video method based on mechanical constraint and time sequence optimization according to any one of claims 1-7.

Images