Shielding scene two-dimensional attitude estimation method and system based on spatio-temporal information

Abstract

The invention discloses a method and system for estimating the two-dimensional attitude of an occlusion scene based on spatio-temporal information, and relates to the field of computer vision, and the method comprises the steps: S1, obtaining a video comprising information of a plurality of human bodies; s2, constructing a two-dimensional attitude estimation model; the model comprises an input module, a visibility prediction and shielding module, a key point reasoning module, a space-time dependency module and a regression layer. S3, inputting the obtained video into a two-dimensional attitude estimation model for training to obtain a trained two-dimensional attitude estimation model; and S4, using the trained two-dimensional attitude estimation model to carry out space-time information-based occlusion scene two-dimensional attitude estimation. According to the two-dimensional attitude estimation model constructed by the method, shielded key points can be accurately reasoned, the smoothness and accuracy of attitude estimation are improved, the interference of high-frequency noise is reduced, and the calculation complexity is reduced while the accuracy of the model is ensured, so that the real-time requirement is met.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and system for estimating two-dimensional posture of an occluded scene based on spatiotemporal information. Background Art

[0002] Traditional two-dimensional pose estimation methods usually have difficulty in dealing with the problem of key point feature loss in occluded scenes, resulting in poor accuracy and continuity of the estimation results, which limits their effectiveness in practical applications. Especially in the fields of real-time monitoring and motion analysis, traditional methods often perform unsatisfactorily when faced with challenges such as invisible key points and high-frequency noise interference in complex dynamic scenes. Therefore, how to improve the robustness and accuracy of the model in occluded environments has become a major problem facing pose estimation technology. With the continuous development of deep learning and computer vision technology, spatiotemporal information modeling has gradually become an important research direction for solving occlusion problems. The introduction of deep learning provides a new solution for the dynamic reasoning of key point features and the fusion of spatiotemporal information. By combining time series information with spatial structure relationships, deep learning methods can significantly enhance the model's adaptability to occluded scenes, thereby improving the accuracy of pose estimation. However, existing technologies still face challenges in building an effective spatiotemporal feature fusion framework. Especially in occluded scenes, how to accurately infer occluded key points and improve the smoothness and accuracy of pose estimation is still an urgent problem to be solved. In addition, the existing methods have limited ability to handle high-frequency noise interference, resulting in poor smoothness of the posture sequence, which further affects the application value of the estimation results. At the same time, how to reduce the computational complexity while ensuring the accuracy of the model to meet real-time requirements is also an important problem facing current technology. Summary of the invention

[0003] In response to the above problems, the present invention proposes a method and system for two-dimensional posture estimation in occluded scenes based on spatiotemporal information. By constructing a two-dimensional posture estimation model including an input module, a visibility prediction and shielding module, a key point reasoning module and a spatiotemporal dependency module, the present invention solves the problems of key point feature loss and low posture estimation accuracy in occluded scenes in existing video two-dimensional posture estimation technology. The method and system can accurately infer the occluded key points, improve the smoothness and accuracy of posture estimation, reduce high-frequency noise interference, ensure the accuracy of the model while reducing the computational complexity to meet real-time requirements.

[0004] On the one hand, the method for estimating the two-dimensional pose of an occluded scene based on spatiotemporal information has the following specific steps:

[0005] S1, obtaining a video including multiple human body information;

[0006] S2, constructing a two-dimensional posture estimation model; the model includes an input module, a visibility prediction and shielding module, a key point reasoning module, a spatiotemporal dependency module and a regression layer;

[0007] The input module identifies and crops individuals in the input video to obtain individual video clips of each individual, processes each individual one by one, and uses a convolutional backbone network to extract key point features of each frame of the current individual video clip; the key points are human body joints;

[0008] The visibility prediction and masking module performs visibility prediction on the input key point features to obtain mask features;

[0009] The key point reasoning module inputs mask features, completes the occluded key points, and outputs enhanced key point features;

[0010] The spatiotemporal dependency module processes the input enhanced key point features frame by frame, extracts the time-dependent update features of each key point of the current frame; extracts the space-dependent update features between each key point of the current frame, and fuses the time-dependent update features and space-dependent update features of each key point respectively, and outputs the enhanced spatiotemporal features frame by frame;

[0011] The regression layer processes the enhanced spatiotemporal features frame by frame to perform two-dimensional posture estimation to obtain a two-dimensional posture estimation sequence of the current individual; processes each individual to obtain a two-dimensional posture estimation sequence of all individuals; and the two-dimensional posture estimation model is optimized using a residual likelihood estimation loss function;

[0012] S3, inputting the acquired video into a two-dimensional posture estimation model to obtain a trained two-dimensional posture estimation model;

[0013] S4, using the trained two-dimensional pose estimation model to perform two-dimensional pose estimation of the occluded scene.

[0014] Preferably, the two-dimensional posture estimation model also includes a discrete cosine filtering module; the discrete cosine filtering module takes a two-dimensional posture estimation sequence as input, converts the two-dimensional posture estimation sequence into discrete cosine coefficients, uses a low-pass filter to retain a preset number of low-frequency discrete cosine coefficients, and then obtains a two-dimensional posture sequence with high-frequency noise removed through a discrete inverse cosine transform, and outputs the two-dimensional posture sequence with high-frequency noise removed as the output of the two-dimensional posture estimation model.

[0015] Preferably, the visibility prediction and masking module performs visibility prediction on the input key point features to obtain mask features, and the specific steps are as follows:

[0016] First, a lightweight multi-layer perceptron module is used to train key point features. Perform visibility prediction to obtain the individual key points Fraction ;

[0017] According to each key point of the individual Score, based on the preset classification threshold to obtain the visibility mask, the formula is as follows:

[0018] ;

[0019] in, Represents the visibility mask; express function; Represents a lightweight multilayer perceptron module; Represents key point features; Indicates the preset classification threshold;

[0020] Then, the visibility mask is multiplied element-wise with the key point feature to obtain the mask feature, and the formula is as follows:

[0021] ;

[0022] in, Represents mask features; Represents element-wise multiplication.

[0023] Preferably, the key point reasoning module adopts network; The network consists of 4 identical The blocks are stacked in sequence; each of the The block includes a multi-head self-attention module and a multi-layer perception module; the The network processing steps are as follows:

[0024] First, the mask features are input into the multi-head self-attention module; the self-attention head of the multi-head self-attention module is expressed as:

[0025] ;

[0026] in, Indicates self-attention head; express function; , and They represent the linear projection layer respectively; Represents the dimension of each mask feature; express The transpose of

[0027] The multi-head self-attention module combines self-attention head, expressed as:

[0028] ;

[0029] in, It indicates long self-attention; represents the mth self-attention head; Represents a linear projection layer;

[0030] Secondly, perform residual connection and layer normalization, the formula is as follows:

[0031] ;

[0032] in, Represents the output result of residual connection and layer normalization; Representation layer normalization operation;

[0033] Then, the multi-layer perception module is input, which is expressed as:

[0034] ;

[0035] in, Represents the input vector The result of performing feed-forward neural network operations; and Respectively represent the linear transformation matrix; and They represent bias terms respectively; express Activation function;

[0036] Finally, residual connection and layer normalization are performed to obtain enhanced key point features. The formula is as follows:

[0037] ;

[0038] in, Represents enhanced key point features.

[0039] Preferably, the enhanced spatiotemporal features are expressed as , as follows:

[0040] , ;

[0041] in, Indicates the joint j of individual i in the current frame The spatiotemporal aggregation characteristics of , Indicates the time span, represents the set of all joints; Indicates a serial operation; Indicates the joint j of individual i in the current frame Time-dependent update features; Indicates the joint j of individual i in the current frame The spatial dependency update features of ; n represents the number of joints.

[0042] Preferably, a first local perception attention module is used to acquire time-dependent update features; the first local perception attention module includes a second Transformer network; the second Transformer network includes four identical The blocks are stacked in order, each The block includes a multi-head self-attention module and a multi-layer perception module; it is represented as follows:

[0043] ;

[0044] in, Represents an individual The enhanced key point features of the j-th joint; Represents local perceptual attention.

[0045] Preferably, a second local perception attention module is used to acquire spatially dependent update features; the second local perception attention module includes a third network; the third The network consists of 4 identical The blocks are stacked in order, each The block includes a multi-head self-attention module and a multi-layer perception module; specifically as follows:

[0046] Based on the semantic structure of human posture, the enhanced key point features are divided into Group, use local-aware attention for each group, expressed as follows:

[0047] , ;

[0048] in, express Group joint indicator set; represents the spatially dependent update feature of the kth group; Indicates local perceptual attention; Indicates the joint j of individual i in the current frame The enhanced key point features, represents the kth group .

[0049] Preferably, the regression layer includes fully connected layers equal to the number of joints, each fully connected layer independently processes the features of a joint and does not share network parameters with other fully connected layers. The regression layer outputs the coordinates of each joint in the current frame to complete two-dimensional posture estimation.

[0050] On the other hand, the system for estimating two-dimensional pose of an occluded scene based on spatiotemporal information includes the following:

[0051] A data acquisition module, used for acquiring a video including a plurality of human body information;

[0052] A model building module, used to build a two-dimensional posture estimation model; the model includes an input module, a visibility prediction and shielding module, a key point reasoning module, a spatiotemporal dependency module and a regression layer;

[0053] A model training module is used to input the acquired video into a two-dimensional posture estimation model for training to obtain a trained two-dimensional posture estimation model;

[0054] A two-dimensional pose estimation module is used to perform two-dimensional pose estimation of occluded scenes using a trained two-dimensional pose estimation model;

[0055] The input module identifies and crops individuals in the input video to obtain individual video clips of each individual, uses a convolutional backbone network to extract key point features of each frame of each individual video clip in turn, and outputs them one by one to the visibility prediction and shielding module;

[0056] The visibility prediction and masking module performs visibility prediction on the input key point features to obtain mask features and outputs them to the key point reasoning module;

[0057] The key point reasoning module completes the occluded key points based on the input mask features, obtains enhanced key point features, and outputs them to the spatiotemporal dependency module;

[0058] The spatiotemporal dependency module processes the input enhanced key point features frame by frame, extracts the time-dependent update features of each key point in the current frame and the space-dependent update features between each key point, and respectively fuses the time-dependent update features and space-dependent update features of each key point to obtain enhanced spatiotemporal features, and outputs them frame by frame;

[0059] The regression layer performs two-dimensional posture estimation based on the enhanced spatiotemporal features processed frame by frame to obtain a two-dimensional posture estimation sequence of the current individual; and processes each individual to obtain a two-dimensional posture estimation sequence of all individuals.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention constructs a two-dimensional posture estimation model including an input module, a visibility prediction and shielding module, a key point reasoning module, a spatiotemporal dependency module, a regression layer and a discrete cosine filtering module. It can accurately infer the occluded key points, improve the smoothness and accuracy of posture estimation, reduce the interference of high-frequency noise, ensure the accuracy of the model while reducing the computational complexity to meet real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The present invention is further described in detail below in conjunction with the accompanying drawings;

[0063] Figure 1 It is a flow chart of a method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of a flow chart of a method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to an embodiment of the present invention;

[0065] Figure 3 The structure block diagram of the system for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present invention is further described below through specific implementation modes.

[0067] like Figure 1 As shown in Figure 1, the method for estimating two-dimensional pose of occluded scenes based on spatiotemporal information is as follows:

[0068] S1, obtaining a video including multiple human body information;

[0069] The video is represented as ,in, Is a given time Video frames containing multiple people, is a predefined time span.

[0070] S2, construct a two-dimensional posture estimation model; the model includes an input module, a visibility prediction and shielding module, a key point reasoning module, a spatiotemporal dependency module and a regression layer. Figure 2 shown.

[0071] Input module: The input module identifies and crops individuals in the input video to obtain individual video clips of each individual. The convolutional backbone network is used to extract the key point features of each frame of each individual video clip in turn, and outputs them one by one to the visibility prediction and masking module, as follows:

[0072] Input module receives video . First use the human body detector in Individuals are identified in the sequence and the bounding box of each detected individual is expanded by 25% to extract the same individual in the sequence to form a cropped video segment ,in, is an individual in the video. Extract key point features of individuals through convolutional backbone network ,in , indicating that at time Individual Posture The key point features of the joints, Indicates at time Individual Posture All key point features of each joint.

[0073] Specifically, the convolutional backbone network is the core part of the convolutional neural network (CNN) in deep learning, which is responsible for extracting features from input data. It usually includes multiple convolutional layers, activation functions, and pooling layers, and extracts features from low-level to high-level by processing the image layer by layer.

[0074] The visibility prediction and shielding module predicts the visibility of the input key point features, obtains the mask features, and outputs them to the key point reasoning module, as follows:

[0075] First use the lightweight multi-layer perceptron module To capture the visibility state of a single key point, a lightweight multi-layer perceptron module is constructed for a given input In the case of Fraction By using Function. The formula is as follows:

[0076] ;

[0077] in, is an unknown number. The score is converted into a probability, where values ​​close to 0 represent invisible keypoints and values ​​close to 1 represent visible keypoints. Score, apply a classification threshold of 0.5 to obtain the visibility mask, which is expressed as , the visibility mask is a length of of Vector. The visibility mask function is as follows:

[0078] ;

[0079] Then, the visibility mask is applied to the key point features to mask out the corresponding features of the invisible key points. With key point features The element-wise multiplication of the mask feature is called The formula is as follows:

[0080] ;

[0081] in, Indicates that and Each pair of corresponding elements in is multiplied, and the obtained mask feature shields the influence of the features of the occluded key points, thereby improving the accuracy of pose estimation.

[0082] Specifically, the lightweight multilayer perceptron module uses , Designed for predicting the visibility status (visible or occluded) of key points of the human body, it performs binary classification on the key point features of the input. Output the visibility probability of each key point to generate a visibility mask, thereby shielding the features of the occluded key points and preventing them from interfering with model reasoning. By explicitly introducing visibility information, It can effectively reduce the negative impact of occlusion on pose estimation and greatly improve the prediction accuracy and robustness in complex scenes.

[0083] Key point reasoning module, based on the input mask features, the key point reasoning module completes the occluded key points, obtains enhanced key point features, and outputs them to the spatiotemporal dependency module, as follows:

[0084] The key point reasoning module inputs mask features using The network is used to infer the occluded key points. The final output features contain the features of all key points. The occluded key points are inferred and the visible key points are optimized. As input, it outputs enhanced keypoint features through visibility-aware attention. .in, The network uses the original 4 identical The blocks are stacked in order as Network. Each block contains a multi-head self-attention module and a multi-layer perception module. It is expressed as:

[0085] ;

[0086] ;

[0087] in, , , is the linear projection layer, For each mask feature Dimension.

[0088] The multi-head self-attention module combines self-attention head, expressed as:

[0089] ;

[0090] in, is the dimension of each self-attention head, It is a linear projection layer.

[0091] Perform residual connection and layer normalization:

[0092] ;

[0093] in, is the output result of residual connection and layer normalization, is the layer normalization operation.

[0094] The multi-layer perception module is expressed as:

[0095] ;

[0096] in, Represents the input vector The result of performing a feed-forward neural network operation. , is the linear transformation matrix; , is the bias term. express Activation function.

[0097] Perform residual connection and layer normalization to obtain enhanced key point features , expressed as:

[0098] ;

[0099] For time span The current individual on the enhanced key point features It can also be expressed as:

[0100] ;

[0101] in, Indicates at time Individual Posture The enhanced key point features of the joints are Indicates at time Individual Posture All key point features of each joint.

[0102] The spatiotemporal dependency module processes the input enhanced key point features frame by frame, extracts the time-dependent update features of each key point in the current frame and the space-dependent update features between each key point, and fuses the time-dependent update features and space-dependent update features of each key point to obtain enhanced spatiotemporal features, which are output frame by frame, as follows:

[0103] The spatiotemporal dependency module is based on the joints in the time span Feature embedding on Separately model the temporal dynamic dependencies and spatial structural dependencies, i.e. Modules and module, thereby capturing the temporal and spatial dependencies between joints, and then deriving the current frame by fusing the captured spatial and temporal information. The aggregated spatiotemporal features of each joint are expressed as:

[0104] ;

[0105] in, represents a concatenation operation, applied separately to each pair of corresponding updated feature labels associated with each joint.

[0106] module. The module identifies the temporal dependencies of each joint, thereby generating another updated feature for each joint in the current frame. In the module, the current frame is captured Each joint The time dynamic dependency of Selectively apply the local-aware attention module .in, The configuration follows the key point inference module Network: Connect 4 identical The blocks are stacked in sequence, each containing a multi-head self-attention module and a multi-layer perception module. The input features pass through these modules in sequence, and each module produces an updated version as the input of the subsequent layer. That is:

[0107] ;

[0108] in, , indicating individual The enhanced key point features of the j-th joint, Indicates the current video frame Previous The updated features of the joints are encoded and embedded in the sequence The time-dependent information of the joint in .

[0109] module. The module also uses the local perception attention module , learns the spatial dependencies between adjacent joints and generates updated features for each joint in the current frame accordingly. In the module, capture the current frame The spatial structure dependencies between internal joints. According to the semantic structure of human posture, the joints are divided into groups, use , expressed as:

[0110] , ;

[0111] in, express Group joint indicator set, Indicates the current video frame Middle The updated feature tag of each joint contains the spatial structure dependency of the joint in the pose.

[0112] pass and modules , captures the spatiotemporal context of each joint in the current frame and obtains the corresponding updated features and Therefore, the current frame can be explicitly calculated Each joint The spatiotemporal aggregation characteristics of :

[0113] , ;

[0114] Output spatiotemporal aggregation features Combining temporal dynamics and spatial structural information enables subsequent models to more fully understand the input data.

[0115] The regression layer performs two-dimensional posture estimation based on the spatiotemporal information features enhanced by frame-by-frame processing to obtain the two-dimensional posture estimation sequence of the current individual; the two-dimensional posture estimation sequence of all individuals is obtained by processing each individual, as follows:

[0116] The regression layer is a fully connected network, inputting spatiotemporal aggregation features , the output is the current frame Middle The coordinates of the joints ,Right now:

[0117] ;

[0118] in, It is a joint-by-joint fully connected layer, that is, for each joint Use a single fully connected layer to process , the fully connected layer of each joint works independently, only processes the features related to the joint, and does not share network parameters with the features of other joints. Process the spatiotemporal aggregation features output by the spatiotemporal dependency module frame by frame to obtain the video clip In time span The pose estimation coordinates of all video frames on the , i.e., a 2D pose sequence .

[0119] The 2D posture estimation model also includes a discrete cosine filter module. The input of the discrete cosine filter module is a 2D posture sequence ,in, is the length of the 2D pose sequence. Converted into discrete cosine coefficients, using Then, a low-pass filter is applied to each joint trajectory to keep only the first Low frequency discrete cosine coefficients , discarding coefficients representing high-frequency noise, ensuring that key time information is preserved, and excluding high-frequency components that may cause data distortion or interference. The discrete inverse cosine transform is performed to obtain the filtered two-dimensional posture sequence. The discrete cosine filtering module removes unnecessary high-frequency noise and retains the temporal information and dynamic characteristics of the original sequence to the greatest extent, making the processed data smoother and more accurate.

[0120] S3, inputting the acquired video into the two-dimensional posture estimation model to obtain a trained two-dimensional posture estimation model;

[0121] S4, using the trained two-dimensional pose estimation model to perform two-dimensional pose estimation of the occluded scene based on spatiotemporal information.

[0122] like Figure 3 As shown, the present invention also discloses a system for estimating two-dimensional posture of an occluded scene based on spatiotemporal information, comprising:

[0123] The data acquisition module 301 is used to acquire a video including multiple human body information.

[0124] The model building module 302 is used to build a two-dimensional posture estimation model; the model includes an input module, a visibility prediction and shielding module, a key point reasoning module, a spatiotemporal dependency module and a regression layer.

[0125] The model training module 303 is used to input the acquired video into the two-dimensional posture estimation model for training to obtain a trained two-dimensional posture estimation model.

[0126] The two-dimensional pose estimation module 304 is used to perform two-dimensional pose estimation of the occluded scene using the trained two-dimensional pose estimation model.

[0127] The input module identifies and crops individuals in the input video to obtain individual video clips of each individual, uses a convolutional backbone network to extract key point features of each frame of each individual video clip in turn, and outputs them one by one to the visibility prediction and shielding module.

[0128] The visibility prediction and masking module performs visibility prediction on the input key point features, obtains mask features, and outputs them to the key point reasoning module.

[0129] The key point reasoning module completes the occluded key points based on the input mask features, obtains enhanced key point features, and outputs them to the spatiotemporal dependency module.

[0130] The spatiotemporal dependency module processes the input enhanced key point features frame by frame, extracts the time-dependent update features of each key point in the current frame and the space-dependent update features between each key point, and respectively fuses the time-dependent update features and space-dependent update features of each key point to obtain enhanced spatiotemporal features, and outputs them frame by frame.

[0131] The regression layer performs two-dimensional posture estimation based on the enhanced spatiotemporal features processed frame by frame to obtain a two-dimensional posture estimation sequence of the current individual; and processes each individual to obtain a two-dimensional posture estimation sequence of all individuals.

[0132] The specific implementation of the system for estimating two-dimensional posture of an occluded scene based on spatiotemporal information is the same as the method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information, and will not be repeated in this embodiment.

[0133] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A method for estimating two-dimensional pose of an occluded scene based on spatiotemporal information, characterized in that: The steps include: S1, obtaining a video including multiple human body information; S2, constructing a two-dimensional posture estimation model; the model includes an input module, a visibility prediction and shielding module, a key point reasoning module, a spatiotemporal dependency module and a regression layer; The input module identifies and crops individuals in the input video to obtain individual video clips of each individual, uses a convolutional backbone network to extract key point features of each frame of each individual video clip in turn, and outputs them one by one to the visibility prediction and shielding module; The visibility prediction and masking module performs visibility prediction on the input key point features to obtain mask features and outputs them to the key point reasoning module; The key point reasoning module completes the occluded key points based on the input mask features, obtains enhanced key point features, and outputs them to the spatiotemporal dependency module; The spatiotemporal dependency module processes the input enhanced key point features frame by frame, extracts the time-dependent update features of each key point in the current frame and the space-dependent update features between each key point, and respectively fuses the time-dependent update features and space-dependent update features of each key point to obtain enhanced spatiotemporal features, and outputs them frame by frame; The regression layer performs two-dimensional posture estimation based on the enhanced spatiotemporal features processed frame by frame to obtain a two-dimensional posture estimation sequence of the current individual; and processes each individual to obtain a two-dimensional posture estimation sequence of all individuals; S3, inputting the acquired video into a two-dimensional posture estimation model for training to obtain a trained two-dimensional posture estimation model; S4, using the trained two-dimensional pose estimation model to perform two-dimensional pose estimation of the occluded scene.

2. The method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to claim 1, characterized in that: The two-dimensional posture estimation model also includes a discrete cosine filtering module; the discrete cosine filtering module takes a two-dimensional posture estimation sequence as input, converts the two-dimensional posture estimation sequence into discrete cosine coefficients, uses a low-pass filter to retain a preset number of low-frequency discrete cosine coefficients, and then obtains a two-dimensional posture sequence with high-frequency noise removed through a discrete inverse cosine transform, and outputs the two-dimensional posture sequence with high-frequency noise removed as the output of the two-dimensional posture estimation model.

3. The method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to claim 1, characterized in that: The visibility prediction and masking module performs visibility prediction on the input key point features to obtain mask features. The specific steps are as follows: First, a lightweight multi-layer perceptron module is used to train key point features. Perform visibility prediction to obtain the individual key points Fraction ; According to each key point of the individual Score, based on the preset classification threshold to obtain the visibility mask, the formula is as follows: ； in, Represents the visibility mask; express function; Represents a lightweight multilayer perceptron module; Represents key point features; Indicates the preset classification threshold; Then, the visibility mask is multiplied element-by-element with the key point feature to obtain the mask feature, as shown below: ； in, Represents mask features; Represents element-wise multiplication.

4. The method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to claim 1, characterized in that: The key point reasoning module adopts network; The network consists of 4 identical The blocks are stacked in sequence; each of the The block includes a multi-head self-attention module and a multi-layer perception module; the The network processing steps are as follows: First, the mask features are input into the multi-head self-attention module; the self-attention head of the multi-head self-attention module is expressed as: ； in, Indicates self-attention head; express function; , and They represent the linear projection layer respectively; Represents the dimension of each mask feature; express The transpose of The multi-head self-attention module combines self-attention head, expressed as: ； in, It indicates long self-attention; represents the mth self-attention head; Represents a linear projection layer; Secondly, perform residual connection and layer normalization, the formula is as follows: ； in, Represents the output result of residual connection and layer normalization; Representation layer normalization operation; Then, the multi-layer perception module is input, which is expressed as: ； in, Represents the input vector The result of performing feed-forward neural network operations; and Respectively represent the linear transformation matrix; and They represent bias terms respectively; express Activation function; Finally, residual connection and layer normalization are performed to obtain enhanced key point features. The formula is as follows: ； in, Represents enhanced key point features.

5. The method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to claim 1, characterized in that: The enhanced spatiotemporal features are expressed as , as follows: ， ； in, Indicates the joint j of individual i in the current frame The spatiotemporal aggregation characteristics of , Indicates the time span, represents the set of all joints; Indicates a serial operation; Indicates the joint j of individual i in the current frame Time-dependent update features; Indicates the joint j of individual i in the current frame The spatial dependency update features of ; n represents the number of joints.

6. The method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to claim 5, characterized in that: The first local perception attention module is used to acquire time-dependent update features; the first local perception attention module includes a second Transformer network; the second Transformer network includes 4 identical The blocks are stacked in order, each The block includes a multi-head self-attention module and a multi-layer perception module; it is represented as follows: ； in, Represents an individual The enhanced key point features of the j-th joint; Represents local perceptual attention.

7. The method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to claim 5, characterized in that: The second local perception attention module is used to acquire the spatial dependent update feature; the second local perception attention module includes a third network; the third The network consists of 4 identical The blocks are stacked in order, each The block includes a multi-head self-attention module and a multi-layer perception module; specifically as follows: Based on the semantic structure of human posture, the enhanced key point features are divided into Group, use local-aware attention for each group, expressed as follows: ， ； in, express Group joint indicator set; represents the spatially dependent update feature of the kth group; Indicates local perceptual attention; Indicates the joint j of individual i in the current frame The enhanced key point features, represents the kth group .

8. The method for estimating two-dimensional posture of an occluded scene based on spatiotemporal information according to claim 1, characterized in that: The regression layer includes fully connected layers equal to the number of joints. Each fully connected layer independently processes the features of a joint and does not share network parameters with other fully connected layers. The regression layer outputs the coordinates of each joint in the current frame to complete two-dimensional posture estimation.

9. A system for estimating two-dimensional pose of an occluded scene based on spatiotemporal information, comprising: A data acquisition module, used for acquiring a video including a plurality of human body information; A model building module, used to build a two-dimensional posture estimation model; the model includes an input module, a visibility prediction and shielding module, a key point reasoning module, a spatiotemporal dependency module and a regression layer; A model training module is used to input the acquired video into a two-dimensional posture estimation model for training to obtain a trained two-dimensional posture estimation model; A two-dimensional pose estimation module is used to perform two-dimensional pose estimation of occluded scenes using a trained two-dimensional pose estimation model; The input module identifies and crops individuals in the input video to obtain individual video clips of each individual, uses a convolutional backbone network to extract key point features of each frame of each individual video clip in turn, and outputs them one by one to the visibility prediction and shielding module; The visibility prediction and masking module performs visibility prediction on the input key point features to obtain mask features and outputs them to the key point reasoning module; The key point reasoning module completes the occluded key points based on the input mask features, obtains enhanced key point features, and outputs them to the spatiotemporal dependency module; The spatiotemporal dependency module processes the input enhanced key point features frame by frame, extracts the time-dependent update features of each key point in the current frame and the space-dependent update features between each key point, and respectively fuses the time-dependent update features and space-dependent update features of each key point to obtain enhanced spatiotemporal features, and outputs them frame by frame; The regression layer performs two-dimensional posture estimation based on the enhanced spatiotemporal features processed frame by frame to obtain a two-dimensional posture estimation sequence of the current individual; and processes each individual to obtain a two-dimensional posture estimation sequence of all individuals.

Images