Video style migration attitude detection method based on graph neural network

Abstract

The invention discloses a video style migration attitude detection method based on a graph neural network, which belongs to the technical field of video processing, and specifically comprises the following steps: preprocessing a video data stream; constructing a dynamic space-time association network, mapping pixels of the video frame into space-time diagram nodes, and converting the original video frame into a target artistic style by aggregating the space-time features of the nodes layer by layer; in the style migration process, a dynamic mask is generated for the area where the human body joint points are located, and the geometric consistency of the joint structures in stylized output is restrained; extracting joint point coordinates of each frame from the stylized video, constructing a space-time trajectory sequence, and generating a feature vector containing space-time dependence; performing multi-dimensional comparison on the feature vector and a preset action mode library, calculating a track offset and a mode matching degree, judging that the action is abnormal according to a threshold value, and outputting a detection result; according to the invention, the cooperative performance of style migration and anomaly detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video style transfer posture detection method based on graph neural network. Background Art

[0002] With the booming development of the Internet, pictures and videos have been deeply integrated into people's lives. Creative artists and designers actively explore and incorporate artistic elements of different styles into their works, giving rise to the highly anticipated field of style transfer, which has shown broad prospects in film and television special effects, virtual reality, etc. Style transfer can transfer the style features of an image, such as color, texture, composition, etc., to another image, making the latter appear to be a re-rendering of the former in a new style.

[0003] However, video data has complex spatiotemporal characteristics and highly nonlinear mapping relationships, which are difficult for traditional machine learning algorithms to effectively capture and process. Moreover, most existing style transfer methods only focus on optimizing the visual quality of a single frame and lack modeling of the consistency of continuous movements, resulting in problems such as human posture distortion and joint mutation after style transfer. Traditional solutions rely on optical flow or temporal smoothing constraints, but are difficult to deal with topological structure distortion in complex motion scenes.

[0004] In recent years, graph neural networks (GNNs) have made some progress in the field of human pose estimation by virtue of their advantages in modeling non-Euclidean data. However, most existing graph convolution methods use a fixed adjacency matrix, which cannot adaptively capture the dynamic changes in joint relationships caused by style transfer, and are not robust enough in occlusion or fast motion scenarios. In addition, spatiotemporal models based on pure convolutions are difficult to take into account both long-range temporal dependencies and local structural features at the same time. Summary of the invention

[0005] The purpose of the present invention is to provide a video style transfer posture detection method based on graph neural network to solve the following technical problems: Most existing graph convolution methods use a fixed adjacency matrix, which cannot adaptively capture the dynamic changes in joint relationships caused by style transfer, and are not robust enough in occlusion or fast motion scenarios.

[0006] The purpose of the present invention can be achieved through the following technical solutions: A video style transfer posture detection method based on graph neural network includes the following steps: Parse the original video frames frame by frame, extract continuous video sequences, dynamically adjust the resolution according to the scene and divide it into local areas to generate pre-processed video data streams; Construct a dynamic spatiotemporal association network to map the pixels of video frames into spatiotemporal graph nodes. The connection weights between nodes are dynamically calculated based on the motion trends of adjacent frames and the complexity of regional content. The original video frames are converted into the target artistic style by aggregating the spatiotemporal features of nodes layer by layer. In the process of style transfer, a dynamic mask is generated for the area where the human body joints are located, and local features of the corresponding areas of adjacent frames are fused based on the mask coverage to constrain the geometric consistency of the joint structure in the stylized output; Extract the coordinates of the joint points of each frame from the stylized video, construct a spatiotemporal trajectory sequence, learn the motion pattern and spatial topological relationship of the joint points through a dynamic spatiotemporal association network, and generate a feature vector containing spatiotemporal dependencies; The feature vector is compared with the preset action pattern library in multiple dimensions, the trajectory offset and pattern matching degree are calculated, the action abnormality is determined according to the threshold and the detection result is output.

[0007] As a further solution of the present invention: the construction of the dynamic spatiotemporal association network specifically includes: The pixels of the video frame are divided into several local areas, each of which is mapped to a graph node. The node attributes include color gradient, motion vector and local texture entropy value. The motion vector is generated by optical flow field calculation, and the texture entropy value reflects the complexity of details in the area. In the time dimension, cross-frame connections are established for corresponding nodes in adjacent frames, and the connection weight is dynamically calculated based on the spatiotemporal correlation of node attributes: if the motion vector directions of nodes in adjacent frames are consistent and the texture entropy value difference is less than the adaptive threshold, the weight is increased, otherwise it is decreased; In the spatial dimension, hierarchical connections are constructed for human joints in the same frame: the first-level connection is defined based on anatomical priors, the second-level connection is defined based on motion synergy, and the weights of the connections between the levels are dynamically adjusted according to the range of joint motion. Through the spatiotemporal graph convolution operation, the dynamic connection features across frames and levels are fused layer by layer to generate spatiotemporal correlation features that combine style transfer and motion perception.

[0008] As a further solution of the present invention: the generation of the dynamic mask includes: Based on the preprocessed video frames, a lightweight posture estimation network is used to predict the human joints in real time, and an initial mask is generated with the joints as the center. The mask radius expands linearly with the joint movement speed. Combining the optical flow information with the joint trajectory prediction model, the deformation parameters of the next frame mask are inferred. The deformation direction is consistent with the joint movement direction, and the deformation amount is proportional to the movement acceleration. In the process of style transfer, the mask coverage area is subject to dual constraints, including geometric constraints and semantic constraints. The geometric constraints strengthen the contour feature extraction of the joints in the mask through a dynamic spatiotemporal association network to suppress the edge blurring caused by stylization; the semantic constraints compare the semantic segmentation results of the mask area before and after style transfer. If the key anatomical structure is lost, the local stylization strength attenuation is triggered. When it is detected that the motion speed of a joint exceeds a threshold or is occluded, a temporary mask is generated based on the mask trajectory interpolation of the historical frames, and the mask range is automatically expanded to cover the motion blur area.

[0009] As a further solution of the present invention: the construction of the spatiotemporal trajectory sequence includes: Extract the coordinates of joint points from each frame of the stylized video, construct the original trajectory sequence in chronological order, and perform segmented denoising on the sequence. Use sliding average filtering to smooth jitter noise in low-speed motion segments; retain the original data for high-speed motion segments and only remove outliers. An adaptive time window is allocated to each joint point, and the window length is dynamically adjusted according to the motion acceleration: the greater the acceleration, the shorter the window length; In the spatiotemporal correlation network, multi-scale feature extraction is performed on the trajectory data in the window: the multi-scale features are fused with the hierarchical connection weights to generate a spatiotemporal feature vector containing local details and global patterns.

[0010] As a further solution of the present invention: the construction of the spatiotemporal trajectory sequence further includes: Extracting visual modality features from the stylized video, including color distribution, edge sharpness, and texture density, and generating a first feature vector through normalization; Extract motion modal features from the spatiotemporal trajectory sequence, including joint displacement, acceleration, and motion direction consistency, and generate a second eigenvector through dynamic time warping; Construct a cross-modal association matrix, align the dimensions of the first eigenvector and the second eigenvector, and fuse the two types of features through a dynamic weight allocation strategy. The weight value is adjusted in real time according to the scene lighting intensity, motion complexity, and stylized interference degree; During the fusion process, if it is detected that the visual modality features are distorted due to style transfer, local feature compensation is performed based on the spatiotemporal continuity of the motion modality features to ensure that the fused feature vector reflects both the visual style and the essence of the action. When performing anomaly detection on the fused feature vector, a conflict resolution mechanism is introduced: when the judgment results of the two modes are inconsistent, the detection results of the motion mode are adopted first, and the results are verified by tracing the consistency of the trajectories of adjacent frames.

[0011] As a further solution of the present invention: extracting the coordinates of the joint points of each frame and the color distribution of the corresponding pixel area from the stylized video, calculating the difference between the motion trajectory offset of the joint points and the color distribution of the stylized area, and generating correction parameters; The node connection weights of the dynamic spatiotemporal association network are optimized by gradient descent according to the correction parameters, and the connection weights of the nodes with trajectory offsets exceeding the threshold value and the adjacent nodes are reduced; For pixel areas where the color distribution difference exceeds the set range, recalculate the spatiotemporal correlation with the corresponding areas of adjacent frames and update the feature aggregation rules of local nodes; The optimized node connection weights and feature aggregation rules are fed back to the dynamic spatiotemporal association network to generate updated style migration results and spatiotemporal trajectory sequences, which are iterated until the trajectory offset and color difference converge to the preset range.

[0012] As a further solution of the present invention: the determination of abnormal action specifically includes: The preset action pattern library contains the spatiotemporal feature vectors, trajectory deviation tolerances, and motion direction constraints of standard actions; Align the spatiotemporal feature vector generated in real time with the features in the pattern library dimension by dimension, and calculate the trajectory offset, direction deviation angle, and motion acceleration difference; If the trajectory deviation exceeds the tolerance, the direction deviation angle is greater than the set angle, or the acceleration difference exceeds the threshold, it is judged as an abnormal action; The judgment result is verified for multi-frame continuity, and the final result is output only when anomalies are detected in multiple consecutive frames.

[0013] As a further solution of the present invention: the pretreatment comprises: Perform dynamic region segmentation on the video frame, divide the interior of the video frame into high-detail area and low-detail area according to the content complexity threshold, and perform local super-resolution enhancement on the high-detail area; Extract the optical flow information of the video frame, generate a motion vector, and mark the high-motion area and the static area; dynamically adjust the preprocessing parameters according to the motion vector, perform noise reduction and blur suppression on the high-motion area, and perform color balance optimization on the static area; The preprocessed video data stream and the motion vector graph are synchronously input into the dynamic spatiotemporal association network to guide the network to optimize the focus on key areas.

[0014] Beneficial effects of the present invention: The present invention effectively solves the problems of traditional methods' difficulty in processing complex spatiotemporal characteristics of videos, lack of modeling of continuous action consistency, inability to adaptively capture dynamic changes in joint relationships, and insufficient robustness in complex scenarios through key technical features such as frame-by-frame analysis of original video frames, construction of dynamic spatiotemporal association networks, generation of dynamic masks, construction of spatiotemporal trajectory sequences, and action anomaly judgment. It achieves high-precision posture recognition, strong stability and reliability, can maintain the geometric consistency of joint structures after style transfer, accurately judge action anomalies, and further improves the robustness of detection results and algorithm performance through multimodal fusion, preprocessing, and evaluation optimization mechanisms. It can iteratively improve the synergistic performance of style transfer and anomaly detection and realize model self-evolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below in conjunction with the accompanying drawings.

[0016] Figure 1 It is a schematic diagram of the structure of the basic network ST-GCN used in the present invention to process the task of human posture detection; Figure 2 is a flow chart of the spatial attention module of the present invention; Figure 3 It is a schematic diagram of the overall network structure of the style transfer posture detection algorithm of the present invention; Figure 4 It is a flow chart of the style transfer network of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Embodiment 1, the present invention is a video style transfer posture detection method based on graph neural network, comprising the following steps: 1.1 Original video frame analysis and continuous video sequence extraction: The original video is parsed frame by frame, and each frame of the video is arranged in chronological order through accurate timestamp and frame number recognition. On this basis, a specific algorithm is used to detect the scene switching points in the video, so as to accurately extract the continuous video sequence. By analyzing multi-dimensional information such as color histograms, brightness changes, and feature point matching between adjacent frames, the algorithm can effectively identify significant changes at the moment of scene switching, ensuring the integrity and accuracy of the continuous video sequence.

[0019] 1.2 Video frame dynamic region segmentation and super-resolution enhancement: For each frame of video, a method based on content complexity calculation is used to perform dynamic region segmentation. Specifically, the gradient amplitude, texture complexity and other features of the image block are calculated and compared with the pre-set content complexity threshold. For areas with complexity higher than the threshold, they are determined to be high-detail areas; otherwise, they are low-detail areas.

[0020] For high-detail areas, local super-resolution enhancement technology is used. This technology is based on a deep learning algorithm. By constructing a neural network model containing multiple convolutional layers and deconvolutional layers, the image blocks in the high-detail areas are extracted and reconstructed. During the training process, the model learns the mapping relationship between a large number of high-resolution images and the corresponding low-resolution images. In practical applications, it can restore low-resolution image blocks in high-detail areas to high resolution, significantly improving the clarity of details.

[0021] 1.3 Optical flow information extraction and motion area marking: The optical flow information of the video frame is extracted using classic optical flow estimation algorithms, such as the Lucas-Kanade algorithm or the Farneback algorithm. These algorithms calculate the motion vector of each pixel by tracking the motion trajectory of feature points between adjacent frames. Based on the obtained motion vectors, a motion vector map is generated.

[0022] Based on the motion vector map, a motion amplitude threshold is set to divide the video frame into high motion areas and static areas. Areas with motion vector amplitudes greater than the threshold are marked as high motion areas, while areas with motion vector amplitudes close to zero are static areas.

[0023] 1.4 Dynamically adjust preprocessing parameters according to motion vector diagram: For high-motion areas, since fast motion easily introduces noise and produces blur, targeted noise reduction and blur suppression algorithms are used. In terms of noise reduction, a noise reduction method based on wavelet transform is used to perform wavelet decomposition on images in high-motion areas, remove noise components in different frequency sub-bands, and then perform wavelet reconstruction. In terms of blur suppression, deconvolution technology is used to perform inverse operations on blurred images according to the direction and size of the motion vector to restore the clear edges of the image.

[0024] For static areas, we focus on color balance optimization. By calculating the color histogram of the static area image, we use the histogram equalization algorithm to adjust the color distribution, making the image color more uniform and rich, and improving the visual effect.

[0025] 1.5 Pre-processed video data stream and motion vector diagram synchronous input: The video data stream obtained through the above series of preprocessing steps is synchronized with the motion vector map to ensure a one-to-one correspondence between the two in the time dimension. Then, the synchronized information is input into the dynamic spatiotemporal association network. The network can perform in-depth analysis and processing of the video content in the spatiotemporal dimension based on the key area information provided by the motion vector map, realize the focus optimization of the key area, and thus improve the accuracy and efficiency of video processing.

[0026] 2.1 Video frame area division and node mapping: Finely divide the pixels of the video frame into many local areas of appropriate size. The size of these areas needs to take into account the complexity of the video content and the subsequent computing efficiency. The best size can usually be determined through experiments, such as dividing the video frame into several small areas of 16x16 pixels. Each local area is mapped to a graph node, thus building the basis of the graph structure.

[0027] Each node is given rich attributes. The color gradient is obtained by calculating the rate of change of the pixel color in the region in the horizontal and vertical directions, which can reflect the color change trend in the region. The motion vector is generated with the help of optical flow field calculation. Common optical flow estimation algorithms such as the Lucas-Kanade algorithm or the Farneback algorithm accurately calculate the movement direction and displacement of each pixel by tracking the motion trajectory of feature points between adjacent frames, and then obtain the motion vector of the region. The texture entropy value is used to reflect the complexity of the details in the region. It is obtained by calculating the information entropy of the distribution of pixel grayscale values ​​in the region. The higher the entropy value, the richer the texture details in the region and the more complex the changes.

[0028] 2.2 Cross-frame connection construction and weight calculation: In the time dimension, cross-frame connections are established for the corresponding nodes of adjacent frames to capture the dynamic changes of the video in the time series. For each pair of corresponding nodes in adjacent frames, the connection weight is not fixed, but is dynamically calculated based on the spatiotemporal correlation of node attributes.

[0029] In the specific calculation process, the motion vector directions of the nodes in adjacent frames are first compared. If the directions of the two are consistent, it means that the region has a relatively stable motion trend in time. At the same time, the texture entropy values ​​of the nodes are compared. If the difference in texture entropy values ​​is less than the threshold set adaptively according to the video content (the threshold can be adjusted dynamically by statistical analysis of the overall texture entropy value of the video combined with the average texture entropy value of the local area), it indicates that the texture details of the region change little in time and have a high spatiotemporal correlation. In this case, the corresponding cross-frame connection weight will be increased, otherwise it will be reduced. Through this dynamic weight adjustment mechanism, the key information transmission of the video in the time dimension can be more accurately reflected.

[0030] 2.3 Hierarchical connection construction and weight adjustment of human joints in the same frame: In the spatial dimension, hierarchical connections are constructed for human joints in the same frame. Based on anatomical prior knowledge, the first-level connection is defined as the connection between adjacent joints, such as shoulder-elbow, elbow-wrist, etc. These connections can directly reflect the basic composition and movement relationship of the human limb structure.

[0031] Based on the synergy of movement, the secondary connection is defined, that is, the connection across limb joints, such as left wrist-right ankle, etc. This connection method takes into account the synergistic movement relationship between different limb parts during the movement of the human body, and can more comprehensively describe the overall movement pattern of the human body.

[0032] The weights of the connections between the layers are dynamically adjusted according to the range of motion of the joints. The range of motion of the joints is calculated by tracking the position changes of the joint points in the video sequence. For joint connections with large range of motion, higher weights are assigned to highlight the important role and dynamic changes of these joints in human movement. For joint connections with small range of motion, the weights are reduced accordingly, so as to reasonably distribute the importance of different joint connections in the constructed hierarchical connection structure.

[0033] 2.4 Spatiotemporal graph convolution operation and feature generation: The constructed dynamic spatiotemporal graph structure is processed layer by layer using spatiotemporal graph convolution operations. Spatiotemporal graph convolution can effectively fuse dynamic connection features across frames and levels. In each layer of convolution operations, specific convolution kernels are used to operate with graph nodes and their connection weights to aggregate the information of neighboring nodes to the central node, thereby achieving feature propagation and fusion.

[0034] During the convolution process, the cross-frame connection information in the time dimension and the hierarchical connection information in the spatial dimension are fully considered, so that the generated features can not only capture the dynamic changes of human body movements in the video, but also integrate the style features of different regions. After multiple layers of spatiotemporal graph convolution operations, spatiotemporal correlation features with both style transfer and motion perception are finally generated. These features can comprehensively and accurately describe the key information of the video content in the spatiotemporal dimension, providing a solid data foundation for subsequent applications such as video analysis, motion recognition, and style transfer.

[0035] 3.1 Initial mask generation: At the beginning of style transfer, a lightweight pose estimation network is used to perform real-time operations based on preprocessed video frame data. The network has been trained with a large amount of human pose data and can quickly and accurately predict the positions of human joints in video frames. These joints serve as key reference points for subsequent operations.

[0036] Generate an initial mask with each predicted joint point as the center. The shape of the mask is usually set to a circle, and the determination of its radius is closely related to the joint movement speed. Specifically, by tracking and analyzing the joint point positions in the current frame and the previous frames, the displacement of the joint per unit time is calculated, and the joint movement speed is obtained. The mask radius is linearly expanded according to the joint movement speed, that is, the faster the joint movement speed, the larger the mask radius expansion. For example, a basic radius value r0 is set in advance. When the joint movement speed is v, the mask radius r=r0+k*v, where k is the proportional coefficient, which can be determined through experimental optimization to ensure that the mask can reasonably cover the movement-related areas around the joint.

[0037] 3.2 Mask deformation parameter inference: Combining optical flow information and joint trajectory prediction model, the deformation parameters of the mask in the next frame are accurately inferred. Optical flow information is obtained through optical flow estimation algorithm, which reflects the movement of pixels in the video frame. The joint trajectory prediction model uses the position information of joint points in historical frames and adopts time series prediction algorithms such as Kalman filtering to predict the position of joints in future frames.

[0038] The deformation direction of the mask is consistent with the direction of joint movement, which is based on the actual situation that the joint movement drives the change of the surrounding area. The size of the deformation is proportional to the motion acceleration. The motion acceleration a is obtained by calculating the change in the joint motion speed of adjacent frames. Suppose the deformation of the mask in the current frame is Δx1, and the deformation of the next frame is Δx2. If the motion acceleration is a, then Δx2=Δx1+m*a, where m is a proportional factor related to the acceleration, which also needs to be optimized through experiments. This setting enables the mask to be adaptively deformed according to the dynamic changes of the joint movement, which is more in line with the actual motion state of the joint in the video sequence.

[0039] 3.3 Dual constraint mechanism: Geometric constraints: During the style transfer process, geometric constraints are imposed on the mask coverage area with the help of a dynamic spatiotemporal association network. The dynamic spatiotemporal association network can effectively capture the key information of the video in the spatiotemporal dimension. For the joint area within the mask, the network enhances the clarity of the joint contour during the stylization process by strengthening the contour feature extraction. For example, the convolutional layer in the network specifically extracts and amplifies the edge features of the joint contour to suppress the edge blurring that may be caused by the stylization operation. Through this geometric constraint, it is ensured that the joint still maintains a clear and accurate geometric contour in the visual presentation after style transfer.

[0040] Semantic constraint: Compare the semantic segmentation results of the mask area before and after style transfer to achieve semantic constraint. Semantic segmentation can divide different objects and areas in the video frame according to semantic categories. For mask areas related to human joints, focus on key anatomical structures, such as the outlines of the five fingers of the hand. If key anatomical structures are found to be missing in the semantic segmentation results after style transfer, the system will automatically trigger the local stylization intensity attenuation mechanism. The specific approach is to reduce the intensity of the stylization algorithm in the mask area so that the area retains the semantic features of the original video frame to a certain extent, avoiding the loss of important human structure information due to excessive stylization.

[0041] 3.4 Temporary mask generation and range adjustment: When it is detected that the movement speed of the joint point exceeds the preset threshold, it indicates that the joint movement is relatively violent, which may cause problems such as motion blur. Or when the joint point is blocked by other objects, in order to ensure that the mask effectively covers the relevant area, an interpolation operation is performed based on the mask trajectory of the historical frame to generate a temporary mask. The interpolation method can use common algorithms such as linear interpolation or spline interpolation to infer the mask shape that the current frame should have based on the position and shape information of the historical mask. At the same time, the mask range is automatically expanded so that it can cover areas where motion blur may occur, ensuring that in these special cases, the style transfer operation can still be accurately performed on the joints and their surrounding areas, ensuring the consistency and accuracy of the style transfer effect.

[0042] 3.5 Local feature fusion and geometric consistency constraints: Based on the mask coverage, local features of the corresponding areas of adjacent frames are fused. By extracting the color, texture, structure and other features in the mask areas corresponding to adjacent frames, these features are organically combined using feature fusion algorithms, such as weighted average fusion or feature fusion networks based on deep learning. During the fusion process, the geometric consistency of the joint structure in the stylized output is strictly constrained. For example, by comparing the positions of the joint points and the shapes of the joint contours of adjacent frames, the weights and methods of feature fusion are adjusted to ensure that in the stylized video sequence, the transition of the joint structure between adjacent frames is natural and smooth, and there will be no sudden changes in geometric shapes or unreasonable deformations, thereby improving the visual quality and realism of the video after style transfer.

[0043] 4.1 Joint point trajectory sequence construction and noise reduction processing: From each frame of the stylized video, the joint point coordinates are accurately extracted using a high-precision joint point detection algorithm. These algorithms are based on deep learning models and are trained on a large amount of video data containing human joint annotations to accurately identify the joint point positions in different scenes and postures. The extracted joint point coordinates are arranged in chronological order to construct the original trajectory sequence. This sequence fully records the movement path of the joint on the video timeline.

[0044] In order to improve the quality of the trajectory sequence, it is subjected to segmented noise reduction. The joint movement speed is calculated by analyzing the displacement of the joint point in unit time. For the low-speed motion segment, since it may cause jitter in the trajectory due to noise interference, a sliding average filtering algorithm is used. This algorithm averages the data in a sliding window of a fixed length to smooth out the jitter noise. For the high-speed motion segment, in order to avoid the loss of key motion information due to excessive smoothing, only outliers are removed. By calculating the mean and standard deviation of the data, points that deviate from the mean by more than a certain multiple of the standard deviation are regarded as outliers and removed.

[0045] 4.2 Adaptive time window allocation and multi-scale feature extraction: An adaptive time window is assigned to each joint point to better capture its characteristics under different motion states. The window length is dynamically adjusted according to the motion acceleration. The greater the acceleration, the faster the joint motion state changes, and the shorter the time window is required to reflect the instantaneous changes of joint motion more timely. The acceleration is obtained by calculating the change in the joint motion speed of adjacent frames. For example, if the joint motion speed of the current frame is v i , the previous frame is v i−1 , then the acceleration a=v i −v i−1 The time window length of each joint point in each frame is determined by the pre-set mapping relationship between acceleration and window length.

[0046] In the spatiotemporal association network, multi-scale feature extraction is performed on the trajectory data within the time window. Convolution kernels or pooling windows of different sizes are used to extract features at different levels of the trajectory data. Smaller scales can capture local detail features, such as small changes in joint movements in a short period of time; larger scales focus on global patterns, such as the movement trends of joints over a longer period of time. The multi-scale features are fused with hierarchical connection weights, which reflect the strength of the association between joint points based on anatomy and motion synergy. The multi-scale features are combined with the connection weights through weighted summation and other methods to generate a spatiotemporal feature vector containing local details and global patterns, providing rich motion information for subsequent analysis and processing.

[0047] 4.3 Multimodal feature extraction and fusion: Visual modality feature extraction: Extract visual modality features from stylized videos, including color distribution, edge sharpness, and texture density. Color distribution is described by calculating the pixel ratio and histogram of different colors in the video frame, reflecting the overall color characteristics of the video. Edge sharpness uses edge detection algorithms such as Sobel operator or Canny operator to calculate the strength and clarity of image edges. Texture density is measured by counting the number and distribution of texture elements in the image. These features are normalized and their values ​​are mapped to the range of [0,1] or [-1,1] to generate the first eigenvector to ensure that the weights of different features are balanced in the subsequent fusion process.

[0048] Motion modal feature extraction: Motion modal features are extracted from the spatiotemporal trajectory sequence, including joint displacement, acceleration, and motion direction consistency. The joint displacement is directly calculated by the difference between the joint coordinates of adjacent frames, and the acceleration is calculated by the velocity change as described above. The consistency of motion direction is determined by comparing the angle between the joint motion directions of adjacent frames. The smaller the angle, the higher the consistency. The dynamic time warping algorithm is used to align motion trajectory sequences of different lengths on the time axis, so that the motion features of different video clips are comparable, and then the second feature vector is generated.

[0049] Feature fusion: Construct a cross-modal association matrix, align the first eigenvector with the second eigenvector, and fuse the two types of features through a dynamic weight allocation strategy. The weight value is adjusted in real time according to the scene illumination intensity, motion complexity, and the degree of stylized interference. For example, when the scene illumination intensity changes greatly, the weight of the visual modality feature is appropriately reduced, because the illumination may affect the accuracy of visual features such as color and edges; when the motion complexity is high, the weight of the motion modality feature is increased to highlight the importance of motion information. During the fusion process, if it is detected that the visual modality feature is distorted due to style transfer, such as severe color deviation or excessive edge blur, local feature compensation is performed based on the spatiotemporal continuity of the motion modality feature. By analyzing the changing rules of the motion modality features of adjacent frames, the reasonable visual features of the affected area are inferred, thereby ensuring that the fused feature vector reflects both the visual style and the essence of the action.

[0050] 4.4 Anomaly detection and conflict resolution: When performing anomaly detection on the fused feature vector, a conflict resolution mechanism is introduced. When the judgment results based on visual modality features and motion modality features are inconsistent, the detection results of motion modality are given priority. This is because motion modality features can better reflect the intrinsic nature of human movements and are relatively less affected by external factors such as style transfer. At the same time, the results are verified by tracing back the trajectory consistency of adjacent frames. Check whether the trajectories of the joints in adjacent frames conform to the normal motion logic and continuity. If there are abnormal jumps or non-compliance with the laws of motion, further analyze the causes and correct the detection results to improve the accuracy and reliability of anomaly detection.

[0051] 4.5 Correction parameter generation and network optimization: Extract the coordinates of the joint points of each frame and the color distribution of the corresponding pixel area from the stylized video, and calculate the motion trajectory offset of the joint points and the color distribution difference of the stylized area. The motion trajectory offset is obtained by comparing the deviation between the joint point trajectory of the current frame and the expected trajectory (such as predicted by historical trajectory or inferred by normal motion model). The color distribution difference is measured by calculating the distance between the color histograms of the corresponding pixel areas before and after stylization, such as using Bhattacharyya distance or chi-square distance. These calculation results are used to generate correction parameters to guide the optimization of the dynamic spatiotemporal association network.

[0052] The node connection weights of the dynamic spatiotemporal association network are optimized by gradient descent according to the correction parameters. For nodes whose trajectory offset exceeds the threshold, the connection weights with adjacent nodes are reduced. This is because trajectory offset may mean that the association between the node and the adjacent nodes is abnormal. By reducing the weight, the impact of abnormal association on the overall performance of the network is reduced. For pixel areas where the color distribution difference exceeds the set range, the spatiotemporal association with the corresponding area of ​​the adjacent frame is recalculated. Using spatiotemporal association algorithms, such as methods based on optical flow and feature matching, the feature aggregation rules of local nodes are updated so that the network can more accurately handle the impact of color changes during the stylization process.

[0053] The optimized node connection weights and feature aggregation rules are fed back to the dynamic spatiotemporal association network to generate updated style transfer results and spatiotemporal trajectory sequences. The above process is iterated continuously, that is, re-extracting features, calculating correction parameters, and optimizing the network until the trajectory offset and color difference converge to the preset range. At this point, the dynamic spatiotemporal association network can more accurately reflect the actual situation of human joint movement while ensuring the style transfer effect, thereby improving the quality and reliability of the stylized video.

[0054] 5.1 Preset action mode library construction: The establishment of a preset action pattern library is the basis of the entire comparison process. The library contains a large amount of information related to standard actions, among which the spatiotemporal feature vectors are extracted by in-depth analysis of videos of various standard actions in different scenarios. Advanced motion capture equipment and professional data analysis software are used to accurately record the spatiotemporal changes of the joints of the human body when performing standard actions, and then after complex algorithm processing, the spatiotemporal feature vectors that can accurately characterize the characteristics of standard actions are generated. The trajectory deviation tolerance is the permissible deviation range set for the joint trajectory of each standard action. It is based on the statistical analysis of a large number of standard action samples, taking into account the slight differences that may exist in actual execution. For example, for the common waving action, after statistics on the trajectory data of 1,000 standard waving actions, it is determined that the trajectory deviation tolerance of the arm joint in a specific direction is ±5 pixels. The motion direction constraints specify the rules that the joint motion direction should follow during the execution of the standard action. By analyzing the standard action video frame by frame, the ideal motion direction of each joint at each stage is determined and expressed as an angle range. For example, in a standard squat, the motion direction of the knee joint during the descending phase should be within an angle of ±15° from the vertical. 5.2 Feature vector dimension-by-dimension alignment and difference calculation: The spatiotemporal feature vector generated in real time is aligned dimension by dimension with the features in the preset action pattern library. Since different spatiotemporal feature vectors may differ in dimensional order and representation, specific alignment algorithms are required to achieve accurate comparison. These algorithms match and adjust each dimension based on the physical meaning of the feature vector and the action representation logic. For example, for the joint point coordinate dimension, ensure that the order of the joint points in the real-time feature vector is consistent with the order of the joint points of the standard action in the preset library, and the coordinate units and reference system are the same. After completing the dimension-by-dimension alignment, the trajectory offset, direction deviation angle, and motion acceleration difference are calculated. The trajectory offset is obtained by calculating the spatial distance between the corresponding joint points of the real-time joint point trajectory and the standard motion trajectory at the same time point.

[0055] The direction deviation angle is obtained by comparing the angle between the real-time joint movement direction and the joint movement direction in the standard action. First, calculate the real-time movement direction vector v real and the standard motion direction vector v std , and then use the vector angle formula to calculate the angle, which is the direction deviation angle.

[0056] The motion acceleration difference is measured by comparing the sum of the absolute values ​​of the differences between the real-time motion acceleration and the standard motion acceleration in each dimension. 5.3 Abnormal action determination: The calculated trajectory offset, direction deviation angle, and motion acceleration difference are compared with the corresponding tolerance, set angle, and threshold in the preset motion pattern library to determine whether the action is abnormal. If the trajectory offset exceeds the tolerance, it means that the joint motion trajectory deviates from the normal range of the standard action, which may indicate that the action is not performed in a standardized manner or an abnormal situation has occurred. For example, in a standard running action, if the trajectory offset of the leg joint exceeds the preset tolerance of ±10 cm, it may mean that the running posture is incorrect or there is a deformation of the movement caused by physical discomfort. When the direction deviation angle is greater than the set angle, it indicates that the joint movement direction does not meet the requirements of the standard action. For example, in a standard boxing punching action, if the arm movement direction deviates from the set angle by more than 30°, it may be that the punching direction is wrong or the force is improper. If the acceleration difference exceeds the threshold, it means that the acceleration change of the joint movement is significantly different from the standard action. This may reflect problems with the strength and speed control of the action. For example, in a standard weightlifting action, if the acceleration difference during the barbell lifting process exceeds the preset threshold, it may be that the athlete's force is uneven or the technical action is wrong. As long as any of the above three conditions is met, it will be judged as an abnormal action. 5.4 Multi-frame continuity verification and final result output: In order to improve the accuracy and reliability of abnormal action judgment and avoid misjudgment due to noise or accidental factors in single-frame data, the judgment results are verified for multi-frame continuity. The system will continuously monitor the action judgment of multiple consecutive frames, and only output the final abnormal action result when multiple consecutive frames (for example, 5 consecutive frames) are abnormal. This is because in actual scenarios, single-frame data may be subject to various interferences, such as light changes, occlusions, etc., resulting in short-term feature abnormalities, but it does not necessarily represent real action abnormalities. Through multi-frame continuity verification, these interference factors can be effectively eliminated to ensure that the output abnormal action results are more credible. Once an abnormality is detected in multiple consecutive frames, the system will trigger the corresponding alarm or recording mechanism to prompt relevant personnel to pay attention to abnormal movements for further analysis and processing.

[0057] For example 2, please refer to Figure 1-Figure 4 As shown, in another preferred embodiment of the present invention, the steps are: Step 1: Obtain training data and preprocess it, and adjust the size of the unified video frame; Step 2 constructs a video style transfer posture detection algorithm based on graph neural network; the video style transfer posture detection algorithm based on graph neural network includes: a style transfer module based on a feedforward convolutional neural network, and a posture recognition module based on spatiotemporal graph convolutional neural network. Use time loss to constrain video consistency, and finally complete the model architecture for generating style transfer videos and anomaly detection; Step 3 uses the trained video style transfer pose detection algorithm to perform style transfer and pose anomaly detection, and uses the dataset to pre-evaluate the temporal consistency of style transfer; Step 4 uses subjective feelings and objective quality indicators to evaluate the results of style transfer and confirm the performance of the model.

[0058] The preprocessing in step 1 includes: selecting videos from different scenes to ensure that the model can cope with content from various environments, ensuring the generalization ability of the model in practical applications, and avoiding the problem that style transfer is only applicable to specific types of videos; since the video frames used as data have inconsistent sizes, the size of the video frames is uniformly adjusted to 640×360; at the same time, pre-calculating the optical flow data of adjacent frames for subsequent temporal consistency constraints to avoid repeated calculations during training and improve efficiency.

[0059] The style transfer module in step 2 is based on a feed-forward convolutional neural network, which consists of two main components: a stylization network and a loss network. The stylization network takes a frame as input and produces its corresponding stylized output. Subsequently, the loss network extracts the features of the stylized output frame and then calculates the loss, which is used to train the stylization network.

[0060] On one hand, these features are used to compute the spatial loss in order to evaluate the style transfer quality in the spatial domain, which is a weighted sum of the content loss and the style loss: ; The content loss extracts high-level features through the pre-trained VGG network to ensure that the semantic content of the generated image is consistent with the original video. l represents the feature extraction layer in VGG-19. The content loss defined at layer l is the mean square error between the feature map of the input frame xt and the feature map of its stylized output frame xt, and the formula is: ; in, represents the feature map of layer l, is the dimension of the feature map of layer l. This content loss is motivated by the observation that high-level features learned by CNNs represent abstract content, which is what is preserved for the original input in the style transfer task. The content loss ensures that the input content and the stylized output are as similar as possible.

[0061] For style transfer, the details need to be stylized according to the reference style image. Therefore, style loss is added to evaluate the style difference between the style image and the stylized frame. In order to better capture the style information, the Gram matrix is ​​used to capture the texture statistical features of the style image (such as color distribution, stroke direction), and fine-grained style transfer is achieved through multi-level features: ; in is the feature map. The style loss is defined as the style image s and the stylized output frame The mean squared error between the Gram matrices of : ; On the other hand, the temporal loss is trained to enforce temporal consistency between stylized outputs. During training, two consecutive input frames x t−1 and two stylized output frames xt t−1 and xt are fed to a loss network to compute the temporal loss, which measures the Euclidean color distance between corresponding pixels with reference to the pre-computed optical flow.

[0062] Specifically, simply applying image style transfer methods to video frames will inevitably introduce flickering artifacts. Therefore, a temporal loss is incorporated to enforce the temporal consistency between adjacent frames. Two consecutive frames are fed into the network simultaneously to measure the temporal consistency. The temporal loss is defined as the mean squared error between the stylized outputs at time t: ; in, and They are the stylized results of the current frame and the previous frame respectively. is the function that warps the stylized output from time t-1 to time t according to the precomputed optical flow. D = H × W × C is the size of the output. Represents the per-pixel confidence of the optical flow: 0 at occluded regions and motion boundaries, 1 otherwise.

[0063] In step 2, the posture recognition module is based on the spatiotemporal graph convolutional neural network, the posture extraction module is based on Alphapose and Poseflow, and the core module is the spatiotemporal graph convolutional autoencoder (ST-GCAE). Among them, AlphaPose is an efficient multi-human posture estimation method that can identify the joint position of each human body in the video. In this architecture, it is used to extract the key points of the human body in each frame, and PoseFlow uses optical flow information to associate cross-frame joints to form a coherent trajectory.

[0064] For the core module, the spatio-temporal graph convolutional autoencoder (ST-GCAE), it is based on the basic network ST-GCN (Spatio-Temporal Graph Convolutional Network). ST-GCN is a deep learning network architecture specially designed for human posture detection tasks, especially suitable for processing spatio-temporal information (such as video or time series data). ST-GCN combines spatial and temporal information and uses graph convolutional network (GCN) to model the spatial structure and temporal dynamics of human key points, thereby achieving efficient detection of human posture. Specifically, its input is a data sequence containing the positions of human joints. The posture of each frame consists of a set of key points (joints), which are connected by the edges of the graph to form graph structure data. For each frame of graph structure data, ST-GCN first applies graph convolution operations to capture spatial relationships. Then, the network convolves data of multiple time steps through TCN to capture the dynamic changes of the human body in the time series. Finally, the network outputs the joint position in each frame and makes accurate posture predictions through comprehensive analysis of spatio-temporal information.

[0065] In ST-GCN, nodes are equivalent to image pixels in traditional convolution. The sampling function is responsible for specifying the range of adjacent nodes involved when performing graph convolution operations on each node. It is expressed by the following formula, where d represents the distance between two nodes and v represents the node feature.

[0066] ; Weight function Indicates that a weight vector is obtained based on a node and its neighborhood , the formula is as follows: ; express In The label in the molecular set label is assigned to the central node. Therefore, the spatial domain graph convolution formula is: ; in: ; is a normalization term, i.e., the cardinality of the corresponding subset, which is used to balance the contribution of different subsets. The sampling function and weight function introduced above are for spatial graphs, and temporal graph convolution also requires a set of sampling functions and weight functions. The basic principle is the same, except that the mapping function of label grouping is redefined, and the rest of the calculation method is the same: ; Based on this, the spatial attention graph convolution module in ST-GCAE consists of 3 GCN operators, one using a static adjacency matrix (A), the second using an adjacency matrix learned globally during training (B), and the third using an adjacency matrix inferred by the attention submodule (C). The three adjacency matrices capture different aspects of the model: (i) using body part connectivity as a prior for node relations, represented using a static adjacency matrix, (ii) dataset-level keypoint relations, captured by the global adjacency matrix, and (iii) sample-specific relations, captured by the inferred adjacency matrix.

[0067] The static adjacency matrix A is fixed and shared by all layers. The input matrix [N, V, C] represents the node features of each graph (or time step), while the static adjacency matrix A[V, V] describes the structural relationship between these nodes. Each element Aij of the matrix represents the connection relationship between node i and node j. Therefore, the static adjacency matrix captures the topological structure between nodes, that is, the structure of the graph is shared between different samples. The static adjacency matrix transmits the information of neighbor nodes to each node, so that the features of each node are integrated with the features of neighbor nodes.

[0068] The role of the global learning adjacency matrix B is to make the connection relationship of the graph structure automatically learned through the training process, rather than just relying on a fixed static adjacency matrix. The global learning adjacency matrix is ​​updated and optimized through training, with the goal of improving the performance of the model on the dataset. The global learnable matrix B is learned separately at each layer and applied equally to all samples during the forward pass. For graphs with fewer nodes (such as dozens of nodes), this globally learnable adjacency matrix does not bring a significant computational burden. The size of the matrix is ​​V×V, so if the number of nodes is small, the size and amount of computation of the adjacency matrix are also small, and the optimization and computation process is relatively efficient.

[0069] The inferred adjacency matrix C is based on an attention mechanism that uses learned weights to compute a sample-specific adjacency matrix that is different for each sample in the batch. For example, for a batch of graphs of size N with V nodes, the inferred adjacency size is [N, V, V], and the inferred adjacency matrix is ​​constructed using a graph self-attention layer. A simple multiplicative attention mechanism is chosen. First, the input is embedded twice using two sets of learned weights.

[0070] and ; in and is the node feature after mapping, is the input node feature, and The first and second sets of weights are given in Figure 1. Then, we transpose one of the embedding matrices and take the dot product between the two and normalize them.

[0071] ; ; Where S is the attention score matrix, which represents the similarity between each pair of nodes. A is the normalized adjacency matrix, which represents the attention weights between nodes in each graph. Then we get the inferred adjacency matrix.

[0072] In general, the functions of the three adjacency matrices are as follows: The static adjacency matrix A provides the basic structure of the human skeleton and helps the model understand the basic spatial relationship between key points. It is mainly responsible for transmitting structured geometric information in posture recognition and providing a stable foundation for the model. The global learning adjacency matrix B is obtained through learning and training. It can dynamically capture the relationship between key points in the global scope and adapt to different postures and changes, thereby enhancing the performance of the model, especially in complex dynamic environments. The attention adjacency matrix C dynamically calculates the weights between nodes through the graph attention mechanism, so that the model can automatically focus on the most relevant key point relationships and flexibly adjust the connection strength to improve the accuracy of key point extraction and recognition.

[0073] As for the temporal convolution layer, its role is to capture the continuity of actions in the temporal dimension. The combination of temporal convolution and graph convolution models normal action patterns in the latent space, allowing anomaly detection to consider both spatial specificity and temporal continuity.

[0074] In ST-GCAE, the loss function used is reconstruction loss (also known as mean square error loss, MSELoss). Its core goal is to measure the difference between the model output and the true target, which is used to constrain the accuracy of the model (ST-GCAE) to reconstruct the pose sequence: ; in is a real human posture sequence, The scalar loss value is the posture sequence reconstructed by the model. The smaller the value, the closer the reconstruction result is to the real data. The input posture sequence is reconstructed by ST-GCAE, forcing the model to learn the potential representation of normal actions. Minimizing the reconstruction error LrecLrec enables the encoder-decoder structure to compress and restore the spatiotemporal features of normal postures. The smaller the L2 error between the reconstructed walking posture sequence and the input posture, the more accurate the modeling of the normal walking mode is.

[0075] In step 3, the consistency of the style transfer algorithm is pre-evaluated. The specific measure is to use a dataset (such as the MPISintel dataset, a large-scale dataset for optical flow estimation and visual flow analysis) to evaluate the optical flow estimation algorithm in computer vision. If the style transfer process destroys the continuity of the optical flow, then a large error will be generated during the evaluation. Therefore, the temporal consistency is pre-judged in order to consider the accuracy of the subsequent evaluation of the style transfer results.

[0076] In step 4, subjective feelings and objective quality indicators mainly include: The style transfer results of this algorithm are compared with other transfer methods (AdaIn, NeuralNeighborStyleTransfer, SANET), and the test results are randomly selected to show the difference in subjective visual effects of the algorithms. Different contents are compared with the same style.

[0077] In the subjective evaluation stage, the evaluator evaluates the style transfer results from the aspects of completeness of stylization effect, style unity, texture refinement, and naturalness of consecutive frames. To further evaluate the stability of the pose after video style transfer, it is necessary to analyze the pose graph and evaluate the pose detection effect of the architecture.

[0078] In the objective quality indicator evaluation stage, the RGB histogram is drawn. By drawing the RGB histogram, it is possible to intuitively see whether the pixel distribution of each color channel (red, green, and blue) of the image has changed significantly before and after style transfer. It can reveal the overall color shift, contrast change, and visual effects brought by style transfer. By superimposing the histograms of multiple image frames and calculating the standard deviation, it is possible to observe whether there is temporal inconsistency in the style transfer process. In addition, AUC (Area Under the Curve) is also an important indicator for objective evaluation. It represents the area under the ROC (Receiver Operating Characteristic) curve. The ROC curve evaluates the performance of the classifier by plotting the relationship between the true positive rate and the false positive rate of the model at different thresholds. The AUC value ranges from 0 to 1. The closer the value is to 1, the stronger the model's ability to distinguish between positive and negative classes. When the AUC value is 0.5, it means that the performance of the model is equivalent to random guessing. The AUC score is used to evaluate the performance of the model in posture recognition. By classifying the human posture in the video frame, it is determined whether a specific action or posture is successfully recognized. The AUC score helps quantify the classification effect of the model under different decision thresholds. In the case where style transfer may affect the posture, AUC can effectively evaluate whether the model can maintain good classification performance under different styles.

[0079] In addition, many other indicators are compared, including warp error, temporal error, peak signal-to-noise ratio (PSNR), mean square error (MSE), structural similarity index (SSIM), and inter-frame structural difference (IFSD). Among them, the warp error focuses on the consistency of image content during spatial transformation, aiming to measure whether the image can maintain the consistency of spatial structure during content migration. By minimizing the warp error, it can effectively ensure that the spatial relationship of the image is preserved during the conversion or reconstruction process, thereby improving the quality of image processing. Temporal error measures the visual difference between adjacent frames. Its core goal is to minimize the temporal changes between adjacent frames to ensure the temporal consistency and visual continuity of the video sequence. PSNR (peak signal-to-noise ratio) is a common indicator for measuring image quality. The larger its value, the smaller the difference between the original image and the reconstructed image, so the image quality is correspondingly higher. Mean square error (MSE) evaluates the accuracy of image reconstruction by calculating the pixel-level difference between the target style image and the generated image. The smaller its value, the higher the quality of image reconstruction. The structural similarity index (SSIM) is a more comprehensive quality assessment indicator that measures the degree of image content preservation by evaluating the similarity of images in terms of brightness, contrast, and structure. The closer the SSIM value is to 1, the more similar the image is visually and structurally to the original image, and the better the quality. The inter-frame structural difference (IFSD) is used to evaluate the structural differences between consecutive video frames, especially focusing on whether there is flickering caused by drastic changes in image content. The smaller the IFSD value, the smoother and more consistent the image sequence, and the more natural the transition between frames, reducing the visual sense of incoherence.

[0080] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A video style transfer posture detection method based on graph neural network, characterized in that: The following steps are involved: Parse the original video frames frame by frame, extract continuous video sequences, dynamically adjust the resolution according to the scene and divide it into local areas to generate pre-processed video data streams; Construct a dynamic spatiotemporal association network to map the pixels of video frames into spatiotemporal graph nodes. The connection weights between nodes are dynamically calculated based on the motion trends of adjacent frames and the complexity of regional content. The original video frames are converted into the target artistic style by aggregating the spatiotemporal features of nodes layer by layer. In the process of style transfer, a dynamic mask is generated for the area where the human body joints are located, and local features are fused for the corresponding areas of adjacent frames based on the mask coverage to constrain the geometric consistency of the joint structure in the stylized output; Extract the coordinates of the joint points of each frame from the stylized video, construct a spatiotemporal trajectory sequence, learn the motion pattern and spatial topological relationship of the joint points through a dynamic spatiotemporal association network, and generate a feature vector containing spatiotemporal dependencies; The feature vector is compared with the preset action pattern library in multiple dimensions, the trajectory offset and pattern matching degree are calculated, the action abnormality is determined according to the threshold and the detection result is output.

2. According to the method of video style transfer posture detection based on graph neural network in claim 1, it is characterized in that: The construction of the dynamic spatiotemporal association network specifically includes: The pixels of the video frame are divided into several local areas, each of which is mapped to a graph node. The node attributes include color gradient, motion vector and local texture entropy value. The motion vector is generated by optical flow field calculation, and the texture entropy value reflects the complexity of details in the area. In the time dimension, cross-frame connections are established for corresponding nodes in adjacent frames, and the connection weight is dynamically calculated based on the spatiotemporal correlation of node attributes: if the motion vector directions of nodes in adjacent frames are consistent and the texture entropy value difference is less than the adaptive threshold, the weight is increased, otherwise it is decreased; In the spatial dimension, hierarchical connections are constructed for human joints in the same frame: the first-level connection is defined based on anatomical priors, the second-level connection is defined based on motion synergy, and the weights of the connections between the levels are dynamically adjusted according to the range of joint motion. Through the spatiotemporal graph convolution operation, the dynamic connection features across frames and levels are fused layer by layer to generate spatiotemporal correlation features that combine style transfer and motion perception.

3. According to the method of video style transfer posture detection based on graph neural network in claim 1, it is characterized in that: The generation of the dynamic mask includes: Based on the preprocessed video frames, a lightweight posture estimation network is used to predict the human joints in real time, and an initial mask is generated with the joints as the center. The mask radius expands linearly with the joint movement speed. Combining the optical flow information with the joint trajectory prediction model, the deformation parameters of the next frame mask are inferred. The deformation direction is consistent with the joint movement direction, and the deformation amount is proportional to the movement acceleration. In the process of style transfer, the mask coverage area is subject to dual constraints, including geometric constraints and semantic constraints. The geometric constraints strengthen the contour feature extraction of the joints in the mask through a dynamic spatiotemporal association network to suppress the edge blurring caused by stylization; the semantic constraints compare the semantic segmentation results of the mask area before and after style transfer. If the key anatomical structure is lost, the local stylization strength attenuation is triggered. When it is detected that the motion speed of a joint exceeds a threshold or is occluded, a temporary mask is generated based on the mask trajectory interpolation of the historical frames, and the mask range is automatically expanded to cover the motion blur area.

4. According to the method of video style transfer posture detection based on graph neural network in claim 1, it is characterized in that: The construction of the spatiotemporal trajectory sequence includes: Extract the coordinates of joint points from each frame of the stylized video, construct the original trajectory sequence in chronological order, and perform segmented denoising on the sequence. Use sliding average filtering to smooth jitter noise in low-speed motion segments; retain the original data for high-speed motion segments and only remove outliers. An adaptive time window is allocated to each joint point, and the window length is dynamically adjusted according to the motion acceleration: the greater the acceleration, the shorter the window length; In the spatiotemporal correlation network, multi-scale feature extraction is performed on the trajectory data in the window: the multi-scale features are fused with the hierarchical connection weights to generate a spatiotemporal feature vector containing local details and global patterns.

5. According to the method of video style transfer posture detection based on graph neural network in claim 4, it is characterized in that: The construction of the spatiotemporal trajectory sequence also includes: Extracting visual modality features from the stylized video, including color distribution, edge sharpness, and texture density, and generating a first feature vector through normalization; Extract motion modal features from the spatiotemporal trajectory sequence, including joint displacement, acceleration, and motion direction consistency, and generate a second eigenvector through dynamic time warping; Construct a cross-modal association matrix, align the dimensions of the first eigenvector and the second eigenvector, and fuse the two types of features through a dynamic weight allocation strategy. The weight value is adjusted in real time according to the scene lighting intensity, motion complexity, and stylized interference degree; During the fusion process, if it is detected that the visual modality features are distorted due to style transfer, local feature compensation is performed based on the spatiotemporal continuity of the motion modality features to ensure that the fused feature vector reflects both the visual style and the essence of the action. When performing anomaly detection on the fused feature vector, a conflict resolution mechanism is introduced: when the judgment results of the two modes are inconsistent, the detection results of the motion mode are adopted first, and the results are verified by tracing the consistency of the trajectories of adjacent frames.

6. The method for detecting postures in video style transfer based on graph neural network according to claim 1, characterized in that: Extract the coordinates of the joint points of each frame and the color distribution of the corresponding pixel area from the stylized video, calculate the difference between the motion trajectory offset of the joint points and the color distribution of the stylized area, and generate correction parameters; The node connection weights of the dynamic spatiotemporal association network are optimized by gradient descent according to the correction parameters, and the connection weights of the nodes with trajectory offsets exceeding the threshold value and the adjacent nodes are reduced; For pixel areas where the color distribution difference exceeds the set range, recalculate its spatiotemporal correlation with the corresponding areas of adjacent frames and update the feature aggregation rules of local nodes; The optimized node connection weights and feature aggregation rules are fed back to the dynamic spatiotemporal association network to generate updated style migration results and spatiotemporal trajectory sequences, which are iterated until the trajectory offset and color difference converge to the preset range.

7. The method for detecting postures in video style transfer based on graph neural network according to claim 1, characterized in that: The determination of abnormal action specifically includes: The preset action pattern library contains the spatiotemporal feature vectors, trajectory deviation tolerances, and motion direction constraints of standard actions; Align the spatiotemporal feature vector generated in real time with the features in the pattern library dimension by dimension, and calculate the trajectory offset, direction deviation angle, and motion acceleration difference; If the trajectory deviation exceeds the tolerance, the direction deviation angle is greater than the set angle, or the acceleration difference exceeds the threshold, it is judged as an abnormal action; The judgment result is verified for multi-frame continuity, and the final result is output only when anomalies are detected in multiple consecutive frames.

8. The method for detecting postures in video style transfer based on graph neural network according to claim 1, characterized in that: The pre-processing comprises: Perform dynamic region segmentation on the video frame, divide the interior of the video frame into high-detail area and low-detail area according to the content complexity threshold, and perform local super-resolution enhancement on the high-detail area; Extract the optical flow information of the video frame, generate a motion vector, and mark the high-motion area and the static area; dynamically adjust the preprocessing parameters according to the motion vector, perform noise reduction and blur suppression on the high-motion area, and perform color balance optimization on the static area; The preprocessed video data stream and the motion vector graph are synchronously input into the dynamic spatiotemporal association network to guide the network to optimize the focus on key areas.

Images