A monocular camera-based three-dimensional motion capture and intelligent analysis system and method

Abstract

The application discloses a monocular camera-based three-dimensional motion capture and intelligent analysis system and method, relates to the technical field of image processing, and comprises an input and preprocessing module, a feature extraction and interactive fusion module, a human key point detection module, a three-dimensional posture reconstruction module and an intelligent analysis module; by constructing a multi-scale feature pyramid structure, the system can perform fine processing on features at different resolutions to obtain a down-sampling feature map; by applying a Mamba model to a human motion capture system, the Mamba model adaptively fuses multi-scale features in a time sequence, and generates a fused multi-scale feature map through convolution and up-sampling operations. The accuracy and robustness of key point detection are improved; through an xLSTM model, the key point information of front and rear frame images is combined to smooth the key point sequence, the time consistency and continuity of key point detection are ensured, and the accuracy and stability of human posture detection are further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a three-dimensional motion capture and intelligent analysis system and method based on a monocular camera. BACKGROUND

[0002] Three-dimensional human motion reconstruction has a wide range of application fields, including virtual reality, augmented reality, human-computer interaction, etc. Accurate and smooth reconstruction of human motion sequences is crucial for three-dimensional human pose and shape estimation in monocular videos. However, it is challenging to achieve smooth results using image-based human reconstruction methods on videos.

[0003] Traditional methods rely on complex three-dimensional scanning or motion capture devices, which are expensive. Reconstruction methods based on monocular videos have the advantages of low cost and easy deployment. Existing technologies attempt to use image-based methods to reconstruct three-dimensional humans from videos by estimating three-dimensional human poses and shapes on each frame of the video respectively. Although these methods can predict reasonable outputs for static images, the accuracy and robustness of key point detection are low due to the lack of modeling of the continuity of human motion in consecutive frames, making it difficult to estimate temporally coherent and smooth three-dimensional human poses and shapes from image sequences. SUMMARY

[0004] The purpose of the present application is to provide a three-dimensional motion capture and intelligent analysis system and method based on a monocular camera, which can improve the accuracy and robustness of key point detection and estimate temporally coherent and smooth three-dimensional human poses and shapes from image sequences.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In a first aspect, the present application provides a three-dimensional motion capture and intelligent analysis system based on a monocular camera, comprising:

[0007] An input and preprocessing module, configured to: pre-process an original image sequence to obtain a target image sequence; the original image sequence is obtained by a monocular camera; the target image sequence includes a plurality of consecutive frame target images;

[0008] The feature extraction and interaction fusion module is configured to: perform feature extraction on each target image to obtain an initial feature map corresponding to each target image; perform multi-scale feature extraction on each initial feature map to obtain a down-sampled feature map at different scales; perform fusion and serialization processing on the down-sampled feature map at each scale by using a Mamba model to obtain an efficient feature map at different scales; perform up-sampling operation on the efficient feature map at each scale to obtain an up-sampled feature map at different scales; fuse the up-sampled feature maps at different scales to obtain a multi-scale feature map corresponding to each target image; and perform feature reconstruction on the multi-scale feature map corresponding to each target image to obtain a final feature map corresponding to each target image.

[0009] The human key point detection module is configured to: perform feature extraction on the final feature map corresponding to each target image to generate a heat map of each key point in the target image; determine the position coordinates of each key point based on the heat map of each key point in the target image; and perform smoothing processing on the position coordinates of each key point, a previous frame image and a next frame image of the target image by using an xLSTM model to obtain a smoothed key point sequence corresponding to each target image.

[0010] The three-dimensional pose reconstruction module is configured to: perform projection processing on the smoothed key point sequence corresponding to each target image to obtain an initial high-dimensional embedding representation corresponding to each target image; perform processing on the initial high-dimensional embedding representation corresponding to each target image by using dynamic spatial graph convolution to obtain a dynamic spatial graph convolution feature corresponding to each target image; perform time series processing on the dynamic spatial graph convolution feature corresponding to each target image by using a Transformer model to obtain a final high-dimensional feature representation of a key point in each target image; and perform three-dimensional mapping processing on the final high-dimensional feature representation of the key point in each target image to obtain a three-dimensional reconstruction result of the key point in each target image.

[0011] The intelligent analysis module is configured to: analyze and evaluate a motion action in a target image sequence according to the three-dimensional reconstruction result of the key point in the target image to obtain an analysis result.

[0012] Optionally, the monocular camera-based three-dimensional motion capture and intelligent analysis system further includes a motion stage division module, and the motion stage division module includes:

[0013] The dedicated feature extraction unit is configured to: perform feature extraction on the final feature map corresponding to each target image by using a third-generation deformable convolutional neural network to obtain a dedicated feature corresponding to each target image.

[0014] a global modeling unit, configured to input the special features corresponding to each of the target images into a Transformer model to obtain global features corresponding to each of the target images;

[0015] a classification unit, configured to classify the actions in each of the target images according to the global features corresponding to each of the target images to obtain a stage classification result of each of the target images.

[0016] Optionally, the intelligent analysis module is configured to analyze and evaluate the motion actions in the target image sequence according to the stage classification result of each of the target images and the three-dimensional reconstruction result of the key points in the target images to obtain an analysis result.

[0017] Optionally, the original image sequence includes a plurality of frames of original images; and the original images are obtained by a monocular camera during the golf motion process of a user.

[0018] Optionally, the feature extraction and interaction fusion module includes a multi-scale feature extraction unit; and the multi-scale feature extraction unit is configured to perform multi-scale feature extraction on the initial feature map by using a multi-scale feature pyramid to obtain down-sampling feature maps at different scales.

[0019] Optionally, the human key point detection module includes a heat map generation unit; and the heat map generation unit is configured to perform feature extraction on the final feature map corresponding to each of the target images by using a multi-layer convolutional neural network to generate a heat map of each key point in the target image; and the multi-layer convolutional neural network is composed of a plurality of convolutional neural networks connected in sequence.

[0020] Optionally, the human key point detection module includes a key point position coordinate determination unit; and the key point position coordinate determination unit is configured to perform argmax operation on the heat map of each key point in the target image to obtain the position coordinates of each key point.

[0021] In a second aspect, the present application provides a monocular camera-based three-dimensional motion capture and intelligent analysis method based on the monocular camera-based three-dimensional motion capture and intelligent analysis system of the first aspect, including:

[0022] performing preprocessing on the original image sequence to obtain a target image sequence; the original image sequence is obtained by a monocular camera; and the target image sequence includes a plurality of continuous frames of target images;

[0023] perform feature extraction on each of the target images to obtain an initial feature map corresponding to each of the target images; for each of the initial feature maps, perform multi-scale feature extraction on the initial feature map to obtain a down-sampled feature map at different scales; using a Mamba model, perform fusion and serialization processing on the down-sampled feature map at each scale to obtain an efficient feature map at different scales; perform up-sampling operation on the efficient feature map at each scale to obtain an up-sampled feature map at different scales; fuse the up-sampled feature maps at different scales to obtain a multi-scale feature map corresponding to each of the target images; perform feature reconstruction on the multi-scale feature map corresponding to each of the target images to obtain a final feature map corresponding to each of the target images;

[0024] perform feature extraction on each of the final feature maps corresponding to the target images to generate a heat map of each key point in the target image; determine the position coordinates of each key point based on the heat map of each key point in the target image; using an xLSTM model, perform smoothing processing on the position coordinates of each key point in the target image, the previous frame image and the next frame image of the target image to obtain a smoothed key point sequence corresponding to each of the target images;

[0025] perform projection processing on the smoothed key point sequence corresponding to the target image to obtain an initial high-dimensional embedding representation corresponding to each of the target images; using dynamic spatial graph convolution, process the initial high-dimensional embedding representation corresponding to each of the target images to obtain a dynamic spatial graph convolution feature corresponding to each frame of target image; using a Transformer model, perform time series processing on the dynamic spatial graph convolution feature corresponding to each of the target images to obtain a final high-dimensional feature representation of the key points in each of the target images; perform three-dimensional mapping processing on the final high-dimensional feature representation of the key points in each of the target images to obtain a three-dimensional reconstruction result of the key points in each of the target images;

[0026] analyze and evaluate the motion actions in the target image sequence according to the three-dimensional reconstruction result of the key points in the target image to obtain an analysis result.

[0027] Optionally, the monocular camera-based three-dimensional motion capture and intelligent analysis method further comprises:

[0028] using a third-generation deformable convolutional neural network, performing feature extraction on each of the final feature maps corresponding to the target images to obtain a dedicated feature corresponding to each of the target images;

[0029] input the dedicated feature corresponding to each of the target images into a Transformer model to obtain a global feature corresponding to each of the target images;

[0030] According to the global feature corresponding to each target image, the action in each target image is classified to obtain a stage classification result of each target image.

[0031] Optionally, the motion action in the target image sequence is analyzed and evaluated according to the three-dimensional reconstruction result of the key points in the target image to obtain an analysis result, and the analysis result specifically includes:

[0032] According to the global feature corresponding to each target image, the action in each target image is classified to obtain a stage classification result of each target image.

[0033] According to the global feature corresponding to each target image, the action in each target image is classified to obtain a stage classification result of each target image.

[0034] The application provides a monocular camera-based three-dimensional motion capture and intelligent analysis system and method, which can refine features to obtain down-sampling feature maps under different resolutions by constructing a multi-scale feature pyramid structure. The Mamba model is applied to a human motion capture system. The Mamba model adaptively fuses multi-scale features in a time sequence by selecting a structured space model (SSM), and generates a fused multi-scale feature map through convolution and up-sampling operation. The accuracy and robustness of key point detection are improved, and the system is particularly suitable for processing complex human motion postures. The time sequence smoothing processing technology of the xLSTM model is combined with the key point information of the front and rear frames of images to smooth the key point sequence, ensuring the time consistency and continuity of key point detection, and further improving the accuracy and stability of human posture detection. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0036] Figure 1 A functional module schematic diagram of a monocular camera-based three-dimensional motion capture and intelligent analysis system according to an embodiment of the present application is provided.

[0037] Figure 2 A functional module schematic diagram of a monocular camera-based three-dimensional motion capture and intelligent analysis system according to an embodiment of the present application is provided.

[0038] Figure 3A flowchart of a monocular camera-based three-dimensional motion capture and intelligent analysis method is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0040] The above purposes, features and advantages of the present application will be more apparent and understandable. The present application will be further described in detail below with reference to the drawings and specific embodiments.

[0041] In an exemplary embodiment, as shown in Figure 1 and Figure 2 , a monocular camera-based three-dimensional motion capture and intelligent analysis system is provided, comprising an input and preprocessing module, a feature extraction and interaction fusion module, a human key point detection module, a three-dimensional pose reconstruction module, a motion stage division module and an intelligent analysis module.

[0042] First stage:

[0043] The input and preprocessing module is configured to preprocess the original image sequence to obtain a target image sequence. The original image sequence is captured by a monocular camera, and the original image sequence includes a plurality of original images. The original image is obtained by the monocular camera capturing the golf motion process of a user. The target image sequence includes a plurality of continuous target images.

[0044] In the first stage (input and preprocessing module), the pose features of the human body are extracted from the original image sequence captured by the monocular camera. The core task of this stage is to preprocess the original image to ensure that the subsequent key point detection and pose reconstruction stages can obtain accurate input. The input and preprocessing process is as follows:

[0045] 1.1 Input data

[0046] The input data is a continuous original image sequence from a monocular camera, represented as:

[0047] I={I1,I1,…,I T};

[0048] Wherein, I tdenotes the single-frame original image obtained at time t, t = 1, 2, …, T, T is the total number of frames of images in the image sequence. The resolution of each original image is H x W, H and W are the height and width of the original image respectively, which constitutes the dynamic change sequence of human body posture in the whole movement process.

[0049] 1.2 Data Preprocessing

[0050] In order to eliminate the influence of different light conditions and camera settings on the image, the original image is first normalized. Assuming that the pixel value range of the original image is [0, 255], the normalization process maps the pixel value to the [0, 1] range, and the normalization formula is:

[0051]

[0052] where I' is the normalized image corresponding to the single-frame original image obtained at time t, μ and σ are the mean and standard deviation of the image pixel value respectively. t

[0053] In order to improve the generalization ability of the model and enhance its robustness under different shooting conditions, a series of data enhancement operations are performed on the input image sequence, including random rotation, scaling, cropping and flipping data enhancement operations:

[0054] 1) Random rotation: randomly rotate the image within the range of [-10°, 10°].

[0055] 2) Scaling: randomly scale the image within the range of [0.8, 1.2].

[0056] 3) Cropping and flipping: randomly crop or horizontally flip the image to ensure that the model can adapt to different angles of view.

[0057] The above data enhancement operations ensure that the model can still robustly extract human posture information when facing different angles, proportions and environmental changes.

[0058] After normalization and data enhancement, the target image sequence is obtained, which is used as the input of the next stage of posture key point detection and three-dimensional posture reconstruction. The input and preprocessing module provides high-quality input data for subsequent posture reconstruction, significantly improving the robustness and accuracy of the overall model.

[0059] Second stage:

[0060] ​The feature extraction and interactive fusion module is configured to: perform feature extraction on each target image to obtain an initial feature map corresponding to each target image; perform multi-scale feature extraction on each initial feature map to obtain a down-sampled feature map at different scales; perform fusion and serialization processing on the down-sampled feature map at each scale by using a Mamba model to obtain an efficient feature map at different scales; perform up-sampling operation on the efficient feature map at each scale to obtain an up-sampled feature map at different scales; fuse the up-sampled feature maps at different scales to obtain a multi-scale feature map corresponding to each target image; and perform feature reconstruction on the multi-scale feature map corresponding to each target image to obtain a final feature map corresponding to each target image.

[0061] The second stage aims to detect two-dimensional key points of human bodies from images captured by monocular cameras. To this end, multi-scale feature extraction is performed using the Mamba model, and combined with time series modeling to ensure the accuracy and continuity of key point detection.

[0062] 2.1 Input data

[0063] The feature extraction and interactive fusion module includes an initial feature extraction unit, which is configured to perform feature extraction on each target image by using a ResNet-50 model to obtain an initial feature map corresponding to each target image.

[0064] First, after the preprocessing stage, the input is a target image sequence I' = {I'1, I'2, …, I' T}. After processing by the ResNet-50 model, the initial feature map obtained is where H' and W' are the height and width of the initial feature map, and C is the number of channels of the initial feature map.

[0065] 2.2 Mamba-based feature engineering

[0066] Step 1: Input feature map

[0067] We input the initial feature map F t generated by the ResNet-50 model into the Mamba model. At this time, the initial feature map F t contains high-level features extracted from the target image. Next, we want to capture detailed information at different scales through multi-scale processing.

[0068] Step 2: Build multi-scale feature pyramid

[0069] The feature extraction and interaction fusion module comprises a multi-scale feature extraction unit; the multi-scale feature extraction unit is configured to: for each initial feature map, performing multi-scale feature extraction on the initial feature map by using a multi-scale feature pyramid to obtain down-sampled feature maps at different scales.

[0070] To achieve multi-scale feature extraction, the initial feature map F t needs to be down-sampled to generate down-sampled feature maps at different scales. Assuming that we want to generate S down-sampled feature maps at different scales, we perform layer-by-layer down-sampling on the initial feature map to obtain down-sampled feature maps at different resolutions. The specific operation of multi-scale feature extraction is as follows:

[0071] (1) First, perform convolution and pooling operations on the initial feature map F t to generate the first down-sampled feature map F

[0072]

[0073] wherein, represents a convolution operation, k1 is the size of the convolution kernel, and Pool represents a down-sampling pooling operation.

[0074] (2) Repeat the above operation to further down-sample to generate subsequent down-sampled feature maps at different scales:

[0075]

[0076] wherein, s = 2, 3,..., S represents different scales; is the down-sampled feature map at the s-th scale, and the resolution of the down-sampled feature map gradually decreases as the scale s increases.

[0077] Through the multi-scale feature pyramid, down-sampled feature maps at multiple scales F are obtained. These down-sampled feature maps capture image information at different spatial resolutions. Each down-sampled feature map at a scale contains features at different levels extracted from the target image, suitable for processing key points of different sizes.

[0078] Step 3: Feature interaction and fusion of down-sampled feature maps at different scales using Mamba (selective state space model) will be interacted through the Mamba model (selective state space model (SSM)), so as to adaptively fuse features at different time points at each scale. The Mamba model serializes the multi-scale features by dynamically adjusting its state transition matrix A, B, C, D, to obtain efficient feature maps, thereby retaining efficient information related to key points. The formula is as follows:

[0079]

[0080] In the above formula, represents the hidden state vector, which is used in the Mamba model (selective state space model, SSM) to describe the information storage of the feature map of a certain scale s at each time step t . Specifically, encodes the information of the previous time step and combines the features of the current time step to form a dynamic state update mechanism, thereby helping the model to process time series data at multiple scales, is a key state representation in time series modeling, which helps the model to effectively capture and process the multi-scale features of human motion, thereby improving the accuracy of key point detection.

[0081] where A, B, C, Δ are learnable parameters:

[0082]

[0083] where l B , l C , l Δ are three linear mappings, and P Δ is a learnable parameter.

[0084] After the Mamba serializes the multi-scale features, the down-sampled feature sequence of scale s is converted into an efficient feature map sequence

[0085] Then, through the up-sampling operation (which is realized by a convolutional layer), these feature maps are restored to the same size as the original resolution H'xW':

[0086]

[0087] where Upsample is the up-sampling operation, is the up-sampled feature map of the s-th scale at time t. Next, the efficient feature maps of different scales are fused to obtain the fused multi-scale feature map at time t

[0088]

[0089] where α s is a learnable weighting coefficient, thereby adaptively focusing on more important scale features. The fused multi-scale feature map is reconstructed by a convolutional layer to generate the final feature map corresponding to each target image For subsequent human key point detection, motion segmentation (motion stage recognition) tasks, the feature reconstruction process can be represented by the following formula:

[0090]

[0091] The third stage:

[0092] The human key point detection module is configured to: perform feature extraction on the final feature map corresponding to each target image to generate a heat map of each key point in the target image; determine the position coordinates of each key point based on the heat map of each key point in the target image; and perform smoothing processing on the position coordinates of each key point in the target image, the previous frame image and the next frame image of the target image by using an xLSTM model to obtain a smoothed key point sequence corresponding to each target image.

[0093] In the second stage, we use the ResNet-50 model and the Mamba model to extract features from the input video (target image sequence). In the previous stage, we not only focused on the multi-resolution characteristics of the image, but also used the Mamba model to realize the interaction of features at different time frames at each resolution. Therefore, the final fused features can effectively serve human key point detection and motion stage division tasks.

[0094] 3.1 Generate key point heat map

[0095] The human key point detection module includes a heat map generation unit, which is configured to: perform feature extraction on the final feature map corresponding to each target image by using a multi-layer convolutional neural network to generate a heat map of each key point in the target image; and the multi-layer convolutional neural network is composed of a plurality of convolutional neural networks connected in sequence.

[0096] The human key point detection and motion stage division task head (task head) is connected in parallel to the final feature map After that, the human key point detection task head f DetectionHead is introduced first.

[0097] f DetectionHead is composed of a multi-layer convolutional neural network (a plurality of CNNs connected in sequence), which generates a heat map H for each key point based on the final feature map i :

[0098]

[0099] where H i is the heat map of the i-th key point, representing the probability distribution of the key point in the image.

[0100] In another exemplary embodiment of the present application, the human body key points can be selected as follows:

[0101] 1) Head region

[0102] Crown: The highest point of the head, used for posture detection.

[0103] Eyes (left eye, right eye): Used for facial expression recognition and head direction determination.

[0104] Ears (left ear, right ear): Assist in determining head rotation and tilt.

[0105] 2) Upper body region

[0106] Shoulders (left shoulder, right shoulder): Key points of the shoulders, usually used to detect the movement posture of the arms.

[0107] Elbows (left elbow, right elbow): Used to track the degree of bending and movement direction of the arms.

[0108] Wrist (left wrist, right wrist): The end point of the hand, used to accurately track hand movements.

[0109] 3) Torso region

[0110] Neck: Connects the head and torso, often serves as a reference point for the central axis of human posture.

[0111] Spine center: Key point in the middle of the torso, helps understand the overall posture and curvature.

[0112] Pelvis (left hip, right hip): These key points are used for movement analysis of the lower body, especially when walking or jumping.

[0113] 4) Lower body region

[0114] Knees (left knee, right knee): Used to capture the bending state and movement direction of the legs.

[0115] Ankles (left ankle, right ankle): Used to capture the landing situation and step changes of the feet.

[0116] Toes (left toe, right toe): Further refine foot movements in gait analysis.

[0117] 5) Hand region

[0118] Fingers (tips of five fingers): Depending on the action analysis application scenario, it can be determined whether to select hand region key points. In the application scenario of capturing hand details (such as gesture recognition or fine operation), the key points of the fingers may need to be detected.

[0119] These key points are widely used in motion capture and pose reconstruction. For example, in golf swing analysis, key points such as shoulders, hips, and knees are crucial for accurately determining the swing pose; in walking analysis, the detection of ankle and knee positions helps evaluate gait stability and balance. By accurately locating these human key points, the system can better perform tasks such as three-dimensional pose reconstruction and motion phase division.

[0120] 3.2 Extracting key point coordinates

[0121] The human key point detection module includes a key point position coordinate determination unit; the key point position coordinate determination unit is used to: perform argmax operation on the heat map of each key point in the target image to obtain the position coordinates of each key point.

[0122] Extracting key point coordinates from heat maps, finding the maximum probability position in the heat map through argmax operation:

[0123] k i =argmax(H i );

[0124] where k i =(x i ,y i ) is the predicted two-dimensional coordinate of the i-th key point, x i is the horizontal coordinate of the i-th key point, and y i is the vertical coordinate of the i-th key point. The key point set of each frame of image is:

[0125] K t ={k1,k2,…,k N};

[0126] where K t is the coordinate of all detected key points in the t-th frame of image, which includes N key points.

[0127] 3.4 Time series modeling and inter-frame smoothing based on xLSTM

[0128] In this step, we use xLSTM to smooth the key points in the time series to ensure consistency and continuity in time. xLSTM uses the key point sequences K t-1 , K t , K t+1 of the previous and subsequent frames to model, and outputs the smoothed key point sequence. The formula is as follows:

[0129]

[0130] where, is the key point sequence after time series modeling and smoothing, that is, the final human key point detection result; K t-1 is the key point coordinate of the t-1 frame image; K t+1 is the key point coordinate of the t+1 frame image. By introducing the exponential gate and matrix memory unit of the xLSTM model, the time series smoothing of the key points can be more effectively processed. The xLSTM model performs well in processing complex time dependence and long sequence tasks, ensuring that the smoothing of the key points has higher stability and robustness.

[0131] The loss function used in the human key point detection process includes key point position loss and smoothing loss The calculation formulas are as follows:

[0132]

[0133] Wherein, k i,GT is the true key point position label.

[0134] The fourth stage:

[0135] The motion stage division module includes a special feature extraction unit, a global modeling unit and a classification unit.

[0136] The special feature extraction unit is configured to: use a third-generation deformable convolutional neural network to extract features from the final feature map corresponding to each target image, to obtain a special feature corresponding to each target image.

[0137] The global modeling unit is configured to: input the special feature corresponding to each target image into a Transformer model to obtain a global feature corresponding to each target image.

[0138] The classification unit is configured to: classify the action in each target image according to the global feature corresponding to each target image, to obtain a stage classification result of each target image.

[0139] Motion segmentation is an important task in sports analysis. In sports such as golf, badminton, tennis, and table tennis, different movements play important roles and have significant meanings. For example, in golf, the "swing motion" is a complex and multi-level process, and small changes in many places can affect the final ball hitting effect. In order to more accurately evaluate the swing technique of athletes and guide the swing motion of athletes, it is particularly important to divide the swing process into different stages. This more detailed division not only helps to improve the understanding of the details of the swing, but also provides more specific and effective technical guidance for coaches and athletes. Here we take the motion segmentation of the golf swing process as an example to explain, similarly, this method can be applied to the segmentation of other sports.

[0140] A complete golf swing can be located in eight key actions, which are: Address, Toe-up, Mid-backswing, Top, Middownswing, Impact, Mid-follow-through, and Finish. Therefore, for a complete video (i.e., image sequence), we need to determine the action category to which each frame of image belongs.

[0141] 4.1 Feature processing based on third generation deformable convolution DCNv3

[0142] In the previous network architecture, a complete image feature has been constructed in the first stage "feature extraction and interaction fusion" to achieve the task of motion segmentation (such as swing stage recognition). Considering that the final feature map constructed before is a general feature serving different tasks, we need to process the final feature map in the motion segmentation task. Specifically, the final feature map is processed by using a deformable convolution network (DCNv3). In order to capture the dynamic motion features in the motion process (such as the swing process), the third generation deformable convolution neural network (DCNv3) is introduced, so as to convert the general feature into a special feature required by the motion segmentation task

[0143]

[0144] where f DCNv3is a third generation deformable convolutional neural network that divides input features into G groups and allows each group of convolutional kernels to adaptively adjust the sampling points in different spatial regions. By adaptively learning the spatial offset of the convolutional kernel, it can better handle local changes in poses, especially suitable for capturing motion features. DCNv3 first calculates the offset Δp k of each convolutional kernel through a convolutional network. This offset is adaptive and dynamically adjusted according to the input feature map

[0145]

[0146] The convolution process of DCNv3 can be represented as:

[0147]

[0148] where w g is the weight of the gth group, is the modulation scalar, x g is the feature of the gth group; y(p0) represents the output feature value at position p0, which is the output result obtained after the deformable convolution operation. This value is calculated by convolving the input feature x g of different groups g with their corresponding convolution kernel offset and modulation coefficient ; k represents the index of the convolution kernel, which is used to represent the kth convolution kernel in each group g. Since the convolution operation may contain multiple kernels, k is used to traverse these convolution kernels and perform convolution operation at each position, and the modulation coefficient is adjusted. In summary, y(p0) in the formula is the output result obtained by calculating the input feature x g through multiple groups of convolution kernels, offsets and modulation coefficients. And k is responsible for indexing different convolution kernels in each group of convolution kernels.

[0149] 4.2 Feature temporal correlation modeling based on Transformer

[0150] The specialized features processed by DCNv3 can capture subtle changes in space, but the division of the swing phase not only depends on the features of a single frame of image, but also needs to model the dependence of the time series globally. To achieve this, we introduce the Transformer model. The Transformer model has strong global modeling capability, especially when dealing with sequence data, it can effectively capture long-range dependencies between time steps through self-attention mechanism. The following is the feature sequence processed by DCNv3 ​Input into the multi-layer Transformer model for global modeling of time step features. The self-attention mechanism in the Transformer model captures the association between different time steps by calculating the similarity between time steps. For each time step t, the self-attention mechanism obtains the global context information of the time step by calculating the relationship of the query matrix Q t , the key matrix K t and the value matrix V t :

[0151]

[0152] where Z t is the global feature obtained after Transformer processing, which contains the dependency between time steps.

[0153] 4.3 Motion segmentation task head

[0154] After obtaining the global features {Z1, Z2, …, Z T} processed by the Transformer, we need to design a task head to perform the motion phase segmentation task, i.e., to determine the category to which each frame of image in the image sequence belongs. Therefore, the task head will use a classifier to complete classification based on the global features Z1, Z2, …, Z T , so as to determine which phase of motion each time step image is in.

[0155] Specifically, we perform phase classification on the global feature of each time step through a fully connected network, outputting the probability of the image at that time belonging to different motion phases. The task of this network is to assign a phase label (e.g., preparation phase, starting phase, acceleration phase, swing completion phase, etc.) to each time step based on the input feature:

[0156]

[0157] where, is the phase classification result of time step t, W c and b c are the weights and biases of the fully connected layer.

[0158] 4.4 Motion segmentation boundary optimization

[0159] Through the classifier, the prediction result of the motion phase to which each time step image belongs is obtained, but in order to ensure that the boundaries between different motion phases obtained by segmentation are reasonable and smooth, the present application further introduces dynamic programming to optimize the phase boundaries. The dynamic programming algorithm is used to minimize the global cost function of phase classification, ensuring that the boundaries between phases are coherent and natural. The goal of this algorithm is to find the optimal phase boundary point, so that the classification result has a smooth transition over the entire time sequence.

[0160]

[0161] where, is a penalty function using second-order difference penalty to ensure smooth transition of boundary changes:

[0162]

[0163] After the above steps, the output of the final model is the phase classification result of each time step The motion phase division result can be used for subsequent motion analysis or further swing phase detection and adjustment. represent the positions of the motion phase boundaries, which are the optimal phase dividing points found by the dynamic programming algorithm. These boundary points ensure smooth transition of the segmented motion phases over the entire time series and maintain reasonable phase changes. These boundary will affect the motion phase classification result of each time step, making the phase division result more natural and continuous. t represents the phase classification result of the t-th time step image. Specifically, t is the motion phase class label output by the classifier at a certain time, indicating the motion phase to which the current time step belongs (such as the preparation phase, swing phase, etc.). This classification result is the output of the final motion phase detection, and after optimization of the boundary adjustment, it can be used for subsequent motion analysis. In summary, is the boundary position of the motion phase segmentation, and t is the predicted motion phase class of each time step. By optimizing can ensure smooth and accurate segmentation of the motion phase.

[0164] Fifth stage:

[0165] The three-dimensional pose reconstruction module is configured to: project the smoothed key point sequence corresponding to the target image to obtain an initial high-dimensional embedding representation corresponding to each target image; process the initial high-dimensional embedding representation corresponding to each target image using dynamic spatial graph convolution to obtain a dynamic spatial graph convolution feature corresponding to each target image; perform time series processing on the dynamic spatial graph convolution feature corresponding to each target image using a Transformer model to obtain a final high-dimensional feature representation of the key points in each target image; and perform three-dimensional mapping processing on the final high-dimensional feature representation of the key points in each target image to obtain a three-dimensional reconstruction result of the key points in each target image.

[0166] Three-dimensional human pose reconstruction is to recover the three-dimensional human joint coordinates from two-dimensional human key points, which is crucial for applications such as motion analysis and virtual reality. For this purpose, we use a three-dimensional pose reconstruction algorithm based on graph neural network GNN and Transformer: dynamic graph convolution-Transformer (DGCN-Transformer) architecture.

[0167] 5.1 Input and initial embedding

[0168] In the third stage of human key point detection, the two-dimensional key point sequence of each frame image is obtained, In order to perform high-dimensional embedding on two-dimensional key points, the application uses a linear layer to project these key points to obtain the initial high-dimensional embedding representation X of the two-dimensional key points at time t t :

[0169]

[0170] where W0 and b0 are learnable parameters.

[0171] 5.2 Dynamic spatio-temporal graph convolution-Transformer architecture

[0172] Graph convolutional network (GCN) can capture local spatial relationships, which makes it very suitable for representing and remembering the relationship between human key points, thus achieving better three-dimensional pose reconstruction. However, traditional graph convolutional neural networks rely on a pre-defined adjacency matrix, which limits its flexibility. In order to enable the model to dynamically update the graph structure, we use dynamic graph convolution to replace the traditional graph convolution operation, allowing the model to adaptively adjust the structure of the graph and update the adjacency matrix A based on the input data. Specifically, the adjacency matrix A representing the relationship between graph nodes is no longer pre-defined, but is based on the input high-dimensional embedding vector X t which is dynamically calculated as:

[0173]

[0174] where φ(·) is a function that calculates the similarity of features (such as Euclidean distance, Mahalanobis distance), and here we use cosine similarity. Therefore, we give the dynamic spatial graph convolution formula as follows:

[0175]

[0176] where D dynamic is the degree matrix calculated according to the dynamic adjacency matrix .

[0177] After dynamic graph convolution processing, the dynamic graph convolution features of each frame are obtained is input into the Transformer architecture to model the temporal correlation, and the final high-dimensional feature representation X of the two-dimensional key points in each target image is obtained final,t :

[0178] X final,t =Attention time (Q time,t ,K time,t ,V time,t )

[0179] wherein, In our DGCN-Transformer architecture, the DGCN is responsible for learning and modeling the spatial correlation of the input data, and the Transformer is responsible for learning and modeling the temporal correlation of the input data. Therefore, the output X final,t fully integrates and mines the spatial correlation and temporal correlation of the input two-dimensional key point sequence. Next, it is mapped to a three-dimensional coordinate through a linear regression head:

[0180]

[0181] wherein W r and b r are learnable parameters, is the reconstruction result of the human key points in the time t frame.

[0182] In order to strengthen the capture of temporal and spatial correlation, we introduce an adaptive spatio-temporal loss (Adaptive Spatio-Temporal Loss) in the proposed dynamic spatio-temporal graph convolution-spatio-temporal Transformer architecture During the model training process, the adaptive spatio-temporal loss is used to train the three-dimensional human pose reconstruction model, which is composed of a linear layer, a dynamic spatio-temporal graph convolution, a Transformer model and a linear regression head. This loss function can dynamically adjust the weight of each time step and spatial position:

[0183]

[0184] wherein w t,n is the adaptive weight of each time step and spatial position, which is dynamically learned by the model, and X 3D,t,n is the real three-dimensional coordinate of the human key points in the time t frame.

[0185] The sixth stage:

[0186] The intelligent analysis module is configured to: analyze and evaluate the motion action in the target image sequence according to the three-dimensional reconstruction result of the key points in the target image, and obtain an analysis result.

[0187] The intelligent analysis module is configured to analyze and evaluate the motion action in the target image sequence according to the target image stage classification result and the three-dimensional reconstruction result of the key points in the target image, and obtain an analysis result.

[0188] The application further provides an application scenario of the monocular camera-based three-dimensional motion capture and intelligent analysis system. Specifically, the monocular camera-based three-dimensional motion capture and intelligent analysis system can be applied in a golf motion analysis scenario.

[0189] This stage applies the aforementioned monocular camera-based human motion capture technology to golf motion analysis, particularly calculating the X-factor in a golf swing. The X-factor is an important parameter in golf kinematics, which reflects the difference in rotation angle between the shoulder and the pelvis during the swing, and is usually used to evaluate the player's trunk rotation ability and efficiency when hitting the ball.

[0190] 6.1 Calculation of X-factor

[0191] The X-factor refers to the difference in rotation angle between the shoulder and the hip during the golf swing. A larger X-factor is usually associated with greater swing power and longer hitting distance. The calculation steps are as follows:

[0192] (1) Key point extraction:

[0193] The shoulder key points include the left shoulder point and the right shoulder point:

[0194] Left shoulder point: S L =[x SL ,y SL ,z SL ];

[0195] Right shoulder point: S L =[x SR ,y SR ,z SR ]。

[0196] The hip key points include the left hip point and the right hip point:

[0197] Left hip point: H L =[x HL ,y HL ,z HL ];

[0198] Right hip point: H R =[x HR ,y HR ,z HR ]。

[0199] In the formula, x SL ,ySL , z SL are the coordinates of the left shoulder point in the x, y, and z directions, respectively; x SR , y SR , z SR are the coordinates of the right shoulder point in the x, y, and z directions, respectively; x HL , y HL , z HL are the coordinates of the left hip point in the x, y, and z directions, respectively; x HR , y HR , z HR are the coordinates of the right hip point in the x, y, and z directions, respectively.

[0200] (2) Define shoulder and hip vectors:

[0201] V shoulder = S R - S L ;

[0202] V hip = H R - H L ;

[0203] where V shoulder represents the shoulder vector, and V hip represents the hip vector.

[0204] (3) Project onto the horizontal plane: To calculate the horizontal rotation angle, project the above shoulder and hip vectors onto the horizontal plane (usually the XY plane):

[0205]

[0206] where v sx = x SR - x SL , v sy = y SR - y SL ; v hx = x HR - x HL , v hy = y HR - y HL ; denotes the shoulder vector projection, denotes the hip vector projection.

[0207] (4) Calculate the angle of the vectors:

[0208] θ shoulder = arctan 2(v sy , v sx );

[0209] θ hip = arctan 2(v hy , v hx );

[0210] where θ shoulder represents the shoulder angle and θ hip represents the hip angle.

[0211] (5) Calculate the X-factor:

[0212] X factor = θ shoulder - θ hip ;

[0213] When the X-factor X factor is positive, it indicates that the shoulder is rotating backward relative to the hip. When the X-factor X factor is negative, it indicates that the shoulder is rotating forward relative to the hip.

[0214] (6) Time series analysis:

[0215] For each time frame t, calculate the corresponding X-factor X factor , and obtain the X-factor change curve throughout the swing. At the same time, calculate the X-factor at the key moments:

[0216] Top of Backswing: usually the moment when the X-factor reaches the maximum value.

[0217] Impact: observe the regression degree of the X-factor to assess the energy release efficiency.

[0218] 6.2 Motion analysis

[0219] Through the above, accurate motion capture results can be obtained, and indicators such as X-factor can be calculated based on the motion capture results. Based on these indicators, more detailed analysis of the motion process can be carried out. For example, draw the curve of X-factor changing with time, intuitively display the rotation difference in the swing process; compare the X-factor curves of different swing attempts or different players horizontally; train the model to predict the optimal X-factor range and improve the hitting effect; identify abnormal patterns and provide real-time feedback and correction suggestions. Practical applications include but are not limited to:

[0220] (1) Technical evaluation and improvement:

[0221] Power output: by analyzing the maximum value and change rate of the X-factor, the power output ability of the player is evaluated.

[0222] Coordination of movements: Observe the synchronization of shoulder and hip rotations to identify potential issues of movement incoordination.

[0223] Swing efficiency: Evaluate the efficiency of energy transfer and release through changes in X-factor at different stages.

[0224] (2) Injury prevention:

[0225] Risk of over-rotation: Excessive X-factor may increase the risk of lower back injury, and timely adjustments can prevent injury.

[0226] Asymmetry detection: Find the difference in left and right side rotations to guide the player to targeted training and balance.

[0227] (3) Personalized training program development:

[0228] Targeted training: Develop training plans to improve core strength and flexibility based on X-factor analysis results.

[0229] Progress tracking: Evaluate training effectiveness and progress by continuously monitoring changes in X-factor.

[0230] In summary, by designing a monocular camera-based three-dimensional human key point capture method and applying it to golf motion analysis, the player's motion details can be captured and analyzed with high precision, especially the key X-factor indicators. This provides a scientific basis and effective means for golf technical training, performance evaluation and injury prevention, promoting the intelligentization and data-driven of golf motion analysis.

[0231] The present application has the following beneficial effects:

[0232] 1. The present application first applies the Mamba model to human motion capture systems. By constructing a multi-scale feature pyramid structure, the system can refine features at different resolutions; the Mamba model further selects a state space model to adaptively fuse multi-scale features in time series, and generates a fused feature map through convolution and up-sampling operations. This multi-scale feature interaction fusion technology significantly improves the accuracy and robustness of key point detection, especially suitable for processing complex human motion postures.

[0233] 2. To address the limitations of traditional graph convolutional networks (GCN) in handling spatial relationships between human key points, the present application proposes a DGCN-Transformer architecture. This architecture allows the system to dynamically adjust the adjacency matrix based on input data through dynamic graph convolution, enhancing the ability to model spatial correlations of human poses. Combined with the global modeling capability of the Transformer model for time series, it achieves more accurate three-dimensional pose reconstruction, especially suitable for motion analysis and virtual reality applications.

[0234] 3. To address the need for stage division in complex motion processes such as swings, the present application introduces the third generation deformable convolutional network (DCNv3). DCNv3 captures the local changes in poses during the motion process by adaptively learning the spatial offset of the convolution kernel, enabling precise handling of subtle differences in actions. Combined with the modeling of temporal correlations by the Transformer, it achieves fine division of motion stages, providing more accurate technical support for motion analysis.

[0235] 4. To address the consistency of key point detection over time series, the present application introduces a time series smoothing technique based on xLSTM. xLSTM combines key point information from previous and subsequent frames, using its innovative exponential gating and matrix memory unit to smooth the key point sequence, ensuring temporal consistency and coherence of key point detection. The application of this technology further improves the accuracy and stability of human pose detection.

[0236] 5. The present application adopts a multi-task learning framework, integrating human key point detection, motion stage division, and three-dimensional pose reconstruction into the same framework. By sharing the underlying feature extraction network (such as ResNet-50 and Mamba model), different tasks can effectively share information, improving the efficiency and performance of the overall system. In addition, a joint loss function is introduced between tasks, optimizing the loss of multiple tasks simultaneously, making the results of each task more consistent and improving the robustness and accuracy of the entire system.

[0237] 6. The present application forms a complete system framework from input data acquisition, preprocessing, feature extraction, key point detection, motion stage division, to three-dimensional pose reconstruction. Adaptive processing mechanisms and dynamic adjustment strategies are introduced at each stage to ensure the adaptability and stability of the system in different usage scenarios. The optimization design of the whole process makes the system not only suitable for single scenarios, but also can be extended to various complex application environments, such as sports training, medical rehabilitation, virtual reality, etc.

[0238] 7. The system architecture of the present application adopts a modular design, and each functional module can be independently run or combined with other modules. This design greatly improves the scalability of the system, enabling flexible adjustments to the system according to specific application requirements. For example, for different sensor data types, corresponding feature extraction and alignment modules can be flexibly introduced or replaced. This scalability not only extends the life cycle of the system, but also increases its applicability in different application scenarios, ensuring the system's continued competitiveness in future technological developments.

[0239] 8. In terms of application, the present application accurately captures three-dimensional human motion through a monocular camera and applies it to precise and intelligent analysis of golf, achieving comprehensive analysis of the swing trajectory and posture of a golfer. The system not only reconstructs the three-dimensional motion trajectory of key points of the human body through a monocular camera, but also generates key biomechanical parameters such as X-Factor and Tight Factor. Through this precise motion capture and analysis, the system can provide precise technical guidance for golf beginners and help coaches develop personalized training programs, significantly improving the training efficiency and scientific nature of golf.

[0240] Based on the same inventive concept, the present application also provides a monocular camera-based three-dimensional motion capture and intelligent analysis method for implementing the above-mentioned monocular camera-based three-dimensional motion capture and intelligent analysis system. The problem-solving implementation scheme provided by this method is similar to the implementation scheme described in the above system, so the specific limitations in one or more monocular camera-based three-dimensional motion capture and intelligent analysis method embodiments provided below can refer to the limitations of the monocular camera-based three-dimensional motion capture and intelligent analysis system described above, and will not be repeated here.

[0241] In one exemplary embodiment, as shown in Figure 3 a monocular camera-based three-dimensional motion capture and intelligent analysis method includes:

[0242] S1: Preprocess the original image sequence to obtain a target image sequence; the original image sequence is obtained by a monocular camera; the target image sequence includes a plurality of consecutive frame target images;

[0243] S2: performing feature extraction on each of the target images to obtain an initial feature map corresponding to each of the target images; for each of the initial feature maps, performing multi-scale feature extraction on the initial feature map to obtain a down-sampled feature map at different scales; using a Mamba model, performing fusion and serialization processing on the down-sampled feature map at each scale to obtain an efficient feature map at different scales; performing up-sampling operation on the efficient feature map at each scale to obtain an up-sampled feature map at different scales; fusing the up-sampled feature maps at different scales to obtain a multi-scale feature map corresponding to each of the target images; performing feature reconstruction on the multi-scale feature map corresponding to each of the target images to obtain a final feature map corresponding to each of the target images;

[0244] S3: performing feature extraction on the final feature map corresponding to each of the target images to generate a heat map of each key point in the target image; determining the position coordinates of each key point based on the heat map of each key point in the target image; using an xLSTM model, performing smoothing processing on the position coordinates of each key point in the target image, the previous frame image and the next frame image of the target image to obtain a smoothed key point sequence corresponding to each of the target images;

[0245] S4: performing projection processing on the smoothed key point sequence corresponding to the target image to obtain an initial high-dimensional embedding representation corresponding to each of the target images; using dynamic spatial graph convolution, processing the initial high-dimensional embedding representation corresponding to each of the target images to obtain a dynamic spatial graph convolution feature corresponding to each frame of target image; using a Transformer model, performing time series processing on the dynamic spatial graph convolution feature corresponding to each of the target images to obtain a final high-dimensional feature representation of the key points in each of the target images; performing three-dimensional mapping processing on the final high-dimensional feature representation of the key points in each of the target images to obtain a three-dimensional reconstruction result of the key points in each of the target images;

[0246] S5: analyzing and evaluating the motion action in the target image sequence according to the three-dimensional reconstruction result of the key points in the target image to obtain an analysis result.

[0247] As an optional implementation, the monocular camera-based three-dimensional motion capture and intelligent analysis method further comprises:

[0248] using a third-generation deformable convolutional neural network to perform feature extraction on the final feature map corresponding to each of the target images to obtain a dedicated feature corresponding to each of the target images;

[0249] inputting the dedicated feature corresponding to each of the target images into a Transformer model to obtain a global feature corresponding to each of the target images;

[0250] According to the global feature corresponding to each target image, the action in each target image is classified to obtain a stage classification result of each target image.

[0251] S5 specifically comprises: according to the stage classification result of each target image and the three-dimensional reconstruction result of the key points in the target image, analyzing and evaluating the motion action in the target image sequence to obtain an analysis result.

[0252] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0253] The principles and implementation modes of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method and its core idea of the present application. Meanwhile, for those skilled in the art, the specific implementation modes and application ranges can be changed according to the idea of the present application. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A monocular camera based three-dimensional motion capture and intelligent analysis system, characterized in that, The monocular camera-based three-dimensional motion capture and intelligent analysis system comprises: An input and preprocessing module, configured to: pre-process an original image sequence to obtain a target image sequence; the original image sequence is obtained by a monocular camera; and the target image sequence comprises a plurality of continuous frame target images; A feature extraction and interactive fusion module, configured to: perform feature extraction on each target image to obtain an initial feature map corresponding to each target image; perform multi-scale feature extraction on each initial feature map to obtain a down-sampled feature map at different scales; perform fusion and serialization processing on the down-sampled feature map at each scale by using a Mamba model to obtain an efficient feature map at different scales; perform up-sampling operation on the efficient feature map at each scale to obtain an up-sampled feature map at different scales; fuse the up-sampled feature maps at different scales to obtain a multi-scale feature map corresponding to each target image; and perform feature reconstruction on the multi-scale feature map corresponding to each target image to obtain a final feature map corresponding to each target image; A human key point detection module, configured to: perform feature extraction on the final feature map corresponding to each target image to generate a heat map of each key point in the target image; determine the position coordinates of each key point based on the heat map of each key point in the target image; and perform smoothing processing on the position coordinates of each key point in the target image, a previous frame image and a next frame image of the target image by using an xLSTM model to obtain a smoothed key point sequence corresponding to each target image; A three-dimensional pose reconstruction module, configured to: perform projection processing on the smoothed key point sequence corresponding to each target image to obtain an initial high-dimensional embedding representation corresponding to each target image; perform processing on the initial high-dimensional embedding representation corresponding to each target image by using dynamic spatial graph convolution to obtain a dynamic spatial graph convolution feature corresponding to each frame target image; perform time series processing on the dynamic spatial graph convolution feature corresponding to each target image by using a Transformer model to obtain a final high-dimensional feature representation of a key point in each target image; and perform three-dimensional mapping processing on the final high-dimensional feature representation of the key point in each target image to obtain a three-dimensional reconstruction result of the key point in each target image; An intelligent analysis module, configured to: analyze and evaluate a motion action in the target image sequence according to the three-dimensional reconstruction result of the key point in the target image to obtain an analysis result.

2. The monocular camera-based three-dimensional motion capture and intelligent analysis system according to claim 1, wherein, The monocular camera-based three-dimensional motion capture and intelligent analysis system further comprises a motion stage division module, and the motion stage division module comprises: A dedicated feature extraction unit, configured to: perform feature extraction on the final feature map corresponding to each target image by using a third-generation deformable convolutional neural network to obtain a dedicated feature corresponding to each target image; A global modeling unit, configured to: input the dedicated feature corresponding to each target image into a Transformer model to obtain a global feature corresponding to each target image. The classification unit is configured to classify the action in each target image according to the global feature corresponding to each target image, and obtain a stage classification result of each target image.

3. The monocular camera-based three-dimensional motion capture and intelligent analysis system according to claim 2, wherein, The intelligent analysis module is configured to analyze and evaluate the motion action in the target image sequence according to the stage classification result of each target image and the three-dimensional reconstruction result of the key points in the target image, and obtain an analysis result.

4. The monocular camera-based three-dimensional motion capture and intelligent analysis system according to claim 1, wherein, The original image sequence includes a plurality of frames of original images; the original images are obtained by a monocular camera during the golf movement of a user.

5. The monocular camera-based three-dimensional motion capture and intelligent analysis system according to claim 1, wherein, The feature extraction and interaction fusion module includes a multi-scale feature extraction unit; the multi-scale feature extraction unit is configured to perform multi-scale feature extraction on the initial feature map of each target image by using a multi-scale feature pyramid, and obtain a down-sampling feature map at different scales.

6. The monocular camera-based three-dimensional motion capture and intelligent analysis system according to claim 1, wherein, The human key point detection module includes a heat map generation unit; the heat map generation unit is configured to perform feature extraction on the final feature map corresponding to each target image by using a multi-layer convolutional neural network, and generate a heat map of each key point in the target image; the multi-layer convolutional neural network is composed of a plurality of convolutional neural networks connected in sequence.

7. The monocular camera-based three-dimensional motion capture and intelligent analysis system according to claim 1, wherein, The human key point detection module includes a key point position coordinate determination unit; the key point position coordinate determination unit is configured to perform argmax operation on the heat map of each key point in the target image, and obtain the position coordinates of each key point.

8. A monocular camera based three-dimensional motion capture and intelligent analysis method based on the monocular camera based three-dimensional motion capture and intelligent analysis system according to any one of claims 1-7, characterized in that, The three-dimensional motion capture and intelligent analysis method based on a monocular camera includes: The original image sequence is preprocessed to obtain a target image sequence; the original image sequence is obtained by a monocular camera; the target image sequence includes a plurality of continuous frames of target images; Feature extraction is performed on each target image to obtain an initial feature map corresponding to each target image; multi-scale feature extraction is performed on the initial feature map to obtain a down-sampling feature map at different scales; a Mamba model is used to perform fusion and serialization processing on the down-sampling feature map at each scale to obtain an efficient feature map at different scales; up-sampling operation is performed on the efficient feature map at each scale to obtain an up-sampling feature map at different scales; the up-sampling feature maps at different scales are fused to obtain a multi-scale feature map corresponding to each target image; feature reconstruction is performed on the multi-scale feature map corresponding to each target image to obtain a final feature map corresponding to each target image; Feature extraction is performed on the final feature map corresponding to each target image to generate a heat map of each key point in the target image; the position coordinates of each key point are determined based on the heat map of each key point in the target image; an xLSTM model is used to perform smoothing processing on the position coordinates of each key point, the previous frame of image and the next frame of image of the target image, to obtain a smoothed key point sequence corresponding to each target image; Projecting the smoothed key point sequence corresponding to the target image to obtain an initial high-dimensional embedding representation corresponding to each target image; processing the initial high-dimensional embedding representation corresponding to each target image using dynamic spatial graph convolution to obtain a dynamic spatial graph convolution feature corresponding to each target image; processing the dynamic spatial graph convolution feature corresponding to each target image in time sequence using a Transformer model to obtain a final high-dimensional feature representation of the key points in each target image; performing three-dimensional mapping processing on the final high-dimensional feature representation of the key points in each target image to obtain a three-dimensional reconstruction result of the key points in each target image; According to the three-dimensional reconstruction result of the key points in the target image, the motion action in the target image sequence is analyzed and evaluated to obtain an analysis result.

9. The monocular camera-based three-dimensional motion capture and intelligent analysis method according to claim 8, wherein, The three-dimensional motion capture and intelligent analysis method based on a monocular camera further comprises: extracting features from the final feature map corresponding to each target image using a third-generation deformable convolutional neural network to obtain a dedicated feature corresponding to each target image; inputting the dedicated feature corresponding to each target image into a Transformer model to obtain a global feature corresponding to each target image; classifying the action in each target image according to the global feature corresponding to each target image to obtain a stage classification result of each target image.

10. The monocular camera-based three-dimensional motion capture and intelligent analysis method according to claim 9, wherein, According to the three-dimensional reconstruction result of the key points in the target image, the motion action in the target image sequence is analyzed and evaluated to obtain an analysis result, specifically comprising: According to the three-dimensional reconstruction result of the key points in the target image, the motion action in the target image sequence is analyzed and evaluated to obtain an analysis result, specifically comprising: According to the three-dimensional reconstruction result of the key points in the target image, the motion action in the target image sequence is analyzed and evaluated to obtain an analysis result, specifically comprising:

Images