A volleyball court three-dimensional reconstruction and player motion capture method

Abstract

The present disclosure provides a volleyball court three-dimensional reconstruction and player motion capture method, which is suitable for the analysis task of the interaction between players and the court in the volleyball scene. First, the court reconstruction and camera estimation are performed: the three-dimensional geometric structure of the input volleyball video sequence is reconstructed to obtain the three-dimensional point cloud representation and the camera extrinsic information of the court; then the player posture estimation is performed: through video frame human region detection cropping and feature extraction, human key point positioning and coordinate decoding, human posture parameter regression and time consistency constraint, the three-dimensional posture of the volleyball player is estimated; finally, the volleyball court geometry and the global motion trajectory consistent with the spatial structure thereof are output. Through the joint three-dimensional modeling of the volleyball court and the player, a stable, continuous and directly observable three-dimensional motion process and scene structure can be obtained under the condition of single video input, which is convenient for analyzing the volleyball technical action and its spatial relationship.

Description

Technical Field

[0001] This disclosure relates to the field of full-body motion capture in computer vision, and more specifically, to a method for 3D reconstruction of a volleyball court and motion capture of athletes. Background Technology

[0002] With the development of computer vision and 3D reconstruction technologies, video-based global human motion capture has gradually become an important supporting technology in fields such as virtual reality, augmented reality, digital human modeling, film and animation, sports analysis, and intelligent referee assistance. This type of technology aims to recover the spatial posture of the human body in the world coordinate system and its temporal trajectory from monocular or multi-view video data, thereby achieving precise modeling and continuous reconstruction of human motion.

[0003] In team ball sports such as volleyball, human motion capture not only serves the analysis of sports performance, but is also directly related to the determination of athletes' positions, the evaluation of technical movements, and the objective recording of the competition process, which puts forward higher requirements for the spatial scale consistency and temporal continuity of the reconstruction results.

[0004] Current research on global human motion capture has evolved from "human-centered motion recovery" to "cooperative reconstruction of human body, scene, and camera." Early and mainstream research typically focused on human motion, estimating human poses in input video sequences by combining multi-view geometry, visual odometry, or deep learning models, and constructing continuous human motion trajectories in the temporal dimension. These methods, on the one hand, recover camera motion through traditional geometric constraints and stitch together the local human reconstruction results using a world coordinate system to obtain the global human trajectory; on the other hand, with the development of deep neural networks, researchers have gradually introduced temporal neural networks or spatiotemporal Transformer models, using large-scale data to learn the spatiotemporal patterns of human motion, achieving end-to-end prediction of the human's global pose and motion process in world space. Some methods also incorporate human dynamics or physical priors, such as foot contact constraints and motion smoothing constraints, to enhance the physical plausibility and temporal continuity of the motion results. However, in highly dynamic sports scenarios such as volleyball, where multiple people participate, there are frequent situations such as jumping, sudden stops, turning, and multiple people obstructing the view during the movement. Traditional human-centered modeling methods still face significant challenges in terms of global-scale stability and long-term temporal consistency.

[0005] Building upon this foundation, recent research has further integrated 3D scene reconstruction, exploring a global motion capture framework that simultaneously recovers human pose, camera trajectory, and environmental geometry. Related methods improve the world-scale consistency and physical plausibility of human trajectories by introducing scene geometry as spatial constraints and anchoring human motion results to the real-world coordinate system. For example, some methods utilize human geometry or statistical characteristics in scale inference during scene reconstruction to alleviate scale uncertainty when relying solely on visual information; others constrain the spatial relationships between the human body and the environment or scene semantic information by introducing human-environment contact relationships or scene semantic information, thereby enhancing the physical plausibility of human motion results; still others attempt to simultaneously predict scene depth, normal vectors, semantic information, and human pose parameters within a unified network framework, and leverage feature hinting mechanisms and lightweight model design to enable the system to possess near-real-time inference capabilities. Through these approaches, human motion is no longer estimated in isolation but forms a collaborative modeling relationship with scene geometry and camera motion, further improving the spatial consistency and physical plausibility of global motion capture results. For sports scenarios with clear court structure constraints, such as volleyball, scene geometry (such as the court plane, sideline areas, and mesh structure) provides a stable and repeatable spatial reference for human movement. This makes human-scene collaborative modeling particularly important in such applications.

[0006] Despite ongoing research, existing global human motion capture and 3D reconstruction methods still face several technical limitations in terms of accuracy, stability, and system practicality in complex real-world scenarios, making it difficult to directly serve the video analysis needs of spatially dependent sports such as volleyball. In volleyball matches, the movement, jumps, landing points, line touches, and net crossings of athletes all have close three-dimensional spatial relationships with the court structure. Traditional 2D videos can only provide the motion process from a planar perspective, making it difficult to intuitively present the true spatial position and trajectory of the human body and the court. Furthermore, due to the influence of shooting angle, camera transitions, and movement speed, the movements in the video often exhibit jitter, instability, or discontinuity, further increasing the difficulty of observing and analyzing technical movements. Therefore, there is an urgent need for a unified method that can simultaneously complete the 3D reconstruction of the volleyball court and athletes under a single video input condition, achieving a coordinated expression of human motion and court spatial structure, transforming ordinary video into a stable, continuous, and intuitively observable 3D motion process, thereby improving the intuitiveness and accuracy of volleyball motion analysis. Summary of the Invention

[0007] The purpose of this disclosure is to provide a method for volleyball court 3D reconstruction and athlete motion capture. By performing unified 3D modeling and collaborative reconstruction of the volleyball court and athletes, the method can realize the overall expression of human movement and the spatial relationship of the court in the video. This allows ordinary volleyball match videos to intuitively present a stable and continuous 3D motion process, thereby facilitating accurate observation and analysis of volleyball technical movements and their spatial relationship with the court.

[0008] In general, a method for 3D reconstruction of a volleyball court and motion capture of athletes is provided, including: S1 Field 3D Reconstruction and Camera Estimation: Based on volleyball match videos, targeting sports scenarios with clear field structure constraints, the 3D geometric structure of the volleyball field is reconstructed through a process of video input and keyframe construction, camera pose estimation, camera pose refinement and joint feature learning, local 3D geometric estimation in the camera coordinate system, learnable global spatial mapping and geometric alignment, multi-view point cloud fusion and scene reconstruction output. This process yields the field 3D point cloud in a unified world coordinate system and the corresponding camera extrinsic parameters. S2 Athlete Pose Estimation: Targeting the rapid movement, frequent jumping, and simultaneous presence of multiple athletes in volleyball match videos, based on the input video, through video frame human body region detection and cropping and feature extraction, human body key point localization and coordinate decoding, human pose parameter regression and time consistency constraint process, the three-dimensional pose estimation of athletes based on a parametric human body model is performed to obtain the pose parameters, shape parameters, and translation parameters of each athlete in the camera coordinate system. S3 Joint Optimization: In volleyball scenarios, there are clear spatial constraints between athletes and the playing field, and frequent airborne, landing and rapid posture changes occur during the movement. The input video and the athlete's body posture, volleyball court geometry information and camera parameters obtained from S1 and S2 are jointly optimized to establish a consistent spatial relationship between athletes, volleyball court and camera under a unified world coordinate system. The final output includes the volleyball court geometry and the global human motion posture and trajectory consistent with its spatial structure.

[0009] The specific method for video input and keyframe construction is as follows: Let the input video sequence be: in, Let H represent the RGB color image of frame t, where H and W are the height and width of the image, respectively. The time interval is... The sampling method selects a set of keyframes: ,in, This indicates the index of the selected keyframe. Indicates the sequence number is The image frames; this sampling method is suitable for long-term continuous shooting scenarios in volleyball match videos, which helps to reduce redundant calculations while ensuring time coverage.

[0010] Normalization preprocessing is performed on all keyframes: ,in and These represent the pixel mean and standard deviation of the dataset, respectively; the final result is a normalized keyframe sequence of length N. As input: .

[0011] The processes of camera pose estimation, camera pose refinement and joint feature learning, and local 3D geometric estimation in the camera coordinate system are as follows: The camera pose estimation first performs image analysis on each frame. Perform image block division: Where M is the segmented non-overlapping image patch. The number of [variables]. Introduce a set of learnable camera latent variables: Where N is the number of frames in the input sequence, and each frame is an image Corresponding to a camera parameter Image patch features are concatenated with camera latent variables and then input into the camera pose prediction network. Predicting the initial camera pose : Camera pose From translation vector and rotation vector express: The translation vector and rotation vector are defined as follows: , , Using the unit quaternion form, we have By introducing independent camera latent variables for each frame in the sequence dimension and performing joint optimization, the camera pose estimation process can adapt to the actual shooting conditions in volleyball match scenarios, such as long shooting time, limited perspective changes, but local shaking.

[0012] The camera pose refinement and joint feature learning introduces a multi-task joint network to refine and optimize the camera pose while simultaneously learning geometric feature representations. Introducing latent variables for camera pose refinement: Image features, initial camera pose, and latent variables are jointly input into the network. : ,in The optimized camera pose is derived from the translation vector. and rotation vector composition; Let D represent the set of implicit feature vectors of a point cloud in a local space, where D is the feature dimension; The local 3D geometry estimation in the camera coordinate system uses the set of implicit feature vectors of point clouds in the local space obtained by camera pose refinement and joint feature learning. Input into geometry decoder Generate a dense point map in the camera coordinate system and its corresponding confidence level: ,in This represents a 3D point map in the camera coordinate system of the i-th frame. This represents the 3D coordinates of each pixel in the corresponding confidence map. The generated local 3D point map covers the geometry around the athlete's body and the entire visible volleyball court plane, sideline area, and other stable geometric structures in the scene.

[0013] The specific method for reconstructing the three-dimensional geometry of a volleyball court through the learnable global spatial mapping and geometric alignment process is as follows: The learnable global spatial mapping and geometric alignment define a mapping network from local geometric features to global geometric features. Given the set of implicit feature vectors of point clouds in the local space and the optimized camera pose, complete the latent space mapping: ; Then through the global geometry decoder Obtain the geometric point map in the world coordinate system: ,in Represents the point cloud coordinates in the world coordinate system. This indicates the spatial confidence level of the point cloud; The multi-view point cloud fusion and site reconstruction output is a point cloud from all views, using a confidence threshold. Perform filtering and merging: ; Further sparsification is achieved using voxel filtering:

[0014] The final output includes multi-frame camera pose parameters. A 3D global point cloud of a volleyball game scene in a unified world coordinate system. And the point cloud after global geometric mapping corresponding to each viewpoint. .

[0015] The method for human body region detection, cropping, and feature extraction in the video frame is as follows: The keyframe sequence obtained based on the video input and keyframe construction For each frame of image Perform human detection to obtain the bounding boxes of one or more athletes in the video frame. The corresponding human image region is obtained by cropping each athlete according to the bounding box. Scale it to a uniform resolution This detection and cropping process is applicable to scenarios in volleyball where multiple players move simultaneously and there is partial occlusion. Feeding human images into a feature extraction network: ,in It represents the high-dimensional semantic features of the human body in image space.

[0016] The specific method for locating and decoding key human body points is as follows: A unified point positioning mechanism is adopted to maintain the spatial continuity and numerical stability of key point prediction under the conditions of frequent large-amplitude movements such as jumping, soaring, and rapid limb swings in volleyball. Let any three-dimensional point in the standard human body space be... For each query point, through the dot-coded network: ; Points are mapped to a high-dimensional feature space and then jointly input with image features into the localization network. : ,in Represents a two-dimensional heat map. Represents a three-dimensional voxel heatmap; The heatmap is decoded into continuous coordinates using the Soft-Argmax operation, where the locations of two-dimensional key points are... Three-dimensional relative coordinates By combining the camera parameters, the 3D key points in the camera coordinate system are obtained: ,in This is the back projection function based on the camera's intrinsic and extrinsic parameters.

[0017] The process of regressing human posture parameters and constraining time consistency is as follows: The complete set of key points is obtained by summing the key points in the camera coordinate system of each frame obtained from the human body key point localization and coordinate decoding: ; Based on the explicit modeling characteristics of parametric human body models in terms of human structure and kinematic relationships, a parametric regression network is used. Estimate human posture parameters: ,in These parameters are the joint rotation parameters, human body shape parameters, and human body global translation parameters, which together form a complete three-dimensional human body posture result.

[0018] Finally, to ensure the continuity and consistency of the athlete's posture in the video sequence, a temporal smoothing constraint is introduced: This suppresses posture jitter in volleyball match videos caused by rapid movement, occlusion switching, or detection errors, ensuring that the athlete's posture estimation results maintain reasonable changes between consecutive frames.

[0019] The specific method for obtaining the athlete's global motion posture and trajectory consistent with the spatial relationship of the volleyball court is as follows: First, we initialize and unify the representation of the joint variables: Let the case be in frame t: The camera parameters are: Human posture parameters are: The scene point cloud is represented as: in Indicates joint rotation parameters; Indicates human body shape parameters; This represents the global pose of the human body in the world coordinate system; Next, set a consistency constraint between the human body and the scene point cloud: The set of points on the surface of the human body model is represented as: To constrain the spatial consistency between volleyball players' bodies and the playing field, a point cloud distance constraint is introduced: ; Then, set regularization constraints for human posture and shape: Introducing parameter regularization terms for parametric human body models: , in: , The initial estimate is represented by two regularization terms, which are used to optimize the variables and ensure that the human posture generated during the optimization process remains natural and reasonable, preventing it from deviating too far from the initial posture or from becoming distorted due to rapid movement and short-term occlusion.

[0020] And two-dimensional reprojection consistency constraints: Projecting the three-dimensional joints of the human body onto the image plane: ,in: and the detected two-dimensional key points Apply constraints: This ensures that the 3D human posture remains consistent with image observation from all perspectives in the volleyball match video. Set global camera consistency constraints: Introducing a camera smoothing term: To suppress the instability of posture caused by camera movement, shaking or perspective switching during volleyball match video shooting; Thus, the objective function is jointly optimized: ,in These are the weight coefficients for each loss term. The optimal solution is obtained through iterative optimization. The joint optimization process simultaneously constrains human posture, scene geometry, and camera motion within a unified framework, enabling the results of each submodule to mutually correct each other and converge collaboratively.

[0021] Final output: Optimized global athlete pose sequence: Camera trajectory consistent with the volleyball court: ; Merged site location cloud: This allows for the acquisition of global human motion capture results that are consistent in scale and continuous in time within a volleyball game scenario.

[0022] The technical effects to be achieved by the embodiments of the present invention are as follows: This method is designed for volleyball scenarios. By tightly coupling human posture, court geometry, and camera parameters in a unified world coordinate system, it introduces point cloud consistency constraints, parameter regularization of the parameterized human model, 2D reprojection loss, and global camera loss to achieve unified 3D modeling and collaborative reconstruction of the volleyball court and athletes. This enables ordinary volleyball match videos to intuitively present a stable and continuous 3D motion process, thus facilitating accurate observation and analysis of volleyball techniques and their spatial relationship with the court. Attached Figure Description

[0023] The above and other objects and features of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings.

[0024] Figure 1 This is a schematic diagram illustrating the architecture of a volleyball court 3D reconstruction and athlete motion capture method according to an embodiment of the present disclosure.

[0025] Figure 2 This is a schematic diagram illustrating the effect of a volleyball court 3D reconstruction and athlete motion capture method according to an embodiment of the present disclosure. Detailed Implementation

[0026] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0027] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become clear upon understanding the disclosure of this application.

[0028] As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more.

[0029] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0030] In the specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, the element may be directly "on" another element, directly "connected to," or "bonded to" the other element, or one or more other elements may be present in between. Conversely, when an element is described as being "directly on" another element, "directly connected to," or "directly bonded to" another element, no other elements may be present in between.

[0031] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0032] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0033] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0034] Figure 1 This is a schematic diagram illustrating a method for three-dimensional reconstruction of a volleyball court and motion capture of athletes according to an embodiment of the present disclosure.

[0035] To achieve the aforementioned objectives, the present invention employs the following technical framework: Figure 1 As shown.

[0036] By combining high-precision local human pose estimation with 3D scene reconstruction from video, a robust global human motion capture and human-scene collaborative reconstruction method is proposed.

[0037] The technical solution includes the following steps: S1 3D Scene Reconstruction and Camera Estimation: Based on volleyball match videos, the 3D geometry of the volleyball scene is reconstructed to obtain the 3D point cloud of the scene in a unified world coordinate system and the corresponding camera extrinsic parameters. S2 Human Pose Estimation: Based on volleyball match videos, this method performs 3D pose estimation of athletes using a parametric human body model, obtaining the pose parameters, shape parameters, and translation parameters of each athlete in the camera coordinate system. S3 Joint Optimization: Jointly optimizes the input and the human pose, scene geometry information, camera parameters, etc. obtained from S1 and S2 to obtain the volleyball scene geometry and the global human motion pose and trajectory consistent with its spatial structure.

[0038] Unlike traditional loosely coupled or alternating optimization schemes, this method tightly couples human posture, court geometry, and camera parameters in a unified world coordinate system. It introduces point cloud consistency constraints, parameter regularization of the parameterized human model, 2D reprojection loss, and global camera loss to achieve unified 3D modeling and collaborative reconstruction of the volleyball court and athletes. This enables ordinary volleyball match videos to intuitively present a stable and continuous 3D motion process, thus facilitating accurate observation and analysis of volleyball techniques and their spatial relationship with the court.

[0039] Specifically, this includes estimating human 3D pose and global motion trajectory from video sequences, performing 3D geometric reconstruction of the scene based on visual information, and aligning and fusing the human body and the scene, joint training, and global optimization to achieve global collaborative reconstruction of human motion and scene details in complex real-world scenarios. The implementation steps are illustrated in the flowchart below. Figure 1 As shown.

[0040] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] Step 1: Scene 3D Reconstruction This step is used to reconstruct the 3D scene point cloud and estimate the camera pose from video keyframes.

[0042] S11: Video Input and Keyframe Construction This step is used to perform temporal sampling and standardization preprocessing on the input video data to build the foundation data for subsequent camera pose estimation and 3D reconstruction.

[0043] Suppose the input video sequence is: in, Let H represent the RGB color image of frame t, where H and W are the height and width of the image, respectively. The time interval is... The sampling method selects a set of keyframes:

[0044] in, This indicates the index of the selected keyframe. Indicates the sequence number is Image frames. To eliminate the impact of lighting differences on network inference, all keyframes undergo normalization preprocessing:

[0045] in and These represent the pixel mean and standard deviation of the dataset, respectively.

[0046] Finally, a normalized keyframe sequence of length N is obtained. As input: S12: Initial camera pose estimation This step aims to directly predict the initial camera pose parameters from each viewpoint using video frame images, providing a rough spatial constraint for subsequent 3D geometric estimation.

[0047] First, process each frame of image. Perform image block division: Where M is the segmented non-overlapping image patch The number of [variables]. Introduce a set of learnable camera latent variables: Where N is the number of frames in the input sequence, and each frame is an image Corresponding to a camera parameter Image patch features are concatenated with camera latent variables and then input into the camera pose prediction network. Predicting the initial camera pose : Camera pose From translation vector and rotation vector express: The translation vector and rotation vector are defined as follows: To avoid gimbal lock issues, a unit quaternion form is used. .

[0048] S13: Camera Pose Refinement and Joint Feature Learning Considering that the initial camera pose may contain errors, this step introduces a multi-task joint network to refine and optimize the camera pose and simultaneously learn geometric feature representations.

[0049] Introducing latent variables for camera pose refinement: Image features, initial camera pose, and latent variables are jointly input into the network. : in The optimized camera pose is derived from the translation vector. and rotation vector composition; Let D represent the set of implicit feature vectors of point clouds in local space, where D is the feature dimension.

[0050] Through joint training, the network can use camera pose error to inversely constrain and optimize geometric features, thereby improving the stability and accuracy of subsequent reconstruction.

[0051] S14: Local 3D geometric estimation in camera coordinate system This step generates a depth map or point map representation for each viewpoint based on local features, thus obtaining the three-dimensional geometric structure in the camera coordinate system.

[0052] Using the local geometric feature token obtained in S13 Input into geometry decoder Generate a dense point map in the camera coordinate system and its corresponding confidence level: in This represents a 3D point map in the camera coordinate system of the i-th frame. This represents the corresponding confidence map. The 3D coordinates of each pixel in the dot plot are as follows: S15: Learnable global spatial mapping and geometric alignment Since point maps from different perspectives are located in different camera coordinate systems, this step introduces a learnable geometric mapping network to transform them to a unified world coordinate system.

[0053] Define a mapping network from local geometric features to global geometric features. Input geometric implicit features and camera pose to complete latent space mapping: Then through the global geometry decoder Obtain the geometric point map in the world coordinate system: in Represents the point cloud coordinates in the world coordinate system. This represents the spatial confidence level of the point cloud.

[0054] The learnable method avoids the high sensitivity to camera pose accuracy in traditional geometric projection and enhances the system's robustness to camera errors.

[0055] S16: Multi-view point cloud fusion and scene reconstruction output This step fuses 3D point clouds generated from multiple perspectives to construct a complete 3D reconstruction model of the scene.

[0056] For point clouds from all viewpoints, use a confidence threshold. Perform filtering and merging: Further sparsification is achieved using voxel filtering: The final output includes multi-frame camera pose parameters. 3D global point cloud Point clouds after global geometric mapping from various perspectives .

[0057] Step 2: Human Pose Estimation This module is used to estimate the position and pose parameters of human body 3D key points from video keyframes. By constructing a unified human body point localization network, it achieves continuous prediction of human skeleton key points and dense surface points. Combined with pose parameter regression and time consistency constraints, it completes the stable reconstruction of human body 3D pose and global motion in video, providing human motion priors for subsequent global motion capture and human-scene joint optimization.

[0058] S21: Human Region Detection, Cropping, and Feature Extraction in Video Frames Based on the keyframe sequence obtained in S11 For each frame of image Perform human detection and obtain human bounding boxes. The human body image region is obtained by cropping based on the bounding box. Scale it to a uniform resolution

[0059] Feeding human images into a feature extraction network: in It represents the high-dimensional semantic features of the human body in image space.

[0060] S22: Human Keypoint Localization (Continuous Point Prediction Mechanism) and Coordinate Decoding To support the simultaneous prediction of standard skeletal joints and dense human body points, this invention employs a unified point localization mechanism: Let any three-dimensional point in the human body's standard space be... For each query point, through the dot-coded network: Points are mapped to a high-dimensional feature space and then jointly input with image features into the localization network. : in Represents a two-dimensional heat map. This represents a three-dimensional voxel heatmap.

[0061] The heatmap is decoded into continuous coordinates using the Soft-Argmax operation, where the locations of two-dimensional key points are... Three-dimensional relative coordinates By combining the camera parameters, the 3D key points in the camera coordinate system are obtained: in This is the back projection function based on the camera's intrinsic and extrinsic parameters.

[0062] S23: Human Posture Parameter Regression and Time Consistency Constraints The complete set of keypoints is obtained by summing the keypoints in the camera coordinate system of each frame obtained in S22: Then use the parameter regression network Estimate human posture parameters: in These parameters are the joint rotation parameters, human body shape parameters, and human body global translation parameters, which together form a complete three-dimensional human body posture result.

[0063] Finally, to ensure the pose continuity in the video sequence, a temporal smoothing constraint is introduced: Improve the stability of human movement over time.

[0064] Step 3: Joint Training and Optimization Based on the results of human pose estimation and scene reconstruction, this module jointly optimizes human pose parameters, scene geometry, and camera parameters to obtain global human motion pose and trajectory consistent with the scene space relationship.

[0065] S31: Initialization and Unified Representation of Joint Variables Let the case be in frame t: The camera parameters are: Human posture parameters are: The scene point cloud is represented as: in Indicates joint rotation parameters; Indicates human body shape parameters; It represents the global pose of the human body in the world coordinate system.

[0066] S32: Human-scene point cloud consistency constraint The set of points on the surface of the human body model is represented as: To constrain spatial consistency between the human body and the scene, a point cloud distance constraint is introduced: This constraint is used to suppress human hovering or excessive penetration of scene geometry.

[0067] S33: Human Posture and Shape Regularization Constraints To avoid non-physical jitter in human pose during joint optimization, a parameterized human model parameter regularization term is introduced: in: , This is the initial estimate.

[0068] S34: Consistency Constraints for Two-Dimensional Reprojection Projecting the three-dimensional joints of the human body onto the image plane: in: and the detected two-dimensional key points Apply constraints: This is used to ensure consistency between human posture and image observation.

[0069] S35: Global Camera Consistency Constraints To ensure the continuity of the camera trajectory over time, a camera smoothing term is introduced: This reduces the impact of camera shake on the trajectory of human movement.

[0070] S36: Joint Optimization Objective Function Taking into account the above constraints, a joint optimization objective is constructed: in These are the weight coefficients for each loss term. The optimal solution is obtained through iterative optimization. S37: Global Human Motion Trajectory Output Final output: Optimized global human pose sequence: Camera trajectory consistent with the scene: The merged scene point cloud: This results in global human motion capture with consistent scale and continuous time.

[0071] By introducing a tightly coupled joint optimization mechanism of human posture, scene geometry and camera parameters, this invention realizes unified three-dimensional modeling and collaborative reconstruction of volleyball court and athletes, enabling ordinary volleyball match videos to intuitively present a stable and continuous three-dimensional motion process, thereby facilitating accurate observation and analysis of volleyball technical movements and their spatial relationship with the court.

[0072] While some embodiments of this disclosure have been shown and described, those skilled in the art will understand that modifications may be made to these embodiments without departing from the principles and spirit of this disclosure, which are defined by the claims and their equivalents.

Claims

1. A method for three-dimensional reconstruction of a volleyball court and motion capture of athletes, characterized in that, include: S1 3D reconstruction of the court and camera estimation: Based on volleyball video, the 3D geometric structure of the volleyball court is reconstructed through the following processes: S11 video input and keyframe construction, S12 camera pose estimation, S13 camera pose refinement and joint feature learning, S14 local 3D geometric estimation in the camera coordinate system, S15 learnable global spatial mapping and geometric alignment, and S16 multi-view point cloud fusion and court reconstruction output. This process obtains the 3D point cloud of the court in a unified world coordinate system and the corresponding camera extrinsic parameters. S2 Athlete Pose Estimation: Based on the input video, through the process of human body region detection and cropping and feature extraction in video frames, human body key point localization and coordinate decoding, human pose parameter regression and time consistency constraint, the three-dimensional pose estimation of athletes based on the parametric human body model is performed to obtain the pose parameters, shape parameters and translation parameters of each athlete in the camera coordinate system. S3 Joint Optimization: Based on the input video and the pose parameters, shape parameters, and translation parameters of the athlete's body posture obtained from S1 and S2, the 3D point cloud of the volleyball court, and the corresponding camera extrinsic parameters, a consistent spatial relationship is established between the athlete, the volleyball court, and the camera in a unified world coordinate system. The final output includes the volleyball court geometry and the global motion posture and trajectory of the athletes, which are consistent with its spatial structure. The processes of camera pose estimation, camera pose refinement and joint feature learning, and local 3D geometric estimation in the camera coordinate system are as follows: The camera pose estimation first processes each frame in the normalized keyframe sequence. Perform image block division: Where M is the segmented non-overlapping image patch. The number of cameras; introduce a set of learnable camera latent variables: Where N is the number of frames in the input sequence, and each frame is an image Corresponding to a camera parameter The image patch features are concatenated with the camera latent variables and then input into the camera pose prediction network. Predicting the initial camera pose Camera pose From translation vector and rotation vector express: The translation vector and rotation vector are defined as follows: , , Using the unit quaternion form, we have ; The camera pose refinement and joint feature learning introduces a multi-task joint network to refine and optimize the camera pose while simultaneously learning geometric feature representations. Introducing latent variables for camera pose refinement: Image patch features, initial camera pose, and refined latent variables of camera pose are jointly input into the network. ,in The optimized camera pose is derived from the translation vector. and rotation vector composition; Let D represent the set of implicit feature vectors of a point cloud in a local space, where D is the feature dimension; The local 3D geometry estimation in the camera coordinate system uses the set of implicit feature vectors of point clouds in the local space obtained by camera pose refinement and joint feature learning. Input into geometry decoder Generate a dense point map in the camera coordinate system and its corresponding confidence level: ,in This represents a 3D point map in the camera coordinate system of the i-th frame. This represents the 3D coordinates of each pixel in the corresponding confidence map. ; The specific methods for learnable global spatial mapping and geometric alignment, multi-view point cloud fusion, and scene reconstruction output are as follows: The learnable global spatial mapping and geometric alignment are defined as a mapping network from local geometric features to global geometric features. Given the set of implicit feature vectors of point clouds in the local space and the optimized camera pose, complete the latent space mapping: ; Then through the global geometry decoder Obtain the geometric point map in the world coordinate system: ,in Represents the point cloud coordinates in the world coordinate system. This indicates the spatial confidence level of the point cloud; The multi-view point cloud fusion and site reconstruction output is a point cloud from all views, using a confidence threshold. Perform filtering and merging: ; Further sparsification is achieved using voxel filtering: The final output includes multi-frame optimized camera pose parameters. A 3D global point cloud of a volleyball court in a unified world coordinate system And the point cloud after global geometric mapping corresponding to each viewpoint. ; The specific method for joint optimization is as follows: S31: Initialization and Unified Representation of Joint Variables Let the case be in frame t: The camera parameters are: Human posture parameters are: The scene point cloud is represented as: in Indicates joint rotation parameters; Indicates human body shape parameters; This represents the global pose of the human body in the world coordinate system; S32: Human-scene point cloud consistency constraint The set of points on the surface of the human body model is represented as: To constrain spatial consistency between the human body and the scene, a point cloud distance constraint is introduced: This constraint is used to suppress human body levitation or excessive penetration of scene geometry; S33: Human Posture and Shape Regularization Constraints To avoid non-physical jitter in human pose during joint optimization, a parameterized human model parameter regularization term is introduced: in: , This is the initial estimate; S34: Consistency Constraints for Two-Dimensional Reprojection Projecting the three-dimensional key points of the human body onto the image plane: in: and the detected two-dimensional key points Apply constraints: Used to ensure consistency between human posture and image observation; S35: Global Camera Consistency Constraints To ensure the continuity of the camera trajectory over time, a camera smoothing term is introduced: This reduces the impact of camera shake on the trajectory of human movement; S36: Joint Optimization Objective Function Taking into account the above constraints, a joint optimization objective is constructed: in The optimal solution is obtained through iterative optimization, where the weights of each loss term are used. S37: Global Human Motion Trajectory Output Final output: Optimized global human pose sequence: Camera trajectory consistent with the scene: The merged scene point cloud: This results in global human motion capture with consistent scale and continuous time.

2. The method for three-dimensional reconstruction of a volleyball court and motion capture of athletes as described in claim 1, characterized in that, The specific method for video input and keyframe construction is as follows: Let the input video sequence be: in, This represents the RGB color image of frame t, where H and W are the height and width of the image, respectively; the time interval is... The sampling method selects a set of keyframes: ,in, This indicates the index of the selected keyframe. Indicates the sequence number is Image frames; Normalization preprocessing is performed on all keyframes: ,in and These represent the pixel mean and standard deviation of the dataset, respectively; the final result is a normalized keyframe sequence of length N. As input: .

3. The method for three-dimensional reconstruction of a volleyball court and motion capture of athletes as described in claim 2, characterized in that, The method for human body region detection, cropping, and feature extraction in the video frame is as follows: The keyframe sequence obtained based on the video input and keyframe construction For each frame of image Perform human detection to obtain the bounding boxes of one or more athletes in the video frame. The corresponding human image region is obtained by cropping each athlete according to the bounding box. Scale it to a uniform resolution This detection and cropping process is applicable to scenarios in volleyball where multiple players move simultaneously and there is partial occlusion. Feeding human images into a feature extraction network: ,in It represents the high-dimensional semantic features of the human body in image space.

4. The method for three-dimensional reconstruction of a volleyball court and motion capture of athletes as described in claim 3, characterized in that, The specific method for locating and decoding the human body's key points and coordinates involves employing a unified point positioning mechanism to maintain the spatial continuity and numerical stability of key point prediction under the conditions of frequent jumping, aerial maneuvers, and rapid limb swings in volleyball. This is achieved by assuming a standard human body space... Any three-dimensional point in is For each query point, through the dot-coded network: ; Points are mapped to a high-dimensional feature space and then jointly input with image features into the localization network. ,in Represents a two-dimensional heat map. Represents a three-dimensional voxel heatmap; The heatmap is decoded into continuous coordinates using the Soft-Argmax operation, where the locations of two-dimensional key points are... Three-dimensional relative coordinates By combining the camera parameters, the 3D key points in the camera coordinate system are obtained: ,in This is the back projection function based on the camera's intrinsic and extrinsic parameters.

5. The method for three-dimensional reconstruction of a volleyball court and motion capture of athletes as described in claim 4, characterized in that, The process of regressing human posture parameters and constraining time consistency is as follows: The complete set of key points is obtained by summing the key points in the camera coordinate system of each frame obtained from the human body key point localization and coordinate decoding: ; Then use the parameter regression network Estimate human posture parameters: ,in These parameters are the joint rotation parameters, human body shape parameters, and the global pose of the human body in the world coordinate system, which together constitute a complete three-dimensional human body pose result. Finally, to ensure the continuity and consistency of the athlete's posture in the video sequence, a temporal smoothing constraint is introduced: This suppresses posture jitter in volleyball match videos caused by rapid movement, occlusion switching, or detection errors, ensuring that the athlete's posture estimation results maintain reasonable changes between consecutive frames.

Images