Indoor golfer-human body-club interaction recognition method and system based on visual base model prior features
By introducing neural networks based on prior features of the visual fundamental model and physical theorems, the accuracy problem of human-club interaction recognition in indoor golf simulators was solved, achieving high-precision recognition under occlusion and ambiguity conditions, and ensuring the physical rationality and temporal continuity of the recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHENGDU XIAOJIUTIE SPORTS DEVELOPMENT CO LTD
- Filing Date
- 2026-05-07
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies struggle to accurately identify and reconstruct the three-dimensional human posture and club trajectory during a user's swing in indoor golf simulators. Furthermore, traditional methods neglect the physical interaction between the human body and the club, leading to inaccurate identification results or results that violate common sense physics.
We employ a method based on prior features of a visual fundamental model. By combining a cross-modal topological prior injection graph network and a spatiotemporal implicit kinematic energy field model with neural Hamiltonian ordinary differential equations, we simulate the dynamic coupling mechanical process between the human body and the golf club. We then use a phase-aware dynamic time warping attention mechanism to aggregate features and achieve the recognition of time-continuous interactive states.
It effectively eliminates physical violations in traditional methods, improves anti-occlusion and anti-blurring capabilities, ensures the physical rationality and temporal continuity of recognition results, and reduces coordinate drift and jitter in long-term prediction.
Smart Images
Figure CN122493362A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and computer vision, and in particular to an indoor golf human-club interaction recognition method and system based on prior features of a visual basic model. Background Technology
[0002] In recent years, indoor golf simulators have experienced rapid development and widespread adoption due to their lack of weather and venue limitations. The core technological challenge of such systems lies in accurately identifying and reconstructing the three-dimensional human posture and high-speed trajectory of the club during the user's swing, and deeply understanding the physical interaction between the human body (especially the hands) and the club.
[0003] To address this challenge, existing technologies have mainly followed two paths, but each has its own substantial technical bottlenecks.
[0004] The first type is the traditional optical marker motion capture system. This type of system requires users to wear tight-fitting clothing covered with reflective beads, and multiple infrared cameras capture the position of the markers. However, this invasive approach not only severely disrupts the natural comfort of the swing, but also involves expensive equipment and a cumbersome calibration process, making it difficult to popularize in consumer-grade indoor golf settings.
[0005] The second category is label-free vision methods based on deep learning. These methods typically combine "human pose estimation" and "club pose estimation" as two separate regression tasks. While avoiding interference from physical labels, their separate computational paradigm suffers from the following fundamental flaws: Ignoring the core physical topology coupling. This type of method does not model the most fundamental physical constraint of "both hands gripping the club tightly", which makes it easy to predict results that violate basic physical common sense, such as "the club detaches from the hands" or "the club passes through the body and enters the body" due to body obstruction or motion ambiguity at high-speed downswing and impact.
[0006] The lack of understanding of continuous physical time is a significant issue. Traditional deep networks (such as CNNs or LSTMs) only empirically fit pixel-level features at discrete time points and lack an understanding of the laws governing kinetic and potential energy conversion (i.e., the law of conservation of energy). Once severe occlusion occurs for several consecutive frames, the network loses its tracking ability, resulting in obvious trajectory jumps and jitter.
[0007] In summary, there is an urgent need in this field for a new interactive recognition architecture that does not rely on external labels, can fundamentally understand the human-stick mechanical coupling, and integrates prior physical knowledge with spatiotemporal continuous reasoning capabilities. Summary of the Invention
[0008] To overcome the aforementioned deficiencies of the prior art, this invention provides an indoor golf human-club interaction recognition method and system based on prior features of a visual basic model, in order to solve the problems existing in the background art.
[0009] This invention provides the following technical solution: an indoor golf human-club interaction recognition method and system based on prior features of a visual basic model, comprising the following steps: Acquire multi-view swing video stream data of the test object in an indoor golf scene; The multi-view swing video stream data is input into the visual basic model to extract a general spatial semantic prior feature field with a global receptive field; The general spatial semantic prior feature field is input into a cross-modal topological prior injection graph network. By jointly representing human joints and club key points, a unified interaction graph structure is constructed to obtain the static physical constraint interaction features in the spatial dimension. The static physical constraint interaction features are input into the spatiotemporal implicit kinematic energy field model, which embeds neural Hamiltonian ordinary differential equations to simulate the dynamic coupling mechanical evolution process between the human body and the club, and obtain a time-continuous interactive state flow that conforms to the conservation of dynamics. The dynamic time warping attention mechanism with phase awareness is used to aggregate features of the continuous temporal interactive state flow to obtain the golf human-club interaction recognition result.
[0010] Furthermore, the node update formula for the unified interaction graph structure in the cross-modal topology prior injection graph network is expressed as: in, Indicates the first The node feature matrix of the layered graph network represents the joint implicit representation of the human body and the club node; Represents a nonlinear activation function; These are learnable modal balance coefficients used to dynamically adjust the weights of topological graph information and visual prior information; This represents a dynamic adjacency matrix with self-connections. This represents the topological constraint matrix of a fixed human kinematic skeleton. This indicates that the implicit features depend on the current swing. The probability matrix of dynamic interaction between human hand and club grip; yes The degree matrix; It is the first The learnable weight matrix of the layer; This represents a multi-head cross-attention function; This represents the general spatial semantic prior feature field extracted from the visual base model; As a query vector As key vectors and value vectors.
[0011] Furthermore, the spatiotemporal implicit kinematic energy field model abstracts the human-stick system into a Lagrange-Hamiltonian dynamic system, whose internal neural Hamiltonian ordinary differential equation evolution formula is expressed as: in, Indicates time The generalized coordinate vector of the system state at any given moment includes the three-dimensional coordinates of the human joints and the geometric center and direction vector of the club. Indicates and The corresponding generalized momentum vector; and These represent the identity matrix and the zero matrix, respectively. Indicates that the parameter is The Hamiltonian scalar energy function parameterized by the deep neural network represents the total energy of the system; The partial derivative operator with respect to generalized coordinates and generalized momentum; The Jacobian matrix represents the holographic physical contact constraint imposed between the hand and the grip of the golf club; Let represent the Lagrange multiplier vector, and let represent the implicit contact force required to maintain the physical constraint of the two hands gripping the bar.
[0012] Furthermore, the step of obtaining a time-continuous interactive state flow that conforms to dynamic conservation includes: To address the temporal discreteness between acquired video frames, a symplectic integrator is used to perform forward difference solutions on the neural Hamiltonian ordinary differential equation. While preserving the system phase space volume, the missing trajectory caused by high-speed motion between two frames is reconstructed, ensuring the energy conservation characteristics of the temporally continuous interactive state flow in long-term time-series prediction and reducing coordinate drift errors caused by occlusion.
[0013] Furthermore, in the step of using a phase-aware dynamic time warping attention mechanism to aggregate features of the temporally continuous interactive state flow, the formula for calculating the attention weight is: in, Indicates time Target characteristics and time Unnormalized attention scores between contextual features; and These represent the query vector and key vector obtained by mapping from a time-continuous interactive state stream, respectively. and It is a linear projection matrix; For feature channel dimensions; Indicates the current time The discrete phase classification results of the golf swing represent the setup, backswing, downswing, impact, and follow-through phases in sequence. The dynamic time warping modulation function based on a Gaussian distribution is defined as follows: ,in To experience the variance in the field during the hitting phase. hour To capture high-frequency interactive mutations during the preparation phase, the minimum value is selected. hour Take the maximum value to establish long-range spatiotemporal dependencies.
[0014] Furthermore, the step of extracting the general spatial semantic prior feature field includes: The multi-view swing video stream data is divided into a sequence of image blocks; Feature representations of image patches are extracted using a visual Transformer that has been unsupervised pre-trained on ultra-large-scale natural images; The underlying weights of the visual Transformer are frozen, and the top-level feature mapping is decoupled from high-dimensional features to three-dimensional golf ball physical features through low-rank adaptive fine-tuning, generating a general spatial semantic prior feature field.
[0015] An indoor golf human-club interaction recognition system based on prior features of a visual fundamental model includes: The data acquisition module is used to acquire multi-view swing video stream data of the test object in an indoor golf scene; The prior feature extraction module is used to input the multi-view swing video stream data into the visual base model and extract a general spatial semantic prior feature field with a global receptive field. The topology prior injection module is used to input the general spatial semantic prior feature field into the cross-modal topology prior injection graph network, and to construct a unified interaction graph structure by jointly representing human joints and club key points to obtain static physical constraint interaction features in the spatial dimension. The implicit energy field module is used to input the static physical constraint interaction features into the spatiotemporal implicit kinematic energy field model. The model embeds neural Hamiltonian ordinary differential equations to simulate the dynamic coupling mechanical evolution process between the human body and the club, and obtains a time-continuous interactive state flow that conforms to the conservation of dynamics. The dynamic aggregation module is used to aggregate features of the continuous time-interactive state flow using a phase-aware dynamic time warping attention mechanism to obtain a golf human-club interaction recognition model.
[0016] Furthermore, the implicit energy field module is specifically used for: Calculate the generalized coordinate vector and generalized momentum vector based on the topology of human joints and the geometric features of the club. The Hamiltonian energy function of the system is approximated by inputting generalized coordinates and momentum into a fully connected neural network. By combining the Jacobi contact constraint to calculate the Lagrange multiplier, the action and reaction forces caused by the hand grip are derived, which forces the model to strictly limit the predicted trajectory of the club to the three-dimensional spatial boundary of the hands.
[0017] An electronic device, characterized in that it comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the indoor golf human-club interaction recognition method based on prior features of a visual basic model as described in any one of the above.
[0018] A computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the indoor golf human-club interaction recognition method based on prior features of a visual basic model as described in any one of the above.
[0019] The technical effects and advantages of this invention are as follows: Physical rationality: By introducing the neural Hamiltonian ordinary differential equation and Lagrange contact constraints, the laws of classical mechanics are embedded in the deep learning model, and the energy conservation and hand-rod coupling constraints are enforced from the mathematical level, thus completely eliminating physical violations such as "rod through mold" or "hand-rod separation" that are common in traditional methods.
[0020] Anti-occlusion and anti-blurring capabilities: By utilizing the global spatial semantic prior provided by the Visual Foundation Model (VFM) and combining it with the dynamic adjacency matrix of the cross-modal topological graph network, the model can still reconstruct a reasonable interaction state based on topological constraints and physical inertia even when severe occlusion or motion blur is caused by high-speed swings.
[0021] Spatiotemporal continuity and accuracy: The symplectic integrator is used to solve the neural Hamiltonian ordinary differential equation, which ensures the conservation of phase space volume and can reconstruct smooth, energy-conserving continuous motion trajectories between discrete video frames, significantly reducing coordinate drift and jitter in long-term prediction.
[0022] Temporally Adaptive Aggregation: The phase-aware dynamic time warping attention mechanism dynamically adjusts the temporal receptive field according to different stages of the swing. It maintains extremely high temporal resolution at the moment of impact to capture abrupt changes, and expands the receptive field during the preparation phase to extract steady-state features, thus balancing computational efficiency and recognition sensitivity. Attached Figure Description
[0023] Figure 1A flowchart illustrating an indoor golf human-club interaction recognition method based on prior features of a visual basic model, provided for an embodiment of this application; Figure 2 A structural block diagram of an indoor golf human-club interaction recognition system based on prior features of a visual basic model, provided for an embodiment of this application; Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0024] The technical solutions in the embodiments of this application will be described in detail and accurately below with reference to the accompanying drawings.
[0025] In indoor golf human-equipment interaction recognition tasks, traditional techniques relying on 2D image pixels or ordinary spatiotemporal convolutional networks are trapped in the "curse of dimensionality" and "physical amnesia" dilemma due to limitations such as indoor lighting conditions, background interference, self-occlusion of the body during the swing, and severe motion blur caused by clubhead speeds exceeding 150 miles per hour. For example, existing algorithms often use a separate HRNet to detect human joints and another ResNet to detect the club. When the club disappears from the image due to motion blur or is occluded by the torso, ResNet immediately fails; and the hand position predicted by HRNet ignores the common sense that the palm should be firmly gripping the handle.
[0026] To overcome this technical limitation, the fundamental solution proposed in this application is to introduce a "Visual Foundation Model (VFM)" pre-trained with hundreds of millions of parameters to provide powerful anti-interference underlying features. Instead of directly performing coordinate regression, a graph neural network is used to establish a topological primitive of "human body-ball stick". These primitives are then fed into a "neural Hamiltonian ordinary differential equation" network that simulates classical physics theorems. The physical laws are used to automatically deduce the motion trajectory that is not visible to the eye in the latent space.
[0027] Please see Figure 1 , Figure 1 A flowchart illustrating an indoor golf human-club interaction recognition method based on prior features of a visual basic model, provided for embodiments of this application, includes the following steps: S100: Acquire multi-view swing video stream data of the test object in an indoor golf scene.
[0028] For example, an indoor golf simulator is typically surrounded by synchronized high-speed industrial cameras (e.g., global shutter cameras shooting at 240 FPS) positioned directly in front, to the sides, and optionally above. Hardware triggers ensure hard-synchronized acquisition of the multi-view cameras. The acquired raw data is characterized as follows: in It is the frame sequence length. It refers to the number of camera angles. and These are the height and width of the image, respectively. Before inputting the data into the algorithm, joint calibration is performed on all cameras to obtain the camera intrinsic parameter matrices. With extrinsic matrix This allows us to construct a three-dimensional world coordinate system.
[0029] S200: Input the multi-view swing video stream data into the visual basic model and extract the general spatial semantic prior feature field with a global receptive field.
[0030] For example, visual foundation models (VFMs, such as DINOv2 or SAM architectures) can precisely lock foregrounds with object-level semantics by performing self-supervised contrastive learning on large-scale unlabeled images, thus ignoring background noise.
[0031] The specific steps are as follows: S210: Transfer each frame of image Cut into sizes Non-overlapping image patches are expanded and mapped into sequence vectors.
[0032] S220: After adding positional and viewpoint encodings, the input is fed into the VisionTransformer (ViT) network with frozen bottom weights. Since directly fine-tuning the ViT with hundreds of millions of parameters is computationally expensive and prone to catastrophic forgetting, this application innovatively introduces a Low-Rank Adaptation (LoRA) layer.
[0033] S230: Through low-rank matrix decomposition, i.e. .
[0034] Where the matrix .
[0035] The VFM network outputs high-dimensional continuous feature maps, which are then aggregated and reduced in dimensionality to obtain a general spatial semantic prior feature field. .
[0036] This feature field not only includes low-level semantics such as edges and textures, but also contains powerful spatial prior logic that distinguishes different parts of the human body from the various components of a golf club (shaft and clubhead).
[0037] S300: Input the general spatial semantic prior feature field into the cross-modal topological prior injection graph network to obtain the static physical constraint interaction features of the spatial dimension.
[0038] For example, a human golf swing is not a collection of scattered pixels, but rather a topological system conforming to the skeletal kinematic chain. In this step, a heterogeneous graph is defined. Among them, nodes Corresponding to major joints in the human body (such as the shoulder, elbow, and wrist), nodes This corresponds to the geometric key points of the golf club (grip end, shaft midpoint, clubhead). The node feature updates across modal graph networks do not simply involve passing messages between nodes, but rather execute an innovative fusion mechanism based on the following formula: In this formula, It is of utmost importance. According to human anatomy, it is a constant; while The connection weight between the wrist node and the club grip node represents the connection weight in relation to the current graph state. The function. If the hand is occluded from a certain viewpoint, a normal network cannot locate the hand; however, because the graph network of this invention uses a dynamic adjacency matrix to create an elastic "invisible spring (topological constraint)" between the torso nodes, arm nodes, and club nodes, and the second cross-modal attention mechanism... Continuously from a global perspective Search for supporting evidence in the hidden features The system robustly reconstructs the spatial distribution of the obscured hand or cue. The output is a static physical constraint interaction feature. .
[0039] S400: Input the static physical constraint interaction features into the spatiotemporal implicit kinematic energy field model to obtain a time-continuous interactive state flow that conforms to dynamic conservation.
[0040] For example, this is the core Physics-AI integration area of this invention. A golf swing is essentially the process of converting the body's internal energy into the club's kinetic energy, counteracting gravitational potential energy. Instead of performing discrete RNN tracking on continuous video sequences, this system constructs a spatiotemporal implicit kinematic energy field.
[0041] S410: Use a differentiable decoder to extract the aforementioned... Mapped to the system's generalized coordinates (Including angle and position) and generalized momentum (Includes angular velocity and linear velocity information mapping).
[0042] S420 is used to construct a deep fully connected network to fit the Hamiltonian: S430: Its evolution is controlled using neural Hamiltonian ordinary differential equations. To enforce the physical law that the hand and the club grip must remain in contact, a contact constraint equation is defined. The derivative yields the constrained Jacobian matrix. The Lagrange multipliers in the formula The computation is implicitly performed by the self-differentiating molecular modules within the network. Because the system follows calculus-level physical derivations, when encountering highly motion-blurred frames, the neural network no longer blindly searches through pixels. Instead, it adapts to system inertia (generalized momentum) and physical constraints (Lagrange multipliers), smoothly and accurately calculating the most mechanically logical coordinate state for the next moment along the energy surface, generating a time-continuous interactive state flow. .
[0043] Meanwhile, a symptotic integral algorithm based on symplectic group geometry is used instead of standard Euler ordinary differential differentiation, thus ensuring the area conservation form in phase space. This is strictly maintained. Therefore, no matter how much the motion playback is slowed down, the trajectory flow generated by the system is time-continuous, smooth, and energy-conserving.
[0044] S500: The phase-aware dynamic time warping attention mechanism is used to aggregate features of the continuous time-interactive state flow to obtain a golf human-club interaction recognition model.
[0045] For example, a golf swing is a highly non-linear temporal process. During the address phase, the movement remains relatively unchanged for several seconds; however, during the downswing and impact phase, the movement changes dramatically within milliseconds. If a fixed attention window is used... This will cause the high-frequency motion at the moment of impact to be smoothly filtered out (causing the disappearance of the peak of the swing trajectory).
[0046] This embodiment designs a phase-aware dynamic time-warping attention (PA-DTWA): firstly, time slices are predicted using a lightweight classification head. Swing phase Then calculate the adjusted temporal self-attention weights: when When classified as "the moment of impact", the model automatically uses a lookup table to adjust the time variance. Reduced to a minimum value (e.g.) At this point, the exponential decay function changes abruptly, and the network only assigns values to the nearest neighbor time. Non-zero weights enable extremely high temporal resolution, capturing instantaneous axial deformation and rapid wrist rotation features. Conversely, during the "preparation phase," the scale is expanded. (like The network absorbs a wide range of contextual information and extracts steady-state features. After the aforementioned time-weighted aggregation, a nonlinear mapping layer outputs the final high-difficulty motion capture coordinate system and joint torque analysis data.
[0047] S600: Loss Function and Model Training Strategy This model belongs to an end-to-end training framework, and the loss function is designed as a linear sum of three parts: .
[0048] Data consistency loss : The mean square error between the reprojected 2D / 3D coordinates output by the calculation model and a small number of ground truth values or pseudo-labels.
[0049] Physical consistency loss The punishment does not conform to the trajectory of Newton's laws of motion. This requires that the internal energy change of the system at adjacent time points must be equivalent to the external work done. and the punishment for breaking the rules .
[0050] Topological loss : Utilizing the property that the length of the human limbs and the rigid length of the club in the spatial skeleton cannot change during the movement, the rate of change of the penalty length is determined.
[0051] Please see Figure 2 , Figure 2 The structural block diagram of the indoor golf human-club interaction recognition system based on prior features of a visual fundamental model provided in this application embodiment is shown. The system includes: a data acquisition module 100, a prior feature extraction module 200, a topology prior injection module 300, an implicit energy field module 400, and a dynamic aggregation module 500. These modules are highly coupled through an internal hardware and software bus, and GPU resources are scheduled in parallel. The data acquisition module 100 is responsible for connecting to external multi-view high frame rate cameras using an asynchronous flow control strategy; the prior feature extraction module 200 has a built-in LoRA-tuned tensor computation acceleration engine; the topology prior injection module 300 and the implicit energy field module 400 collaboratively construct the back-end processing pipeline for physical graph inference; the dynamic aggregation module 500 performs streaming segmentation and phase attention analysis on the time-series sequence, ultimately outputting 3D panoramic uncalibrated motion capture and dynamic parameter diagnostic results. Further details regarding repeated derivations are omitted here.
[0052] Please see Figure 3, Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. The electronic device may include a processor, a communication interface, a memory, and at least one communication bus. The communication bus is used to enable communication between these components. The processor can be an integrated circuit chip with extremely high tensor operation processing capabilities; it can be a general-purpose processor, including a CPU, a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices. The memory stores the aforementioned low-level drivers and core model computer-readable instructions.
[0053] This application also provides a computer-readable storage medium storing a set of binary or bytecode instructions. When the instruction set is loaded by a high-performance computer cluster and run on a processor, its execution mechanism strictly replicates the steps detailed and formulaically described in the foregoing method embodiments, thus completely solving the problem of recognizing high-speed interactive objects in an indoor golf environment.
[0054] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. For those skilled in the art, this application is not limited to golf simulator scenarios, but can also be extended to fields with high-speed human-machine interaction characteristics such as baseball and ice hockey, and can be implemented by adjusting physical constants or Hamiltonian topology. Any equivalent substitutions, equivalent derivations of topological structures, hardware-based improvements to algorithm operators, etc., made within the spirit and innovative mathematical-physical principles of this application should be unequivocally included within the scope of protection of this application.
[0055] It should be noted that similar reference numerals and letters in the accompanying drawings indicate similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
Claims
1. A method for recognizing human-club interaction in indoor golf based on prior features of a visual fundamental model, characterized in that, include: Acquire multi-view swing video stream data of the test object in an indoor golf scene; The multi-view swing video stream data is input into the visual basic model to extract a general spatial semantic prior feature field with a global receptive field; The general spatial semantic prior feature field is input into a cross-modal topological prior injection graph network. By jointly representing human joints and club key points, a unified interaction graph structure is constructed to obtain the static physical constraint interaction features in the spatial dimension. The static physical constraint interaction features are input into the spatiotemporal implicit kinematic energy field model, which embeds neural Hamiltonian ordinary differential equations to simulate the dynamic coupling mechanical evolution process between the human body and the club, and obtain a time-continuous interactive state flow that conforms to the conservation of dynamics. The dynamic time warping attention mechanism with phase awareness is used to aggregate features of the continuous temporal interactive state flow to obtain the golf human-club interaction recognition result.
2. The indoor golf human-club interaction recognition method based on prior features of a visual basic model according to claim 1, characterized in that, The node update formula for the unified interaction graph structure in the cross-modal topological prior injection graph network is expressed as follows: in, Indicates the first The node feature matrix of the layered graph network represents the joint implicit representation of the human body and the club node; Represents a nonlinear activation function; These are learnable modal balance coefficients used to dynamically adjust the weights of topological graph information and visual prior information; This represents a dynamic adjacency matrix with self-connections. This represents the topological constraint matrix of a fixed human kinematic skeleton. This indicates that the implicit features depend on the current swing. The probability matrix of dynamic interaction between human hand and club grip; yes The degree matrix; It is the first The learnable weight matrix of the layer; This represents a multi-head cross-attention function; This represents the general spatial semantic prior feature field extracted from the visual base model; As a query vector As key vectors and value vectors.
3. The indoor golf human-club interaction recognition method based on prior features of a visual basic model according to claim 1, characterized in that, The spatiotemporal implicit kinematic energy field model abstracts the human-stick system into a Lagrange-Hamiltonian dynamic system, whose internal neural Hamiltonian ordinary differential equation evolution formula is expressed as: in, Indicates time The generalized coordinate vector of the system state at any given moment includes the three-dimensional coordinates of the human joints and the geometric center and direction vector of the club. Indicates and The corresponding generalized momentum vector; and These represent the identity matrix and the zero matrix, respectively. Indicates that the parameter is The Hamiltonian scalar energy function parameterized by the deep neural network represents the total energy of the system; The partial derivative operator with respect to generalized coordinates and generalized momentum; The Jacobian matrix represents the holographic physical contact constraint imposed between the hand and the grip of the golf club; Let represent the Lagrange multiplier vector, and let represent the implicit contact force required to maintain the physical constraint of the two hands gripping the bar.
4. The indoor golf human-club interaction recognition method based on prior features of a visual basic model according to claim 3, characterized in that, The steps for obtaining a time-continuous interactive state flow that conforms to dynamic conservation include: To address the temporal discreteness between acquired video frames, a symplectic integrator is used to perform forward difference solutions on the neural Hamiltonian ordinary differential equation. While preserving the system phase space volume, the missing trajectory caused by high-speed motion between two frames is reconstructed, ensuring the energy conservation characteristics of the temporally continuous interactive state flow in long-term time-series prediction and reducing coordinate drift errors caused by occlusion.
5. The indoor golf human-club interaction recognition method based on prior features of a visual basic model according to claim 1, characterized in that, In the step of using a phase-aware dynamic time warping attention mechanism to aggregate features of the temporally continuous interactive state flow, the formula for calculating the attention weight is: in, Indicates time Target characteristics and time Unnormalized attention scores between contextual features; and These represent the query vector and key vector obtained by mapping from a time-continuous interactive state stream, respectively. and It is a linear projection matrix; For feature channel dimensions; Indicates the current time The discrete phase classification results of the golf swing represent the setup, backswing, downswing, impact, and follow-through phases in sequence. The dynamic time warping modulation function based on a Gaussian distribution is defined as follows: ,in To experience the variance in the field during the hitting phase. hour To capture high-frequency interactive mutations during the preparation phase, the minimum value is selected. hour Take the maximum value to establish long-range spatiotemporal dependencies.
6. The indoor golf human-club interaction recognition method based on prior features of a visual basic model according to claim 1, characterized in that, The steps for extracting the general spatial semantic prior feature field include: The multi-view swing video stream data is divided into a sequence of image blocks; Feature representations of image patches are extracted using a visual Transformer that has been unsupervised pre-trained on ultra-large-scale natural images; The underlying weights of the visual Transformer are frozen, and the top-level feature mapping is decoupled from high-dimensional features to three-dimensional golf ball physical features through low-rank adaptive fine-tuning, generating a general spatial semantic prior feature field.
7. An indoor golf human-club interaction recognition system based on prior features of a visual basic model, characterized in that, include: The data acquisition module is used to acquire multi-view swing video stream data of the test object in an indoor golf scene; The prior feature extraction module is used to input the multi-view swing video stream data into the visual base model and extract a general spatial semantic prior feature field with a global receptive field. The topology prior injection module is used to input the general spatial semantic prior feature field into the cross-modal topology prior injection graph network, and to construct a unified interaction graph structure by jointly representing human joints and club key points to obtain static physical constraint interaction features in the spatial dimension. The implicit energy field module is used to input the static physical constraint interaction features into the spatiotemporal implicit kinematic energy field model. The model embeds neural Hamiltonian ordinary differential equations to simulate the dynamic coupling mechanical evolution process between the human body and the club, and obtains a time-continuous interactive state flow that conforms to the conservation of dynamics. The dynamic aggregation module is used to aggregate features of the continuous time-interactive state flow using a phase-aware dynamic time warping attention mechanism to obtain a golf human-club interaction recognition model.
8. The indoor golf human-club interaction recognition system based on prior features of a visual fundamental model according to claim 7, characterized in that, The implicit energy field module is specifically used for: Calculate the generalized coordinate vector and generalized momentum vector based on the topology of human joints and the geometric features of the club. The Hamiltonian energy function of the system is approximated by inputting generalized coordinates and momentum into a fully connected neural network. By combining the Jacobi contact constraint to calculate the Lagrange multiplier, the action and reaction forces caused by the hand grip are derived, which forces the model to strictly limit the predicted trajectory of the club to the three-dimensional spatial boundary of the hands.
9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the indoor golf human-club interaction recognition method based on prior features of a visual fundamental model as described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the indoor golf human-club interaction recognition method based on prior features of a visual basic model as described in any one of claims 1 to 6.