Real-time badminton action detection system and device based on MediaPipe and Motion Bidirectional Encoder Representation Transformer

Abstract

A real-time badminton action detection system, consisting of: a video recording module configured to continuously record video images at a frame rate of at least thirty frames per second; a pose estimation processing unit configured to detect and output two-dimensional skeletal landmark coordinates for a variety of body joints, including at least wrists, elbows, shoulders, hips, knees and ankles, from each video frame; a Motion Bidirectional Encoder Representation Transformer (Motion-BERT) configured to receive sequential skeleton landmark coordinates over a defined time window and encode motion trajectories using multi-head self-attention mechanisms across past and future frames; and a classification controller module operationally coupled to the Motion-BERT, wherein the classification controller module comprises a dense neural network with a softmax output layer configured to generate real-time probability distributions over a variety of badminton-specific action classes, the end-to-end system being configured to produce recognition results with a processing latency of less than 100 milliseconds; and wherein the video recording module comprises a high-speed digital camera with a wide-angle lens positioned at a point on the perimeter of the court, the camera being calibrated with intrinsic and extrinsic parameters for perspective correction, and wherein the system includes a calibration routine that aligns detected skeletal landmarks with a reference badminton court coordinate system.

Description

Technical field

[0001] The present invention belongs to the field of computer vision, artificial intelligence, and machine learning, and in particular to the real-time recognition of human actions in high-speed racket sports. Specifically, the invention describes a system and a device for recognizing badminton-specific movements using MediaPipe-based pose estimation and a Motion Bidirectional Encoder Representation Transformer (Motion-BERT) for temporal motion encoding. The invention also relates to a physical device structure that integrates image acquisition hardware, computing processors, and optimized model execution units for low-latency operation in real-time sports analysis, coaching, and broadcast applications. BACKGROUND OF THE INVENTION

[0002] Human action recognition has been extensively researched in fields such as surveillance, healthcare, gaming, and sports analytics. In the sports context, the real-time recognition of specific athletic movements is particularly important for applications such as automatic annotation, performance monitoring, tactical analysis, and decision support. Conventional action recognition techniques, such as convolutional neural networks combined with recurrent models like long short-term memory (LSTM), require extensive training, exhibit high inference latency, and are often not transferable to different topics and environments.

[0003] Pose estimation frameworks like MediaPipe have revolutionized skeletal tracking by enabling simple, real-time detection of human body features, including joint coordinates. However, analyzing temporal dependencies across frames remains a challenge, especially in fast-paced sports like badminton, where movements are rapid, complex, and context-dependent. Standard recurrent or convolutional timing models fail to capture bidirectional dependencies, which are crucial for distinguishing between visually similar but contextually distinct actions.

[0004] The Transformer architecture has proven promising for modeling sequential data, as it uses attention mechanisms to capture dependencies over time. The Motion-BERT variant optimizes Transformer specifically for motion dynamics, thus enabling bidirectional analysis of skeletal pose sequences. Although the state of the art shows some progress, the systems are often still too computationally intensive for real-time use or are limited by the lack of context-sensitive temporal encoding.

[0005] Therefore, there is an urgent need for a lightweight, precise and low-latency system tailored to detect badminton actions that works effectively on edge devices such as smartphones or embedded GPU modules while maintaining high temporal fidelity.

[0006] In the field of human action recognition, particularly within the specific context of racket sports like badminton, the technological landscape has advanced considerably over the past decade. Researchers and developers have persistently sought to design frameworks capable of reliably interpreting athletic movements in real time, but this path has been fraught with limitations in accuracy, latency, and scalability. Action recognition itself has its roots in the broader field of computer vision research, where the initial goal was to classify common human activities such as walking, running, sitting, or waving. These traditional systems were designed for use in relatively controlled environments where movement variability and action speed were manageable.However, when extended to highly dynamic sports such as badminton, where actions take place within fractions of a second and often involve slight variations in movement patterns, such systems do not achieve the precision and responsiveness required for practical use.

[0007] Early approaches relied heavily on hand-crafted features and traditional machine learning classifiers. Methods such as oriented gradient histograms (HOGs), motion history images (MHIs), and optical flow-based descriptors were commonly used to capture the spatiotemporal features of human movement. Once the features were extracted, they were typically fed into classifiers such as support vector machines (SVMs) or random forests. While these approaches were interpretable and computationally efficient, they were inherently limited in their generalizability to complex, nonlinear movement patterns. In badminton, where a smash and a clear involve similar preparatory movements but can diverge in the finishing phase, hand-crafted features often failed to accurately distinguish between the two actions.Furthermore, these systems required labor-intensive feature development and tended to suffer performance degradation with noisy or cluttered backgrounds.

[0008] With the advent of deep learning, convolutional neural networks (CNNs) and recurrent neural networks (RNNs), particularly long short-term memory (LSTM) networks, came to dominate the field of action recognition. CNNs offered robust mechanisms for extracting hierarchical visual features, while LSTMs provided the ability to capture sequential dependencies in video frames. For sports applications, several attempts were made to integrate skeletal pose data into CNN-LSTM pipelines to temporally process joint coordinates for classification. While this hybrid approach achieved better recognition rates than hand-crafted methods, it also introduced new challenges. CNN-LSTM architectures were computationally intensive, required long training times, and often exhibited latencies that made real-time deployment impossible.Their unidirectional nature meant that temporal dependencies were only captured forward, thus missing the contextual benefits of motion analysis that considers both past and future images. This shortcoming is particularly pronounced in badminton, where anticipatory and deceptive movements are central to the sport. For example, the start of a drop shot and a clear can appear almost identical until the shuttlecock is released, making bidirectional context crucial for accurate detection.

[0009] In parallel, pose estimation frameworks evolved into a promising method for abstracting human movement from raw video data. OpenPose was one of the first and most influential tools for extracting skeletal features from 2D images and videos. Its ability to provide joint coordinates in real time opened up new possibilities for action recognition systems, as it reduced the reliance on pixel-level raw video data and instead incorporated human kinematics. However, OpenPose was computationally intensive and required high-end GPUs even for reasonably real-time performance. Furthermore, judder and inaccuracies occurred when joints were obscured or athletes moved at very high speeds. In badminton, where the racket arm often crosses the torso or moves rapidly across the frame, OpenPose frequently failed to track joints consistently.These shortcomings limited its usability in mobile or embedded systems, which are becoming increasingly important for practical use in sports analysis.

[0010] Researchers attempted to improve temporal modeling beyond LSTMs using Temporal Convolutional Networks (TCNs). These networks employed one-dimensional convolutions over time to capture dependencies within sequential data. While TCNs achieved some improvements in modeling temporal relationships, they were fundamentally unidirectional, capturing either past-to-future or future-to-past dependencies without integrating both directions simultaneously. This lack of bidirectional awareness meant the models could not fully contextualize ambiguous motion sequences. Furthermore, TCNs required careful tuning of receptive fields and often struggled with long-range dependencies, limiting their effectiveness in scenarios where entire action sequences spanned several seconds.

[0011] Attention mechanisms and transformer architectures marked another significant breakthrough. By concentrating computational resources on the most relevant parts of a sequence, transformers could capture complex temporal dependencies without being limited by the sequential nature of RNNs. Applied to skeletal-based action recognition, transformers showed promising results, particularly in their ability to model relationships between distant images in a video sequence. However, early applications of transformers in this area were designed as general-purpose sequence models and were not optimized for motion dynamics. Consequently, they tended to overfit when trained on relatively small sports datasets and were computationally too large for real-time use on resource-constrained devices.Its performance was impressive in offline batch processing, but its lack of ability to deliver real-time inference limited its adoption in live sports environments where immediate feedback is essential.

[0012] Another research focus was on three-dimensional action recognition models that combined pose and temporal embedding. These systems used 3D convolutions to directly process spatiotemporal information from video data. While they offered the advantage of integrating spatial and temporal features into a unified framework, they were computationally prohibitive. Such models required enormous computing resources for both training and inference and were therefore unsuitable for use in real-time scenarios, especially on edge devices. For badminton, where fast frame-by-frame processing is essential to keep pace with the speed of the game, 3D CNN-based models were impractical despite their theoretical accuracy advantages.

[0013] The drawbacks of these existing solutions highlight the inherent tension between accuracy, latency, and computational power in detecting athletic actions. Systems based on hand-crafted functions were too simplistic to handle complex, deceptive movements. CNN-LSTM models improved accuracy but were too slow and unidirectional. OpenPose-based pipelines enabled skeleton tracking but were computationally intensive and prone to landmark jitter. TCNs offered efficiency but lacked bidirectional context. Transformers added powerful attention-based modeling but were prone to overfitting and excessive computational cost. Finally, 3D CNNs provided a comprehensive spatiotemporal representation but were too resource-intensive for real-world applications.

[0014] Another aspect of the challenge is feasibility. Most existing systems were developed in controlled laboratory environments with access to powerful computer hardware. Transferring these systems to the real world of badminton training centers, live broadcast studios, or court analysis required a streamlined solution that could run on mobile devices or embedded systems without sacrificing accuracy. For real-time usability, the systems also needed to deliver results within tight latency times, ideally under 100 milliseconds, so that coaches, players, and commentators could react immediately to the insights. None of the existing solutions fully met this requirement.

[0015] Furthermore, extensibility was a problem with many earlier systems. Most were trained and optimized for general action recognition datasets like Kinetics or UCF-101, which do not adequately capture the subtle differences between sport-specific actions. Directly applying such systems to badminton often led to poor results because the unique movement dynamics of this sport were not well represented in the training data. Adapting these models to badminton required extensive retraining, often with insufficient data, which exacerbated the problems of overfitting and limited generalizability.

[0016] The technical background reveals an evolution from simple, hand-crafted models to deep learning and attention-based architectures. Each step brings improvements but also new bottlenecks. The lack of a specialized, lightweight, and bidirectional system tailored to the fast, deceptive, and complex movements of badminton remains evident. This gap underscores the need for a novel solution that not only integrates efficient pose estimation with advanced temporal modeling but also ensures low-latency, real-time operation on practical devices. By addressing these issues, the present invention closes a critical gap in the prior art and enables precise, responsive, and scalable badminton action recognition for real-world applications. Object of the invention

[0017] The main objective of the invention is to provide a real-time badminton action recognition system that detects shots such as serves, smashes, drops, clears, and net shots with high accuracy and a latency of less than 100 ms. A further objective is to integrate MediaPipe-based skeletal pose extraction with Motion-BERT-based temporal coding to combine efficient pose tracking with advanced motion-aware sequence modeling. The invention is also intended to enable practical use on mobile devices, embedded processors, and edge computing environments, providing coaches, athletes, and broadcasters with immediate, reliable action recognition data. Summary of the invention

[0018] The invention presents a novel system and device that combines real-time skeletal landmark extraction with bidirectional motion encoding for robust badminton motion detection. The system continuously captures video data via a camera module, processes it using a MediaPipe-based pose estimator, and sequences the extracted joint coordinates. The Motion-BERT encoder receives these sequences and applies bidirectional temporal attention to capture motion patterns over time. A dense classification head then outputs real-time action labels such as "serve," "smash," or "clear," along with confidence scores.

[0019] The invention also provides a special device structure in which the system is embedded. The device includes an integrated high-speed camera, a computing unit such as an ARM-based CPU with embedded GPU cores or an NVIDIA Jetson system-on-chip, a hardware-accelerated AI inference engine, and a display or communication interface for outputting detected actions. The device is housed in a lightweight, robust enclosure suitable for installation in sports training centers, field-side analysis setups, or even as a portable accessory for coaches.

[0020] Key technical contributions include pose smoothing algorithms to reduce jitter during fast movements, optimized quantization of the Motion-BERT model for real-time inference at or above 30 frames per second, and extensibility for additional sports or action categories.

[0021] The present invention pursues several objectives: It addresses the weaknesses of existing action recognition technologies while simultaneously introducing new features specifically optimized for high-speed racket sports such as badminton. One of the main objectives of the invention is to provide a real-time recognition framework capable of accurately identifying and classifying badminton-specific actions such as serves, smashes, drops, clears, and net shots. In contrast to conventional systems that suffer from latency and inconsistent recognition, this invention aims for end-to-end inference within 100 milliseconds, thus ensuring immediate applicability in live coaching, performance monitoring, and broadcast scenarios.

[0022] Another key objective of the invention is the integration of lightweight yet highly precise skeletal tracking using MediaPipe with advanced temporal sequence modeling based on a Motion Bidirectional Encoder Representation Transformer (Motion-BERT). By combining efficient pose estimation with bidirectional temporal encoding, the invention overcomes the limitations of unidirectional recurrent networks and heavy convolutional models, thus capturing the nuanced movement dynamics that characterize similar-looking actions in badminton. The bidirectional motion modeling enables the system to contextualize both past and future movements, thereby ensuring superior recognition accuracy even in the case of deceptive or anticipatory movements.

[0023] Another objective of the invention is to develop a solution that can be used on edge computing devices such as smartphones, embedded GPU platforms, or dedicated hardware accelerators. Existing systems are often limited to laboratory setups due to their computational intensity. This invention, however, aims to enable practical applicability through the use of model optimization techniques such as quantization, pruning, and runtime acceleration. This ensures that the system functions effectively in real-world environments such as sports academies, training centers, or directly at the edge of the playing field during a game.

[0024] Another objective of the invention is to create an expandable and modular action recognition framework that is not limited to badminton but can also be adapted to other sports or movement-intensive fields. The architecture is designed so that new action categories can be integrated by retraining the Motion-BERT model on domain-specific datasets. This enables applications in tennis, squash, basketball, martial arts, or even health monitoring. This generalizability extends the usefulness of the invention beyond its immediate focus on badminton.

[0025] The invention also aims to improve the robustness of action recognition under challenging conditions such as occlusion, rapid player movements, or inconsistent lighting. By introducing pose smoothing algorithms and noise treatment mechanisms, the invention ensures that skeletal data remains stable and reliable. This improves classification accuracy in real-world scenarios where ideal recording conditions cannot always be guaranteed.

[0026] Another objective of the invention is to enhance the overall experience for athletes, coaches, and spectators by providing actionable insights in real time. Through integration with live dashboards, video annotation tools, or broadcast overlays, the invention enables intelligent feedback, performance statistics, and improved media presentations. This objective aims not only to improve training efficiency but also to transform the way sports are experienced by the audience. BRIEF DESCRIPTION OF THE FIGURE

[0027] These and other features, aspects, and advantages of the present invention will be better understood if the following detailed description is read with reference to the accompanying drawing, in which the same symbols consistently represent the same parts. The following applies: Fig.Figure 1 shows a block diagram of a real-time badminton action detection system and device based on MediaPipe and a representation of a bidirectional motion encoder. Transformer

[0028] Experts will also recognize that the elements in the drawing are shown for the sake of simplicity and are not necessarily to scale. For example, the flowcharts illustrate the process by highlighting the main steps to enhance understanding of the aspects of this disclosure. Furthermore, with regard to the design of the device, one or more components of the device may be represented in the drawing by conventional symbols, and the drawing may show only the specific details relevant to understanding the embodiments of this disclosure, so as not to clutter the drawing with details that are readily apparent to those skilled in the art after reading this description. Detailed description of the invention

[0029] For a better understanding of the inventive principles, reference is made below to the embodiment shown in the drawing, which is described in specific terminology. However, this does not limit the scope of the invention. Changes and further modifications of the illustrated system, as well as further applications of the inventive principles, are possible, as would normally occur to a person skilled in the art in the field of invention.

[0030] It is clear to the person skilled in the art that the preceding general description and the following detailed description are exemplary and explanatory of the invention and are not intended as a limitation of it.

[0031] References in this specification to “an aspect”, “another aspect”, or similar expressions mean that a particular feature, structure, or property described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, occurrences of the expressions “in one embodiment”, “in another embodiment”, and similar expressions in this specification may all refer to the same embodiment, but need not.

[0032] The terms "includes," "include," or other variations thereof are intended to cover non-exclusive inclusion, such that a process or method that includes a list of steps may not only contain those steps but may also include other steps not expressly listed or inherent in such process or method. Likewise, the statement "includes..." in the case of one or more devices, subsystems, elements, structures, or components does not, without further limitations, preclude the existence of other devices, subsystems, elements, structures, components, or additional devices, subsystems, elements, structures, or components.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by a person skilled in the art in the field of the invention. The system, methods, and examples provided here serve only for illustration and are not to be construed as a limitation.

[0034] Embodiments of the present disclosure are described in detail below with reference to the attached drawing.

[0035] Referring to Figure 1, a block diagram of a real-time badminton action detection system and device based on MediaPipe and Motion Bidirectional Encoder Representation Transformer is shown. The System 100 comprises: a video capture module (102) configured to continuously capture video frames at a frame rate of at least 30 frames per second; a pose estimation unit (104) implemented using a MediaPipe engine and configured to detect and output two-dimensional skeletal landmark coordinates for a variety of body joints, including at least wrists, elbows, shoulders, hips, knees, and ankles, from each video frame;a Motion Bidirectional Encoder Representation Transformer (Motion-BERT) (106) configured to receive sequential skeletal landmark coordinates over a defined time window and to encode motion trajectories using multi-head self-attention mechanisms across past and future images; and a classification controller module (108) operationally coupled to the Motion-BERT, wherein the classification controller module comprises a dense neural network with a softmax output layer configured to generate real-time probability distributions over a variety of badminton-specific action classes, the end-to-end system being configured to produce recognition results with a processing latency of less than 100 milliseconds.

[0036] In one embodiment, the pose estimation processing unit (104) further comprises a smoothing subsystem configured to apply a temporal low-pass filter algorithm to successive skeletal landmark coordinates to reduce jitter and noise caused by occlusion, motion blur, or rapid racket arm movements, and wherein the smoothing subsystem dynamically adjusts the filter coefficients based on a detected velocity profile of the joints, such that minimal delay occurs during fast strokes while maintaining stability during steady-state or transition phases.

[0037] In one embodiment, Motion-BERT (106) comprises a multi-layered bidirectional encoder architecture, wherein each encoder layer includes self-attention heads configured to compute temporal attention weights over pose sequences with a length between 32 and 128 frames, and wherein motion dynamics-specific position encodings are added to the pose vectors prior to attention computation to enable the encoder to distinguish between cyclic, repetitive movements and sudden, impact-intensive movements characteristic of badminton strokes.

[0038] In one embodiment, the classification controller module (108) is trained using a dataset of annotated badminton actions, wherein the dataset includes segmented pose sequences labeled "serve", "smash", "drop", "clear" and "net shot", and wherein the classification controller module is further fine-tuned using transfer learning from a general dataset of sporting actions, so that the system can utilize general human movement priorities while also specifically adapting to badminton-specific kinematics.

[0039] In one embodiment, the system is deployed on an edge computing device comprising an ARM-based central processing unit, a graphics processing unit or a dedicated tensor processing accelerator, a storage subsystem configured to buffer at least five seconds of sequential pose data, and a power supply designed to maintain continuous inference for at least two hours, with the Motion-BERT model running on the device being optimized by quantization and clipping to maintain detection accuracy while reducing the computational load to enable real-time performance.

[0040] In one embodiment, the video recording module (102) comprises a high-speed digital camera with a wide-angle lens positioned at a point on the perimeter of the playing field, wherein the camera is calibrated with intrinsic and extrinsic parameters for perspective correction, and wherein the system includes a calibration routine that aligns detected skeletal landmarks with a reference coordinate system of a badminton court, thereby enabling the detection of actions in the context of the player's spatial position relative to the net and the boundaries of the playing field.

[0041] In one embodiment, the Motion-BERT(106) encoder is also equipped with a sliding window mechanism that maintains overlapping frame sequences for continuous action detection. Overlapping sequences allow the system to detect transition phases between successive strikes, thus enabling the detection of compound actions, such as a defensive block followed by an immediate net kill, without loss of temporal continuity.

[0042] In one embodiment, the classification controller module (108) outputs a confidence value, expressed as a normalized probability, in addition to the categorical label of the action, and the system also includes a thresholding mechanism that dynamically adjusts decision thresholds based on the confidence distribution, so that uncertain classifications trigger a fallback mode in which multiple candidate labels with ordered probabilities are presented, thereby improving interpretability for coaching and analysis applications.

[0043] In one embodiment, the system is configured for integration with a visualization interface that includes a real-time dashboard. The interface is configured to overlay detected actions, timestamps, and trust values ​​onto live or recorded video streams and is also configured to store metadata for later retrieval, indexing, and performance analysis of multiple games.

[0044] In one embodiment, the system is extendable to sports other than badminton, with the Motion-BERT encoder being trained using domain adaptation techniques to recognize actions in sports such as tennis, squash, or table tennis. This adaptation involves freezing lower-level encoder weights trained on general pose dynamics, while fine-tuning higher-level attention weights to capture domain-specific stroke mechanics, thus enabling action recognition across multiple sports within the same system architecture.

[0045] The detailed description of the present invention relates to a real-time badminton action detection system that integrates a video recording module, a pose estimation subsystem, a Motion Bidirectional Encoder Representation Transformer (Motion-BERT), and a classification engine configured for low-latency operation. The invention is implemented through a sequence of algorithmic and architectural steps that ensure highly accurate action detection while maintaining processing delays of less than 100 milliseconds. The following description explains the algorithmic flow and technical operation of the invention with reference to the claimed system components.

[0046] The algorithm begins by capturing continuous video signals using a high-speed digital camera that achieves frame rates of at least 30 frames per second. The captured images are then transmitted to the pose estimation subsystem, which uses the MediaPipe framework to extract two-dimensional skeletal landmarks of the subject. In each image, the algorithm identifies at least seventeen major body joints, including wrists, elbows, shoulders, hips, knees, and ankles. The skeletal coordinates are output in normalized form relative to the image, ensuring consistent landmark positioning across different image resolutions and viewing angles. To address jitter and instability in landmark detection caused by rapid racket movements, partial occlusion, or lighting fluctuations, the algorithm includes a smoothing stage.This stage applies a temporal low-pass filter to successive landmark coordinates and dynamically adjusts the filter coefficients to the velocity profile of each joint. For example, if a rapid wrist movement characteristic of a smash is detected, the filter minimizes the smoothing to preserve motion fidelity, while for a static posture, the filter increases the smoothing to stabilize the detected landmarks.

[0047] The pre-processed skeletal coordinates are then transferred to a sliding window buffer, which collects pose sequences across defined time windows. The length of each window can vary between 32 and 128 frames, allowing the system to capture sufficient motion for accurate classification of both fast and long shots. The sliding window design ensures overlapping sequences, thus maintaining temporal continuity across successive actions. This prevents the loss of transition phases between shots, for example, when a player executes a defensive block immediately followed by a shot on goal. Each sequence extracted from the buffer serves as input for the Motion-BERT module.

[0048] In the Motion-BERT encoder, skeletal pose vectors are first augmented with positional codings specifically designed for motion dynamics. Unlike conventional positional codings, which merely index the sequence of frames, these codings incorporate motion-based information such as relative displacement and joint velocity. This allows the encoder to distinguish between repetitive, cyclical actions like leg movement patterns and sudden, impactful movements like a smash. The encoder consists of several bidirectional layers, each containing multi-headed self-attention mechanisms. For each input sequence, the self-attention operation calculates similarity values ​​between all frame pairs within the sequence. In this way, the encoder identifies the temporal relationships between frames that may be widely separated in time but correlated in terms of motion.For example, the start of a smash swing and the moment of shuttle contact might be separated by dozens of frames, but the encoder can link these events through high attention weights. The encoder's bidirectional nature ensures that contextual information from past and future frames is incorporated into the representation. This allows the system to resolve ambiguities in deceptive movements. For instance, the initial movements of a clear and a drop shot may appear nearly identical. However, by considering subsequent movement cues, the encoder can distinguish between the two actions with greater certainty.

[0049] The coded output of the Motion-BERT is then passed to the classification controller module, which consists of a dense neural network with one or more hidden layers and a final softmax output layer. The dense network aggregates the encoder's motion-aware embeddings and computes a probability distribution across predefined action categories such as serve, smash, drop, clear, and net shot. The softmax layer ensures that the probabilities are normalized across all categories and that the label with the highest probability is output as the recognized action. Furthermore, the classification controller module generates a confidence score representing the certainty of the prediction. To manage uncertain classifications, the algorithm includes a dynamic thresholding mechanism.If the highest probability does not exceed the confidence threshold, the system triggers a fallback mode in which the two or three best candidate labels are evaluated and displayed. This ensures interpretability for downstream applications such as coaching analyses.

[0050] To further improve accuracy and efficiency, the classification controller module is trained using a badminton-specific dataset containing annotated pose sequences for each stroke type. Training begins with transfer learning, where the Motion-BERT encoder is first trained on a general sports action dataset to capture broad movement priorities. Subsequently, the encoder and classifier are fine-tuned using the badminton dataset to specialize the model in sport-specific kinematics. This two-stage training strategy minimizes the risk of overfitting while maximizing recognition performance for badminton strokes.

[0051] The entire algorithm is implemented on an edge computing device optimized for real-time use. The hardware consists of an ARM-based processor coupled with a graphics processing unit (GPU) or a tensor accelerator to support parallelized inference. The Motion-BERT model is further optimized by quantization, pruning, and runtime acceleration libraries to ensure computational efficiency without compromising accuracy. The device's memory subsystem buffers several seconds of sequential pose data, enabling smooth and uninterrupted operation even with fluctuating input rates. The processing pipeline, from single-frame capture to action classification, is executed in less than 100 milliseconds, ensuring true real-time operation.

[0052] In another embodiment, the system includes a calibration routine that maps skeletal landmarks to a reference badminton court coordinate system. During initialization, intrinsic and extrinsic camera parameters are estimated to correct perspective distortions. This allows the detected actions to be contextualized within the spatial characteristics of the court, for example, to distinguish between a forehand clear near the baseline and a net throw in the forecourt. The calibration also enables integration with tactical analysis modules that rely on spatial positioning in addition to action recognition.

[0053] The output of the classification controller module is transferred to a visualization interface that overlays detected actions, confidence scores, and timestamps directly onto live or recorded video streams. The metadata generated by the system, including detected action sequences and associated probabilities, is logged in a structured database and can subsequently be retrieved, indexed, and used for performance analysis. The system can also be synchronized with cloud-based platforms to enable long-term storage and cross-player performance comparisons.

[0054] In another embodiment, the system architecture can be extended to sports other than badminton. By freezing lower-level encoder weights that capture general movement dynamics and retraining higher-level attention weights based on sport-specific datasets, the system can be adapted to detect tennis strokes, squash rallies, or table tennis smashes. This modular adaptability ensures that the invention is not limited to a single sport but represents a generalizable platform for real-time, position-based action detection.

[0055] Through this detailed algorithmic framework, the invention achieves a balance between accuracy, latency, and computing power, overcoming the limitations of existing CNN-LSTM, OpenPose-based, or pure Transformer systems. The integration of MediaPipe with Motion-BERT, along with optimization for edge deployment and scalability across domains, makes the invention a technically advanced and practically applicable solution for real-time badminton action detection.

[0056] In one embodiment, the invention comprises four functional modules. The first is the input acquisition module, which includes a high-resolution digital camera capable of at least 60 frames per second to ensure smooth tracking of rapid badminton movements. The camera can be mounted on a tripod or integrated into a portable device.

[0057] The second module is the Pose Estimation module, implemented using MediaPipe's Pose engine. This module extracts the 2D skeletal coordinates of at least 17 joints from each captured image, including wrists, elbows, shoulders, hips, knees, and ankles. The system also applies smoothing filters to minimize jitter caused by occlusion, motion blur, or inconsistent image capture.

[0058] The third module is the Motion-BERT encoder, which operates with a sliding window of sequential pose vectors. This encoder applies bidirectional temporal attention, allowing the system to understand not only immediately past movements but also predictable future movements, thus distinguishing between actions such as a fake drop and a smash setup.

[0059] The fourth module is the classification header, which uses a dense neural network with a softmax output layer. This classifier assigns probabilities to predefined action labels, thus generating a real-time recognition result with confidence metrics.

[0060] In another embodiment, the invention takes the form of a special device structure. The device consists of a carbon fiber reinforced housing containing the following: 1. A wide-angle, high-speed digital camera for continuous motion capture. 2. A primary processing unit consisting of an ARM Cortex-A78 CPU integrated with a GPU or a dedicated tensor accelerator. 3. A cooling system that maintains thermal stability during continuous inference at 30 fps or more. 4. A display panel or wireless transmission module for outputting live action detection results to smartphones, tablets, or larger dashboards. 5. An internal rechargeable power supply is optimized for two hours of continuous operation, making the device suitable for use on the sidelines.

[0061] The device operates in real time, receiving raw video streams, calculating skeletal poses, encoding temporal dynamics, and classifying actions with an end-to-end latency of less than 100 milliseconds. The system can also include cloud synchronization capabilities for storing annotated performance data for further analysis.

[0062] The present invention relates to the technical fields of computer vision, artificial intelligence, and machine learning, and in particular to systems for the real-time recognition of human actions for use in sports analysis. The invention specifically addresses the recognition of fine-grained athletic movements in fast-paced racket sports, with a focus on badminton, through the integration of lightweight pose estimation frameworks and bidirectional temporal motion modeling. The system utilizes MediaPipe for the efficient extraction of skeletal landmarks and employs a Motion Bidirectional Encoder Representation Transformer (Motion-BERT) to capture temporal dependencies in sequential pose data, thereby enabling precise, real-time classification of complex actions.The invention lies at the intersection of deep learning, edge computing and intelligent human-computer interaction, and offers a deployable solution for improving player performance, enhancing coaching methods, enriching sports broadcasts and extending action recognition to other areas such as health monitoring, gaming and surveillance.

[0063] The drawing and the preceding description show examples of embodiments. Those skilled in the art will recognize that one or more of the described elements can be combined to form a single functional element. Alternatively, certain elements can be divided into several functional elements. Elements of one embodiment can be added to another embodiment. For example, the sequence of the processes described here can be changed and is not limited to the manner described here. Furthermore, the actions of a flowchart need not be implemented in the sequence shown; nor does it necessarily have to be performed by all actions. Actions that are not dependent on other actions can also be performed in parallel with the other actions. The scope of the embodiments is in no way limited by these specific examples.Numerous variations are possible, whether explicitly stated in the specification or not, such as differences in structure, dimensions, and material use. The range of embodiments is at least as broad as specified in the following claims.

[0064] Advantages, further benefits, and problem solutions have been described above with reference to specific embodiments. However, the advantages, benefits, problem solutions, and all components that can lead to an advantage, benefit, or solution occurring or becoming more apparent are not to be construed as critical, necessary, or essential features or components of individual or all claims. REFERENCES 100 A real-time system and device for the detection of badminton actions based on MediaPipe and Motion Bidirectional Encoder Representation Transformer. 102 Video recording module 104 Unit for Pose Estimation 106 Transformer for the representation of bidirectional motion encoders 108 Classification Controller Module

Claims

[1] A real-time badminton action detection system consisting of: a video recording module configured to continuously record video images at a frame rate of at least thirty frames per second; a pose estimation processing unit configured to detect and output two-dimensional skeletal landmark coordinates for a variety of body joints, including at least wrists, elbows, shoulders, hips, knees and ankles, from each video frame; a Motion Bidirectional Encoder Representation Transformer (Motion-BERT) configured to receive sequential skeleton landmark coordinates over a defined time window and encode motion trajectories using multi-head self-attention mechanisms across past and future frames; and a classification controller module operationally coupled to the Motion-BERT, wherein the classification controller module comprises a dense neural network with a softmax output layer configured to generate real-time probability distributions over a variety of badminton-specific action classes, the end-to-end system being configured to produce recognition results with a processing latency of less than 100 milliseconds; and wherein the video recording module comprises a high-speed digital camera with a wide-angle lens positioned at a point on the perimeter of the court, the camera being calibrated with intrinsic and extrinsic parameters for perspective correction, and wherein the system includes a calibration routine that aligns detected skeletal landmarks with a reference badminton court coordinate system. [2] System according to claim 1, wherein the pose estimation processing unit further comprises a smoothing subsystem configured to apply a temporal low-pass filter algorithm to successive skeletal landmark coordinates to reduce jitter and noise caused by occlusion, motion blur or rapid racket arm movements, and wherein the smoothing subsystem dynamically adjusts the filter coefficients based on a detected velocity profile of the joints such that minimal delay occurs during fast strokes while maintaining stability during steady or transition phases. [3] System according to claim 1, wherein the Motion-BERT comprises a multi-layered bidirectional encoder architecture, wherein each encoder layer contains self-attention heads configured to compute temporal attention weights over pose sequences of length between 32 and 128 frames, and wherein position-specific encodings are added to the pose vectors prior to the attention computation for motion dynamics in order to enable the encoder to distinguish between cyclic, repetitive movements and sudden, jerky movements characteristic of badminton strokes. [4] System according to claim 1, wherein the classification controller module is trained on a data set of annotated badminton actions, the data set comprising segmented pose sequences labeled "serve", "smash", "drop", "clear" and "net shot", and wherein the classification controller module is further fine-tuned using transfer learning from a general data set of sporting actions so that the system can utilize general human movement priorities while also specifically adapting to badminton-specific kinematics. [5] System according to claim 1, wherein the system is deployed on an edge computing device comprising an ARM-based central processing unit, a graphics processing unit or a dedicated tensor processing accelerator, a storage subsystem configured to buffer at least five seconds of sequential pose data, and a power supply designed to sustain continuous inference for at least two hours, wherein the Motion-BERT model running on the device is optimized by quantization and clipping to maintain detection accuracy while reducing the computational load to enable real-time performance. [6] System according to claim 1, wherein the Motion-BERT encoder is further equipped with a sliding window mechanism that maintains overlapping frame sequences for continuous action detection, wherein overlapping sequences enable the system to detect transition phases between successive strikes and thus enable the detection of compound actions such as a defensive block followed by an immediate net kill without loss of temporal continuity. [7] System according to claim 1, wherein the classification controller module outputs a confidence value expressed as a normalized probability in addition to the categorical label of the action, and wherein the system also includes a threshold mechanism that dynamically adjusts decision thresholds based on the confidence distribution, such that uncertain classifications trigger a fallback mode in which multiple candidate labels with ordered probabilities are presented, thereby improving interpretability for coaching and analysis applications. [8] System according to claim 1, wherein the system is configured for integration with a visualization interface comprising a real-time dashboard, the interface being configured to display detected actions, timestamps and trust ratings on live or recorded video streams, and further configured to store metadata for later retrieval, indexing and performance analysis across multiple games. [9] System according to claim 1, wherein the system is extendable to sports other than badminton, wherein the Motion-BERT encoder is trained using domain adaptation techniques to detect actions in sports such as tennis, squash or table tennis, wherein the adaptation includes freezing lower-level encoder weights trained on general pose dynamics while fine-tuning higher-level attention weights to capture domain-specific stroke mechanics, thereby enabling the detection of actions in multiple sports within the same system architecture.

Images