Sports fitness auxiliary application based on combination of AR glasses analysis and Transform

Abstract

The invention relates to the technical field of physical fitness assistance, in particular to a physical fitness assistance application based on combination of AR glasses analysis and Transform, comprising a camera module, a sensor module, AR glasses, a wireless communication module and a processing module; wherein the camera module collects a user motion video stream, the sensor module synchronously records acceleration, angular velocity and direction data, the processing module operates a Transform model in real time through a parallel computing architecture, and the AR glasses superposes three-dimensional visual guidance information to a real scene by adopting a waveguide optical technology; the camera module carries out image noise cleaning, then image features are extracted through a convolutional network, and feature normalization processing is carried out; after the sensor module removes noise through the low-pass filter, a sliding window extracts time sequence features, and feature normalization processing is carried out; the two features are fused and spliced, time sequence modeling is carried out based on Transform, action behaviors are output, calculation with a standard action template is carried out, suggestions are proposed and displayed in AR glasses, and exercise data are recorded and reversely input to a Transform model for recording.

Description

Technical Field

[0002] The present invention relates to the technical field of sports fitness assistance, and particularly relates to a sports fitness assistance application based on the combination of AR glasses analysis and Transformer. Background Art

[0003] In recent years, with the improvement of health awareness and the popularization of smart wearable devices, the sports fitness assistance technology has developed rapidly. Traditional fitness training mainly relies on on-site coaching by coaches or users watching teaching videos by themselves. However, this method has problems such as insufficient real-time feedback and difficulty in ensuring the standardization of movements. The accumulation of incorrect training postures over a long time may lead to sports injuries, reduce the training effect, and even cause chronic musculoskeletal diseases. Existing fitness assistance technologies mainly include motion capture systems based on inertial sensors and motion analysis solutions based on computer vision. The inertial sensor solutions (such as smart bracelets, sports belts, etc.) can collect data such as acceleration and angular velocity for analyzing the movement frequency and amplitude. However, their spatial perception ability is limited, and it is difficult to accurately evaluate the joint angles and body postures of complex movements. The vision-based solutions (such as smartphone apps or fixed cameras) can provide richer spatial information, but they are limited by shooting angles, lighting conditions, and computing resources, and it is difficult to achieve stable and low-latency analysis in mobile scenarios.

[0004] In recent years, augmented reality (AR) technology has provided a new interaction method for fitness assistance. Some commercial AR glasses have tried to combine with motion guidance functions. However, existing solutions mostly use rule engines or simple machine learning models, with insufficient recognition accuracy for complex movements and lack of personalized adaptive adjustment capabilities. At the same time, traditional methods usually process sensor data or visual data independently, and fail to fully utilize the complementary advantages of multi-modal information, resulting in limited comprehensiveness and robustness of motion evaluation.

[0005] However, many existing fitness applications only rely on mobile phone cameras or handheld devices and cannot comprehensively capture the movement details of users. The action recognition algorithms are not advanced enough to accurately identify and correct complex fitness movements. Many applications can only provide feedback after the event and cannot correct actions in real time when users are exercising. The real-time feedback system often has a large delay, affecting the user experience. Existing applications mainly rely on a single data source (such as a camera), ignoring the importance of multi-sensor fusion and unable to comprehensively understand the user's motion state and posture. The prompts and guidance of many applications are preset, lacking personalized customization and interactivity and unable to target the user's actual situation.

[0006] In view of the above situation, in order to overcome the above technical problems, the present invention designs a sports fitness assistance application based on the combination of AR glasses analysis and Transformer, which solves the above technical problems. Summary of the Invention

[0007] The technical objective to be achieved by the present invention is to design a sports fitness assistance application based on the combination of AR glasses analysis and Transformer, to achieve augmented reality action correction, and to adjust incorrect actions or postures.

[0008] To achieve the above technical objective, the present invention provides the following technical solutions: A sports fitness assistance application based on the combination of AR glasses analysis and Transformer, characterized in that it includes a camera module, a sensor module, AR glasses, a wireless communication module, and a processing module; wherein the camera module collects the user's motion video stream, the sensor module synchronously records acceleration, angular velocity, and direction data, the processing module runs the Transformer model in real time through a parallel computing architecture, and the AR glasses use waveguide optical technology to superimpose three-dimensional visualization guidance information onto the real scene; After the camera module performs image noise cleaning, a convolutional network extracts image features and performs feature normalization processing; after the sensor module removes noise through a low-pass filter, a sliding window is used to extract temporal features and perform feature normalization processing; the two features are fused and spliced, and the base Transformer performs temporal modeling to output action behaviors, calculates with the standard action template, proposes suggestions and displays them in the AR glasses, and records the exercise data and inputs it back to the Transformer model for recording.

[0009] Preferably, the camera module includes a three-stage processing process: first, a noise cleaning algorithm is used to perform light compensation and noise elimination on the original video frame; then, a convolutional network is used to extract the two-dimensional spatial coordinate features of 17 key points of the human body, and finally, the key point coordinates are standardized through a feature normalization unit. Preferably, the camera module converts the pixel coordinates into normalized proportional coordinates relative to the torso based on the user's height, and establishes a 128-dimensional feature vector including the spatial relationship of joint points. This vector also encodes the confidence scores of each joint point for subsequent feature fusion weighting.

[0010] Preferably, the sensor module adopts a multi-stage signal processing architecture: the first-stage hardware filter circuit performs 20Hz low-pass filtering on the original IMU signal to eliminate high-frequency noise; the second-stage software filter layer solves the problem of sensor drift during movement; the third-stage feature extraction layer uses a sliding time window for processing, with a 50% overlap between adjacent windows. Each window extracts time-domain features (including mean, variance, and zero-crossing rate) and frequency-domain features (the amplitudes of 5 main frequency components calculated by FFT), and finally generates a temporal feature vector containing 15 features. All features are unified and calibrated according to the principles of sports biomechanics.

[0011] Preferably, the processing module adopts a spatio-temporal alignment cross-modal fusion strategy: in the time dimension, the image feature frames are strictly aligned with the sensor data through a hardware-level synchronization signal, and the error is controlled within ±5 ms; in the feature dimension, the 128-dimensional spatial feature vector output by the camera module is concatenated with the 15-dimensional temporal feature vector of the sensor module to form a 143-dimensional fusion feature.

[0012] Preferably, the Transformer model adopts a four-layer improved Transformer encoder architecture: each layer contains 12 heads of self-attention mechanisms, where the first 6 heads are specifically used to process spatial visual features, the last 6 heads focus on analyzing the sensor temporal features, and finally, cross-attention is used to achieve information interaction between modalities.

[0013] Preferably, the processing module establishes a dynamically updated standard action template library: each standard template contains the motion trajectory function of each joint point and the angle threshold of the key phase points under ideal conditions; during evaluation, an improved dynamic time warping algorithm is used to calculate the multi-dimensional difference degree between the user's action and the template on the basis of time alignment, including: joint angle deviation (accurate to 0.5°), motion speed ratio, and trajectory smoothness; the feedback information generation module gives graded prompts according to the difference degree: the first-level prompt (deviation <5°) only displays the AR green mark, the second-level prompt (5° ≤ deviation <10°) superimposes a yellow direction arrow, and the third-level prompt (deviation ≥10°) triggers a red flashing warning and a voice warning; all feedback information is accurately anchored to the user's actual joint position through spatial registration technology.

[0014] Preferably, the auxiliary application also includes a built-in dual-loop update mechanism in the system: the short-term loop performs offline analysis on the user's training data every day, calculates the 7-day moving average of each action index, and automatically adjusts the fault tolerance threshold of the template; the long-term loop performs model fine-tuning once a month, injects the user's historical correct action features as new positive samples into the contrast learning framework of the Transformer model, and uses the elastic weight consolidation algorithm to prevent catastrophic forgetting during the update; at the same time, a user-specific motion profile is established to record the change trends of 12 biomechanical indexes including joint range of motion and strength symmetry. These data are synchronized to the cloud analysis platform after differential privacy processing.

[0015] Preferably, the AR glasses adopt a hierarchical rendering AR display strategy: the bottom layer displays the real-time motion tracking skeleton, the middle layer superimposes the ideal position marks (colored spheres) of the key action nodes, and the top layer displays the error correction guidance (dynamic arrows / words); the display system intelligently adjusts the information density through the eye movement tracking module and automatically expands the detailed data panel when it detects that the user is gazing at a certain joint.

[0016] Preferably, the auxiliary application also includes a safety monitoring and emergency processing module, which continuously calculates the joint load index and immediately initiates the protection protocol when an abnormal impact load is detected; the fall detection algorithm analyzes the sudden change pattern of the IMU signal (acceleration>3g and angular velocity>200° / s), triggering an emergency response within 300ms; the scene perception unit uses a semantic segmentation network to identify dangerous factors in the training environment (such as slippery ground, obstacles), and displays highlighted warnings through AR; all safety events automatically generate a report containing the sensor data before and after 10 seconds, and send it to the preset emergency contact through the wireless module, and store the encrypted complete event record locally.

[0017] The beneficial effects of the present invention are as follows: (1) Traditional fitness auxiliary equipment (such as smart bracelets and mobile phone apps) usually requires users to manually operate the screen or buttons, which will interfere with the training process and even increase the risk of injury. However, the AR glasses used in the present invention integrate high-precision gesture recognition and noise reduction voice interaction modules. Users only need simple gesture sliding or voice commands (such as "switch action" and "adjust viewing angle") to control the system without pausing training or touching the device.

[0018] (2) Users can use palm rotation gestures to enlarge the action correction prompts displayed in AR, or query the current training data through voice, maintaining movement continuity throughout the process. In addition, the built-in eye tracking technology of the glasses can assist in identifying the user's focus and automatically call up relevant data panels (such as heart rate and calories burned), further reducing the need for active interaction. This contactless, immersive interaction mode greatly improves the fluency and safety of training. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] The above and other aspects of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 It is a schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0022] like Figure 1As shown in the figure, the present invention relates to an innovative sports fitness assistance system, the core of which is to combine augmented reality (AR) technology with an advanced Transformer deep learning model to provide users with real-time and accurate exercise guidance. The system mainly consists of five hardware modules: a high-precision camera module, a multi-axis sensor module, AR display glasses, a wireless communication module, and a high-performance processing module. These modules work together to form a complete intelligent fitness assistance closed-loop system.

[0023] In terms of the hardware architecture, the system adopts a modular design to achieve optimal performance. The camera module is equipped with a 120° wide-angle lens and an RGB-D depth sensor, and captures the user's exercise video stream at a frame rate of 60fps to ensure that the complete range of motion is captured. The sensor module integrates a nine-axis inertial measurement unit (IMU), including a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, and synchronously records the user's acceleration, angular velocity, and direction data at a sampling frequency of 100Hz. The AR glasses adopt the latest waveguide optical display technology, with a light transmittance of 85% and a field of view of 40°, and can naturally superimpose clear three-dimensional visualization guidance information onto the user's real field of vision. The wireless communication module supports dual-mode transmission of Wi-Fi 6 and Bluetooth 5.2 to ensure the stability of real-time data synchronization. The processing module adopts a heterogeneous computing architecture, integrating a GPU, an NPU, and a CPU, and providing up to 15TOPS of computing power support for the Transformer model.

[0024] In the data processing flow, the system realizes fine multi-modal feature extraction and fusion. The camera module first processes the original video frames using a deep learning-based image denoising algorithm to eliminate the influence of environmental light changes and motion blur. Subsequently, the spatial features of 17 key points of the human body are extracted through an optimized lightweight HRNet convolutional neural network, including the two-dimensional coordinates and confidence scores of each joint. These features are processed by Z-score normalization to form a 128-dimensional feature vector. The sensor module adopts a four-level signal processing flow: hardware-level analog filtering to remove high-frequency noise, digital Kalman filtering to eliminate sensor drift, a sliding window (1-second window length, 50% overlap) to extract time-domain and frequency-domain features, and finally Min-Max normalization to output a 15-dimensional time-series feature vector.

[0025] The core innovation of the system lies in its Transformer-based multi-modal fusion and action analysis architecture. Visual and sensor features are first strictly aligned in the spatio-temporal dimension, ensuring synchronization accuracy within ±5ms through hardware timestamps. The feature fusion layer concatenates the two types of features into a 143-dimensional joint feature vector and adds learnable modality identity encoding and position encoding. The improved Transformer model adopts a four-layer encoder structure, where the self-attention mechanism is designed as a "sub-modal - interaction" dual-path architecture: the first 6 heads focus on processing spatial visual features, the latter 6 heads analyze temporal sensor features, and finally, cross-attention is used to achieve cross-modal information fusion. The model output includes action classification results and predictions of the three-dimensional spatial coordinates of each joint point.

[0026] The real-time feedback mechanism of the system constructs a complete guidance closed-loop. The Dynamic Time Warping (DTW) algorithm is used to calculate the difference between the user's action and the standard template. When the joint angle deviation exceeds the preset threshold, the AR glasses will immediately project colored guiding marks at the corresponding positions: green indicates a standard action, a yellow arrow indicates a deviation of 5° - 10°, and a red flashing warning mark indicates a serious error (deviation ≥ 10°). All feedback information is accurately anchored in the real space position through SLAM technology to ensure the accuracy of the guidance. The system also innovatively developed a "virtual mirror" function to help users observe their back actions through 3D reconstruction from the rear view.

[0027] Another important feature of the system is its continuous learning ability. After each training, the user's action data and system feedback are encrypted and stored, and participate in model optimization through the federated learning framework. This design not only protects user privacy but also enables the system to continuously adapt to the user's personal characteristics. Through empirical testing, the accuracy of this system in recognizing common fitness actions reaches 98.7%, and the real-time feedback delay is controlled within 80ms, significantly superior to existing solutions. Its innovative multi-modal fusion architecture and natural AR interaction method provide a new technical paradigm for the field of intelligent fitness assistance.

[0028] The camera module adopts an advanced three-stage processing flow to ensure the accurate acquisition and analysis of motion data. In the first stage, the system uses a deep learning-based noise cleaning algorithm to eliminate the influence of environmental light changes through an adaptive light compensation module, and at the same time adopts an improved non-local mean denoising technique to effectively remove Gaussian noise and motion blur in video frames. In the second stage, a lightweight HRNet convolutional neural network is deployed. This network is specially optimized for motion data and can real-time detect and extract the two-dimensional spatial coordinate features of 17 key points of the human body, including parts such as the head and main joints of the limbs, and at the same time output the confidence scores of each joint point. In the final feature processing stage, the system converts the original pixel coordinates into a normalized coordinate system centered on the human torso through a feature normalization unit, and uses the Z-score normalization method to eliminate the influence of individual body shape differences, and finally generates a 128-dimensional feature vector containing spatial position relationships and confidence weights, providing high-precision visual input data for subsequent multi-modal feature fusion.

[0029] The camera module adopts an intelligent coordinate conversion algorithm based on ergonomics. First, it accurately identifies the reference point of the user's torso center through a deep learning model, and then uses the user's actual height as the normalization reference to convert the pixel coordinates in the original image into a normalized proportional coordinate system centered on the torso (range [-1,1]). During this process, the system dynamically calculates the spatial position relationship of each joint point relative to the torso, and comprehensively considers the proportional characteristics of various parts of the human body to ensure the comparability of motion data of users with different body shapes. The finally generated 128-dimensional feature vector not only accurately encodes the normalized coordinate positions (x,y) of 17 joint points, but also includes the motion confidence scores (range 0-1) of each joint point. This confidence is calculated in real time by the neural network according to factors such as image quality and occlusion. In the subsequent multi-modal feature fusion stage, the system adaptively weights the visual features based on these confidence scores, significantly improving the robustness and accuracy of data processing in complex motion scenarios.

[0030] The sensor module adopts an intelligent three-level signal processing architecture to ensure the accuracy and reliability of motion data acquisition. At the first-level hardware processing level, the module is built-in with a high-performance analog filtering circuit, and a Butterworth low-pass filter with a cut-off frequency of 20Hz is used to effectively filter out high-frequency noise and circuit interference in the IMU raw signal, while retaining the key frequency band information of human motion. At the second-level software processing layer, an improved adaptive Kalman filtering algorithm is deployed. This algorithm can real-time monitor the sensor drift characteristics, and significantly reduce common gyroscope drift and accelerometer offset problems during motion by dynamically adjusting the process noise parameter and the observation noise covariance matrix, ensuring the long-term stability of attitude solution.

[0031] In the third - level feature extraction stage, the system processes data using a sliding window with a duration of 1 second. The window step size is 0.5 seconds (i.e., 50% overlap), and multi - dimensional motion features are synchronously extracted within each window. Time - domain analysis includes 6 features such as motion mean (reflecting the basic trend), variance (characterizing the motion amplitude), and zero - crossing rate (indicating the action frequency). Frequency - domain analysis calculates the signal power spectrum through the Fast Fourier Transform (FFT) and extracts the normalized amplitude features of the first 5 main frequency components. The system also innovatively introduces 4 derivative features based on motion biomechanics, including joint angular velocity and motion - plane stability index. All 15 - dimensional features are strictly unified in physical units to ensure the comparability of data from different sensors and different dimensions. The finally generated time - series feature vector not only retains the detailed features of the original motion signal but also meets the standardized requirements of human motion analysis, providing high - quality input data for subsequent Transformer modeling.

[0032] The processing module adopts an innovative spatio - temporal alignment cross - modal fusion strategy to achieve precise collaborative processing of visual and inertial data. In terms of time synchronization, the processor provides an accurate global clock signal for the camera module and the IMU sensor module through a dedicated hardware synchronization interface. The time - stamp interpolation compensation technology is used to ensure that the time - alignment error between the two types of data is strictly controlled within the range of ±5ms, fully meeting the time - series accuracy requirements of human motion analysis. At the feature - fusion level, the processor first performs dimensionality - expansion processing on the 128 - dimensional spatial feature vector (including normalized joint - point coordinates and confidence scores) output by the camera module, and at the same time normalizes and scales the 15 - dimensional time - series feature vector (including time - domain, frequency - domain, and biomechanics features) of the sensor module. Through an intelligent feature - stitching algorithm, the two types of features are aligned and merged in a dedicated memory buffer to form a 143 - dimensional cross - modal fusion feature vector. To enhance the feature - representation ability, the system also automatically injects a 2 - dimensional modality - identification vector ([1,0] represents visual features, [0,1] represents sensor features), enabling the subsequent Transformer model to distinguish feature information from different sources. This refined fusion strategy not only retains the unique characteristics of each modality but also realizes in - depth information complementarity, providing a more comprehensive data basis for action recognition.

[0033] The Transformer model adopts a four-layer improved encoder architecture specifically designed for motion analysis, and realizes the efficient processing of multi-modal features through an innovative attention allocation mechanism. Each layer of the encoder contains 12 independent attention heads, and a sub-modal processing strategy is adopted: the first 6 attention heads are dedicated to parsing the spatial visual features extracted by the camera module, focusing on capturing the geometric relationships and motion trajectories between human joint points; the last 6 attention heads focus on processing the temporal features provided by the sensor module, and deeply analyze dynamic information such as motion speed and rhythm. On the basis of hierarchical processing, the model introduces a cross-modal cross-attention mechanism, and establishes a dynamic association between visual spatial features and sensor temporal features through a learnable attention weight matrix, realizing the deep interaction and complementarity of the two modal information. This architecture design significantly improves the model's ability to understand complex motion patterns, while maintaining high computational efficiency, ensuring real-time performance on embedded devices.

[0034] The processing module constructs an intelligent dynamic action evaluation system, the core of which is a standardized action template library established based on professional sports biomechanics. The template library adopts a hierarchical storage architecture, and each standard action template contains accurate parameters in three dimensions: first, the ideal motion trajectory function of each joint point modeled by B-spline curves, which can accurately describe the spatial path characteristics of the standard action; second, the angle threshold matrix of key phase points, which defines the allowable range of angles of each joint at different time phases during the action; finally, the motion dynamics parameters, including time domain features such as standard speed curves and acceleration distributions. The processor adopts incremental learning technology, allowing coaches to add new professional action templates or adjust existing parameters at any time through the management interface.

[0035] In the real-time evaluation stage, the system adopts an improved multi-scale dynamic time warping algorithm (MS-DTW). This algorithm first realizes the time alignment of the user's action sequence and the template through dynamic programming, and then calculates three-dimensional difference metrics: 1) joint angle deviation, which uses quaternion interpolation to achieve spatial angle calculation with an accuracy of 0.5°; 2) motion speed ratio, which analyzes the percentage difference between the speeds of each stage of the user's action and the standard value; 3) trajectory smoothness, which evaluates the smoothness of the action path through curvature analysis. After normalization processing, the evaluation results generate a comprehensive difference score (0-100 points).

[0036] Based on the evaluation results, the system's intelligent feedback engine implements a three-level progressive prompting strategy: for minor deviations (difference < 15%, corresponding angular deviation < 5°), only the relevant joints are highlighted in semi-transparent green in the AR field of view to avoid disturbing the user; medium deviations (15% - 30%, 5° - 10°) trigger yellow dynamic arrows for guidance, and the length and direction of the arrows reflect the magnitude of adjustment in real time; in case of severe deviations, a red pulsating warning box is immediately activated, and at the same time, a directional voice warning (such as pay attention to keeping your waist straight) is played through spatial audio technology. All feedback elements achieve sub-centimeter spatial registration accuracy through visual-inertial fusion SLAM technology, ensuring the precise alignment of virtual prompts with real body parts. The system also innovatively introduces a deviation heat map mode, which intuitively shows the error distribution of various parts of the whole body with color gradients to help users quickly locate problem areas.

[0037] This evaluation feedback mechanism has been verified by actual measurements of professional athletes. In scenarios such as weight training and yoga, it can average improve the action standardization by 42% and significantly reduce the risk of sports injuries. At the same time, the system will encrypt and store the evaluation data of each training for generating an individual sports ability development curve to provide data support for long-term training planning.

[0038] The described auxiliary application innovatively designs a double-loop adaptive update mechanism to achieve the continuous evolution and personalized adaptation of the system's capabilities. The short-term update loop has a cycle of 24 hours and is automatically started during the idle period at night to conduct multi-dimensional offline analysis of the training data of the day: calculate the 7-day moving average of each action index of the user through the sliding window algorithm and dynamically adjust the personalized tolerance threshold of the standard template; use the anomaly detection algorithm to identify the unique motion pattern characteristics of the user and generate an adaptive parameter matrix. The long-term update loop performs a deep optimization once a month. Through the federated learning framework, the historical correct action characteristics of the user within three months (verified by sports experts) are used as high-quality positive samples and injected into the contrastive learning framework of the Transformer model. During the model update process, the improved elastic weight consolidation (EWC++) algorithm is adopted. By imposing regularization constraints on important parameters, the retention rate of the original recognition ability exceeds 95% when new knowledge is injected.

[0039] The system synchronously constructs a complete user personalized sports profile system, continuously tracking 12 core biomechanical indicators: including the range of motion of six major joints (shoulder, elbow, wrist, hip, knee, ankle), the left-right side strength symmetry coefficient, the core stability index and other key parameters. These data are managed through a time series database, and sliding average filtering is used to eliminate measurement noise, and a visualizable ability development curve is generated. To protect user privacy, all data uploaded to the cloud are processed with differential privacy with ε = 0.5, adding Gaussian noise that complies with the ISO / IEC 29100 standard, achieving strict privacy protection while ensuring the accuracy of data analysis. The cloud analysis platform will aggregate group data, identify common patterns of training bottlenecks, and provide guidance for the evolution direction of the system. This double-loop mechanism enables the system to adapt to the immediate needs of users in the short term and achieve long-term intelligent evolution, ultimately forming a truly personalized digital fitness assistant.

[0040] The AR glasses adopt an innovative hierarchical rendering display strategy to clearly present motion guidance information through three-dimensional visual hierarchy design. At the bottom layer of the display architecture, the system renders the user's current motion skeleton model in real time with blue lines at 50% transparency, providing a basic action reference framework. The middle layer display uses colored spheres to mark the ideal positions of key action nodes, where green spheres represent joints that have reached the standard, and orange spheres indicate nodes that need fine-tuning. Each sphere is accurately anchored in the real space position through SLAM technology. The top dynamic guidance layer, according to the real-time analysis results, superimposes red direction arrows (indicating the adjustment direction) and concise text prompts (such as raise the elbow by 5 cm) at joints where the deviation exceeds the threshold. All guidance elements come with smooth entry animations to avoid visual interference.

[0041] The eye movement tracking module integrated in the system detects the user's gaze point at a frequency of 120 Hz. When the user continuously gazes at a certain joint for more than 0.5 seconds, a fan-shaped data panel will automatically expand at that position, displaying in detail the current joint angle, deviation value and improvement suggestions. Through the intelligent information density adjustment algorithm, the system will dynamically control the display intensity of each layer according to the user's line of sight focus: the transparency of the skeleton in the non-gazed area is increased to 70%, while the relevant guidance for the gazed joint enhances the display brightness and expands the size to ensure the efficient transmission of key information. This hierarchical progressive display method not only ensures the integrity of the guidance information but also avoids AR interface overload, and it can reduce the user's cognitive load by 40% through actual measurement.

[0042] The auxiliary application integrates an intelligent security monitoring and emergency handling system, which constructs an all-round motion protection system through multi-sensor fusion technology. The core of the system is a real-time joint load monitoring algorithm that calculates the joint load index (JLI) of each major joint 60 times per second, comprehensively considering three dimensions: instantaneous pressure value, cumulative fatigue degree, and movement normativity. When an abnormal impact load is detected (such as the impact force generated by the sudden drop of the barbell during bench press exceeding the safety threshold), the system will activate a three-level protection protocol within 100 ms: first, send a red flashing warning through the AR glasses, and at the same time trigger electro-pulse muscle stimulation (through wearable devices) to cause protective contraction, and automatically reduce the resistance level of the surrounding fitness equipment.

[0043] The advanced fall detection module adopts a multi-stage analysis strategy: in the first stage, the IMU signal is monitored in real time through a hardware accelerator, and when a characteristic mutation is detected (acceleration > 3g and angular velocity > 200° / s), a pre-alarm is immediately triggered; in the second stage, the fall posture is confirmed by combining visual analysis within 200 ms; in the third stage, the danger level of the user's surrounding environment is comprehensively evaluated. The entire judgment process is completed within 300 ms, which is 2 times faster than the traditional solution. The system is also equipped with an environmental perception unit, which is equipped with a lightweight semantic segmentation neural network (inference speed 15 fps), and can real-time identify risk factors in the training venue, including liquid on the ground, abnormal placement of equipment, etc., and highlight the dangerous areas in the form of pulse halos through the AR interface.

[0044] All security events will trigger a complete emergency response process: the system automatically saves multi-modal sensing data (including videos, IMUs, environmental parameters, etc.) 10 seconds before and after the event, generates an encrypted report that meets medical standards, and sends it to the preset emergency contacts through a multi-channel transmission protocol (simultaneously attempting Wi-Fi / Bluetooth / cellular networks). Local storage adopts AES-256 encryption and blockchain verification technology to ensure the integrity and immutability of event records. The system also innovatively adds an accident review function, and users can reproduce the whole process of the event in three dimensions through the AR interface after safety to analyze the cause of the accident. After laboratory tests, the warning accuracy rate of this security system for typical fitness injury events reaches 99.2%, and the response delay is controlled within the range of human nerve reflex time (150 - 300 ms), greatly improving the training safety factor.

[0045] First, the AR glasses capture the user's motion images from multiple angles to obtain continuous image frames. Subsequently, the obtained images are subjected to noise cleaning. After normalization, a convolutional network is used to extract relevant features of the images. At the same time, the sensor captures the user's motion data, and a low-pass filter is used to remove high-frequency noise. After normalization, the sliding window technique is used to extract temporal features. The image features and sensor features are spliced together to form a multi-modal input. The Transformer model is used to perform temporal modeling on the multi-modal features to identify the motion pattern. Subsequently, the trained Transformer model is used for action recognition, and based on the recognition results, the real-time feedback situation is provided. By comparing the recognized user action features with the standard action templates, specific error types (such as incorrect postures, insufficient action amplitudes, etc.) are identified. According to the recognized error types, corresponding correction suggestions are matched. Finally, the correction suggestions are displayed in real time in the user's field of view through the AR glasses, and combined with the voice feedback system, multi-modal real-time guidance is provided. At the same time, the exercise data, including habits, etc., will be fed back into the Transformer model in reverse, making the model more personalized.

[0046] For those of ordinary skill in the art, various modifications to the present disclosure will be apparent, and the general principles defined herein can be applied to other variations without departing from the scope of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein. Although one or more exemplary embodiments of the present disclosure have been described with reference to the accompanying drawings, those of ordinary skill in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A sports fitness assistance application based on the combination of AR glasses analysis and Transformer, characterized in that, It includes a camera module, a sensor module, AR glasses, a wireless communication module, and a processing module; Among them, the camera module collects the user's motion video stream, the sensor module synchronously records acceleration, angular velocity, and direction data, the processing module runs the Transformer model in real time through a parallel computing architecture, and the AR glasses use waveguide optical technology to superimpose three-dimensional visualization guidance information onto the real scene; After the camera module performs image noise cleaning, a convolutional network extracts image features and performs feature normalization processing; After the sensor module removes noise through a low-pass filter, a sliding window is used to extract temporal features and perform feature normalization processing; The two features are fused and spliced, and the base Transformer performs temporal modeling to output action behaviors, calculates with the standard action template, proposes suggestions and displays them in the AR glasses, and records the exercise data and inputs it back to the Transformer model for recording.

2. The sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, wherein: The camera module includes a three-stage processing process: first, a noise cleaning algorithm is used to perform light compensation and noise elimination on the original video frame; then, the two-dimensional spatial coordinate features of 17 human key points are extracted through a convolutional network, and finally, the key point coordinates are standardized through a feature normalization unit.

3. The sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, wherein: The camera module converts the pixel coordinates into normalized proportional coordinates relative to the torso based on the user's height, and establishes a 128-dimensional feature vector containing the spatial relationship of joint points. This vector also encodes the confidence scores of each joint point for subsequent feature fusion weighting.

4. The sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, characterized in that: The sensor module adopts a multi-level signal processing architecture: the first-level hardware filter circuit performs 20Hz low-pass filtering on the original IMU signal to eliminate high-frequency noise; the second-level software filter layer solves the sensor drift problem during the movement process; The third-level feature extraction layer uses a sliding time window for processing, with an overlap of 50% between adjacent windows. Each window extracts temporal features (including mean, variance, zero-crossing rate) and frequency-domain features (the amplitudes of 5 main frequency components calculated by FFT), and finally generates a temporal feature vector containing 15 features. All features are unified and calibrated according to the principles of sports biomechanics.

5. The sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, characterized in that: The processing module adopts a spatio-temporal alignment cross-modal fusion strategy: in the time dimension, through a hardware-level synchronization signal, the image feature frame is strictly aligned with the sensor data, and the error is controlled within ±5ms; in the feature dimension, the 128-dimensional spatial feature vector output by the camera module is spliced with the 15-dimensional temporal feature vector of the sensor module to form a 143-dimensional fusion feature.

6. The sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, characterized in that: The Transformer model adopts a four-layer improved Transformer encoder architecture: each layer contains 12 heads of self-attention mechanisms, among which the first 6 heads are specifically used to process spatial visual features, the last 6 heads focus on analyzing sensor temporal features, and finally cross-attention is used to achieve information interaction between modalities.

7. An exercise fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, characterized in that: The processing module establishes a dynamically updated standard action template library: each standard template contains the motion trajectory functions of each joint point and the angle thresholds of key phase points under ideal conditions; during evaluation, an improved dynamic time warping algorithm is used to calculate the multi-dimensional difference degree between the user's action and the template on the basis of time alignment, including: joint angle deviation (accurate to 0.5°), motion speed ratio, and trajectory smoothness; the feedback information generation module gives graded prompts according to the difference degree: the first-level prompt (deviation < 5°) only displays the AR green mark, the second-level prompt (5° ≤ deviation < 10°) superimposes a yellow direction arrow, and the third-level prompt (deviation ≥ 10°) triggers a red flashing warning and a voice warning; all feedback information is accurately anchored to the user's actual joint position through spatial registration technology.

8. A sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, characterized in that: The auxiliary application also includes a built-in dual-loop update mechanism in the system: the short-term loop performs offline analysis on the user's training data every day, calculates the 7-day moving average of each action index, and automatically adjusts the fault tolerance threshold of the template; The long-term loop performs model fine-tuning once a month, injects the historical correct action features of the user as new positive samples into the contrast learning framework of the Transformer model, and uses the elastic weight consolidation algorithm to prevent catastrophic forgetting during the update; at the same time, a user's personalized motion profile is established to record the change trends of 12 biomechanical indicators including joint range of motion and strength symmetry. These data are synchronously sent to the cloud analysis platform after differential privacy processing.

9. The sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, characterized in that: The AR glasses adopt a hierarchical rendering AR display strategy: the bottom layer displays the real-time motion tracking skeleton, the middle layer superimposes the ideal position marks (colored spheres) of key action nodes, and the top layer displays error correction guidelines (dynamic arrows / words); the display system intelligently adjusts the information density through the eye movement tracking module and automatically expands the detailed data panel when it detects that the user is gazing at a certain joint.

10. The sports fitness assistance application based on the combination of AR glasses analysis and Transformer according to claim 1, wherein: The auxiliary application also includes a safety monitoring and emergency handling module. The safety monitoring and emergency handling module continuously calculates the joint load index and immediately activates the protection protocol when an abnormal impact load is detected; the fall detection algorithm analyzes the sudden change pattern of the IMU signal (acceleration > 3g and angular velocity > 200° / s) and triggers an emergency response within 300 ms; the scene perception unit uses a semantic segmentation network to identify the risk factors in the training environment (such as slippery ground and obstacles) and highlights the warning through AR display; all safety events automatically generate a report containing the sensing data 10 seconds before and after, and send it to the preset emergency contact through the wireless module, while locally storing the encrypted complete event record.

Images