Physical education method and system based on multimedia and holographic AR fusion

Abstract

The invention relates to the technical field of physical education, in particular to a physical education method and system based on multimedia and holographic AR fusion, and the method comprises the following steps: collecting the motion data of a trainee based on a multi-source sensor array, constructing a motion parameterized model, and generating a holographic projection generation module through the cooperation with a standard motion database; a holographic AR teaching image is generated in real time, an environment perception module is combined to carry out adaptive projection on illumination and spatial layout, a spatial positioning module is used to establish coordinate mapping between a trainee and a virtual image, and a multi-channel feedback device is started to output visual, tactile and auditory signals. And finally, comparing the action parameters with the standard action data to automatically generate a training report containing wrong action labels and correction suggestions. According to the method, the motion capture precision and the interactive immersion can be effectively improved, and efficient and extensible technical support is provided for modern physical education and training.

Description

Technical Field

[0001] The present invention relates to the field of sports teaching technology, and in particular to a sports teaching method and system based on the fusion of multimedia and holographic AR. Background Art

[0002] With the continuous development of modern physical education and sports training, how to efficiently and accurately collect trainees' movements, analyze them in real time and provide effective corrective guidance in a timely manner has become an important issue that needs to be urgently solved in the field of physical education.

[0003] Although there are some sports teaching solutions based on motion capture or virtual reality technology on the market, the motion capture accuracy is insufficient or the sensor fusion is insufficient, making it difficult to obtain a stable and fine-grained motion parameterized model, and unable to achieve effective coordinate mapping between the real sports field and the virtual image. This leads to inaccurate projection position of holographic or AR teaching content in the venue, limited interactivity and immersion, and a lack of systematic multi-channel feedback mechanism. Common solutions only provide a single visual prompt or a simple audio signal, and are unable to provide instant and targeted corrections to trainees through multi-dimensional means such as touch and hearing. Summary of the invention

[0004] The present invention provides a sports teaching method and system based on the fusion of multimedia and holographic AR.

[0005] The physical education teaching method based on the integration of multimedia and holographic AR includes the following steps: S1: Collect the trainee's real-time motion data through a multi-source sensor array and construct a motion parameterized model including three-dimensional spatial coordinates, joint angles and motion trajectories; S2: Input the action parameterized model into the holographic projection generation module, and combine it with the standard action database to generate dynamic holographic AR teaching content with multiple layers of transparency; S3: Based on the real-time lighting parameters and spatial layout data obtained by the environment perception module, the projection path of the dynamic holographic AR teaching content is optimized to generate interactive holographic images that adapt to the on-site environment; S4: Establish the coordinate mapping relationship between the real sports field and the holographic image through the spatial positioning module, and use the multi-channel feedback device to synchronously output visual, tactile and auditory guidance signals; S5: Compare the difference data between the trainee's motion parameterized model and the standard motion database in real time, and generate a multimodal training report containing incorrect motion annotations and correction suggestions.

[0006] Optionally, the S1 includes: S11, Selection and Deployment of Multi-Source Sensor Array: Select a multi-source sensor array for capturing three-dimensional spatial coordinates, joint angles, and motion trajectories. The multi-source sensor array includes an inertial measurement unit, an optical motion capture device, and an electromyography sensor. Reasonably deploy it according to the trainee's activity range and expected motion characteristics, and ensure that each sensor has an interference-free data transmission environment; S12, Sensor Initialization and Calibration: Initialize and calibrate the multi-source sensor array before use; S13, Real-Time Data Acquisition and Synchronization: When the trainee performs training actions, the multi-source sensor array simultaneously starts real-time data acquisition.

[0007] S14, Data Fusion and Preprocessing: Perform multi-dimensional correction and denoising processing on the real-time data of the multi-source sensor array through a fusion algorithm.

[0008] S15, Construction of Action Parameterization Model: Convert the real-time data after fusion and preprocessing into a quantifiable parameterization model.

[0009] Optionally, the S2 includes: S21, Reception and Loading of Action Parameterization Model: Obtain three-dimensional spatial coordinates, joint angles, and motion trajectory data from the generated action parameterization model, and import them into the holographic projection generation module; S22, Standard Action Database Matching and Retrieval: Retrieve the standard action template that best matches the trainee's action from the pre-established standard action database, highlighting the differences and commonalities between this action and the standard template.

[0010] S23, Holographic Rendering Design with Multiple Layers of Transparency: In the holographic projection generation module, visualize the trainee's action and the standard action at multiple levels to form a dynamic comparison and auxiliary information containing multiple layers of transparency.

[0011] S24, Dynamic Generation and Combination of Holographic Images: Combine the multi-layer transparency rendering scheme with the matched standard action template to form holographic AR teaching content for visual comparison and difference display.

[0012] Optionally, the S3 includes: S31, Environmental Perception Data Acquisition: Obtain real-time lighting parameters and spatial layout data through the environmental perception module, and integrate them to generate environmental perception data, providing environmental basic information for subsequent projection path optimization and holographic image generation; S32, Environmental data analysis and parameter extraction: Analyze the environmental perception data, extract key projection parameters, including the effective projection area, light compensation value, and occlusion information, and integrate the key projection parameters to form an environmental parameter set, taking into account both the display quality of the holographic image and the interaction requirements of the trainees.

[0013] S33, Projection path optimization: Based on the environmental parameter set and the requirements of the holographic AR teaching content, plan the optimal projection path and projection method to ensure the clarity and interactivity of the holographic image in the on-site environment.

[0014] Optionally, S3 further includes: S34, Environmentally adaptive holographic image generation: Based on the projection path optimization results and combined with the display requirements of the dynamic holographic AR teaching content, generate an interactive holographic image in the on-site environment.

[0015] S35, Integration of interaction and feedback mechanism: Embed an interaction function in the generated holographic image to achieve the intuitive perception and instant feedback of the trainees on the holographic image.

[0016] Optionally, S4 includes: S41, Calibration and calibration of the venue and the holographic coordinate system: Establish a unified and traceable coordinate mapping relationship between the real sports venue and the holographic image through the spatial positioning module.

[0017] S42, Trainee real-time position and attitude tracking: Use the spatial positioning module to accurately locate the trainees and maintain coordination with the holographic image.

[0018] Optionally, S4 further includes: S43, Deployment of multi-channel feedback devices: Deploy the visual, tactile, and auditory feedback devices required in the sports teaching scenario and connect them to the system.

[0019] S44, Synchronous output of multi-channel feedback signals: According to the real-time position and attitude information of the trainees, synchronously push visual, tactile, and auditory guidance signals to the multi-channel feedback devices.

[0020] Optionally, S5 includes: S51, Action parameter acquisition and difference analysis: Compare the action parameterized model of the trainees with the reference actions in the standard action database in real time to identify action deviation points; S52, Incorrect action annotation: Visually annotate and classify the identified action deviation points for subsequent analysis and feedback.

[0021] Optionally, S5 further includes: S53, Correction Suggestion Generation: Automatically or semi-automatically generate personalized action correction suggestions based on the results of the difference analysis and the guiding strategies in the standard action database.

[0022] S54, Multi-modal Training Report Output: Generate a multi-modal training report based on the mis-action annotations and correction suggestions.

[0023] A sports teaching system based on the integration of multimedia and holographic AR is used to implement the above-mentioned sports teaching method based on the integration of multimedia and holographic AR, and includes the following modules: Multi-source Sensor Array Module: Collect the real-time motion data of the trainee and output action parameterized information including three-dimensional spatial coordinates, joint angles, and motion trajectories; Holographic Projection Generation Module: Receive the action parameterized information and generate dynamic holographic AR teaching content with multiple transparency layers in combination with the reference templates in the standard action database; Environmental Perception Module: Obtain the light intensity and spatial layout data of the training venue, analyze and extract the effective projection area, light compensation value, and occlusion information, and output an environmental parameter set; Projection Path Optimization Module: Based on the environmental parameter set and the requirements of the holographic AR teaching content, use a preset genetic algorithm to plan the optimal projection path; Spatial Positioning Module: Used to establish the coordinate mapping relationship between the real motion venue and the holographic image, and real-time track the position and posture of the trainee; Multi-channel Feedback Device: Includes an AR head-mounted display, a wearable vibrating armband, and bone conduction headphones, and is used to synchronously output visual, tactile, and auditory cue signals to assist the trainee in identifying and correcting action deviations; Action Comparison and Report Module: Compare the action parameterized model of the trainee with the difference data in the standard action database in real time, and generate a multi-modal training report containing mis-action annotations and correction suggestions.

[0024] Advantages of the present invention: In the present invention, the real-time motion data of the trainee (including three-dimensional spatial coordinates, joint angles, and motion trajectories) is comprehensively collected and fused through a multi-source sensor array to construct an action parameterized model. Compared with the traditional method that only uses a single sensor or pure vision means, this solution can provide more stable, accurate, and fine-grained motion parameters, laying a quantifiable foundation for subsequent comparison with standard actions and error annotation, enabling coaches and trainees to accurately locate problems and efficiently correct action details.

[0025] The present invention combines a holographic projection generation module and a spatial positioning module, and through holographic AR dynamic content with multiple transparencies, optimized projection paths, and real-time environment perception, it realizes self-adaptive adjustment of the lighting and spatial layout of the real training site. While establishing the coordinate mapping between the real site and the virtual image, this solution uses multi-layer rendering methods such as a reference layer, a comparison layer, and an auxiliary layer, enabling trainees to vividly observe the differences between their own actions and the standard actions. The optimized projection paths and multi-dimensional interaction design enable the holographic image to maintain high visibility and accuracy in complex training environments, greatly enhancing the training efficiency and learning experience.

[0026] The present invention closely combines visual, tactile, and auditory devices, and through multi-channel synchronous output, it gives vibration warnings, audio prompts, or highlighted holographic images in a timely manner when there are action deviations of the trainees. This multi-modal feedback not only enhances the immediacy and pertinence of motion guidance, but also generates a multi-modal training report containing error action annotations and correction suggestions based on real-time comparison in the S5 stage, providing trainees with a record of teaching results that can be traced and quantified. By continuously collecting action data and integrating feedback information, this solution can comprehensively monitor and iteratively optimize the training effect, meeting the personalized teaching and review needs of trainees at different levels and with different requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only those of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0028] Figure 1 It is a schematic flowchart of the method according to an embodiment of the present invention; Figure 2 It is a schematic flowchart of the system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The following will describe the present invention in detail in conjunction with the drawings and specific embodiments. At the same time, it should be noted here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments. For some well-known technologies, those skilled in the art can also adopt other alternative methods for implementation; moreover, the drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0030] It should be noted that in the specification, the mention of "an embodiment", "embodiment", "exemplary embodiment", "some embodiments", etc. indicates that the described embodiments may include specific features, structures or characteristics, but not necessarily every embodiment includes such specific features, structures or characteristics. Additionally, when combining embodiments to describe specific features, structures or characteristics, the implementation of such features, structures or characteristics in combination with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the relevant art.

[0031] Generally, terms can be understood at least in part from their use in context. For example, at least in part depending on the context, the term "one or more" as used herein can be used to describe any feature, structure or characteristic in a singular sense, or can be used to describe a combination of features, structures or characteristics in a plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey a set of exclusive factors, but rather can alternatively, at least in part depending on the context, allow for the existence of other factors that are not necessarily explicitly described.

[0032] As Figure 1 shown, a sports teaching method based on the integration of multimedia and holographic AR includes the following steps: S1: Collect real-time motion data of the trainee through a multi-source sensor array, and construct an action parameterization model including three-dimensional space coordinates, joint angles and motion trajectories; S2: Input the action parameterization model into the holographic projection generation module, and generate dynamic holographic AR teaching content including multiple layers of transparency in combination with the standard action database; S3: Based on the real-time lighting parameters and spatial layout data obtained by the environment perception module, optimize the projection path of the dynamic holographic AR teaching content, and generate an interactive holographic image adapted to the on-site environment; S4: Establish a coordinate mapping relationship between the real sports field and the holographic image through the spatial positioning module, and use a multi-channel feedback device to synchronously output visual, tactile and auditory guidance signals; S5: Compare the difference data between the trainee's action parameterization model and the standard action database in real time, and generate a multi-modal training report including error action annotations and correction suggestions.

[0033] S1 includes: S11, Selection and layout of the multi-source sensor array: Select a multi-source sensor array for capturing three-dimensional space coordinates, joint angles and motion trajectories. The multi-source sensor array includes an inertial measurement unit, an optical motion capture device and an electromyography sensor, and is reasonably laid out according to the trainee's activity range and expected action characteristics, and ensure that each sensor has an interference-free data transmission environment; S12, Sensor initialization and calibration: Initialize and calibrate the multi-source sensor array before use, specifically including: Perform zero-bias correction and gravity direction identification on the inertial measurement unit; Calibrate the origin and scale of the coordinate system for the optical motion capture device; Collect the reference electromyogram signal and confirm the amplification factor for the electromyography sensor; To ensure the consistency and accuracy of the subsequent collected motion data; S13, Real-time data acquisition and synchronization: When the trainee performs training actions, the multi-source sensor array simultaneously starts real-time data acquisition, including: The inertial measurement unit collects acceleration, angular velocity, and orientation data; The optical motion capture device captures the spatial position information of the trainee's key joints and limbs; The electromyography sensor records the muscle activities of the key muscle groups; The sampling frequencies of various sensors are aligned through a unified synchronization clock to ensure that the data collected by each sensor corresponds to the motion state at the same moment.

[0034] S14, Data fusion and preprocessing: Perform multi-dimensional correction and denoising processing on the real-time data of the multi-source sensor array through a fusion algorithm, including: Use the Kalman filter or weighted average method to fuse the pose data of the inertial measurement unit and the optical motion capture device to reduce the noise impact brought by a single sensor; Perform band-pass filtering and amplitude normalization on the electromyogram signal to remove environmental interference and differences between different muscle states; Segment and label the joints of the fused data according to the trainee's body structure model to form time-series data that can reflect the dynamic movement of each joint.

[0035] S15, Construction of the motion parameterization model: Convert the real-time data that has been fused and preprocessed into a quantifiable parameterization model, specifically including: Record the position changes of each joint during the training process in the form of three-dimensional space coordinates and form a continuous motion trajectory; Calculate the angular information of each joint changing with time to obtain a multi-dimensional joint angle curve; Integrate the three-dimensional space coordinates, joint angles, and motion trajectories in chronological order into a motion parameterization model to provide basic data support for the subsequent generation of holographic AR teaching content and motion analysis.

[0036] S2 includes: S21, Receiving and Loading the Action Parameterization Model: Obtain three-dimensional spatial coordinates, joint angles, and motion trajectory data from the generated action parameterization model, and import them into the holographic projection generation module to lay a data foundation for the subsequent generation of holographic AR teaching content. Specifically, it includes: Parse and read the action parameterization model file or data stream; Confirm the coordinate system, time series, and joint marking information of the model; Load the model into the rendering engine of the holographic projection generation module to complete data docking; S22, Matching and Retrieving the Standard Action Database: Retrieve the standard action template that best matches the trainee's action from the pre-established standard action database, highlighting the differences and commonalities between this action and the standard template. Specifically, it includes: Read various sports action parameterization templates stored in the standard action database; Extract features from the trainee's action parameterization model based on action feature parameters (such as joint rotation range, trajectory pattern, action rhythm, etc.); Compare the trainee's action features with the standard action templates in the database, calculate the similarity, and screen out the standard template that best matches the trainee's current action; Determine the standard action reference information that needs to be highlighted in the subsequent holographic projection content according to the comparison results.

[0037] S23, Holographic Rendering Design with Multiple Layers of Transparency: In the holographic projection generation module, visualize the trainee's action and the standard action at multiple levels to form a dynamic comparison and auxiliary information with multiple layers of transparency. Specifically, it includes: Define the multi-layer rendering logic: Comparison layer: Display the standard action as a whole; Alignment layer: Present with a differentiated effect in areas that overlap or deviate significantly from the standard action; Auxiliary layer: Overlay prompt marks for key joints or action paths; Set the transparency, visual effects, and overlay order of each layer to ensure that there is no line-of-sight interference during dynamic display; Configure trigger conditions for each layer. For example, when the action deviation exceeds the threshold, automatically enhance the alignment layer to highlight the warning.

[0038] S24, Dynamic Generation and Combination of Holographic Images: Combine the multi-layer transparency rendering scheme with the matched standard action template to form holographic AR teaching content for visual comparison and difference display. Specifically, it includes: Synchronously render the trainee's real-time action and the standard action according to the time series and action trajectory; Dynamically adjust the display intensity and visibility status of the control layer, comparison layer, and auxiliary layer to highlight the action differences that need attention currently; Overlay a frame-by-frame or continuous playback effect on the generated holographic image to provide data and visual output for subsequent environmental projection optimization and interactive operations.

[0039] S3 includes: S31, Environmental perception data collection: Obtain real-time lighting parameters and spatial layout data through the environmental perception module, and integrate them to generate environmental perception data, providing environmental basic information for subsequent projection path optimization and holographic image generation. Specifically, it includes: Start the lighting sensor and spatial scanning device of the environmental perception module to collect the lighting intensity, light source position distribution, and environmental reflectivity parameters in the current training site in real time; Use a depth sensor or lidar to obtain the positions, areas, and obstacle distributions of projectable surfaces in the training site; Integrate the collected lighting parameters and spatial layout data according to a unified timestamp to generate environmental perception data; S32, Environmental data analysis and parameter extraction: Analyze the environmental perception data, extract key projection parameters. The key projection parameters include the effective projection area, lighting compensation value, and occlusion information. Integrate the key projection parameters to form an environmental parameter set to balance the display quality of the holographic image and the interactive needs of the trainees. Specifically, it includes: S321, Environmental perception data preprocessing: Obtain the environmental perception data set output from step S31, including lighting intensity, light source position, training site spatial layout, and object distribution information; Perform denoising processing on the lighting data, perform coordinate alignment and format standardization on the spatial scanning data to ensure the accuracy of the data and its fusion in the same coordinate system; Perform synchronous correction on the timestamp according to different types of data sources (depth camera, lidar, or lighting sensor, etc.) to generate spatio-temporally aligned data for analysis.

[0040] S322, Effective projection area identification: Based on the site layout information, divide the site plane or curved surface into several sub-regions, and identify the smooth projectable regions in combination with the spatial coordinates of the obstacles; Perform area measurement and inclination analysis on the projectable plane or curved surface to exclude regions with excessive inclination or severe edge distortion; Output the spatial markers of the effective projection area, and record the vertex coordinates and available areas of each projectable region.

[0041] S323, Lighting compensation value calculation: Perform multi-point measurement and distribution analysis on the collected site lighting intensity and light source position to identify high-brightness areas and low-brightness areas; Calculate the amount of brightness compensation to be increased or decreased in each area according to the contrast and brightness requirements of the projection display; Mark special parameters for high-reflection areas to avoid glare or overexposure caused by excessive projection; Record and output the light compensation values of each area (such as brightness gain coefficient, contrast correction coefficient, etc.).

[0042] S324, Occlusion area and blind area detection: Combine fixed obstacles (walls, equipment, etc.) and moving obstacles (trainees themselves, other people or sports equipment) in the training site, and determine potential occlusion areas through depth information or lidar data; Mark the spatial boundaries of each occlusion area and possible temporal changes (if the obstacles move or the trainees walk); Identify the projection blind areas caused by the site structure, and avoid or take other compensation measures during subsequent projection path planning.

[0043] S325, Integration and output of environmental parameter set: Integrate three key projection parameters, namely the effective projection area, light compensation value, and occlusion information, to form an "environmental parameter set"; This parameter set contains data that is decision-making significant for subsequent projection path planning, such as coordinates of the projectable area, recommended brightness adjustment amount, list of occlusions or blind areas to be avoided; Output the environmental parameter set to provide complete environmental data support for projection path optimization (S33).

[0044] S33, Projection path optimization: Based on the environmental parameter set and the requirements of holographic AR teaching content, plan the optimal projection path and projection method to ensure the clarity and interactivity of the holographic image in the on-site environment, specifically including: S331, Generation of initial population: According to the environmental parameter set output by S325, combined with the basic requirements of holographic AR teaching content (such as resolution, frame rate, visible interaction range, etc.), generate several candidate projection paths randomly or based on heuristic methods; Each path contains specific parameters such as the orientation of the projection device, focal length setting, and deformation compensation method, forming the individual coding in the genetic algorithm; Initialize a population of a certain scale (such as N paths) to ensure sufficient coverage of the search space.

[0045] S332, Fitness function design: Define a fitness function to comprehensively evaluate the following indicators: Whether it can cover the effective projection area and meet the visualization requirements; Whether it matches the light compensation value to avoid overexposure or underexposure; Whether it reasonably avoids occlusion areas and blind areas to ensure continuous display of the holographic image; The degree of fit with the interaction range of the trainee, and whether the resolution and frame rate requirements for the holographic content meet the standards.

[0046] By performing a weighted sum of the above indicators, the fitness score of each individual projection path is obtained.

[0047] S333, Iterative Evolution and Operator Application: Adopt the main operation operators of the standard genetic algorithm: selection, crossover, and mutation; Selection: According to the fitness ranking, eliminate individuals with low fitness and retain or copy individuals with high fitness; Crossover: Perform a gene crossover operation on the retained individuals, randomly exchange a part of the parameters of two projection paths, and generate new candidate paths; Mutation: Randomly perturb some parameters of the newly generated individuals (such as the angle of the projection device, the deformation compensation method), so that the algorithm has better global search ability; Repeat the iteration for several generations (such as 20 - 50 generations), and continuously retain better projection schemes in each generation.

[0048] S334, Optimal Projection Path Output: After reaching the preset number of evolutionary generations or the fitness no longer significantly improves, select the individual with the highest fitness as the final projection path scheme; This scheme includes the device layout method within the specific projectable area, the lighting compensation strategy, and the projection trajectory for avoiding occlusion areas; Output the optimal projection path and return the corresponding projection parameters (such as angle, brightness compensation value, deformation correction parameters, etc.) together for subsequent holographic image generation and interaction.

[0049] S3 also includes: S34, Environment - Adaptive Holographic Image Generation: Based on the projection path optimization results and combined with the display requirements of the dynamic holographic AR teaching content, generate an interactive holographic image in the on - site environment, specifically including: According to the optimal projection parameters calculated by S33, perform picture cropping, distortion correction, and brightness adjustment on the dynamic holographic AR teaching content to adapt to the geometric shape and light intensity of the space surface; Perform real - time rendering on different display levels (control layer, comparison layer, auxiliary layer, etc.), ensuring that there is no over - exposure in the high - light area and no detail loss in the shadow area for each layer; Track the interaction position of the trainee, and ensure that the key action area is always clearly visible by correcting the projection focus and angle.

[0050] S35, Integration of Interaction and Feedback Mechanism: Embed an interaction function in the generated holographic image to achieve the intuitive perception and instant feedback of the trainee to the holographic image, specifically including: Set interaction methods such as gesture recognition, touch detection, or voice commands, and continuously collect the position information and movement changes of the trainee through the environmental perception module; When the trainee interacts with the holographic image, the display area or information level of the holographic image is adjusted in real time. For example, when the trainee touches the marked joint, the corresponding auxiliary layer is automatically enlarged; Through linkage with the multi-channel feedback device, the interaction result is fed back to visual, auditory, or tactile signals, and the guiding method and content presentation of the holographic image are dynamically updated according to the accuracy of the trainee's actions and the learning progress.

[0051] S4 includes: S41, calibration and calibration of the venue and the holographic coordinate system: Establish a unified and traceable coordinate mapping relationship between the real sports venue and the holographic image through the spatial positioning module, specifically including: Start the spatial positioning module, collect the coordinates of the ground, boundaries, and key reference objects of the real sports venue, and generate a venue coordinate reference model; Select several spatial calibration points and map their positions to the virtual coordinate system used by the holographic projection generation module respectively; Align the venue coordinate reference model with the virtual coordinate system through the least squares method to obtain a coordinate transformation matrix or calibration parameters; Save this coordinate transformation information in the system for subsequent real-time positioning and image rendering calls.

[0052] S42, real-time position and attitude tracking of the trainee: Use the spatial positioning module to accurately position the trainee and maintain coordination with the holographic image, specifically including: Deploy identifiable marking points (such as infrared reflection balls, active positioning tags, etc.) on the trainee's clothing, equipment, or training equipment, and collect the trainee's position in real time through a camera array or infrared positioning device; Convert the received marking point coordinates to the virtual coordinate system of the holographic projection according to the coordinate transformation matrix in S41 to ensure that the actual position of the trainee is consistent with the image position in the holographic environment; Quickly calculate the trainee's body posture (such as joint angles, limb orientations) to form real-time updatable posture information, providing an accurate reference for subsequent multi-channel feedback signal triggering.

[0053] S4 also includes: S43, deployment of multi-channel feedback devices: Deploy visual, tactile, and auditory feedback devices required in the sports teaching scenario and connect them to the system, specifically including: Visual feedback device: Use an AR head-mounted display with real-time holographic drawing capabilities to support directly overlaying teaching images within the wearer's field of vision; The installation position is centered on the trainee's head to ensure the stability of the device and controllable viewing angle during exercise; Cooperating with the data interface of the system main control unit, it can update and render holographic contents such as the comparison layer, alignment layer, and auxiliary layer of the training actions in real time.

[0054] Tactile feedback device: Wearable vibrating armbands are arranged at key positions on the trainee's two arms or waist, and built-in vibrating motors are used to generate tactile signals with different frequencies and intensities; Through linkage with the motion capture and comparison algorithms, when it is detected that the trainee's action deviates or a specific rhythm reminder is needed, a vibration reminder is triggered; Adjust according to personal body size during installation so that the vibrating motor can closely adhere to the skin without affecting the normal performance of actions.

[0055] Auditory feedback device: Use bone conduction headphones to provide personalized auditory guidance in the training environment without affecting the perception of surrounding environmental sounds; When voice commands, rhythm sounds, or reminder sounds are needed, the system main control unit wirelessly sends audio signals to the bone conduction headphones; Adopt a wrapping or mounting wearing method to ensure that the headphones are not easy to slip off during the trainee's movement and ensure the conduction efficiency.

[0056] System connection and layout integration: Connect the AR head-mounted display, wearable vibrating armbands, and bone conduction headphones to the system main control unit respectively through wireless communication; According to the data of the training action range and spatial positioning modules (S41, S42), the dynamic position and posture of the trainee are monitored in real time, and the triggering timing and content of the three feedback devices are uniformly managed; After testing and debugging, ensure that the output of visual, tactile, and auditory signals is coordinated with each other and consistent with the rendering rhythm of the holographic image, achieving an immersive and multi-channel training guidance effect.

[0057] S44, Synchronous output of multi-channel feedback signals: According to the real-time position and posture information of the trainee, synchronously push visual, tactile, and auditory guidance signals to the multi-channel feedback device, specifically including: S441, Feedback trigger condition recognition: Real-time monitor the joint angles, position trajectories, and action rhythms of the trainee through the spatial positioning module, and compare them with the action parameters in the standard action database; When it is detected that the action deviates or a specific action node is completed, a corresponding feedback trigger signal is generated.

[0058] S442, Feedback scheme selection and instruction generation: Determine the priority of tactile, auditory, or visual feedback according to the type and severity of the trigger signal; For example, if only a slight reminder is needed, a short-term low-frequency vibration is performed through a wearable vibration armband; if the deviation is large, the corresponding joint area is highlighted in the AR head-mounted display, and a reminder sound is played through the bone conduction headphones; The feedback instructions are encapsulated into a unified data format and sent to the control modules of each feedback device.

[0059] S443, real-time synchronization and multi-channel collaboration: using a unified system clock or synchronization signal to ensure that visual, tactile and auditory feedback take effect at the same time to avoid delays or misalignment; Adjust the visualization effect of the multi-layer holographic image on the AR head-mounted display (such as magnifying the deviated joint position), trigger the wearable vibration armband to vibrate locally, and play correction instructions or preset sound effects through bone conduction headphones; If the trainee obtains correction by adjusting the movements, the system immediately captures the new posture data and removes or weakens the trigger signal, dynamically closing the feedback channel that is no longer needed; S444, feedback data recording and subsequent analysis: recording key information such as the time point of each trigger, feedback type and duration in the training log to facilitate subsequent analysis and evaluation of the training process; Through the study of the correlation between feedback records and the speed of action improvement, data support is provided for subsequent training program optimization and training strategy adjustment; Combined with historical data from multi-source sensors, the intervention effects of touch, vision, and hearing at different movement stages are judged to further improve the multi-channel feedback mechanism.

[0060] S5 includes: S51, action parameter acquisition and difference analysis: real-time comparison of the trainee's action parameterized model with the reference action in the standard action database to identify action deviation points, including: Receiving the trainee's real-time motion parameterized model and the matched standard motion template from steps S1 and S2; Compare key indicators such as joint angles, motion trajectories, and action rhythms frame by frame or continuously in time series; Calculate the difference value and determine whether it exceeds the preset motion accuracy threshold, and focus on recording the part that exceeds the threshold; S52, incorrect action marking: visually mark and classify the identified action deviation points to facilitate subsequent analysis and feedback, including: Add error marks to joint angles, limb positions and movement trajectories where there are obvious deviations; Grading or categorizing according to different types of deviations (such as excessive angle, unstable swing, wrong action sequence, etc.); Record the annotation time, action frame index, and deviation value to form a traceable list of incorrect actions.

[0061] S5 also includes: S53, generating correction suggestions: Automatically or semi-automatically generate personalized action correction suggestions based on the results of the difference analysis and the guiding strategies in the standard action database, specifically including: Retrieve the correction strategies and training suggestions for the corresponding error types from the standard action database; Perform secondary matching or fine-tuning on the correction suggestions according to the trainee's personal physiological parameters or training goals; Correspond the correction suggestions with the specific incorrect action marks to form a correction plan for each error point, including detailed information such as adjusting angles, rhythms, or muscle force application methods.

[0062] S54, outputting a multi-modal training report: Generate a multi-modal training report based on the incorrect action annotations and correction suggestions, specifically including: Integrate the list of incorrect actions and correction suggestions generated in steps S52 and S53, and arrange them into a charted report; Embed screenshots of key action frames or holographic image segments in the report to visually display the deviations and correction points; Attach voice or text explanations to provide intuitive references and further guidance for trainees or coaches; Output the finally generated multi-modal training report in the form of an electronic document or system interface, and synchronously store it in the training record database for subsequent playback, comparison, and review.

[0063] As Figure 2 shown, a sports teaching system based on the integration of multimedia and holographic AR is used to implement the above-mentioned sports teaching method based on the integration of multimedia and holographic AR, including the following modules: Multi-source sensor array module: Collect the real-time motion data of the trainee and output action parameterized information including three-dimensional space coordinates, joint angles, and motion trajectories; Holographic projection generation module: Receive the action parameterized information and generate dynamic holographic AR teaching content with multiple transparency layers in combination with the reference templates in the standard action database; Environmental perception module: Obtain the light intensity and spatial layout data of the training venue, analyze and extract the effective projection area, light compensation value, and occlusion information, and output the environmental parameter set; Projection path optimization module: Based on the environmental parameter set and the requirements of the holographic AR teaching content, use a preset genetic algorithm to plan the optimal projection path; Spatial positioning module: Used to establish the coordinate mapping relationship between the real motion venue and the holographic image, and perform real-time tracking of the trainee's position and posture; Multi-channel feedback device: including an AR head-mounted display, a wearable vibrating armband, and bone conduction headphones, which are used to synchronously output visual, tactile, and auditory cue signals to assist the trainee in identifying and correcting movement deviations; Movement comparison and reporting module: It compares the difference data between the trainee's movement parametric model and the standard movement database in real time and generates a multi-modal training report with error movement annotations and correction suggestions.

[0064] The present invention covers any substitutions, modifications, equivalent methods, and solutions made on the essence and scope of the present invention. To enable the public to have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention. However, those skilled in the art can fully understand the present invention without the description of these details. Additionally, to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0065] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A sports teaching method based on the integration of multimedia and holographic AR, characterized in that, It includes the following steps: S1: Collect the real-time motion data of the trainee through a multi-source sensor array, and construct an action parameterization model including three-dimensional space coordinates, joint angles, and motion trajectories; S2: Input the action parameterization model into the holographic projection generation module, and generate dynamic holographic AR teaching content with multiple layers of transparency in combination with the standard action database; S3: Based on the real-time lighting parameters and spatial layout data obtained by the environment perception module, optimize the projection path of the dynamic holographic AR teaching content to generate an interactive holographic image adapted to the on-site environment; S4: Establish a coordinate mapping relationship between the real motion field and the holographic image through the space positioning module, and use a multi-channel feedback device to synchronously output visual, tactile, and auditory guidance signals; S5: Compare the difference data between the trainee's action parameterization model and the standard action database in real time, and generate a multi-modal training report including error action annotations and correction suggestions.

2. The sports teaching method based on the integration of multimedia and holographic AR according to claim 1, characterized in that The S1 includes: S11, Selection and layout of the multi-source sensor array: Select a multi-source sensor array for capturing three-dimensional space coordinates, joint angles, and motion trajectories. The multi-source sensor array includes an inertial measurement unit, an optical motion capture device, and an electromyography sensor, and arrange them reasonably according to the trainee's activity range and expected action characteristics; S12, Sensor initialization and calibration: Initialize and calibrate the multi-source sensor array before use; S13, Real-time data collection and synchronization: When the trainee performs training actions, the multi-source sensor array starts real-time data collection simultaneously; S14, Data fusion and preprocessing: Perform multi-dimensional correction and denoising processing on the real-time data of the multi-source sensor array through a fusion algorithm; S15, Construction of the action parameterization model: Convert the real-time data after fusion and preprocessing into a quantifiable parameterization model.

3. The sports teaching method based on the integration of multimedia and holographic AR according to claim 2, characterized in that, The S2 includes: S21, Receiving and loading of the action parameterization model: Obtain three-dimensional space coordinates, joint angles, and motion trajectory data from the generated action parameterization model, and import them into the holographic projection generation module; S22, Matching and retrieval of the standard action database: Retrieve the standard action template that best matches the trainee's action from the pre-established standard action database, highlighting the differences and commonalities between this action and the standard template; S23, Holographic rendering design with multiple layers of transparency: In the holographic projection generation module, visualize the trainee's action and the standard action at multiple levels to form a dynamic comparison and auxiliary information including multiple layers of transparency; S24, Dynamic generation and combination of holographic images: Combine the multi-layer transparency rendering scheme with the matched standard action template to form holographic AR teaching content for visual comparison and difference display.

4. The sports teaching method based on the integration of multimedia and holographic AR according to claim 3, wherein, The S3 includes: S31, Collection of environment perception data: Obtain real-time lighting parameters and spatial layout data through the environment perception module, and integrate them to generate environment perception data; S32, Analysis of environment data and extraction of parameters: Analyze the environment perception data, extract key projection parameters, and the key projection parameters include the effective projection area, lighting compensation value, and occlusion information. Integrate the key projection parameters to form an environment parameter set; S33. Projection path optimization: Based on the environmental parameter set and the requirements of holographic AR teaching content, plan the optimal projection path and projection method.

5. The sports teaching method based on the integration of multimedia and holographic AR according to claim 4, characterized in that, The above S3 also includes: S34. Environment-adaptive holographic image generation: Based on the projection path optimization result, combined with the display requirements of dynamic holographic AR teaching content, generate interactive holographic images in the on-site environment; S35. Integration of interaction and feedback mechanism: Embed interaction functions in the generated holographic images to achieve intuitive perception and instant feedback of the holographic images by the trainees.

6. The sports teaching method based on the integration of multimedia and holographic AR according to claim 5, characterized in that, The above S4 includes: S41. Calibration and calibration of the venue and holographic coordinate system: Establish a unified and traceable coordinate mapping relationship between the real motion venue and the holographic image through the spatial positioning module; S42. Real-time position and attitude tracking of the trainee: Use the spatial positioning module to locate the trainee and maintain coordination and correspondence with the holographic image.

7. The sports teaching method based on the integration of multimedia and holographic AR according to claim 6, characterized in that, The above S4 also includes: S43. Deployment of multi-channel feedback devices: Deploy visual, tactile, and auditory feedback devices required in the sports teaching scenario; S44. Synchronous output of multi-channel feedback signals: According to the real-time position and attitude information of the trainee, synchronously push visual, tactile, and auditory guidance signals to the multi-channel feedback device.

8. The sports teaching method based on the integration of multimedia and holographic AR according to claim 7, characterized in that, The above S5 includes: S51. Action parameter acquisition and difference analysis: Compare the action parameterized model of the trainee with the reference actions in the standard action database in real time to identify action deviation points; S52. Marking of incorrect actions: Visually mark and classify the identified action deviation points.

9. The sports teaching method based on the integration of multimedia and holographic AR according to claim 8, characterized in that, The above S5 also includes: S53. Generation of correction suggestions: Automatically or semi-automatically generate personalized action correction suggestions according to the difference analysis results and the guidance strategy of the standard action database; S54. Output of multi-modal training reports: Generate multi-modal training reports based on the incorrect action marking and correction suggestions.

10. A sports teaching system based on the integration of multimedia and holographic AR, which is used to implement the sports teaching method based on the integration of multimedia and holographic AR according to any one of claims 1-9, characterized in that, It includes the following modules: Multi-source sensor array module: Collect the real-time motion data of the trainee and output action parameterized information including three-dimensional space coordinates, joint angles, and motion trajectories; Holographic projection generation module: Receive the above action parameterized information and generate dynamic holographic AR teaching content with multiple transparency levels in combination with the reference template in the standard action database; Environment perception module: Obtain the light intensity and spatial layout data of the training venue, analyze and extract the effective projection area, light compensation value, and occlusion information, and output the environmental parameter set; Projection path optimization module: Based on the above environmental parameter set and the requirements of holographic AR teaching content, use the preset genetic algorithm to plan the optimal projection path; Spatial positioning module: Used to establish a coordinate mapping relationship between the real motion venue and the holographic image and perform real-time tracking of the trainee's position and attitude; Multi-channel feedback device: Includes an AR head-mounted display, a wearable vibrating armband, and bone conduction headphones, which are used to synchronously output visual, tactile, and auditory prompt signals to assist the trainee in identifying and correcting action deviations; Action comparison and reporting module: Compare the difference data between the trainee's action parameterized model and the standard action database in real time and generate a multi-modal training report containing incorrect action marking and correction suggestions.

Images