An edge device-oriented multi-modal sports teaching large model distillation deployment method

Abstract

The present application relates to the technical field of artificial intelligence, in particular to a multi-modal sports teaching large model distillation deployment method for edge devices; comprising the following steps: constructing a multi-modal sports teaching standardized data set; building a multi-modal sports teaching teacher reasoning architecture and completing full-amount pre-training; building a sports-specific three-layer knowledge transfer framework; obtaining parameters of two types of edge devices, namely embedded AI modules and high-performance gateways; deploying the reasoning architecture that has completed hardware adaptation to edge hardware; using the edge device to locally collect real-time video and voice of students and complete pre-processing, relying on the local reasoning architecture to independently complete all operations, and outputting action correction results, voice and text teaching guidance on site; adjusting the loss weight of the three-layer knowledge transfer framework to iteratively optimize the overall operation effect; through the above-mentioned mode, the whole process reasoning delay is low, the network transmission and cloud queuing delay are eliminated, real-time correction can be realized, and the real-time interaction demand of the sports teaching scene is met.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for deploying a large-scale multimodal sports teaching model for edge devices. Background Technology

[0002] With the deep integration of multimodal artificial intelligence technology and sports teaching scenarios, large-scale multimodal sports teaching models that integrate video footage, human skeleton timelines, voice explanations, and text lesson plans are rapidly developing. These large-scale models can simultaneously complete the entire teaching process, including human movement recognition, movement deviation judgment, standardized teaching guidance generation, and real-time voice error correction. They can replace traditional human coaches to achieve large-scale and personalized sports teaching guidance and are widely applicable to scenarios such as school playgrounds, gyms, and outdoor training grounds.

[0003] The current mainstream implementation method is cloud-based centralized inference: the multimodal sports teaching model is deployed on a cloud server, and the student movement videos, audios and skeletal data collected on site are uploaded to the cloud for calculation, and then the teaching guidance results are sent back to the terminal for display.

[0004] However, under the existing cloud inference model, the inference latency is too high in the entire process of video uploading, network transmission, cloud queuing processing and result feedback. In weak network environments, it can even be as high as several seconds, which makes real-time error correction and instant guidance functions basically unusable in actual use and cannot meet the real-time interactive needs of sports teaching scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a multimodal sports teaching large-scale model distillation deployment method for edge devices, aiming to solve the technical problem in the existing cloud inference mode that the inference latency is too high in the entire process of video uploading, network transmission, cloud queuing processing and result feedback, even reaching several seconds in weak network environment, which makes real-time error correction and instant guidance functions basically unusable in actual use and cannot meet the real-time interactive needs of sports teaching scenarios.

[0006] To achieve the above objectives, the present invention employs a multimodal sports teaching large-scale model distillation deployment method for edge devices, comprising the following steps: A standardized dataset for multimodal physical education teaching was constructed. The dataset includes videos of physical movements, time-series sequences of human skeletons, teaching audio, standard lesson plan texts, and annotations of correct and incorrect movements. Multimodal time-series alignment and unified encoding were completed, and training and validation subsets were divided. A multimodal physical education teaching teacher reasoning framework was built and fully pre-trained. Single-modal low-level features, cross-modal attention matrix, and teaching task prediction probability were extracted as knowledge transfer supervision signals. A three-layer knowledge transfer framework specifically for sports is built. It adopts a multi-type error constraint to simultaneously complete single-modal feature matching, cross-modal semantic alignment, and teaching task output fitting training until the accuracy of motion recognition and teaching guidance output on the validation set and the error of teacher architecture meet the standards, and generates a lightweight inference computing architecture. Acquire parameters for two types of edge devices: embedded AI modules and high-performance gateways. Perform differentiated quantization compression on the visual branch (INT8) and the voice / text branch (INT4), redundant channel pruning, multi-operator fusion, and edge-specific inference format conversion on the lightweight inference architecture. The hardware-adapted inference architecture is deployed to edge hardware. The resource scheduler keeps the motion recognition core program running in the background and dynamically loads and releases the speech and text teaching module as needed. The computing resources are allocated according to the priority of tasks such as action correction and lesson plan generation. The system utilizes edge devices to locally collect real-time video and audio data from students and performs pre-processing. It independently completes all calculations based on the local inference architecture and outputs action correction results and voice and text teaching guidance on-site. Simultaneously, it collects inference latency and recognition accuracy indicators and iteratively optimizes the overall calculation effect by adjusting the loss weight of the three-layer knowledge transfer framework.

[0007] Among the steps involved in constructing a standardized multimodal physical education teaching dataset, which includes videos of physical movements, temporal sequences of human skeletons, teaching audio, standard lesson plan texts, and annotations of correct and incorrect movements, and completing multimodal temporal alignment and unified encoding, and dividing the dataset into training and validation subsets: Collect action footage, human skeletal dynamic trajectory, instructional audio, standardized teaching text plans, and materials for marking correct and incorrect movements for various sports including track and field, ball games, gymnastics, and martial arts; The timelines of four types of materials—video, skeleton, audio, and text—are uniformly calibrated to eliminate time misalignment issues between different modalities. Standardized coding is performed on the corrected multimodal materials to unify the data storage and expression formats of various materials; The processed multimodal data is split into a subset for training and a subset for performance evaluation according to a fixed ratio.

[0008] Among the steps, the following steps are involved: building a multimodal physical education teacher inference framework and completing full pre-training; extracting single-modal low-level features, cross-modal attention matrix, and teaching task prediction probability as knowledge transfer supervision signals: Build an integrated physical education teaching basic reasoning framework and use the reasoning framework to process video, skeleton, voice and text simultaneously; The reasoning framework is fully trained using a subset of the divided training materials, enabling it to recognize motion and provide instructional guidance. Lock all runtime parameters of the trained inference architecture and stop updating or adjusting the parameters. We extract three types of content—underlying information of single-type materials, multimodal fusion and correlation information, and teaching judgment output reference values—as reference standards for subsequent knowledge transfer training.

[0009] In the process of building a three-layer knowledge transfer framework specifically for sports, and simultaneously completing single-modal feature matching, cross-modal semantic alignment, and teaching task output fitting training by integrating multiple types of error constraints, until the accuracy of motion recognition and teaching guidance output on the validation set meets the teacher architecture error standards, a lightweight inference computing architecture is generated: A sports-specific knowledge transfer processing framework was constructed, consisting of three layers: single-modal matching, cross-modal alignment, and teaching output fitting. A lightweight basic reasoning architecture with a simple structure and lower computational load is built as the target carrier for knowledge transfer training; Based on the extracted reference standards, training is carried out in layers to align single-modal low-level information, match multimodal semantic associations, and fit teaching guidance output content. The training effect is continuously tested using a subset of validation materials. Training stops when the difference between the recognition and guidance output of the lightweight architecture and the basic inference architecture meets the preset standard, thus obtaining the lightweight inference computing architecture.

[0010] Among them, in the steps of acquiring parameters of two types of edge devices, namely embedded AI modules and high-performance gateways, and performing differential quantization compression of the visual branch INT8 and the speech and text branch INT4 on the lightweight inference architecture, redundant channel pruning, multi-operator fusion, and edge-specific inference format conversion: Read the computing power limit, available memory, and storage space of the embedded AI computing module and the high-performance edge gateway; The visual computing branch that processes video and skeletal information uses INT8 precision quantization compression, while the branch that processes speech and text information uses INT4 precision quantization compression to achieve differentiated lightweight processing. Remove redundant computational channels unrelated to sports recognition within the weighted inference architecture and merge continuously repetitive basic computational steps; The processed inference architecture is converted into a dedicated runtime file format adapted to the underlying acceleration library of edge hardware.

[0011] Among them, the steps of deploying the hardware-adapted inference architecture to edge hardware, using a resource scheduler to keep the motion recognition core program resident, dynamically loading and releasing the speech and text teaching module on demand, and allocating computing resources according to the priority of tasks such as action error correction and lesson plan generation are as follows: Deploy the lightweight inference architecture, which has undergone hardware adaptation and modification, into embedded AI modules and high-performance gateways; Start the hardware resource scheduling program and set the human motion recognition core computing program to a continuous resident running state; A dynamic start / stop mechanism is set up for the voice broadcast and text lesson plan generation modules, which are only loaded when teaching guidance is needed and released after use; The computational priority of the two types of tasks, real-time action error correction and teaching plan generation, is distinguished, and hardware computing power is allocated to the action error correction task first.

[0012] The process involves: utilizing edge devices to locally collect real-time video and audio data from trainees and performing pre-processing; independently completing all calculations based on a local inference architecture; and outputting action error correction results and audio / text teaching guidance locally. Simultaneously, it involves collecting inference latency and recognition accuracy metrics, and then iteratively optimizing the overall computational effect by adjusting the loss weights of the three-layer knowledge transfer framework. The edge device uses its built-in camera and audio recording equipment to capture real-time images of trainees' movements and on-site audio, and performs pre-processing such as image noise reduction and audio filtering. The preprocessed multimodal data is sent to the local lightweight inference architecture, without transmitting the original motion data to the external cloud, and the entire calculation process is completed independently. Based on the calculation results, the system directly outputs the judgment content of the student's movement deviation and simultaneously generates physical education teaching guidance in both voice broadcast and text form. The system continuously records operational metrics such as single-operation time and action recognition accuracy, adjusts the constraint strength of each level of the three-layer knowledge transfer framework based on these metrics, and iteratively optimizes the overall computational performance.

[0013] This invention discloses a method for deploying a large-scale multimodal sports teaching model for edge devices, comprising the following steps: constructing a standardized multimodal sports teaching dataset, which includes sports action videos, human skeleton time-series sequences, teaching audio, standard lesson plan text, and action correctness annotations, and completing multimodal time-series alignment and unified encoding, dividing the dataset into training and validation subsets; building a multimodal sports teaching teacher inference architecture and completing full pre-training, extracting single-modal low-level features, cross-modal attention matrices, and teaching task prediction probabilities as knowledge transfer supervision signals; building a sports-specific three-layer knowledge transfer framework, simultaneously completing single-modal feature matching, cross-modal semantic alignment, and teaching task output fitting training by integrating multiple error constraints, until the accuracy of motion recognition and teaching guidance output on the validation set meets the teacher architecture error standard, generating a lightweight inference computing architecture; acquiring parameters of two types of edge devices: embedded AI modules and high-performance gateways, and performing... The lightweight inference architecture performs differentiated quantization compression for the visual branch (INT8) and the speech / text branch (INT4), redundant channel pruning, multi-operator fusion, and edge-specific inference format conversion. The hardware-adapted inference architecture is deployed to edge hardware. A resource scheduler is resident in the motion recognition core program, dynamically loading and releasing the speech / text teaching module as needed. Computational resources are allocated based on task priority for action correction and lesson plan generation. Real-time video and speech data from students are locally collected from edge devices and pre-processed. All computations are completed independently using the local inference architecture, outputting action correction results and speech / text teaching guidance locally. Inference latency and recognition accuracy metrics are collected simultaneously, and the loss weights of the three-layer knowledge transfer framework are iteratively adjusted to optimize the overall computational effect. Through these methods, low inference latency is achieved throughout the entire process, eliminating network transmission and cloud queuing delays, enabling real-time error correction, and meeting the real-time interactive needs of sports teaching scenarios. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the steps of the multimodal sports teaching large-scale model distillation deployment method for edge devices according to the present invention.

[0016] Figure 2 This is a flowchart of steps S100 of the present invention.

[0017] Figure 3 This is a flowchart of steps S200 of the present invention.

[0018] Figure 4 This is a flowchart of steps S300 of the present invention.

[0019] Figure 5 This is a flowchart of steps S400 of the present invention.

[0020] Figure 6 This is a flowchart of steps S500 of the present invention.

[0021] Figure 7 This is a flowchart of steps S600 of the present invention. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0023] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0025] Please see Figures 1-7 This invention provides a method for deploying a large-scale multimodal physical education teaching model for edge devices, comprising the following steps: S100: Construct a standardized multimodal physical education teaching dataset. The dataset includes videos of physical movements, time-series sequences of human skeletons, teaching audio, standard lesson plan texts, and annotations of correct and incorrect movements. Multimodal time-series alignment and unified encoding are completed, and training and validation subsets are divided.

[0026] In this embodiment, a standardized multimodal physical education teaching dataset is constructed. The dataset includes videos of physical movements, temporal sequences of human skeletons, teaching audio, standard lesson plan texts, and annotations of correct and incorrect movements. Multimodal temporal alignment and unified encoding are completed, and the dataset is divided into training and validation subsets. The specific process is as follows: S101: Collect action footage, human skeletal dynamic trajectory, instructional audio, standardized teaching text plans, and materials for marking correct and incorrect movements for various sports including track and field, ball games, gymnastics, and martial arts; S102: Unify and correct the timelines of four types of materials: video, skeleton, audio, and text, and eliminate time misalignment issues between different modal materials; S103: Perform standardized coding processing on the corrected multimodal materials to unify the data storage and expression format of various materials; S104: Divide all processed multimodal data into a subset for training and a subset for performance evaluation according to a fixed ratio.

[0027] In the above process, action footage, human skeletal dynamic trajectories, instructional audio, standardized teaching text, and action correctness / incorrectness markers for various sports such as track and field, ball games, gymnastics, and martial arts are collected. Then, the timelines of the four types of materials—video, skeleton, audio, and text—are uniformly calibrated to eliminate time misalignment issues between different modal materials. Next, standardized encoding processing is performed on the calibrated multimodal materials to unify the data storage and expression formats of various materials. Then, all the processed multimodal materials are split into subsets for training and subsets for effect verification according to a fixed ratio. Throughout the entire process of constructing the dataset, invalid materials with blurry images, excessive audio noise, or incomplete skeletal trajectories are removed to ensure the overall quality of the dataset.

[0028] S200: Build a multimodal physical education teaching teacher reasoning framework and complete full pre-training, extracting single-modal low-level features, cross-modal attention matrix, and teaching task prediction probability as knowledge transfer supervision signals.

[0029] In this embodiment, a multimodal physical education teacher inference framework is built and fully pre-trained. Single-modal low-level features, cross-modal attention matrices, and teaching task prediction probabilities are extracted as knowledge transfer supervision signals. The specific process is as follows: S201: Build an integrated physical education teaching basic reasoning framework and use the reasoning framework to process video, skeleton, voice and text simultaneously; S202: Use the partitioned training material subset to conduct full training on the reasoning architecture, enabling the reasoning architecture to have motion recognition and instructional output capabilities; S203: Lock all runtime parameters of the trained inference architecture and stop updating or adjusting the parameters; S204: Extract three types of content respectively: the underlying information of single-type materials, the multimodal fusion and correlation information, and the teaching judgment output reference value as the reference standard for subsequent knowledge transfer training.

[0030] In the above process, an integrated physical education teaching basic reasoning framework is built, and the reasoning framework is used to process video, skeleton, voice and text simultaneously; then, the reasoning framework is fully trained using the divided training material subset, so that the reasoning framework has the ability to recognize motion and output teaching guidance; then, all the running parameters of the trained reasoning framework are locked and no longer updated or adjusted; then, three types of content are extracted as reference standards for subsequent knowledge transfer training: the underlying information of single-type materials, the multimodal fusion association information, and the teaching judgment output reference value.

[0031] S300: Build a three-layer knowledge transfer framework specifically for sports. It adopts a multi-class error constraint to simultaneously complete single-modal feature matching, cross-modal semantic alignment, and teaching task output fitting training until the accuracy of motion recognition and teaching guidance output on the validation set meets the teacher architecture error, and generates a lightweight inference computing architecture.

[0032] In this implementation, a three-layer knowledge transfer framework specifically for sports is constructed. It employs a multi-class error constraint integration approach to simultaneously complete single-modal feature matching, cross-modal semantic alignment, and teaching task output fitting training until the accuracy of motion recognition and teaching guidance output on the validation set meets the teacher architecture error standards, generating a lightweight inference computing architecture. The specific process is as follows: S301: Construct a sports-specific knowledge transfer processing framework with a three-layer structure: single-modal matching, cross-modal alignment, and teaching output fitting. S302: Build a lightweight basic reasoning architecture with a simple structure and lower computational load as the target carrier for knowledge transfer training; S303: Based on the extracted reference standards, training is carried out in layers to align single-modal low-level information, match multimodal semantic associations, and fit teaching guidance output content. S304: Continuously use a subset of validation materials to test the training effect. When the difference between the recognition and guidance output of the lightweight architecture and the basic inference architecture meets the preset standard, training is stopped, and the lightweight inference computing architecture is obtained.

[0033] In the above process, a sports-specific knowledge transfer processing framework with a three-layer structure consisting of unimodal matching, cross-modal alignment, and teaching output fitting is constructed. A lightweight basic reasoning architecture with a simple structure and lower computational load is also constructed as the target carrier for knowledge transfer training. Based on various extracted reference standards, training is carried out in layers to align unimodal underlying information, match multimodal semantic associations, and fit teaching guidance output content. The training effect is then continuously tested using a subset of validation materials. Training stops when the recognition and guidance output gap between the lightweight architecture and the basic reasoning architecture meets the preset standard, thus obtaining the lightweight reasoning computation architecture.

[0034] S400: Acquires parameters of two types of edge devices: embedded AI modules and high-performance gateways, and performs differential quantization compression of the visual branch INT8 and the voice and text branch INT4 on the lightweight inference architecture, redundant channel pruning, multi-operator fusion, and edge-specific inference format conversion.

[0035] In this embodiment, parameters of two types of edge devices—embedded AI modules and high-performance gateways—are acquired. Differential quantization compression (INT8 for the visual branch and INT4 for the speech / text branch), redundant channel pruning, multi-operator fusion, and edge-specific inference format conversion are then performed on the lightweight inference architecture. The specific process is as follows: S401: Read the computing power limit, available memory, and storage space of the embedded AI computing module and high-performance edge gateway; S402: The visual computing branch that processes video and skeletal information uses INT8 precision quantization compression, and the branch that processes speech and text information uses INT4 precision quantization compression to complete differentiated lightweight processing. S403: Remove redundant computation channels unrelated to sports recognition within the reduced quantization inference architecture and merge continuously repetitive basic computation steps; S404: Converts the processed inference architecture into a dedicated runtime file format adapted to the underlying acceleration library of edge hardware.

[0036] In the above process, the computing power limit, available memory, and storage space of the embedded AI computing module and the high-performance edge gateway are read; then, the visual computing branch that processes video and skeletal information is quantized and compressed with INT8 precision, and the branch that processes voice and text information is quantized and compressed with INT4 precision, to complete differentiated lightweight processing; then, redundant computing channels unrelated to sports recognition in the lightweight inference architecture are removed, and continuous and repetitive basic computing steps are merged; then, the processed inference architecture is converted into a dedicated runtime file format adapted to the underlying acceleration library of the edge hardware; the power consumption of the edge device is monitored in real time throughout the lightweight processing process, and when the power consumption exceeds the preset threshold, the frame rate of image processing is automatically reduced and the non-core computing branches of voice and text are temporarily shut down.

[0037] S500: Deploy the hardware-adapted inference architecture to edge hardware, use a resource scheduler to keep the motion recognition core program running, dynamically load and release the speech and text teaching module as needed, and allocate computing resources according to the priority of tasks such as action correction and lesson plan generation.

[0038] In this embodiment, the hardware-adapted inference architecture is deployed to edge hardware. A resource scheduler is used to keep the motion recognition core program resident, dynamically load and release the speech-text teaching module as needed, and allocate computing resources based on task priorities for action correction and lesson plan generation. The specific process is as follows: S501: Deploy the lightweight inference architecture, which has undergone hardware adaptation and modification, into embedded AI modules and high-performance gateways; S502: Start the hardware resource scheduling program and set the human motion recognition core computing program to a continuous resident running state; S503: A dynamic start / stop mechanism is set for the voice broadcast and text lesson plan generation modules, which are loaded only when teaching guidance is needed and released after use; S504: Differentiate the computational priorities of two types of tasks: real-time motion correction and teaching plan generation, and prioritize the allocation of hardware computing power to the motion correction task.

[0039] In the above process, the lightweight inference architecture, after hardware adaptation and modification, is deployed to the embedded AI module and high-performance gateway. Then, the hardware resource scheduling program is started to set the human motion recognition core computing program to a continuous resident running state. A dynamic start-stop mechanism is set for the voice broadcast and text lesson plan generation modules, which are loaded only when teaching guidance is needed and released after use. By distinguishing the computing priorities of real-time action correction and teaching plan generation tasks, hardware computing power is prioritized for action correction tasks. In addition, the edge hardware locally caches various standard motion action feature templates to reduce the hardware resource consumption caused by repeated calculations.

[0040] S600: It utilizes edge devices to locally collect real-time video and audio data from students and completes pre-processing. It independently completes all calculations based on the local inference architecture and outputs action correction results and voice and text teaching guidance on-site. It also collects inference latency and recognition accuracy indicators and iteratively optimizes the overall calculation effect by adjusting the loss weight of the three-layer knowledge transfer framework.

[0041] In this implementation, edge devices are used to locally collect real-time video and audio data from trainees and perform pre-processing. All calculations are completed independently using a local inference architecture, and action correction results and audio / text teaching guidance are output locally. Simultaneously, inference latency and recognition accuracy metrics are collected, and the loss weights of the three-layer knowledge transfer framework are adjusted in reverse to iteratively optimize the overall computational effect. The specific process is as follows: S601: Real-time acquisition of trainees' movement images and on-site audio through the edge device's built-in camera and audio recording device, and pre-processing of image noise reduction and audio filtering; S602: Sends preprocessed multimodal data into the local lightweight inference architecture, without transmitting raw motion data to the external cloud, and independently completes the entire calculation process; S603: Based on the calculation results, directly output the judgment content of the student's movement deviation, and simultaneously generate voice broadcast and text-based sports teaching guidance; S604: Continuously records operational metrics such as single-operation time and action recognition accuracy, adjusts the constraint strength of each level of the three-layer knowledge transfer framework based on the metrics, and iteratively optimizes the overall operation effect.

[0042] In the above process, the edge device's built-in camera and audio recording equipment are used to collect students' movement images and on-site audio in real time, and pre-processing such as image noise reduction and audio filtering is completed. Then, the pre-processed multimodal data is sent to the local lightweight inference architecture, without transmitting the original movement data to the external cloud, and completes the entire calculation process independently. Based on the calculation results, the judgment content of student movement deviation is directly output, and voice broadcast and text-based sports teaching guidance are generated simultaneously. The system continuously records the operation indicators such as the single calculation time and the accuracy of movement recognition, and adjusts the constraint strength of each level of the three-layer knowledge transfer framework according to the indicators, and iteratively optimizes the overall calculation effect. In the above, the single response time of the entire local inference calculation is controlled within 100ms, which meets the real-time interactive needs of sports teaching.

[0043] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0044] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for deploying a large-scale multimodal physical education teaching model for edge devices, characterized in that, Includes the following steps: A standardized dataset for multimodal physical education teaching was constructed. The dataset includes videos of physical movements, time-series sequences of human skeletons, teaching audio, standard lesson plan texts, and annotations of correct and incorrect movements. Multimodal time-series alignment and unified encoding were completed, and training and validation subsets were divided. A multimodal physical education teaching teacher reasoning framework was built and fully pre-trained. Single-modal low-level features, cross-modal attention matrix, and teaching task prediction probability were extracted as knowledge transfer supervision signals. A three-layer knowledge transfer framework specifically for sports is built. It adopts a multi-type error constraint to simultaneously complete single-modal feature matching, cross-modal semantic alignment, and teaching task output fitting training until the accuracy of motion recognition and teaching guidance output on the validation set and the error of teacher architecture meet the standards, and generates a lightweight inference computing architecture. Acquire parameters for two types of edge devices: embedded AI modules and high-performance gateways. Perform differentiated quantization compression on the visual branch (INT8) and the voice / text branch (INT4), redundant channel pruning, multi-operator fusion, and edge-specific inference format conversion on the lightweight inference architecture. The hardware-adapted inference architecture is deployed to edge hardware. The resource scheduler keeps the motion recognition core program running in the background and dynamically loads and releases the speech and text teaching module as needed. The computing resources are allocated according to the priority of tasks such as action correction and lesson plan generation. The system utilizes edge devices to locally collect real-time video and audio data from students and performs pre-processing. It independently completes all calculations based on the local inference architecture and outputs action correction results and voice and text teaching guidance on-site. Simultaneously, it collects inference latency and recognition accuracy indicators and iteratively optimizes the overall calculation effect by adjusting the loss weight of the three-layer knowledge transfer framework.

2. The multimodal physical education teaching large-scale model distillation deployment method for edge devices as described in claim 1, characterized in that, In constructing a standardized multimodal physical education teaching dataset, which includes videos of physical movements, temporal sequences of human skeletons, teaching audio, standard lesson plan texts, and annotations of correct and incorrect movements, and completing multimodal temporal alignment and unified encoding, and dividing the dataset into training and validation subsets: Collect action footage, human skeletal dynamic trajectory, instructional audio, standardized teaching text plans, and materials for marking correct and incorrect movements for various sports including track and field, ball games, gymnastics, and martial arts; The timelines of four types of materials—video, skeleton, audio, and text—are uniformly calibrated to eliminate time misalignment issues between different modalities. Standardized coding is performed on the corrected multimodal materials to unify the data storage and expression formats of various materials; The processed multimodal data is split into a subset for training and a subset for performance evaluation according to a fixed ratio.

3. The multimodal physical education teaching large-scale model distillation deployment method for edge devices as described in claim 1, characterized in that, In the steps of building a multimodal physical education teaching teacher inference framework and completing full pre-training, extracting unimodal low-level features, cross-modal attention matrix, and teaching task prediction probability as knowledge transfer supervision signals: Build an integrated physical education teaching basic reasoning framework and use the reasoning framework to process video, skeleton, voice and text simultaneously; The reasoning framework is fully trained using a subset of the divided training materials, enabling it to recognize motion and provide instructional guidance. Lock all runtime parameters of the trained inference architecture and stop updating or adjusting the parameters. We extract three types of content—underlying information of single-type materials, multimodal fusion and correlation information, and teaching judgment output reference values—as reference standards for subsequent knowledge transfer training.

4. The multimodal physical education teaching large-scale model distillation deployment method for edge devices as described in claim 1, characterized in that, In building a three-layer knowledge transfer framework specifically for sports, the process involves simultaneously completing single-modal feature matching, cross-modal semantic alignment, and teaching task output fitting training by integrating multiple error constraints, until the accuracy of motion recognition and teaching guidance output on the validation set meets the teacher architecture error standards, and finally generating a lightweight inference computing architecture. A sports-specific knowledge transfer processing framework was constructed, consisting of three layers: single-modal matching, cross-modal alignment, and teaching output fitting. A lightweight basic reasoning architecture with a simple structure and lower computational load is built as the target carrier for knowledge transfer training; Based on the extracted reference standards, training is carried out in layers to align single-modal low-level information, match multimodal semantic associations, and fit teaching guidance output content. The training effect is continuously tested using a subset of validation materials. Training stops when the difference between the recognition and guidance output of the lightweight architecture and the basic inference architecture meets the preset standard, thus obtaining the lightweight inference computing architecture.

5. The multimodal physical education teaching large-scale model distillation deployment method for edge devices as described in claim 1, characterized in that, In the process of acquiring parameters for two types of edge devices—embedded AI modules and high-performance gateways—and performing differential quantization compression (INT8 for the visual branch and INT4 for the speech and text branch), redundant channel pruning, multi-operator fusion, and edge-specific inference format conversion on the lightweight inference architecture: Read the computing power limit, available memory, and storage space of the embedded AI computing module and the high-performance edge gateway; The visual computing branch that processes video and skeletal information uses INT8 precision quantization compression, while the branch that processes speech and text information uses INT4 precision quantization compression to achieve differentiated lightweight processing. Remove redundant computational channels unrelated to sports recognition within the weighted inference architecture and merge continuously repetitive basic computational steps; The processed inference architecture is converted into a dedicated runtime file format adapted to the underlying acceleration library of edge hardware.

6. The multimodal physical education teaching large-scale model distillation deployment method for edge devices as described in claim 1, characterized in that, In the process of deploying the hardware-adapted inference architecture to edge hardware, using a resource scheduler to keep the motion recognition core program resident, dynamically loading and releasing the speech-text teaching module on demand, and allocating computing resources according to the priority of tasks such as action correction and lesson plan generation: Deploy the lightweight inference architecture, which has undergone hardware adaptation and modification, into embedded AI modules and high-performance gateways; Start the hardware resource scheduling program and set the human motion recognition core computing program to a continuous resident running state; A dynamic start / stop mechanism is set up for the voice broadcast and text lesson plan generation modules, which are only loaded when teaching guidance is needed and released after use; The computational priority of the two types of tasks, real-time action error correction and teaching plan generation, is distinguished, and hardware computing power is allocated to the action error correction task first.

7. The multimodal physical education teaching large-scale model distillation deployment method for edge devices as described in claim 1, characterized in that, In the process of using edge devices to locally collect real-time video and audio data from trainees and perform pre-processing, relying on the local inference architecture to independently complete all calculations, and outputting action error correction results and voice and text teaching guidance on-site; and simultaneously collecting inference latency and recognition accuracy indicators, and iteratively optimizing the overall calculation effect by adjusting the loss weights of the three-layer knowledge transfer framework: The edge device uses its built-in camera and audio recording equipment to capture real-time images of trainees' movements and on-site audio, and performs pre-processing such as image noise reduction and audio filtering. The preprocessed multimodal data is sent to the local lightweight inference architecture, without transmitting the original motion data to the external cloud, and the entire calculation process is completed independently. Based on the calculation results, the system directly outputs the judgment content of the student's movement deviation and simultaneously generates physical education teaching guidance in both voice broadcast and text form. The system continuously records operational metrics such as single-operation time and action recognition accuracy, adjusts the constraint strength of each level of the three-layer knowledge transfer framework based on these metrics, and iteratively optimizes the overall computational performance.

Images