A Multimodal Temporal Neural Network Method and System for Perceiving Sports Training Load Quality

By using modal quality assessment and cross-modal attention mechanisms to dynamically adjust the weights of the loss function, the problem of identification stability of multi-source motion time series data is solved, and high accuracy and reliability of sports training load identification are achieved.

CN122904043BActive Publication Date: 2026-11-17GUANGDONG UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611383702.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-09-08
Publication Date
2026-11-17
Estimated Expiration
2046-09-08

AI Technical Summary

Technical Problem

Existing methods for identifying sports training load struggle to handle asynchronous, missing, and unevenly sourced multi-source motion time-series data, resulting in insufficient identification stability and blurred boundaries. Furthermore, they lack credibility evaluation of modal quality and class intervals.

Method used

By employing modal quality assessment, unified time-axis weighted window aggregation, multimodal temporal feature tensor construction, and quality-aware cross-modal attention mechanism, the loss function weights are dynamically adjusted to generate stable training load state classification labels and their confidence scores.

Benefits of technology

It improves the accuracy and robustness of training load identification, provides reliable identification results, adaptively handles data quality fluctuations, and enhances the ability to identify difficult samples.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122904043B_ABST
    Figure CN122904043B_ABST
Patent Text Reader

Abstract

This invention discloses a multimodal temporal neural network method and system for perceiving the quality of sports training load. The method includes: receiving multi-source motion time-series data generated during sports training; evaluating the usability of each modality at each time step; generating quality scores, validity masks, and missing data markers; performing quality-weighted window aggregation based on a unified time axis and constructing a multimodal temporal feature tensor by combining time position encoding; inputting the tensor into a neural network model to extract joint training load features; dynamically adjusting the class weights based on the recent recognition difficulty of each load category during the model training phase; and outputting training load state classification labels and confidence scores during the inference phase. The invention also discloses a multimodal temporal neural network system, electronic device, and storage medium for perceiving the quality of sports training load. This invention solves the problem of insufficient recognition stability in existing neural network models.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of computer system technology based on specific computational models, specifically a method and system for perceiving the quality of sports training load using a multimodal temporal neural network. Background Technology

[0002] Sports training load identification is a crucial step in intelligent sports training analysis. Its purpose is to determine the athlete's training load status based on multi-source time-series data generated during exercise. Data available in sports training scenarios typically includes time-series data on body surface heat distribution, facial pulse correlation, and environmental parameters. These data sources differ, exhibiting variations in sampling frequency, time delay, noise type, missing patterns, and stability. Therefore, directly employing single-modal recognition, fixed threshold judgment, or simple feature concatenation methods makes it difficult to obtain stable training load identification results.

[0003] Existing methods for identifying sports training load typically suffer from the following shortcomings: First, different modal data are difficult to directly correspond on the time axis. Conventional resampling or interpolation processes usually only consider numerical continuity and do not fully reflect the reliability of each modal data at different time steps. Second, during exercise, occlusion, posture changes, lighting fluctuations, and local sampling interruptions can easily occur, leading to a decrease in the quality or loss of some modalities in local time periods. Existing models usually do not input modal quality and missing states as learnable information into the neural network. Third, training load states have continuous changing characteristics, and the boundaries between adjacent load categories are unclear. In particular, the number of samples for transitional load states and high-intensity load states is relatively small, and conventional classification models tend to favor the category with a higher sample proportion. Fourth, existing models mostly output deterministic classification results and lack a confidence evaluation mechanism that combines modal quality, class interval, and attention distribution.

[0004] Therefore, this paper provides a multimodal temporal neural network method and system for perceiving the quality of sports training load, which enables it to perform quality encoding, missing modeling, time alignment, cross-modal fusion, and boundary-sensitive classification on asynchronous, missing, and quality-imbalanced multi-source sports temporal data, thereby improving the accuracy, robustness, and reliability of sports training load identification. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a multimodal temporal neural network method and system for perceiving sports training load quality, in order to solve the problem of insufficient recognition stability of existing neural network models when multi-source sports time-series data have inconsistent sampling frequencies, local missing data, modal quality fluctuations, and ambiguous load category boundaries.

[0006] The technical solution to achieve the above objectives is: One of the present inventions, a multimodal temporal neural network method for perceiving the quality of sports training load, includes: Step S1: Modal quality evaluation is performed on the multi-source motion time series data generated during sports training to obtain the quality score and validity mask of each modality at each time step. The multi-source motion time series data includes body surface heat distribution time series, facial pulse correlation time series and environmental parameter time series. Step S2: Perform quality-weighted window aggregation on multi-source motion time series data based on a unified time axis, and set missing data or data with quality scores below a preset threshold. Step S3: The aligned multi-source motion time series data, quality scores, validity masks, missing markers and time position codes are concatenated to generate a multimodal time series feature tensor. Step S4: Input the multimodal temporal feature tensor into the multimodal temporal fusion neural network model composed of modality-specific coding units, gated recurrent units, residual connection structures, and quality-aware cross-modal attention units to obtain joint training load features; Step S5: During model training, dynamically adjust the class weight coefficients in the loss function based on the recent recognition difficulty of each training load state category. Step S6: After training is completed, receive the target sports training time series data, use the trained multimodal time series fusion neural network model to perform inference, and output the corresponding training load state classification label and its confidence score. The confidence score is calculated based on the class interval between the highest and second-highest class probabilities and the modality quality score of the training load state, and is used to quantify the stability and reliability of the training load identification results.

[0007] Preferably, in step S1, the modal quality evaluation includes: For the temporal sequence of body surface heat distribution, a first quality score is generated based on the visibility of the target body region, the stability of the regional temperature, and the amplitude of changes between consecutive frames. For facial pulse-related timing, a second quality score is generated based on the stability of the facial region of interest, waveform periodicity, and signal-noise level. For environmental parameter time series, a third quality score is generated based on sampling integrity and sampling continuity; The first quality score, the second quality score, and the third quality score are normalized, and a corresponding validity mask is generated based on a preset quality threshold.

[0008] Preferably, in step S2, the quality-weighted window aggregation includes: At any target time step on a unified time axis, determine the candidate sampled data for each mode within the time window adjacent to that target time step; Candidate sample data with quality scores below a preset threshold are removed, and the quality scores of the remaining candidate sample data are used as weights to calculate the alignment features of the modality at the target time step. When there is no valid candidate sampling data in the adjacent time window, a missing marker is set for the modality at the target time step, and it is filled with at least one of the features of the preceding valid time step, local statistical features, or model-generated features.

[0009] Preferably, in step S3, the multimodal temporal feature tensor includes a time dimension, a modality dimension, and a feature dimension; The feature dimensions include alignment features, quality scores, validity masks, missing markers, and temporal location encodings for each modality. Missing labels are used as additional feature channels input to a multimodal temporal fusion neural network model to distinguish between real observation data and compensated data.

[0010] Preferably, in step S4, the modal-specific encoding unit includes: a thermal distribution spatial encoder, a pulse correlation timing encoder, and an environmental parameter encoder; A thermal distribution spatial encoder is used to extract spatial relationship features between different body regions in the temporal sequence of thermal distribution on the body surface; A pulse-related temporal encoder is used to extract periodic and temporal variation features in facial pulse-related temporal sequences. An environmental parameter encoder is used to extract the trend characteristics of environmental parameters over time. The quality-aware cross-modal attention unit includes: a quality embedding layer, a modality attention layer, and a temporal attention layer; The quality embedding layer is used to map quality scores, validity masks, and missing markers into quality embedding vectors; The modal attention layer is used to determine the fusion weights of different modalities based on the quality embedding vector; The temporal attention layer is used to determine the temporal fusion weights based on the overall quality of different time steps, and to obtain the joint training load features based on the modality fusion weights and temporal fusion weights.

[0011] Preferably, in step S5, dynamically adjusting the class weight coefficients in the loss function includes: Calculate the true class prediction probability for each training load state category in the current training batch or several historical training batches. The class confidence statistics of each training load state category are updated by moving average, and the recent recognition difficulty of the corresponding category is determined by the inverse vector of the class confidence statistics. Higher class weights are assigned to training load state categories that are more difficult to identify recently, and a dynamic class weight loss function is constructed by combining the hard sample modulation term.

[0012] Preferably, in step S6, outputting the corresponding training load state classification label and its confidence score includes: Determine the training load state classification label corresponding to the highest category probability based on the training load state category probability; Confidence scores are generated based on the highest class probability, the class interval between the highest and second-highest class probabilities, modality quality scores, and attention weight distribution entropy. When the confidence score is lower than the preset confidence threshold, a low confidence flag or a rejection flag is output for external sports training analysis systems to review or delay the decision.

[0013] The second invention relates to a multimodal temporal neural network system for perceiving the quality of sports training load, comprising: The multi-source time-series data receiving module is used to receive multi-source motion time-series data generated during sports training. The multi-source motion time-series data includes body surface heat distribution time-series, facial pulse correlation time-series, and environmental parameter time-series. The modal quality assessment module is used to generate quality scores and validity masks based on the availability of each modal data at each time step. The temporal alignment and missing data marking module is used to establish a unified time axis, perform quality-weighted window aggregation on multi-source motion time-series data with different sampling frequencies, and set missing data or data with quality scores below a preset threshold. The multimodal tensor construction module is used to concatenate aligned multi-source motion temporal data, quality scores, validity masks, missing markers, and temporal location codes into a multimodal temporal feature tensor. A dedicated modality encoding module is used to extract feature representations corresponding to different modalities. The quality-aware cross-modal fusion module is used to calculate modal fusion weights and temporal fusion weights based on quality scores, validity masks, and missing data markers, and to generate joint training load features. The load state classification module is used to output the probability of training load state categories based on the joint training load characteristics; The dynamic weight training module is used to dynamically adjust the class weights based on the recent recognition difficulty of each training load class during the model training phase. The confidence output module is used to generate recognition confidence based on the training load state category probability, category interval, modality quality score, and attention weight distribution, and output the training load recognition result.

[0014] The third invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a multimodal temporal neural network method for perceiving the quality of sports training load.

[0015] The fourth invention provides a computer-readable storage medium storing a computer program thereon, wherein when the computer program is executed by a processor, it implements the steps of a multimodal temporal neural network method for perceiving sports training load quality.

[0016] Compared with the prior art, the beneficial effects of the present invention are: Adaptive quality perception: Encodes data quality into learnable features, enabling the model to distinguish between high-confidence and low-confidence data, thus improving recognition robustness; Precise temporal fusion: Based on quality-weighted temporal alignment and cross-modal attention mechanism, dynamic fusion of multi-source data is achieved, avoiding feature mixing; Balanced category recognition: Dynamically adjusts loss weights to improve the ability to identify load states with low sample proportions and high recognition difficulty; Confidential results output: Confidence scores are generated by combining modality quality, class margin, and attention distribution to provide a reliable basis for training decisions. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the multimodal temporal neural network method for perceiving the quality of sports training load according to the present invention; Figure 2 This is a block diagram of the multimodal temporal neural network system for perceiving the quality of sports training load according to the present invention; Figure 3 This is a schematic diagram of the construction of multimodal temporal feature tensors in an embodiment of the present invention; Figure 4 This is a schematic diagram of the network structure of the quality-sensing cross-modal fusion module in an embodiment of the present invention; Figure 5 This is a schematic diagram of dynamic category weight training in an embodiment of the present invention; Figure 6 This is a schematic diagram of confidence output in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the multimodal temporal neural network method for perceiving the quality of sports training load includes: Step S1: Perform modal quality evaluation on the multi-source motion time series data generated during sports training to obtain the quality score and validity mask of each modality at each time step.

[0020] In this embodiment, the multi-source motion time series data includes body surface heat distribution time series, facial pulse correlation time series, and environmental parameter time series; Temporal sequence of body surface heat distribution is used to characterize the heat distribution characteristics of different body regions of an athlete over time; Facial pulse-related temporal sequence is used to characterize pulse-related changes obtained from facial videos or other non-contact methods. Environmental parameter time series is used to characterize the changes in temperature, humidity or other environmental parameters in the training environment; Each modal data has a corresponding sampling timestamp, and the original modal data are not required to have the same sampling interval; in, The time series of body surface heat distribution can be obtained by dividing the human body region from infrared thermal imaging data; the time series of facial pulse correlation can be obtained by extracting the subtle color change features of the skin in visible light facial video; and the time series of environmental parameters can be obtained by sampling from environmental sensors. The above data sources are only used to illustrate the formation method of multi-source motion time series data and do not constitute a limitation on the data acquisition equipment.

[0021] In this embodiment, modal quality evaluation includes: For the temporal sequence of body surface heat distribution, a first quality score is generated based on the visibility of the target body region, the stability of the regional temperature, and the amplitude of changes between consecutive frames. For facial pulse-related timing, a second quality score is generated based on the stability of the facial region of interest, waveform periodicity, and signal-noise level. For environmental parameter time series, a third quality score is generated based on sampling integrity and sampling continuity; The first quality score, the second quality score, and the third quality score are normalized, and a corresponding validity mask is generated based on a preset quality threshold.

[0022] Step S2: Perform quality-weighted window aggregation on multi-source motion time series data based on a unified time axis, and set missing data or data with quality scores below a preset threshold.

[0023] In this embodiment, the quality-weighted window aggregation includes: At any target time step on a unified time axis, determine the candidate sampled data for each mode within the time window adjacent to that target time step; Candidate sample data with quality scores below a preset threshold are removed, and the quality scores of the remaining candidate sample data are used as weights to calculate the alignment features of the modality at the target time step. When there is no valid candidate sampling data in the adjacent time window, a missing marker is set for the modality at the target time step, and it is filled with at least one of the features of the preceding valid time step, local statistical features, or model-generated features.

[0024] Step S3 involves concatenating the aligned multi-source motion time series data, quality scores, validity masks, missing markers, and time position codes to generate a multimodal time series feature tensor.

[0025] In this embodiment, the multimodal temporal feature tensor includes a time dimension, a modality dimension, and a feature dimension; The feature dimensions include alignment features, quality scores, validity masks, missing markers, and temporal location encodings for each modality. Missing labels are used as additional feature channels input to a multimodal temporal fusion neural network model to distinguish between real observation data and compensated data.

[0026] In this embodiment, Figure 3 This illustrates the construction of the multimodal temporal feature tensor of the present invention: Time dimension: Several time steps distributed along a uniform time axis (e.g.) , , ..., Each time step corresponds to one standard sampling point; Modal dimension: It includes 3 independent modal channels, corresponding to the body surface heat distribution time series, facial pulse related time series, and environmental parameter time series, respectively; Feature Dimensions: The features of each modality at each time step consist of 5 parts: Alignment features ( ): Original data features after quality-weighted aggregation; Quality rating ( ): The quality quantization value of this mode at the current time step; Validity mask ( ): A binary value that indicates whether the data for this time step is available; Missing markers ( ): Marks whether the data at this time step is a binary value containing missing / filled values; Time location encoding ( ): Used to distinguish positional features in time sequence. The entire tensor can be represented as ,in, For the first Each modality at time step Alignment features, For modal indexing, For time step indexing.

[0027] Step S4: Input the multimodal temporal feature tensor into the multimodal temporal fusion neural network model composed of a modality-specific encoding unit, a gated recurrent unit, a residual connection structure, and a quality-aware cross-modal attention unit to obtain joint training load features.

[0028] In this embodiment, the modal-specific coding unit includes: a thermal distribution spatial encoder, a pulse correlation timing encoder, and an environmental parameter encoder; A thermal distribution spatial encoder is used to extract spatial relationship features between different body regions in a temporal sequence of thermal distribution on the body surface; the thermal distribution spatial encoder includes at least one of a one-dimensional convolutional network, a two-dimensional convolutional network, or a graph convolutional network. A pulse-related temporal encoder is used to extract periodic and temporal variation features in facial pulse-related temporal data. The pulse-related temporal encoder includes at least one of a gated recurrent unit, a long short-term memory network, a temporal convolutional network, or a Transformer encoder. An environmental parameter encoder is used to extract the trend features of environmental parameters changing over time; the environmental parameter encoder includes at least one of a fully connected network, a temporal convolutional network, or a gated recurrent network. The quality-aware cross-modal attention unit includes: a quality embedding layer, a modality attention layer, and a temporal attention layer; The quality embedding layer is used to map quality scores, validity masks, and missing markers into quality embedding vectors; The modal attention layer is used to determine the fusion weights of different modalities based on the quality embedding vector; The temporal attention layer is used to determine the temporal fusion weights based on the overall quality of different time steps, and to obtain the joint training load features based on the modality fusion weights and temporal fusion weights.

[0029] In this embodiment, Figure 4 This is a schematic diagram of the network structure of the quality-sensing cross-modal fusion module in this invention. The module consists of four core units: The modal-specific coding unit includes: a thermal distribution spatial encoder, a pulse correlation timing encoder, and an environmental parameter encoder; Thermal distribution spatial encoder: Employs convolutional networks (1D / 2D convolution or graph convolution) to extract spatial relationship features between different body regions on the body surface; Pulse-related temporal encoder: GRU, LSTM (Long Short-Term Memory Network) or Transformer (attention-based sequence model) encoders are used to extract the periodic changes and temporal dynamic features of the pulse signal; Environmental parameter encoder: Uses a fully connected network or a temporal convolutional network to extract the trend features of environmental parameters; Gated Recurrent Unit (GRU): Models the temporal features after single-modal encoding in the time dimension, enhancing the retention of information at key time nodes; Residual connection structure: used to alleviate the gradient vanishing problem during network training and improve feature transfer efficiency; The quality-aware cross-modal attention unit includes: a quality embedding layer, a modality attention layer, and a temporal attention layer; Quality embedding layer: maps quality scores, validity masks, and missing markers into quality embedding vectors that can be used in computation; Modal attention layer: Dynamically calculates the fusion weights of different modalities based on the quality embedding vector; Temporal attention layer: Determines the fusion weight of the time dimension based on the overall quality of each time step; Finally, based on the modality and time fusion weights, the joint training load features are output.

[0030] Step S5: During model training, dynamically adjust the class weight coefficients in the loss function based on the recent recognition difficulty of each training load state category.

[0031] In this embodiment, dynamically adjusting the class weight coefficients in the loss function includes: In the current training batch or several historical training batches, the true class prediction probability of each training load state category is calculated; wherein, the training load state categories include at least low load state, medium load state, transitional load state, and high intensity load state; The class confidence statistics of each training load state category are updated by moving average, and the recent recognition difficulty of the corresponding category is determined by the inverse vector of the class confidence statistics. Higher class weights are assigned to training load state categories that are more difficult to identify recently, and a dynamic class weight loss function is constructed by combining the hard sample modulation term.

[0032] In this embodiment, Figure 5 The training process for dynamic class weights is demonstrated, including: Enter the predicted probability of the current batch and the historical confidence statistics of the categories; The class confidence statistic for each training load state category is updated using a moving average. The recent identification difficulty of each category is determined by the inverse vector of the statistics, and higher weights are assigned to categories with high identification difficulty (such as excessive load and high intensity load). A dynamic weight loss function is constructed by combining the hard sample modulation term. The core formula is: Class weight = (1 - Class confidence statistic) / mean of class confidence statistic.

[0033] Step S6: After training is completed, the target sports training time series data is received, and the trained multimodal time series fusion neural network model is used for inference to output the corresponding training load state classification label and its confidence score. The confidence score is calculated based on the training load state category probability, the category interval between the highest category probability and the second highest category probability, and the modality quality score, and is used to quantify the stability and reliability of the training load identification result.

[0034] In this embodiment, the output of the corresponding training load state classification label and its confidence score includes: Determine the training load state classification label corresponding to the highest category probability based on the training load state category probability; Confidence scores are generated based on the highest class probability, the class interval between the highest and second-highest class probabilities, modality quality scores, and attention weight distribution entropy. When the confidence score is lower than the preset confidence threshold, a low confidence flag or a rejection flag is output for external sports training analysis systems to review or delay the decision.

[0035] In the embodiments, the identification confidence score is used to characterize the credibility of the identification results of the current training load; When the highest category probability is high, the category interval is large, the modality quality score is high, and the attention weight distribution is concentrated, the recognition confidence increases. When the highest category probability is low, the category interval is small, the modality quality score is low, or the attention weight distribution is scattered, the recognition confidence decreases.

[0036] In this embodiment, Figure 6 The confidence level output process is demonstrated: Input the training load state category probabilities (highest probability, second highest probability), category margin (difference between highest and second highest probabilities), modality quality score, and attention weight distribution entropy; Calculate the confidence score using the formula: Confidence score = ×Highest category probability+ ×Category Spacing+ ×Modal quality score+ ×(1-attention distribution entropy), where, , , and The weights are: the highest category probability, the category interval (the difference between the highest and second highest category probabilities), the modality quality score (the average quality of each sampled modality data), and (1 - attention weight distribution entropy). The lower the entropy value, the more focused the attention, and the higher the score. Output rules: When the confidence score is greater than or equal to the preset confidence threshold, output the training load classification label and confidence score; when the confidence score is less than the threshold, output a "low confidence flag" or "rejection flag" for external sports training analysis system to review.

[0037] like Figure 2 As shown, the multimodal temporal neural network system for perceiving sports training load quality includes: a multi-source temporal data receiving module 1, a modal quality evaluation module 2, a temporal alignment and missing labeling module 3, a multimodal tensor construction module 4, a modal-specific encoding module 5, a quality-perceived cross-modal fusion module 6, a load state classification module 7, a dynamic weight training module 8, and a confidence output module 9.

[0038] The multi-source time-series data receiving module 1 is used to receive multi-source motion time-series data generated during sports training. The multi-source motion time-series data includes body surface heat distribution time-series, facial pulse correlation time-series, and environmental parameter time-series.

[0039] Modal quality assessment module 2 is used to generate quality scores and validity masks based on the availability of each modal data at each time step.

[0040] The timing alignment and missing data marker module 3 is used to establish a unified time axis, perform quality-weighted window aggregation on multi-source motion time series data with different sampling frequencies, and set missing data markers for missing data or data with quality scores below a preset threshold.

[0041] The multimodal tensor construction module 4 is used to concatenate the aligned multi-source motion time series data, quality scores, validity masks, missing markers and time position codes into a multimodal time series feature tensor.

[0042] Modal-specific encoding module 5 is used to extract feature representations corresponding to different modes through modal-specific encoding units. The modal-specific encoding units include a thermal distribution spatial encoder, a pulse correlation temporal encoder, and an environmental parameter encoder.

[0043] The quality-aware cross-modal fusion module 6 is used to calculate modal fusion weights and temporal fusion weights based on quality scores, validity masks, and missing markers, and to generate joint training load features.

[0044] The load state classification module 7 is used to output the probability of the training load state category based on the joint training load characteristics.

[0045] In this embodiment, the joint training load features are input into the load state classification module 7 to obtain the training load state category probability. The training load state category probability is used to characterize the probability that the current sample belongs to each training load state category. The training load state categories include at least low load state, medium load state, transitional load state, and high intensity load state.

[0046] In one specific implementation, the training load state categories can correspond to low load recovery zone, aerobic training zone, aerobic-anaerobic transition zone, and high-intensity anaerobic zone.

[0047] The dynamic weight training module 8 is used to dynamically adjust the class weights during the model training phase based on the recent recognition difficulty of each training load state class.

[0048] The confidence output module 9 is used to generate recognition confidence based on the training load state category probability, category interval, modality quality score and attention weight distribution, and output the training load recognition result.

[0049] An electronic device, Figure 7 This is a schematic diagram of the electronic device. The electronic device 30 includes a memory 32, a processor 31, and a computer program stored in the memory and executable on the processor. When the processor 31 executes the program, it implements the sports training load quality perception multimodal temporal neural network method.

[0050] like Figure 7 As shown, the electronic device 30 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0051] Bus 33 includes a data bus, an address bus, and a control bus.

[0052] The memory 32 may include volatile memory, such as RAM (random access memory) 321 and / or cache memory 322, and may further include ROM (read-only memory) 323.

[0053] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0054] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the sports training load quality perception multimodal temporal neural network method.

[0055] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed through I / O (input / output) interface 35. Furthermore, the model-generated electronic device 30 can also communicate with one or more networks (e.g., LAN (Local Area Network), WAN (Wide Area Network), and / or public networks, such as the Internet) via network adapter 36. Network adapter 36 communicates with other modules of the model-generated electronic device 30 via bus 33. Other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (Rapid Array of Independent Disks) systems, tape drives, and data backup storage systems, can be used in conjunction with the model-generated electronic device 30.

[0056] The foregoing mentions several units / modules or sub-units / modules of an electronic device, but this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in a single unit / module. Conversely, the features and functions of a single unit / module described above can be further divided and embodied by multiple units / modules.

[0057] A computer-readable storage medium having a computer program stored thereon, wherein the steps of a sports training load quality perception multimodal temporal neural network method are implemented when the computer program is executed by a processor.

[0058] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0059] In a possible implementation, the present invention can also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to execute a method for implementing a multimodal temporal neural network for perceiving sports training load quality.

[0060] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0061] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal temporal neural network method for perceiving the quality of sports training load, characterized in that, include: Step S1: Modal quality evaluation is performed on the multi-source motion time series data generated during sports training to obtain the quality score and validity mask of each modality at each time step. The multi-source motion time series data includes body surface heat distribution time series, facial pulse correlation time series and environmental parameter time series. Step S2: Perform quality-weighted window aggregation on multi-source motion time series data based on a unified time axis, and set missing data or data with quality scores below a preset threshold. Step S3: The aligned multi-source motion time series data, quality scores, validity masks, missing markers and time position codes are concatenated to generate a multimodal time series feature tensor. Step S4: Input the multimodal temporal feature tensor into the multimodal temporal fusion neural network model composed of modality-specific coding units, gated recurrent units, residual connection structures, and quality-aware cross-modal attention units to obtain joint training load features; Step S5: During model training, dynamically adjust the class weight coefficients in the loss function based on the recent recognition difficulty of each training load state category. Step S6: After training is completed, receive the target sports training time series data, use the trained multimodal time series fusion neural network model to perform inference, and output the corresponding training load state classification label and its confidence score. The confidence score is calculated based on the class interval between the highest and second-highest class probabilities and the modality quality score of the training load state, and is used to quantify the stability and reliability of the training load identification results. In step S4, the modal-specific encoding unit includes: a thermal distribution spatial encoder, a pulse-related temporal encoder, and an environmental parameter encoder; A thermal distribution spatial encoder is used to extract spatial relationship features between different body regions in the temporal sequence of thermal distribution on the body surface; A pulse-related temporal encoder is used to extract periodic and temporal variation features in facial pulse-related temporal sequences. An environmental parameter encoder is used to extract the trend characteristics of environmental parameters over time. The quality-aware cross-modal attention unit includes: a quality embedding layer, a modality attention layer, and a temporal attention layer; The quality embedding layer is used to map quality scores, validity masks, and missing markers into quality embedding vectors; The modal attention layer is used to determine the fusion weights of different modalities based on the quality embedding vector; The temporal attention layer is used to determine the temporal fusion weights based on the overall quality of different time steps, and to obtain the joint training load features based on the modality fusion weights and temporal fusion weights.

2. The multimodal temporal neural network method for perceiving sports training load quality according to claim 1, characterized in that, In step S1, the modal quality evaluation includes: For the temporal sequence of body surface heat distribution, a first quality score is generated based on the visibility of the target body region, the stability of the regional temperature, and the amplitude of changes between consecutive frames. For facial pulse-related timing, a second quality score is generated based on the stability of the facial region of interest, waveform periodicity, and signal-noise level. For environmental parameter time series, a third quality score is generated based on sampling integrity and sampling continuity; The first quality score, the second quality score, and the third quality score are normalized, and a corresponding validity mask is generated based on a preset quality threshold.

3. The multimodal temporal neural network method for perceiving sports training load quality according to claim 1, characterized in that, In step S2, the quality-weighted window aggregation includes: At any target time step on a unified time axis, determine the candidate sampled data for each mode within the time window adjacent to that target time step; Candidate sample data with quality scores below a preset threshold are removed, and the quality scores of the remaining candidate sample data are used as weights to calculate the alignment features of the modality at the target time step. When there is no valid candidate sampling data in the adjacent time window, a missing marker is set for the modality at the target time step, and it is filled with at least one of the features of the preceding valid time step, local statistical features, or model-generated features.

4. The multimodal temporal neural network method for perceiving sports training load quality according to claim 1, characterized in that, In step S3, the multimodal temporal feature tensor includes a time dimension, a modality dimension, and a feature dimension; The feature dimensions include alignment features, quality scores, validity masks, missing markers, and temporal location encodings for each modality. Missing labels are used as additional feature channels input to a multimodal temporal fusion neural network model to distinguish between real observation data and compensated data.

5. The multimodal temporal neural network method for perceiving sports training load quality according to claim 1, characterized in that, In step S5, dynamically adjusting the class weight coefficients in the loss function includes: Calculate the true class prediction probability for each training load state category in the current training batch or several historical training batches. The class confidence statistics of each training load state category are updated by moving average, and the recent recognition difficulty of the corresponding category is determined by the inverse vector of the class confidence statistics. Higher class weights are assigned to training load state categories that are more difficult to identify recently, and a dynamic class weight loss function is constructed by combining the hard sample modulation term.

6. The multimodal temporal neural network method for perceiving sports training load quality according to claim 1, characterized in that, In step S6, the output of the corresponding training load state classification label and its confidence score includes: Determine the training load state classification label corresponding to the highest category probability based on the training load state category probability; Confidence scores are generated based on the highest class probability, the class interval between the highest and second-highest class probabilities, modality quality scores, and attention weight distribution entropy. When the confidence score is lower than the preset confidence threshold, a low confidence flag or a rejection flag is output for external sports training analysis systems to review or delay the decision.

7. A multimodal temporal neural network system for perceiving the quality of sports training load based on the method of any one of claims 1-6, characterized in that, include: The multi-source time-series data receiving module is used to receive multi-source motion time-series data generated during sports training. The multi-source motion time-series data includes body surface heat distribution time-series, facial pulse correlation time-series, and environmental parameter time-series. The modal quality assessment module is used to generate quality scores and validity masks based on the availability of each modal data at each time step. The temporal alignment and missing data marking module is used to establish a unified time axis, perform quality-weighted window aggregation on multi-source motion time-series data with different sampling frequencies, and set missing data or data with quality scores below a preset threshold. The multimodal tensor construction module is used to concatenate aligned multi-source motion temporal data, quality scores, validity masks, missing markers, and temporal location codes into a multimodal temporal feature tensor. A dedicated modality encoding module is used to extract feature representations corresponding to different modalities. The quality-aware cross-modal fusion module is used to calculate modal fusion weights and temporal fusion weights based on quality scores, validity masks, and missing data markers, and to generate joint training load features. The load state classification module is used to output the probability of training load state categories based on the joint training load characteristics; The dynamic weight training module is used to dynamically adjust the class weights based on the recent recognition difficulty of each training load class during the model training phase. The confidence output module is used to generate recognition confidence based on the training load state category probability, category interval, modality quality score, and attention weight distribution, and output the training load recognition result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the sports training load quality perception multimodal temporal neural network method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the sports training load quality perception multimodal temporal neural network method according to any one of claims 1-6.

Citation Information

Patent Citations

  • Non-intrusive power load decomposition method and system based on multi-modal feature learning

    CN120822062A

  • Personalized exercise training evaluation method based on multi-modal perception

    CN121862308A