A sports training recommendation method and system based on multi-modal data fusion

Abstract

This invention discloses a method and system for recommending exercise training based on multimodal data fusion. The method includes the following steps: obtaining the disease type and clinical characteristics of elderly disabled patients, fusing and constructing a role profile to obtain static features; generating and pushing the first exercise training plan based on the role profile; collecting structured data, text data, and image data during exercise training; inputting the data into a collaborative filtering model, combined with the role profile, to generate and push the next exercise training plan. Using this invention, individualized and targeted exercise training plans can be developed, significantly improving the cognitive, emotional, motor, and verbal abilities of elderly disabled patients.

Description

Technical Field

[0001] This invention relates to a method for recommending exercise training based on multimodal data fusion, and also to a corresponding exercise training recommendation system, belonging to the field of healthcare informatics technology. Background Technology

[0002] According to the dual-coding theory in cognitive theory, humans are generally considered to have two main information processing systems: a verbal system and a nonverbal system. These two systems differ in their encoding methods. Research has shown that the brain's memory for visual material is more effective and faster than for semantic memory, suggesting that the nonverbal system may have an advantage in processing image and spatial information. Although the two systems are functionally independent, they can also activate and influence each other.

[0003] In the field of cognition, there exists a theory called constructivism, which posits that knowledge is constructed through interaction between individuals and their environment. Through communication and collaboration with others, individuals can construct and revise their understanding; cognitive conflict arises when individuals encounter new information that conflicts with their existing knowledge. This conflict prompts individuals to re-evaluate and adjust their cognitive structures to adapt to the new information. Moreover, knowledge construction is a continuously evolving process. As individuals accumulate experience and develop cognitive abilities, their knowledge structures also continuously evolve. Knowledge is constructed within specific contexts; therefore, learning in real-world situations better facilitates the understanding and application of knowledge.

[0004] However, current treatments for elderly patients with disabilities do not include motor training methods that integrate information from both the verbal and nonverbal systems. Furthermore, individualized and targeted motor training programs are needed for individuals at risk of disability to improve or mitigate the decline in their cognitive, emotional, motor, and verbal abilities. Summary of the Invention

[0005] The primary technical problem to be solved by this invention is to provide a method for recommending exercise training based on multimodal data fusion.

[0006] Another technical problem to be solved by the present invention is to provide a motion training recommendation system based on multimodal data fusion.

[0007] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0008] According to a first aspect of the present invention, a motion training recommendation method based on multimodal data fusion is provided, comprising the following steps:

[0009] Step 1: For elderly disabled patients, obtain the patient's disease type and clinical characteristics, demographic information, motor survey information and motor behavior characteristics, in order to integrate and construct a role profile and obtain static characteristics;

[0010] Step 2: Based on the user profile, generate and push the first exercise training plan;

[0011] Step 3: Collect structured data, text data, and image data during exercise training;

[0012] Step 4: Input the data from Step 3 into the collaborative filtering model, combine it with the role profile, and generate and push the next sports training plan;

[0013] The collaborative filtering model employs multiple autoencoders and collaborative filtering networks. It uses graph embedding technology to map entities and relationships in the graph structure to a low-dimensional vector space. This is achieved through a Bayesian TransR embedding model, a Bayesian stacked denoising autoencoder, and a Bayesian stacked convolutional autoencoder. This transforms structured data, text data, and image data into structured vectors, text vectors, and image vectors, respectively, thereby obtaining item latent vectors and user latent vectors. These latent vectors are then used for collaborative filtering learning to recommend the next motion training scheme.

[0014] Preferably, step four includes the following sub-steps:

[0015] 1) Identify key variables as one-dimensional, two-dimensional, and three-dimensional data;

[0016] 2) Create a data matrix based on the key variables;

[0017] 3) Add hysteresis features as four-dimensional data;

[0018] 4) Transform it into a feature vector using embedding techniques;

[0019] 5) Based on the feature vectors, generate project latent vectors and user latent vectors for collaborative filtering learning to recommend sports training programs.

[0020] Preferably, in sub-step 4, entities and relations are converted into vectors using the Bayesian TransR knowledge graph embedding method; text data is converted into text vectors using the Bayesian sparse autoencoder embedding method; and image features are extracted using the Bayesian sparse convolutional autoencoder and converted into image vectors.

[0021] Preferably, the multimodal features and role profile features extracted based on the patient's condition are used as independent variable data, and the exercise prescription suggestions issued by the doctor in clinical practice are used as dependent variable, and are input into the collaborative filtering model.

[0022] The collaborative filtering model uses cosine contrast loss as its loss function, which maximizes the similarity between positive sample pairs while minimizing the similarity between negative sample pairs under margin constraints.

[0023] Preferably, in sub-step 1, an individual data space is constructed using one-dimensional data as the W-axis, two-dimensional data as the Y-axis, three-dimensional data as the X-axis, and four-dimensional data as the Z-axis.

[0024] Preferably, the one-dimensional data includes a time variable; the two-dimensional data includes multiple variables describing exercise preferences; the three-dimensional data includes variables describing physical and psychological characteristics; and the four-dimensional data includes post-training state data.

[0025] Preferably, when creating a data matrix based on key variables, different diseases correspond to different key variables; the role profile includes multiple attributes, including cognitive cortex damage, physical function damage, and emotional control damage attributes.

[0026] Preferably, the exercise training recommendation method further includes the following steps:

[0027] Step 5: Collect structured data, text data, and image data multiple times during the exercise training period, following the method in Step 3, to obtain exercise training cycle data;

[0028] Step Six: Determine if this exercise training plan is the Nth one. If yes, proceed to Step Seven; otherwise, return to Step Four and generate the next exercise training plan based on the features in Step Five; where N is a positive integer.

[0029] Step 7: Based on the data collected from the patient in Steps 3 and 5, input the data into the RRN model to iteratively optimize the exercise training program.

[0030] The RRN model is optimized using the following function:

[0031]

[0032] Where θ represents the parameter to be learned; This is a set of tuples observed in the training set, including patients, exercise training protocols, and time series; r ij |t represents the score of patient i for movement j at time t; For r ij |t represents the predicted value; R represents the regularization function.

[0033] Preferably, the RRN model employs an alternating subspace descent strategy, assuming the motion state is fixed, not propagating gradients to these motion training program sequences, while backpropagating gradients of all patient scores to update patient sequence parameters, and then switching between updating user sequences and updating motion training program sequences.

[0034] According to a second aspect of the present invention, a motion training recommendation system based on multimodal data fusion is provided, comprising a processor and a memory, wherein the memory is coupled to the processor and is used to store one or more programs, which, when executed by the processor, cause the processor to implement the motion training recommendation method based on multimodal data fusion.

[0035] Compared with the prior art, the present invention has the following beneficial technical effects:

[0036] 1) By using a fusion analysis system for multimodal data in collaborative filtering and autoencoder learning, we can ensure that information from multiple sources enters the recommendation model, increase the sources of analysis, and enable iterative motion push content based on the patient's own multidimensional motion data. This allows the training to be most suitable for the individual patient and the current state, thereby improving the training effect.

[0037] 2) Construct a training cycle data system, divide the cycle of the sports training process into annual cycle, large cycle, medium cycle and small cycle, create different time series datasets, and add time series dimension features when introducing the model for sports recommendation calculation, so as to make fuller and more regular use of historical sports data to assist in the generation of periodic sports recommendation schemes.

[0038] 3) Construct a multi-level patient profile system to progressively and deeply deconstruct and evaluate the patient's attribute characteristics, behavioral characteristics, and comprehensive characteristics (including but not limited to cognition, emotion, movement, speech, etc.) to ensure that the individual's labeling system has fine granularity, providing a large amount of effective information on patient sequences for subsequent input into the model for accurate movement recommendations;

[0039] 4) By combining the RRN model and using the recurrent network to effectively calculate time-series data, the data on patients' exercise at different time periods can be effectively backtracked, thereby ensuring the consistency and sustainability of exercise recommendations and improving patient compliance. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the motion training recommendation method based on multimodal data fusion in the first embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of multimodal data fusion in the first embodiment of the present invention;

[0042] Figure 3A This is a schematic diagram of the process of generating a motion training scheme based on a collaborative filtering model in the first embodiment of the present invention;

[0043] Figure 3B This is a schematic diagram of the individual data space in the first embodiment of the present invention;

[0044] Figure 3C This is a schematic diagram of the collaborative filtering model in the first embodiment of the present invention;

[0045] Figure 4 This is a flowchart illustrating the motion training recommendation method based on multimodal data fusion in the second embodiment of the present invention.

[0046] Figure 5 This is a schematic diagram of the structure of the RRN (Recurrent Neural Network) model in the second embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of the structure of a motion training recommendation system based on multimodal data fusion in the third embodiment of the present invention. Detailed Implementation

[0048] The technical content of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0049] The technical concept of this invention is to combine content-based recommendation methods with collaborative filtering models, thereby integrating patient or project feature learning and project recommendation processes into a unified framework. First, various deep learning models are used to learn the latent features of patients or projects, and a unified optimization function is constructed using collaborative filtering methods for parameter training. Then, the trained model is used to obtain the final latent vectors of patients and projects, thereby enabling the recommendation of exercise training programs for patients (mainly elderly and disabled patients).

[0050] First Embodiment

[0051] like Figure 1 As shown in Figure 3, the motion training recommendation method based on multimodal data fusion provided in the first embodiment of the present invention includes at least the following steps.

[0052] Step 1: For elderly disabled patients, obtain the patient's disease type and clinical characteristics, demographic information, motor survey information and motor behavior characteristics, in order to integrate and construct a role profile and obtain static characteristics.

[0053] Using a foundational model for exercise training recommendation based on multimodal data fusion, a core profile can be constructed based on the patient's disease type and clinical characteristics. A basic profile can be constructed using the patient's demographic information, exercise survey information, and exercise behavior characteristics. Finally, by fusing the core and basic profiles, a character profile of the patient is created. Static features are obtained during this process.

[0054] It should be noted that the construction method of the above basic model can be found in the prior patent application with application number 202411764001.3, entitled "Profile Construction Method, Exercise Training Recommendation Method and System for People at Risk of Disability", which is briefly described here:

[0055] This basic model was built using data from multiple patients, based on the following steps:

[0056] S1: Determine the key elements of the profiling based on the pathological mechanism;

[0057] S2: Based on key elements, construct a core profile;

[0058] S3: Construct a basic profile based on demographic information, exercise survey information, and exercise behavior characteristics;

[0059] S4: Construct character profiles based on the core profile set and the basic profile set.

[0060] Key elements include cognitive cortex impairment, physical function impairment, and emotional control impairment; the core profile of disabling diseases includes cognitive cortex impairment, physical function impairment, and emotional control impairment, with different attributes having different labels. Label data comes from patient diagnostic information or survey data. Exercise survey information includes exercise preferences, etc.

[0061] like Figure 2 As shown, damage to the cognitive cortex leads to cognitive impairment, manifesting in areas such as complex attention, executive function, learning and memory, and language. Impaired physical function leads to motor disorders, manifesting in areas such as muscle strength, muscle tone, coordination, and balance. Impaired emotional control leads to mental and behavioral disorders, manifesting in psychological aspects such as hallucinations / delusions or depression / anxiety, and behavioral aspects such as aggression / hoarding or depression / wandering. By evaluating these patients' health conditions (disorders or diseases), it is possible to uncover the patient's specific characteristics.

[0062] The label for type A cognitive cortical impairment includes at least:

[0063] A1. Frontal lobe injury: Patients may exhibit mental disorders such as memory loss and apathy, as well as symptoms such as epilepsy, monoplegia, motor aphasia, high fever, excessive sweating, and visual and olfactory abnormalities.

[0064] A2. Parietal lobe injury: Patients may have complex sensory disturbances, body image disturbances, and functional impairments such as calculation in the contralateral limbs.

[0065] A3. Temporal lobe injury: Patients may experience sensory aphasia, nominal aphasia, olfactory hallucinations, epilepsy, mental abnormalities, memory impairment, visual field changes, etc.

[0066] A4. Occipital lobe injury: Patients may experience visual impairment.

[0067] A5. Insular lobe damage: Patients may experience visceral motor and sensory disturbances such as increased salivation, nausea, and a feeling of fullness.

[0068] A6. Limbic lobe lesions: Patients may experience emotional and memory abnormalities, hallucinations and other mental abnormalities, as well as visceral dysfunction.

[0069] A7……

[0070] The label for Class B physical impairment includes at least:

[0071] B1. Headache

[0072] B2. Nausea or vomiting

[0073] B3. Fatigue or drowsiness

[0074] B4. Tremors, rigidity, and bradykinesia

[0075] B5. Dizziness or loss of balance

[0076] B6. Sensory problems, such as blurred vision, tinnitus, bad breath, or changes in smell.

[0077] B7. Sensitive to light or sound

[0078] B8……

[0079] The labels for Category C impairment of emotional control include at least:

[0080] C1. Mood changes or mood fluctuations

[0081] C2. Depression or anxiety

[0082] C3. Difficulty falling asleep

[0083] C4. More sleepy than usual

[0084] C5……

[0085] The core profile is constructed by quantifying and labeling its attribute features. The attributes of the core profile can correspond one-to-one with core key elements, or they can include more than just core key elements. Multiple tags for each attribute constitute a tag lexicon, with different tag lexicons representing different disease courses, injury types, and disability characteristics. In this step, the core profile tags for the patient are determined based on the patient's diagnosed disease type and clinical characteristics.

[0086] By obtaining demographic information, exercise survey information, exercise behavior characteristics, and social participation information through surveys or medical records, the patient's tags are obtained, and a basic profile is constructed.

[0087] Demographic information includes gender, age, and education level; sports survey information includes sports preferences and activity levels. Sports preferences include sports type (e.g., skipping rope, running), sports location (e.g., gym, park), and sports atmosphere (quiet, dynamic). Activity levels include BADL (Basic Activities of Daily Living) and IADL (Instrumental Activities of Daily Living).

[0088] Motor behavior characteristics include exercise frequency and exercise cycle.

[0089] Information on social participation includes participation activities within the family and participation activities in society. Internal family participation activities include family decision-making and child-rearing; social participation activities include work, study, and recreational activities.

[0090] While the basic profile is not a core factor influencing the occurrence of disability, as a fundamental characteristic, it has a synergistic effect on information in the core profile, such as the patient's disease course, injury type, and disability characteristics. Therefore, it is included in the profile system as a potential influencing variable and participates in the analysis.

[0091] The attributes of a basic portrait should include at least:

[0092] Natural attributes: Gender is a widely used tag, as people of different genders have significantly different preferences for different content; moreover, by using natural attribute tags such as age, region, education, occupation, marital status, and children, it is relatively easy to analyze the basic proportion of the patient population.

[0093] Vertical attributes: reflect the patient's exercise needs, such as preferences for different types of exercise;

[0094] Training attributes: Training attributes are also an important attribute category, which helps to determine the patient's exercise intention, exercise cycle, and exercise frequency;

[0095] Tag attributes: When a patient starts using the system and generates the first piece of data, the system can assign them the first tag - new user; subsequently, as the number of patients accumulates, low-frequency patients, active patients, and high-frequency patients can be gradually distinguished.

[0096] The basic attribute features are quantified and labeled to form a basic attribute profile set.

[0097] Through the preceding steps, a core profile set and a basic profile set have been built for each patient. Next, a persona needs to be constructed based on these core and basic profile sets.

[0098] The core profile and attribute profile are parallel. The core profile is the core characteristics of an individual, while the attribute profile is the basic characteristics of an individual. The role profile has a one-dimensional structure based on the basic attribute profile and a two-dimensional structure based on the core profile. As time progresses, the characteristics of the core profile at different points in time (different stages of disease development) form a three-dimensional structure. In this step, the profile sets are superimposed (union) by adding weight differences to form the role profile of the individual.

[0099] The profile sets built based on different features are merged according to predetermined rules. For example, different features are assigned different weights based on their importance, and then weighted summaries are performed to merge the features from the two profile sets. Here, the features in the core profile have a greater weight than the features in the base profile. Specifically, the weighting allocation needs to consider factors such as the feature coverage of each profile set, data consistency, and the rationality of the weighting allocation, to ensure that the combined profile accurately reflects the comprehensive characteristics of different patient groups and does not lose balance due to the overemphasis of certain features.

[0100] The basic user profile set is used as coarse-grained features for individuals, while the core user profile set is used as fine-grained features. These are then merged and overlaid. By merging features of different granularities, user characteristic information can be captured more comprehensively.

[0101] Coarse-grained features refer to broad, general features that provide general information about an individual or object. These features include basic attributes such as age, gender, and occupation, which are helpful in understanding the basic profile of an individual or patient. These features are primarily used in scenarios involving quickly locating an individual's overall profile and serving as basic data classification feature labels. These features aid in initial screening and classification, providing direction for subsequent in-depth analysis.

[0102] Fine-grained features refer to more specific and detailed characteristics, providing deeper information about individuals or objects. These features include specific behaviors, interests, habits, etc., which are crucial for understanding the uniqueness and personalized needs of individuals or objects. Fine-grained features are suitable for fields that require in-depth understanding of the uniqueness of individuals or objects, such as personalized recommendation systems, precision training, and behavioral feedback analysis. These features can capture subtle differences, providing more accurate computation and recommendation support.

[0103] Therefore, coarse-grained features provide general information about the user, while fine-grained features provide more specific details. This combination of coarse and fine-grained feature representation can improve the accuracy of profile mapping; the fine-grained feature selection module can extract key local and fine-grained discriminative features from the object, while the coarse-grained feature selection module can acquire coarse-grained diverse features that provide contextual information, improving the model's discriminative power; by combining features of different granularities, the model can better generalize to different scenarios and conditions.

[0104] Specifically, the basic profile set (coarse-grained features) and the core profile set (fine-grained features) are merged and overlaid. During this process, different weights need to be assigned to different features. When assigning different weights based on the importance of different features, the following conditions must be met: 1) the feature weights in the core profile are greater than the feature weights in the basic profile; 2) the feature coverage, data consistency, and reasonable weight allocation of each feature in the core profile are considered; 3) the feature coverage, data consistency, and reasonable weight allocation of each feature in the basic profile are also considered.

[0105] The core attribute features are quantified and labeled one by one to form a core attribute profile set.

[0106] A character profile is a three-dimensional mapping system that integrates basic and core profile features. The attributes of the core profile include at least cognitive cortex damage, physical function impairment, and emotional control impairment. Obtaining a character profile requires constructing a three-dimensional time slice based on the basic and core profiles. This three-dimensional time slice is obtained through the following steps:

[0107] First, the disease course is divided into stages. Based on the progression of different types of disability diseases in the elderly, the disease course is divided into several stages, including early, middle, and late stages. Each major stage can be further subdivided into more specific sub-stages to provide a more detailed understanding of the disease progression.

[0108] The next step is feature extraction. At each stage, key patient characteristics are extracted, including cognitive function, emotional state, and motor abilities. These characteristics can be measured using standardized assessment tools, such as the Mini Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA), and the Berg Balance Scale.

[0109] Finally, there's the construction of the timeline. A timeline starting from the onset of symptoms (e.g., every month, every two months, every year, etc.) is used to represent the progression of the disease over time. On the timeline (as a temporal dimension), the start and end points of each stage, as well as the key characteristics of each stage, can be marked. The timeline can be linear or non-linear to reflect the complexity and uncertainty of the disease course. Such a timeline helps to more intuitively understand the progression of the disease and provides guidance for treatment and intervention.

[0110] Based on industry consensus, the time-series dimension features in the role profile are quantified and then integrated. Knowledge graph information, including clinician experience, industry diagnostic standards, and expert consensus within the field, is incorporated. Disease progression stages are used as time-series dividing points, linking different disease stages, quantified clinical characteristics, and other dimensions of information in the role profile. Taking Alzheimer's disease (AD) as an example, according to industry consensus, disease progression can be divided into early stage (asymptomatic amyloidosis stage: SCD, MCI stage), intermediate stage (amyloid protein positive + synaptic dysfunction and / or neurodegenerative stage), and late stage (amyloid protein positive + evidence of neurodegenerative disease + severe cognitive decline). Different exercise training programs correspond to different disease stages. The content of the exercise training program is as follows: Figure 2 As shown, rehabilitation includes cognitive function rehabilitation, motor function rehabilitation, rehabilitation of mental and behavioral symptoms, activity and participation rehabilitation, and comprehensive rehabilitation.

[0111] By linking different disease progression stages, quantified clinical characteristics, and other dimensions of information in the role profile, we can use these as multi-dimensional profile features to input the recommended exercise training programs, thus preparing for the delivery of precise exercise training.

[0112] Step 2: Based on the user profile, generate and push the first exercise training plan.

[0113] Based on the patient's profile, an initial exercise training plan is automatically generated. This initial exercise training plan can be generated based on a pre-trained model (such as the collaborative filtering model mentioned later) or designed based on the doctor's experience.

[0114] Step 3: Collect structured data, text data, and image data during exercise training.

[0115] Structured data, text data, and image data of the patient's exercise according to the initial exercise training plan were collected. These structured data, text data, and image data (video or image-based data) were then used as independent variables input to the collaborative filtering model. Specifically, this included, but was not limited to:

[0116] ■ Structured data: motor diagnostic scales, daily living skills assessment scales, demographic data, etc.;

[0117] Text-based data: Clinical diagnosis and case report data from the past two years (including patient motor diagnosis data obtained through medical records, laboratory reports, etc., such as upper limb motor function injury);

[0118] ■Image data: MRI data, CT image data, etc.;

[0119] Step 4: Input the data from Step 3 into the collaborative filtering model, combine it with the role profile, and generate and push the next sports training plan.

[0120] In this embodiment, step four can be repeated until the desired effect is achieved or the patient stops training.

[0121] On one hand, the collaborative filtering model employs multiple autoencoders and collaborative filtering networks, using graph embedding techniques to map entities (such as patients, movement types, physiological indicators, etc.) and relationships in the graph structure to a low-dimensional vector space. This step is achieved through deep learning embedding techniques such as the Bayesian TransR embedding model, the Bayesian stacked denoising autoencoder (SDAE), and the Bayesian stacked convolutional autoencoder (SCAE), transforming structured data, text data, and image data into structured vectors, text vectors, and image vectors, respectively.

[0122] Specifically, an autoencoder is a layer-by-layer unsupervised learning model that mainly consists of two processes: decoding and encoding. It is used to process high-dimensional data as a feature extraction tool, enabling collaborative filtering models to learn low-dimensional representations (latent vectors) of users and items. These latent vectors can capture the inherent relationships between users and items, thereby improving the performance of the recommendation system. Here, the autoencoder fuses three types of information (structured data, textual data, and image data) to achieve multimodal information fusion analysis and obtain item latent vectors. Furthermore, based on user profile data, the autoencoder uses feature extraction to obtain user latent vectors.

[0123] On the other hand, to fully learn auxiliary information features, the collaborative filtering model employs a Bayesian stacked denoising autoencoder (SDAE) to learn the vector representation of text information, a Bayesian stacked convolutional autoencoder (SCAE) to learn the vector representation of image information, and a Bayesian TransR embedding model to learn the vector representation of structural information. SDAE is a deep learning model that extracts high-level features from data through multi-layer unsupervised learning, capturing both semantic and structural information in text, which is particularly useful for handling noise and uncertainty in text data. SCAE is used to learn the vector representation of image information because convolutional neural networks (CNNs) are advantageous in processing image data, capable of capturing the spatial hierarchy and local features of images. By stacking multiple convolutional and pooling layers, SCAE can learn more abstract and high-level image features, helping the model make more robust predictions when faced with the diversity and complexity of image data.

[0124] TransR is an embedding model specifically designed for knowledge graphs, effectively handling complex relationships within them. It maps entities and relationships to different semantic spaces and uses relationship-specific transformation matrices to project entities from the entity space to the relationship space, performing vector operations in the projected space for more accurate knowledge graph completion. This design not only captures multidimensional relationships between entities but also effectively handles incomplete information and noise in knowledge graphs. Furthermore, TransR's flexibility and scalability allow it to adapt to different structured data, improving accuracy and robustness when processing structured data. By transforming text, images, and structured data into a unified vector representation, TransR ensures comparability of data from different sources in the feature space, thus providing strong support for the efficient representation and application of knowledge graphs.

[0125] Following feature extraction, graph embedding techniques facilitate the construction of an individual data space, integrating multi-dimensional data such as time, movement preferences, physiological and psychological characteristics, and post-training states to form a four-dimensional data structure. Furthermore, it integrates structural vectors, text vectors, and image vectors to form item latent vectors. These item latent vectors combine features from multiple data types, providing a more comprehensive representation of the item's characteristics.

[0126] Specifically, such as Figure 3A As shown, the process of generating and pushing the next motion training plan using a collaborative filtering model includes the following sub-steps:

[0127] 1) Identify key variables as one-dimensional, two-dimensional, and three-dimensional data.

[0128] First, it is necessary to identify which variables are important for assessing and predicting a patient's rehabilitation process. Combining cognitive tasks (such as memory and attention tasks) with motor tasks (such as walking and balance training) can improve a patient's motor function and balance. Through exercise, cerebral blood flow and oxygen levels improve, dormant neurons are activated, neuronal growth and connections are promoted, specific neural conduction patterns are formed, and neurotransmitter levels are regulated. Meanwhile, improved cognitive function helps enhance a patient's motor skills and efficiency, strengthens motivation and self-control (the ability to control emotions, speech, etc.), and optimizes motor strategies.

[0129] In summary, different key independent variables need to be identified for patients with different diseases (different brain regions are affected).

[0130] For all patients with cognitive disorders, important common key independent variables include:

[0131] Time (T)

[0132] Type of Exercise

[0133] Exercise intensity

[0134] Duration

[0135] Frequency

[0136] Patient Status

[0137] Physiological indicators (such as heart rate, blood pressure, etc.)

[0138] Psychological indicators (such as anxiety level, depression level, etc.)

[0139] Recovery Progress

[0140] Different key variables are identified for patients with different diseases, including, for example:

[0141] AD patients:

[0142] The MRI ROI (Region of Interest) markers for brain injury indicate damage to the hippocampus, medial temporal lobe, parietal lobe, and frontal lobe.

[0143] Cognitive Ability Scores include memory loss, spatial disorientation, inattention, and executive function impairment.

[0144] PD patients:

[0145] The brain injury MRI ROI (Region of Interest) indicators showed damage to the substantia nigra, striatum, prefrontal cortex, and parietal lobe;

[0146] Activities of Daily Living Indicators (ADLs) include motor impairments, cognitive decline, and sensory abnormalities.

[0147] Hormone Level Score, dopamine level, etc.

[0148] 2) Create a data matrix based on key variables.

[0149] Create a data matrix based on the key variables for each patient, where each row represents a data record at a given time point and each column represents a variable. For example:

[0150] One-dimensional data: [Time (T)]

[0151] Two-dimensional data (exercise preferences): [Type of Exercise]

[0152] Exercise intensity

[0153] Duration

[0154] [Frequency]

[0155] Three-dimensional data (physiological and psychological characteristics): [Heart Rate]

[0156] Blood pressure

[0157] Anxiety Level

[0158] Depression Level

[0159]

[0160] 3) Add hysteresis features as four-dimensional data.

[0161] To capture dependencies in time series data, lag features can be added.

[0162] Four dimensions: [Patient Status and Recovery Progress after Training], for example:

[0163] time Patient status after training Rehabilitation progress T1 (e.g., 10:00 AM on October 3, 2024) good 5％ T2 (e.g., 3 PM on October 3, 2024) good 5％ T3 (e.g., 10:00 AM on October 4, 2024) good 5％ T4 (e.g., 3 PM on October 4, 2024) improve 10％ ... ... ...

[0164] Based on the aforementioned one-dimensional, two-dimensional, three-dimensional, and four-dimensional data, a four-dimensional data structure is formed. That is, the one-dimensional data (time) is used as the W-axis; the two-dimensional data is used as the Y-axis; the three-dimensional data is used as the X-axis; and the four-dimensional data is used as the Z-axis, thus constructing an individual data space.

[0165] 4) Use embedding techniques to convert it into feature vectors.

[0166] By utilizing the hidden layers of deep learning models for internal processing and fusion, four-dimensional data is transformed into feature vector representations through embedding techniques.

[0167] like Figures 3A to 3C As shown, structured embedding transforms the structured data in each patient's individual data space into structured vectors. For example, using the Bayesian TransR knowledge graph embedding method, entities and relations can be converted into vectors that capture the complex relationships between entities.

[0168] Text embedding transforms textual data into text vectors. For example, Bayesian stacked denoising autoencoders (SDAEs) can learn low-dimensional representations of textual data and capture semantic information of the text.

[0169] Image embedding transforms image-like data into image vectors. For example, Bayesian stacked convolutional autoencoders (SCAEs) can extract image features and convert them into vectors.

[0170] Using the methods provided by TransR, SDAE, and SCAE, structural vectors, text vectors, and image vectors are integrated into an item latent vector. This item latent vector is a vector that integrates features from multiple data types, can more comprehensively represent the characteristics of different types of patients, and is a vector that can represent item features.

[0171] 5) Based on the feature vectors, generate project latent vectors and user latent vectors for collaborative filtering learning to recommend sports training programs.

[0172] The Attribute Fusion Graph Convolutional Network (AF-GCN) is used to fuse feature vectors from different modalities, and an attention mechanism is used to weight the importance of different modalities, thus progressively implicitly vectorizing the feature vectors layer by layer.

[0173] Therefore, this embodiment fuses data from different sources (such as text, images, etc.), constructing an individual data space with one-dimensional data as the W-axis, two-dimensional data as the Y-axis, three-dimensional data as the X-axis, and four-dimensional data as the Z-axis. The one-dimensional data includes a time variable, the two-dimensional data includes multiple variables describing exercise preferences, the three-dimensional data includes variables describing physical and psychological characteristics, and the four-dimensional data includes post-training state data. Furthermore, deep learning models are used to extract features from each data type, combining these features with knowledge representations to form a unified feature space. This allows for a better understanding of the patient's interests and needs, achieving multimodal information feature-level fusion (fusion performed after feature extraction and before decision-making). Moreover, deep learning models (such as SDAE) are used to transform information from the patient's (user's) role profile (behavior, preferences, ratings of different training programs, etc.) into user latent vectors, which are vectors representing user characteristics. These vectors, along with item latent vectors, are input into a collaborative filtering network to calculate the user's preferences for different items and generate the final recommendation scheme (exercise training scheme).

[0174] This embodiment employs a collaborative filtering model that combines graph structures and deep learning techniques. The graph structure is used to represent the knowledge base, which includes entities (such as motion categories, user attributes, and user behaviors) and their relationships. Therefore, the graph structure can be used to capture interaction data between users and items, and to learn user preferences and item characteristics.

[0175] The multimodal features (item latent vectors) and role profile features (user latent vectors) extracted based on the patient's condition are used as independent variables, and the first exercise training plan from the previous step (e.g., an exercise prescription issued by a doctor) is used as the dependent variable. These are input into the collaborative filtering model to generate the next exercise training plan. The loss function of the collaborative filtering model is cosine contrastive loss, which is used to maximize the similarity between positive sample pairs while minimizing the similarity between negative sample pairs under margin constraints.

[0176] As mentioned above, graph embedding technology maps each entity and relation in the graph structure of the individual data space to a low-dimensional vector space, which makes them computable and comparable. This enables collaborative filtering models to analyze interaction data between users and items, capture complex nonlinear relationships between patients and items, learn user preferences and item characteristics, and thus predict items that users may be interested in.

[0177] Collaborative filtering models inherently suffer from a cold start problem. In other words, for new patients, due to a lack of historical behavioral data, it's impossible to calculate patient similarity using conventional collaborative filtering models, making personalized recommendations difficult. This results in the model struggling to provide accurate personalized recommendations when first introduced into hospitals. However, for structured data, image data, and text data, collaborative filtering models can predict the preferences of new users or projects by analyzing the behavior of other similar patients or projects. It can quickly recommend exercise training programs for people with similar characteristics (cold start function) using only a small amount of clinically authoritative diagnostic exercise training program data, without requiring extensive training. This is because the collaborative filtering model provided in this embodiment of the invention easily finds similar patients and similar projects based on the new patient's role profile data and multimodal data.

[0178] In other words, the collaborative filtering model provided in this invention leverages the advantages of role profiling and multimodal data fusion. For example, patients with cognitive cortex damage will show abnormal signals in specific brain regions in their MRI images, and similar cognitive dysfunction will be mentioned in clinical diagnoses. Multimodal data corresponding to cognitive cortex damage and physical function impairment show high feature similarity, allowing the model to identify commonalities and regularities in key features across different patients, thereby improving the accuracy of feature similarity judgment. For another example, although two patients may differ in their emotional control impairment, if they are highly similar in both cognitive cortex damage and physical function impairment, they are more likely to be classified as similar patients when recommending exercise training programs, thus enabling the model to recommend suitable exercise training programs for new patients. Utilizing fine-grained core profiles allows the model to accurately grasp the key needs of new patients and identify similar patients and programs.

[0179] Based on the rich information in the character profile, matching and filtering are performed at multiple levels to identify patient groups similar to new patients across multiple dimensions. This allows for the recommendation of exercise training programs more aligned with the new patients' comprehensive characteristics, enhancing the personalization of the recommendations. For example, for patients with severe cognitive cortex damage, exercise training programs focusing on cognitive training and brain function activation are emphasized; while for patients with significant physical functional impairment, training emphasizing the recovery and enhancement of limb motor function is prioritized. This personalized recommendation approach, compared to traditional collaborative filtering models that rely solely on historical behavioral data, better addresses the cold-start problem, providing new patients with accurate and effective exercise training program recommendations to help improve their cognitive, emotional, motor, and verbal abilities.

[0180] Furthermore, based on basic profiles such as medical history and demographic characteristics, and core profiles such as examination results and doctor's recommendations, the generated patient state sequence data, in the process of fusion with different modal data such as imaging data (image data) and scale data (text data), the collaborative filtering model can more fully grasp the relationship between different media modalities through multi-layer attribute fusion, thereby improving the accuracy and generalization ability of exercise training program recommendations.

[0181] Second Embodiment

[0182] Based on the first embodiment, this embodiment combines the recurrent neural network model (i.e., the RRN model) with the collaborative filtering model in the first embodiment to further improve the coherence and sustainability of exercise training program recommendations and enhance patient compliance.

[0183] like Figure 4 The motion training recommendation method based on multimodal data fusion provided in the second embodiment of the present invention includes at least the following steps:

[0184] Steps one through four are the same as in the first embodiment (but step four does not need to be repeated multiple times), and will not be described in detail here.

[0185] Step 5: Collect structured data, text data, and image data multiple times during the exercise training period, following the method in Step 3, to obtain exercise training cycle data.

[0186] The patient exercises according to the exercise training plan multiple times. By collecting data (structured data, text data, and image data) from these multiple exercises in accordance with step three, the exercise training cycle data can be obtained.

[0187] Training cycle data in sports training programs are generally divided into: multi-year cycle data, long-term cycle data, medium-term cycle data, and short-term cycle data.

[0188] Multi-year cycle: Due to the large individual differences among the elderly, specific problems need to be analyzed on a case-by-case basis. For example, (1) each elderly person's physical condition and recovery ability are different, so the exercise rehabilitation cycle will vary from person to person. Generally speaking, elderly people with better physical condition and less disability may recover faster, and a cycle of 2 years can be used. (2) The more severe the disability, the longer the rehabilitation time usually takes. The cycle for elderly people who are completely bedridden is 8 years, and the cycle for elderly people who can take care of themselves partially is 4 years. During the rehabilitation process, it is necessary to continuously monitor the elderly person's physical condition and rehabilitation progress, and make adjustments as needed. This helps to ensure the effectiveness of the rehabilitation plan and shorten the rehabilitation cycle as much as possible.

[0189] A major cycle is generally defined by the current year or a full year, and training plans are formulated accordingly for six months or the entire year. Some years are divided into three major cycles, which is generally related to the classification of sports.

[0190] In training practice, the intermediate cycle is referred to as phase or monthly training, which generally lasts 4 to 8 weeks, and a phase or monthly training plan is developed accordingly.

[0191] In training practice, a short cycle is called a weekly training cycle, which is usually based on a calendar week to develop a weekly training plan. Short cycles of 4 to 10 days can also be arranged. For example, if a small recovery adjustment can be completed in 4 days, then a 4-day recovery cycle can be arranged.

[0192] Step Six: Determine if this exercise training plan is the Nth one. If so, proceed to Step Seven; otherwise, return to Step Four and generate the next exercise training plan based on the features in Step Five.

[0193] N represents the preset number of repetitions, such as 3 times. This step allows us to obtain the dynamic characteristics of the patient during training according to multiple exercise programs, and these characteristics possess a temporal dimension. This design facilitates the RRN model's learning of more complex dynamic evolutionary representations.

[0194] Step 7: Based on the data collected from the patient in Steps 3 and 5, input the data into the RRN model to iteratively optimize the exercise training program.

[0195] In this step, use as follows Figure 5 The RRN model shown can better learn more complex dynamic evolutionary representations. In the RRN model, the historical interaction information between the patient and movement is the key data driving changes in patient preferences and movement states. Therefore, using a co-evolutionary model can capture the evolutionary latent representations of the patient and movement. Here, using the training cycle vectors input to the RRN model, the RRN model uses a recurrent neural network to learn the dynamic feature representations of the patient and movement, obtaining a dynamic evolutionary representation with temporal relationships. The training cycle vector refers to the vector obtained by converting the various attributes (such as time, frequency, intensity, etc.) in the training cycle data into numerical features, and then transforming them through embedding or encoding techniques.

[0196] like Figure 5 As shown, the RRN model uses two long short-term memory networks (LSTM) on the basis of traditional recurrent neural networks (RNN) to learn the temporal changes of patients' exercise preferences and the long-term (e.g., seasonal) evolution of exercise, respectively. Figure 5 In the middle, y is learned through an LSTM network. i y represents the dynamic feature vector of patient i over time (times t, t+1, etc.); y is learned through another LSTM network. j, which represents the dynamic feature vector of motion j over time. Furthermore, y i and y j All are affected by the patient's static characteristics u i and the static characteristics of motion m j The influence of this is to take into account the patient's long-term movement preferences and the static properties of movement, so that the RRN model can learn both the patient's static latent representation and the static latent representation of movement simultaneously.

[0197] In other words, at each time step, the RRN model uses one LSTM network to update the dynamic feature vector by combining the patient's dynamic feature vector from the previous time step and the current static feature vector; it also uses another LSTM network to update the dynamic feature vector by combining the dynamic feature vector from the previous time step and the current static feature vector. This RRN model, which combines static and dynamic features, can simultaneously consider the patient's long-term exercise preferences and short-term changes, as well as the long-term characteristics and short-term changes of the exercise training program.

[0198] Specifically, the RRN model uses an LSTM-based recurrent neural network to model the dynamic changes in the patient and the exercise training program. For patient i and exercise j (in an exercise training program, multiple exercises are included), assuming u i Let m represent the static feature vector of patient i. j Let u represent the static eigenvector of motion j. At time t, assume u it It is the dynamic feature vector of patient i (i.e., y in the figure). i，t ), m jt It is the dynamic feature vector of motion j (i.e., y in the figure). j，t The dynamic characteristics u at time t+1 i，t+1 and m j,t+1 The solution can be obtained by serializing the data using a separate LSTM network, as follows:

[0199] u i,t+1 =g(u it , {r ij|t})

[0200] m j,t+1 =h(m jt , {r ij|t})

[0201] Where g and h are functions to be learned, g represents the function used in the LSTM network to update the dynamic features of the patient; h represents the function used in the LSTM network to update the dynamic features of the exercise training program; r ij|t This represents the score of patient i for motion j at time t+1. The dynamic characteristics u of the patient at time t+1. i，t+1 It uses an LSTM network to analyze the dynamic characteristics of the patient at time t.i， t and response r ij|t The calculated dynamic characteristics m of the motion at time t+1. j，t+1 It uses an LSTM network to determine the dynamic characteristics m of the motion at time t. j，t and response r ij|t The calculations show that the dynamic characteristics of the patient and the exercise training program are updated by the LSTM network based on their state at the previous time point and their current response. Therefore, the RRN model is able to capture the dynamic characteristics that change over time, thus better simulating and predicting the patient's response to different exercise training programs.

[0202] Input the dynamic and static features of patient i (u i,t+1 m j，t+1 u it m jt The model can then predict the patient's current interest (r). ij Predicted value Here, f is also a function that needs to be learned.

[0203] The loss function of the RRN model is the sum of the squared error loss function and the regularization term. Adjusting the parameter θ to minimize the loss function achieves overall optimization of the RRN model. That is, optimization can be performed using the following function:

[0204]

[0205] Therefore, the goal of RRN model optimization is to make the exercise training programs pushed by the model increasingly match the patient's current exercise preferences, exercise level, and exercise state; that is, to make the predictions closer to reality through the parameters generated by training. Here, θ represents the parameters to be learned. R represents the set of (patients, exercise training programs, time series) tuples observed in the training set, and R denotes the regularization function.

[0206] While the objective function and building blocks in the RRN model are quite standard, a simple application of backpropagation cannot easily solve this optimization problem. The key challenge is that each patient's score depends on both the patient's state and the exercise training protocol. Backpropagation through two sequences is computationally forbidden. This problem is mitigated by backpropagating gradients from the patient's motion feedback; however, each score still depends on the patient's state, which in turn affects the entire sequence pushed by the exercise training protocol.

[0207] Therefore, this embodiment of the invention employs an alternating subspace descent strategy. That is, assuming the motion state is fixed, there is no need to propagate gradients to these motion training sequence sequences. Instead, gradients from all patient scores are backpropagated to update patient sequence parameters, and then the process switches between updating the user sequence and updating the motion training sequence. In this way, standard feedforward and backpropagation are performed only once for each patient's motion training sequence push, ultimately achieving model optimization.

[0208] In summary, this invention employs a hybrid recommendation model (collaborative filtering and RRN models) as the core tool for recommending exercise training. Compared to traditional content recommendation models, the collaborative filtering model in the hybrid model can simultaneously learn representations of multi-source data (structured data, textual data, image data, and training cycle data), utilizing multi-dimensional information including structural, content, and image information. Simultaneously, the recurrent neural network in the hybrid model captures both the patient's long-term preference evolution and short-term preference representations. Therefore, by integrating different models, it is possible to effectively assess the exercise dosage, exercise status, and exercise patterns of individual elderly disabled patients, ultimately leading to the delivery of the optimal exercise training program.

[0209] During each training session, data collected from the previous exercise training session is used to generate a new exercise training plan in step seven. This process is repeated until the predetermined goals are achieved, such as the recovery or maintenance of cognitive, emotional, motor, and verbal abilities.

[0210] As an optional step, step eight can be added after step seven: evaluate the training effect based on the data from the previous exercise training plan. If the effect improves, generate and output an exercise training report or end the process directly; if the effect does not improve, return to step seven to regenerate the exercise training plan. The method for evaluating the training effect can be a conventional evaluation method, including evaluation by a doctor, or comparing and evaluating the data from the last exercise training plan with the data from the previous exercise training plan.

[0211] Compared with the prior art, the present invention has the following technical advantages:

[0212] 1) Construct a multi-level patient profile system, and gradually and deeply deconstruct and evaluate the patient's attribute characteristics, behavioral characteristics and comprehensive characteristics to ensure that the individual labeling system has fine granularity, providing a large amount of effective information on patient sequences for subsequent input into the model for accurate motion recommendation;

[0213] 2) An exercise recommendation framework that integrates the algorithmic advantages of multiple models. On the one hand, it uses a fusion analysis system for multimodal data from collaborative filtering and autoencoder learning to ensure that information from multiple sources enters the recommendation model and increases the sources of analysis; on the other hand, it uses a recurrent network to effectively compute temporal data and effectively backtrack the exercise data of patients at different time periods, thereby ensuring the coherence and sustainability of exercise recommendations and improving patient compliance.

[0214] 3) Construct a training cycle data system, divide the cycle of the sports training process into annual cycle, large cycle, medium cycle and small cycle, create different time series datasets, and add time series dimension features when introducing the model for sports recommendation calculation, so as to make fuller and more regular use of historical sports data to assist in the generation of periodic sports recommendation schemes.

[0215] 4) Construct a deep learning-based exercise recommendation system, and build the multi-dimensional exercise training recommendation logic into a system process. This system can continuously recommend exercise content based on the patient's own multi-dimensional exercise data, so that the training can be most suitable for the individual patient and the current state, thereby improving the training effect.

[0216] Third Embodiment

[0217] Based on the aforementioned motion training recommendation method based on multimodal data fusion, the third embodiment of this invention further provides a motion training recommendation system based on multimodal data fusion. For example... Figure 6 As shown, the exercise training recommendation system includes a processor and a memory. The memory is coupled to the processor and stores one or more programs. When these programs are executed by the processor, the processor implements the exercise training recommendation method based on multimodal data fusion as described in the above embodiment.

[0218] The processor controls the overall operation of the exercise training recommendation system to complete all or part of the steps of the aforementioned multimodal data fusion-based exercise training recommendation method. The processor can be a central processing unit (CPU), graphics processing unit (GPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processing (DSP) chip, etc. The memory stores various types of data to support the operation of the exercise training recommendation system. This data may include, for example, instructions for any application or method operating on the exercise training recommendation system, as well as application-related data. The memory can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, etc.

[0219] In another exemplary embodiment, the present invention also provides a computer-readable storage medium including program instructions, which, when executed by a processor, implement the steps of the motion training recommendation method based on multimodal data fusion in any of the above embodiments. For example, the computer-readable storage medium may be the memory including the program instructions, which may be executed by the system's processor to complete the motion training recommendation method based on multimodal data fusion and achieve the same technical effects as the method described above.

[0220] It should be noted that the above embodiments are merely illustrative examples. The technical solutions of the various embodiments can be combined, and the order of the steps can be changed, all of which are within the protection scope of this invention.

[0221] The above provides a detailed description of the motion training recommendation method and system based on multimodal data fusion provided by this invention. Any obvious modifications made by those skilled in the art without departing from the essence of this invention will constitute an infringement of the patent rights of this invention and will incur corresponding legal liability.

Claims

1. A motion training recommendation method based on multimodal data fusion, characterized in that... Includes the following steps: Step 1: For elderly disabled patients, obtain the patient's disease type and clinical characteristics, demographic information, motor survey information and motor behavior characteristics, in order to integrate and construct a role profile and obtain static characteristics; Step 2: Based on the user profile, generate and push the first exercise training plan; Step 3: Collect structured data, text data, and image data during exercise training; Step 4: Input the data from Step 3 into the collaborative filtering model, combine it with the character profile, to generate and push the next exercise training plan; wherein, the collaborative filtering model adopts multiple autoencoders and collaborative filtering networks, and uses graph embedding technology to map entities and relations in the graph structure to a low-dimensional vector space, implemented through Bayesian TransR embedding model, Bayesian stacked denoising autoencoder, and Bayesian stacked convolutional autoencoder, to transform structured data, text data, and image data into structure vectors, text vectors, and image vectors respectively, thereby obtaining item latent vectors and user latent vectors, which are used for collaborative filtering learning to recommend the next exercise training plan; Step 5: Collect structured data, text data, and image data multiple times during the exercise training period, following the method in Step 3, to obtain exercise training cycle data; Step Six: Determine if this exercise training plan is the Nth one. If yes, proceed to Step Seven; otherwise, return to Step Four and generate the next exercise training plan based on the features in Step Five; where N is a positive integer. Step 7: Based on the data collected from the patient in Steps 3 and 5, input the data into the RRN model to iteratively optimize the exercise training program. The RRN model is optimized using the following function: in, This represents the parameters to be learned; The training set is a collection of tuples observed in the training set, which includes patients, exercise training programs, and time series. Indicates the patient In time For sports The rating; for The predicted value; This represents the regularization function.

2. The motion training recommendation method based on multimodal data fusion as described in claim 1, characterized in that... Step four includes the following sub-steps: 1) Identify key variables as one-dimensional, two-dimensional, and three-dimensional data; 2) Create a data matrix based on the key variables; 3) Add hysteresis features as four-dimensional data; 4) Transform it into a feature vector using embedding techniques; 5) Based on the feature vectors, generate project latent vectors and user latent vectors for collaborative filtering learning to recommend sports training programs.

3. The motion training recommendation method based on multimodal data fusion as described in claim 2, characterized in that: In sub-step 4, entities and relations are converted into vectors using the Bayesian TransR knowledge graph embedding method; text data is converted into text vectors using the Bayesian sparse autoencoder embedding method; and image features are extracted using the Bayesian sparse convolutional autoencoder and converted into image vectors.

4. The motion training recommendation method based on multimodal data fusion as described in claim 3, characterized in that: The multimodal features and role profile features extracted based on the patient's condition are used as independent variables, and the exercise prescription suggestions issued by the clinic are used as dependent variables, which are then input into the collaborative filtering model. The loss function of the collaborative filtering model is cosine contrastive loss, which is used to maximize the similarity between positive sample pairs while minimizing the similarity between negative sample pairs under the margin constraint.

5. The motion training recommendation method based on multimodal data fusion as described in claim 4, characterized in that: In sub-step 1, an individual data space is constructed using one-dimensional data as the W-axis, two-dimensional data as the Y-axis, three-dimensional data as the X-axis, and four-dimensional data as the Z-axis.

6. The motion training recommendation method based on multimodal data fusion as described in claim 5, characterized in that: The one-dimensional data includes a time variable; the two-dimensional data includes multiple variables describing exercise preferences; the three-dimensional data includes variables describing physical and psychological characteristics; and the four-dimensional data includes post-training state data.

7. The motion training recommendation method based on multimodal data fusion as described in claim 6, characterized in that: When creating a data matrix based on key variables, different diseases correspond to different key variables; the role profile includes multiple attributes, including cognitive cortex damage, physical function damage, and emotional control damage.

8. The motion training recommendation method based on multimodal data fusion as described in claim 7, characterized in that: The RRN model employs an alternating subspace descent strategy, assuming that the motion state is fixed. It does not propagate gradients to these motion training program sequences, but instead backpropagates gradients of all patient scores to update patient sequence parameters. Then, it switches between updating the user sequence and updating the motion training program sequence.

9. A motion training recommendation system based on multimodal data fusion, characterized in that... The system includes a processor and a memory, wherein the memory is coupled to the processor for storing one or more programs that, when executed by the processor, cause the processor to implement the motion training recommendation method based on multimodal data fusion as described in any one of claims 1 to 8.

Images