Athletic and cognitive ability assessment method, system and device

Abstract

The invention discloses an exercise and cognition ability assessment method, system and device, and the method comprises the steps: obtaining real-time exercise information, including real-time joint angle information, real-time gait information, real-time attention information and real-time decision speed information, of a user; constructing a physiological-behavior coupling model based on a hidden Markov model, and generating a joint evaluation index by inputting real-time motion information; performing feature extraction on the real-time motion information by using a hierarchical feature extraction model to obtain a motion feature vector and a cognitive feature vector; performing fusion processing on the feature vector and the joint evaluation index, and outputting abnormal state information including motion abnormality and cognitive abnormality; behavior abnormity is detected through the dynamic threshold model, a real-time feedback instruction is generated in combination with a preset feedback rule, and finally user training task parameters are dynamically adjusted according to the instruction. According to the method, through multi-dimensional data fusion and a dynamic evaluation mechanism, cooperative monitoring and self-adaptive training regulation and control of motion and cognitive competence are realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system and device for evaluating motor and cognitive abilities. Background Art

[0002] Traditional rehabilitation training relies on hospitals or professional institutions. Due to the limitations of venues, equipment and personnel, it is difficult for patients to participate frequently and continuously. During home stays, due to the lack of real-time monitoring and guidance, training efficiency is low and even functional impairments are aggravated by incorrect movements. Existing technologies mostly use single-dimensional assessments (such as monitoring only gait or attention), which cannot simultaneously capture motor-cognitive coordination disorders (such as the correlation between limb compensation and slow decision-making), and rely on fixed thresholds or general assessment models, which are difficult to adapt to the individual needs of different rehabilitation stages (such as acute phase and recovery phase). In addition, traditional systems often require wearing complex sensors or connecting professional instruments, which cannot be easily deployed at home, and the feedback mechanism is lagging, and abnormal postures or cognitive biases cannot be corrected immediately during training. This can easily lead to patients' self-training at home. The rehabilitation effect is often affected due to the failure to correct movement errors in time, and even secondary injuries. At the same time, the isolation of cognitive training also hinders the reconstruction of motor and neural function. Summary of the invention

[0003] In view of the above problems, the present invention provides a home rehabilitation technology that can integrate multimodal data, dynamically evaluate the motor-cognitive coupling state and regulate training parameters in real time.

[0004] To achieve the above objectives, in a first aspect, the present application provides a method for evaluating motor and cognitive abilities, comprising:

[0005] Acquire the user's real-time motion information, which includes real-time motion data and real-time cognitive data. The real-time motion data includes real-time joint angle information and real-time gait information. The real-time cognitive data includes real-time attention information and real-time decision speed information.

[0006] A physiological-behavioral coupling model is constructed. The physiological-behavioral coupling model is configured to be generated based on a hidden Markov model. Real-time motion information is input into the physiological-behavioral coupling model to obtain joint evaluation indicators. The joint evaluation indicators include a motion complexity curve and a cognitive load index time series.

[0007] Constructing a hierarchical feature extraction model, the hierarchical feature extraction model includes a feature extraction unit and a feature fusion unit, inputting real-time motion information into the feature extraction unit to obtain a motion feature vector and a cognitive feature vector;

[0008] The motion feature vector, the cognitive feature vector and the joint evaluation index are input into the feature fusion unit to obtain abnormal state information, where the abnormal state information includes motion abnormality and cognitive abnormality;

[0009] Input abnormal status information into the dynamic threshold model to detect abnormal user behavior, and generate real-time feedback instructions through preset feedback rules;

[0010] According to the real-time feedback instructions, the user's training task parameters are adjusted in real time.

[0011] In some embodiments, a physiological-behavioral coupling model is constructed, and the physiological-behavioral coupling model is configured to generate the following based on a hidden Markov model:

[0012] Acquire sample motion information, where the sample motion information includes sample motion data and sample cognitive data;

[0013] The sample motion information is subjected to kernel density estimation to obtain joint probability distribution information, which includes the physiological-behavioral pattern of the user at a specific time point;

[0014] Mapping the joint probability distribution information to the latent space of the hidden Markov model;

[0015] and, defining the length of a first preset time window, and dividing the sample motion information into a plurality of first preset time windows in time sequence;

[0016] Performing time alignment and normalization processing on sample motion information in each first preset time window to obtain multiple joint observation vectors;

[0017] Arrange multiple joint observation vectors in time order and generate joint observation sequences to form the observation space of the hidden Markov model;

[0018] Constructing the transition probability matrix and observation probability matrix of the hidden Markov model according to the latent space and the observation space, and training them to obtain a trained physiological-behavioral coupling model;

[0019] The motion complexity curve is configured to be generated by calculating the transition frequency of the latent space and the entropy value of the joint observation sequence, and the cognitive load index time series is configured to be generated by calculating the duration of the latent space and the variance of the joint observation sequence.

[0020] In some embodiments, the motion feature vector is configured to be obtained by:

[0021] Segmenting the real-time motion data according to the second preset time window to obtain a plurality of first time series segments, each time series segment corresponding to the real-time motion data within the second preset time window;

[0022] Extracting a first time domain feature for each first time series segment, the first time domain feature comprising a first mean, a first variance, a first maximum value, a first minimum value, and a first root mean square value;

[0023] For each first time series segment, convert the time domain signal into a frequency domain signal by fast Fourier transform, and extract first frequency domain features, where the first frequency domain features include a first main frequency, a first spectrum energy, and a first spectrum entropy;

[0024] For each first time series segment, extracting a first spatiotemporal feature, the first spatiotemporal feature including a spatial distribution of a motion trajectory, a spatial variation of a motion speed, and a spatial variation of a motion acceleration;

[0025] The first time domain feature, the first frequency domain feature and the first space-time feature are combined to generate a motion feature vector.

[0026] In some embodiments, the cognitive feature vector is configured to be obtained by:

[0027] Performing a third preset time window segmentation on the real-time cognitive data to obtain a plurality of second time series segments, each second time series segment corresponding to a piece of real-time cognitive data within the third preset time window;

[0028] For each second time series segment, extract a second time domain feature, where the second time domain feature includes a second mean, a second variance, a second maximum value, and a second minimum value;

[0029] For each second time series segment, convert the time domain signal into a frequency domain signal by fast Fourier transform, and extract second frequency domain features, where the second frequency domain features include a second main frequency, a second spectrum energy, and a second spectrum entropy;

[0030] For each second time series segment, extract the second spatiotemporal features, where the second spatiotemporal features include spatial changes in attention distribution, spatial changes in decision speed, and spatial changes in cognitive load, and are used to characterize signs of cognitive decline of the user in the spatial dimension;

[0031] The second time domain feature, the second frequency domain feature and the second space-time feature are combined to generate a cognitive feature vector.

[0032] In some embodiments, the motion feature vector, the cognitive feature vector, and the joint evaluation index are input into the feature fusion unit to obtain abnormal state information including:

[0033] Obtaining the user's rehabilitation information, including rehabilitation duration and rehabilitation progress, and initializing the weight distribution rules of the motion feature vector, cognitive feature vector, and joint evaluation index according to the rehabilitation information. The weight distribution rules are configured as early rehabilitation rules, mid-term rehabilitation rules, and late rehabilitation rules;

[0034] Perform feature alignment and normalization on motion feature vectors, cognitive feature vectors, and joint evaluation indicators;

[0035] Perform weighted fusion on the motion feature vector, cognitive feature vector and joint evaluation index after feature alignment and normalization to obtain a fused feature vector;

[0036] The fused feature vector is used to calculate the motion deviation index according to the motion complexity curve, and the motion deviation index includes a posture deviation coefficient, a joint disorder coefficient or a muscle contraction disorder coefficient;

[0037] The fused feature vector is used to calculate the cognitive bias index according to the cognitive load index time series. The cognitive bias index includes the user's slowness coefficient, misunderstanding coefficient, and attention distraction coefficient in the cognitive process.

[0038] Abnormal state information is generated according to the motor deviation index and the cognitive deviation index.

[0039] In some embodiments, inputting abnormal state information into a dynamic threshold model to detect abnormal behavior of a user, and generating a real-time feedback instruction through a preset feedback rule includes:

[0040] Initializing the initial threshold of the dynamic threshold model according to the rehabilitation information, the initial threshold includes a motor bias threshold and a cognitive bias threshold;

[0041] Dynamically adjust the motion deviation threshold and cognitive deviation threshold based on the user's real-time motion information and real-time cognitive data, including:

[0042] Dynamically adjusting the motion bias threshold according to the changing trend of the motion complexity curve, and dynamically adjusting the cognitive bias threshold according to the changing trend of the cognitive load index time series;

[0043] The movement deviation index is compared with the dynamically adjusted movement deviation threshold, and if the movement deviation index exceeds the movement deviation threshold, it is determined to be abnormal movement behavior;

[0044] Compare the cognitive bias index with the dynamically adjusted cognitive bias threshold. If the cognitive bias index exceeds the cognitive bias threshold, it is determined to be cognitive behavioral abnormality.

[0045] Generate test results based on motor behavior abnormalities and cognitive behavior abnormalities;

[0046] Generate preset feedback rules based on the test results;

[0047] The preset feedback rules include movement feedback rules and cognitive feedback rules;

[0048] The motion feedback rules are generated through the following steps:

[0049] generating movement adjustment instructions according to the type and degree of abnormal movement behavior, wherein the movement adjustment instructions include adjusting the intensity, frequency or posture requirements of the training task;

[0050] The cognitive feedback rules are generated through the following steps:

[0051] Generate cognitive adjustment instructions based on the type and degree of cognitive behavioral abnormalities, including adjusting the complexity, attention requirements or decision-making speed requirements of the training tasks;

[0052] Combine motor adjustment instructions and cognitive adjustment instructions to generate real-time feedback instructions.

[0053] In a second aspect, the present application provides a motor and cognitive ability assessment system, applicable to the assessment method described in the first aspect, the system comprising:

[0054] An information acquisition module is used to acquire the user's real-time motion information, which includes real-time motion data and real-time cognitive data. The real-time motion data includes real-time joint angle information and real-time gait information. The real-time cognitive data includes real-time attention information and real-time decision speed information.

[0055] A logic processing module is used to construct a physiological-behavioral coupling model, which is configured to be generated based on a hidden Markov model, input real-time motion information into the physiological-behavioral coupling model to obtain a joint evaluation index, which includes a motion complexity curve and a cognitive load index time series; construct a hierarchical feature extraction model, which includes a feature extraction unit and a feature fusion unit, input real-time motion information into the feature extraction unit to obtain a motion feature vector and a cognitive feature vector; input the motion feature vector, the cognitive feature vector and the joint evaluation index into the feature fusion unit to obtain abnormal state information, which includes motion abnormality and cognitive abnormality;

[0056] An abnormal feedback module is used to input abnormal status information into a dynamic threshold model to detect abnormal user behavior and generate real-time feedback instructions through preset feedback rules;

[0057] The instruction sending unit is used to adjust the user's training task parameters in real time according to the real-time feedback instructions.

[0058] In the third aspect, the present application also provides a motor and cognitive ability assessment device, including a smart terminal, a motion acquisition device, a cognitive acquisition device, an audio playback device and an auxiliary correction device, wherein the smart terminal is equipped with the assessment system described in the second aspect; the motion acquisition device is electrically connected to the smart terminal, and the motion acquisition device is used to collect real-time motion data; the cognitive acquisition device is electrically connected to the smart terminal, and the cognitive acquisition device is used to collect real-time cognitive data; the audio playback device is electrically connected to the smart terminal, and the audio playback device is used to provide audio guidance to the user according to the real-time feedback instructions of the smart terminal; the auxiliary correction device is electrically connected to the smart terminal, and the auxiliary correction device is used to correct the user's movements according to the real-time feedback instructions, so that the user's real-time movements meet the requirements of the training task.

[0059] In some embodiments, the smart terminal is configured as at least one of a tablet, a computer, and a mobile phone; the motion acquisition device is configured as at least one of a camera, a camcorder, an induction mat, a radar, and an infrared sensor; the cognitive acquisition device is configured as at least one of a smart bracelet, a smart helmet, smart glasses, headphones, a camera, a camcorder, a voice interaction device, and a VR device.

[0060] In some embodiments, the sensing floor mat includes a pressure-sensitive capacitor matrix floor and a flexible piezoresistive material yoga mat. The pressure-sensitive capacitor matrix floor is used to capture the user's gait distribution and center of gravity trajectory in real time, and the flexible piezoresistive material yoga mat is used to detect the user's pressure distribution in partitions.

[0061] Different from the prior art, the above technical solution has the following beneficial effects:

[0062] The present invention provides a method, system and device for evaluating movement and cognitive ability, and the method comprises the following steps: obtaining the user's real-time movement information, including real-time joint angle information, real-time gait information, real-time attention information and real-time decision speed information; constructing a physiological-behavioral coupling model based on a hidden Markov model, and generating a joint evaluation index including a movement complexity curve and a cognitive load index time series by inputting real-time movement information; extracting features from real-time movement information using a hierarchical feature extraction model to obtain a movement feature vector and a cognitive feature vector; fusing the feature vector with the joint evaluation index, and outputting abnormal state information including movement abnormality and cognitive abnormality; detecting behavioral abnormality through a dynamic threshold model and generating real-time feedback instructions in combination with preset feedback rules, and finally dynamically adjusting the user's training task parameters according to the instructions. The method realizes the coordinated monitoring and adaptive training regulation of movement and cognitive ability through multi-dimensional data fusion and dynamic evaluation mechanism.

[0063] The above-mentioned records related to the invention content are only an overview of the technical solution of the present application. In order to enable ordinary technicians in the field to more clearly understand the technical solution of the present application, and then implement it according to the text of the specification and the contents recorded in the drawings, and to make the above-mentioned purpose and other purposes, features and advantages of the present application easier to understand, the following is an explanation in combination with the specific implementation mode and drawings of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The drawings are only used to illustrate the principles, implementation methods, applications, characteristics and effects of the specific embodiments of the present invention and other related contents, and shall not be considered as limiting the present application.

[0065] In the drawings of the specification:

[0066] Figure 1 It is a schematic diagram of steps S101 to S106 of the evaluation method described in the specific implementation method;

[0067] Figure 2 It is a schematic diagram of steps S201 to S207 of the evaluation method described in the specific implementation method;

[0068] Figure 3 It is a schematic diagram of steps S301 to S305 of the evaluation method described in the specific implementation method;

[0069] Figure 4 It is a structural schematic diagram of the evaluation system described in the specific implementation method;

[0070] Figure 5 It is a schematic diagram of the structure of the evaluation device described in the specific implementation method.

[0071] The reference numerals in the above drawings are described as follows:

[0072] 1. Evaluation device;

[0073] 11. Evaluation system;

[0074] 111. Information acquisition module;

[0075] 112. Logic processing module;

[0076] 113. Abnormal feedback module;

[0077] 114. Instruction sending unit.

[0078] 12. Intelligent terminal;

[0079] 13. Motion acquisition equipment;

[0080] 14. Cognitive acquisition equipment;

[0081] 15. Audio playback equipment;

[0082] 16. Auxiliary correction equipment. DETAILED DESCRIPTION

[0083] In order to explain in detail the possible application scenarios, technical principles, specific schemes that can be implemented, and the purposes and effects that can be achieved, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0084] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way to form a corresponding implementable technical solution.

[0085] Unless otherwise defined, the technical terms used in this document have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms in this document is only for describing specific embodiments and is not intended to limit this application.

[0086] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" logical relationship.

[0087] In the present application, terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0088] Without further limitations, in this application, the words "include", "comprises", "has" or other similar open-ended expressions used in sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0089] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0090] In the description of the embodiments of the present application, space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0091] The processor described in the embodiments of the present application can be implemented by hardware, firmware, software or a combination thereof, and can use a circuit, a single or multiple application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic Device, PLD), a field programmable gate array (Field Programmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, at least one of a microprocessor, and also includes other physical, biological or chemical structures that can achieve similar or equivalent functions to the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of the present application, or any combination of the steps mentioned therein.

[0092] The computer program involved in the embodiment can be stored in a computer device readable storage medium, which includes but is not limited to a disk, a tape, a magnetic card, a floppy disk, a flash memory, an optical disk, an optical card, a read-only memory (ROM), a random access memory (RAM), an erasable programmable ROM (EPROM) and an electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical or chemical structures that can achieve the same or equivalent functions as the storage media listed above, such as DNA, RNA, protein and other units with information storage capabilities. In a specific embodiment, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiment can be stored in a single medium in a centralized manner, or it can be stored in multiple media in a distributed manner. The memory containing the computer device readable storage medium can be a non-volatile memory or a random access memory. These computer device readable storage media can be built into the device, or they can be connected to the device involved in the embodiment as an external device or a part of an external device. In some embodiments, a memory with a computer device readable storage medium is deployed locally; in other embodiments, a solution of deploying the memory away from the processor may also be adopted, such as a network attached memory accessed via an RF circuit or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), etc., or a suitable combination thereof, as long as the computer device can access the memory. In addition, the computer program involved in the embodiment may be stored in plaintext / ciphertext form, or may be designed as training data, which may be integrated and reorganized through model training and implicitly stored in the parameter state of a deep neural network or other machine learning model.

[0093] See also Figure 1 In a first aspect, this embodiment provides a method for evaluating motor and cognitive abilities, comprising:

[0094] S101, obtaining real-time motion information of the user, where the real-time motion information includes real-time motion data and real-time cognitive data, where the real-time motion data includes real-time joint angle information and real-time gait information, and the real-time cognitive data includes real-time attention information and real-time decision speed information;

[0095] S102, constructing a physiological-behavioral coupling model, wherein the physiological-behavioral coupling model is configured to be generated based on a hidden Markov model, inputting real-time motion information into the physiological-behavioral coupling model, and obtaining a joint evaluation index, wherein the joint evaluation index includes a motion complexity curve and a cognitive load index time series;

[0096] S103, constructing a hierarchical feature extraction model, the hierarchical feature extraction model includes a feature extraction unit and a feature fusion unit, inputting real-time motion information into the feature extraction unit to obtain a motion feature vector and a cognitive feature vector;

[0097] S104, inputting the motion feature vector, the cognitive feature vector and the joint evaluation index into a feature fusion unit to obtain abnormal state information, where the abnormal state information includes motion abnormality and cognitive abnormality;

[0098] S105, inputting the abnormal state information into the dynamic threshold model to detect abnormal behavior of the user, and generating real-time feedback instructions through preset feedback rules;

[0099] S106. Adjust the user's training task parameters in real time according to the real-time feedback instruction.

[0100] In step S101, real-time joint angle information refers to the three-dimensional bending angles of the shoulder, elbow, and knee joints calculated by filtering the skeletal joint points (using the Kalman filter algorithm to eliminate the jitter noise of the collected joint coordinates), which is used to quantify movement coordination and joint mobility. Real-time gait information refers to the extracted step frequency, stride variation coefficient, and support phase / swing phase ratio values ​​extracted through heart rate-motion correlation calibration, which dynamically corrects the heart rate baseline offset according to the three-axis composite amplitude of the accelerometer, and is used to evaluate sports endurance and gait stability. The formula for heart rate-motion correlation calibration is as follows:

[0101] ;

[0102] in, is the corrected heart rate measurement; Refers to raw, uncorrected heart rate measurements; is the motion compensation coefficient, which represents the degree of interference of unit exercise intensity on the heart rate baseline; , , is the instantaneous acceleration component of the triaxial acceleration sensor in the body coordinate system (unit: ), which correspond to the movement intensity of the human body in the forward, lateral and vertical directions respectively.

[0103] Real-time attention information refers to the standard deviation of the fixation duration and the fractal dimension of the scanning path collected by the eye tracker, which is used to characterize the concentration of cognitive resource allocation; real-time decision speed information refers to the average delay time from the presentation to the key press of the user to randomly appearing visual stimuli (such as color changes) under the dual-task paradigm, which is used to quantify the speed of information processing.

[0104] In step S102, the specific contents of generating the physiological-behavioral coupling model based on the hidden Markov model are described in detail below. After inputting the real-time motion information, the implicit state sequence is obtained through Viterbi decoding, the motion complexity curve is generated by the sliding window calculation of the approximate entropy of the joint trajectory between adjacent states (reflecting the attenuation trend of the motion control stability), and the cognitive load index time series is output by the weighted fusion of the normalized pupil diameter (0.6 weight) and the decision speed Z-score (0.4 weight) (values ​​> 1.5 indicate high load risk).

[0105] In step S103, the hierarchical feature extraction model uses a CNN-LSTM hybrid model to extract spatiotemporal features from the raw sensor data and identify subtle movement abnormalities (such as Parkinson's tremor) and signs of cognitive decline. Real-time motion information is extracted through CNN to extract spatial features, and LSTM further captures the gait cycle timing pattern, and finally outputs a motion feature vector (including stride symmetry, joint angular velocity covariance, etc.) and a cognitive feature vector (including gaze entropy, decision delay coefficient of variation, etc.). The motion feature vector is used to characterize the user's motion state and motion pattern, and the cognitive feature vector is used to characterize the user's cognitive state and cognitive load. The steps for obtaining the motion feature vector and cognitive feature vector are described in detail below.

[0106] In step S104, the feature fusion unit uses a random forest classifier. Optionally, abnormal movement is defined as a range of joint movement exceeding the user's historical data. Periodic Fourier frequency shift of interval or gait ,in, The interval is a criterion for judging outliers based on the principle of normal distribution in statistics. Represents the standard deviation calculated based on the user's own joint angle historical data, reflecting the degree of data dispersion.

[0107] Cognitive abnormality was defined as a cognitive load index that exceeded the individual's baseline by 2 standard deviations for 5 consecutive seconds. ) or a sudden increase of more than 30% in distraction, among which, Represents the standard deviation calculated based on the user's own historical baseline of cognitive load.

[0108] In step S105, the dynamic threshold model refers to the calculation of the mean and standard deviation of the real-time indicator based on the user's historical data to generate a personalized threshold interval, such as the motion complexity curve threshold = baseline mean ,in, Represents the composite standard deviation of multi-dimensional fusion indicators based on user history training.

[0109] When the abnormal status information continues to trigger the threshold for more than 3 cycles, the preset feedback rule is triggered. Optionally, the preset feedback rule starts a graded response mechanism, where the first-level feedback (voice prompt) is activated when a single abnormality is detected, and the second-level feedback (reducing the complexity of the training scene) is triggered when motor abnormalities and cognitive abnormalities occur simultaneously.

[0110] In step S106, the training task parameters are dynamically adjusted through real-time feedback instructions, such as extending the cognitive task interval by 200ms, until the abnormal index returns to within the threshold range.

[0111] The method provided in this embodiment realizes the refined joint evaluation of movement and cognitive ability by integrating real-time movement data and cognitive data, constructing a physiological-behavior coupling model and a hierarchical feature extraction model. The movement complexity curve and cognitive load index time series generated by the hidden Markov model, combined with the spatiotemporal features extracted by the CNN-LSTM hybrid model, can accurately capture the decline of movement coordination and abnormal cognitive resource allocation. The dynamic threshold model generates adaptive detection standards based on personalized historical data, and combined with a hierarchical feedback mechanism, it can adjust the training task parameters in real time when the range of joint movement exceeds the limit or the cognitive load index continues to exceed the standard, effectively balancing the training intensity and user ability. The interference of physical activity on heart rate is eliminated through heart rate-motion correlation calibration, and combined with random forest multimodal decision fusion, the accuracy of movement tremor recognition and cognitive overload warning is significantly improved, providing closed-loop adaptive regulation support for rehabilitation training and dual-task ability assessment.

[0112] See also Figure 2 In some embodiments, a physiological-behavioral coupling model is constructed, and the physiological-behavioral coupling model is configured to generate the following based on a hidden Markov model:

[0113] S201, obtaining sample motion information, where the sample motion information includes sample motion data and sample cognitive data;

[0114] S202, performing a kernel density estimation operation on the sample motion information to obtain joint probability distribution information, where the joint probability distribution information includes the physiological-behavioral pattern of the user at a specific time point;

[0115] S203, mapping the joint probability distribution information to the hidden space of the hidden Markov model;

[0116] and, defining the length of a first preset time window, and dividing the sample motion information into a plurality of first preset time windows in time sequence;

[0117] S204, performing time alignment and normalization processing on sample motion information in each first preset time window to obtain multiple joint observation vectors;

[0118] S205, arranging the multiple joint observation vectors in chronological order and generating a joint observation sequence to form an observation space of a hidden Markov model;

[0119] S206, constructing a transition probability matrix and an observation probability matrix of a hidden Markov model according to the hidden space and the observation space, and performing training to obtain a trained physiological-behavioral coupling model;

[0120] S207. The motion complexity curve is configured to be generated by calculating the transition frequency of the latent space and the entropy value of the joint observation sequence, and the cognitive load index time series is configured to be generated by calculating the duration of the latent space and the variance of the joint observation sequence.

[0121] In step S201, the sample motion data includes sample joint angle information and sample gait information, and the sample cognitive data includes sample attention information and sample decision speed information. Preferably, the sample motion information comes from the behavioral data of a specific user group in a preset experimental environment, and the number of samples is not less than The time span of each group of samples is minute.

[0122] In step S202, the kernel density estimation method is a non-parametric method for estimating the probability density function. The kernel function of the kernel density estimation method is a Gaussian kernel function with a bandwidth of Determined by Silverman's rule, the calculation formula is:

[0123] ;

[0124] in, is the first sample standard deviation, which represents the degree of dispersion of data in a single feature dimension (such as speed, heart rate). is the number of samples. When the bandwidth is determined by the Silverman rule and a multidimensional joint probability distribution is finally generated, the Gaussian kernel function of each dimension will be based on the dimension itself. Adjust the degree of smoothing.

[0125] The joint probability distribution information is represented as a multidimensional probability density function ,in They represent the feature dimensions of sample motion information and sample cognitive information respectively.

[0126] In step S203, the hidden space of the hidden Markov model usually represents the potential state of the system, and the joint probability distribution information is mapped to the hidden space of the hidden Markov model, and the joint probability distribution information is associated with the hidden state. hidden states, each hidden state Represents the physiological-behavioral pattern of a user at a specific point in time.

[0127] The mapping method is based on the state partitioning of the K-means clustering algorithm, clustering the joint probability distribution information into hidden states, the clustering objective function is:

[0128] ;

[0129] in, For the The center point of the hidden state.

[0130] Define the length of the first preset time window as , and dynamically adjust according to the periodic characteristics of user behavior, the adjustment range is , which helps in processing time series data.

[0131] In step S204, since the real-time motion data and real-time cognitive data may come from different sensors or acquisition devices, there may be inconsistent timestamps. Therefore, the data needs to be time-aligned to ensure that the data in the same time window corresponds to the same moment. The real-time motion data and real-time cognitive data are normalized to eliminate the dimensional differences between different features, such as scaling the data to the range of [0,1] or [-1,1]. The time alignment method is time warping based on linear interpolation, and the normalization method is Z-score normalization. The calculation formula is:

[0132] ;

[0133] in, is the sample mean, is the second sample standard deviation, which represents the standard deviation of the data in the current first preset time window in each feature dimension. Each feature dimension is calculated separately in each first preset time window. For example, if the heart rate data within a certain time window fluctuates greatly, the heart rate characteristics within the window Time alignment and normalization help improve the stability and accuracy of the model, ensuring that different features in the same time window (such as motion acceleration and EEG signals) are comparable after normalization, and avoiding the impact of dimensional differences on model training.

[0134] In step S205, the observation space includes a joint observation sequence of the real-time motion data and the real-time cognitive data. An observation vector refers to a multidimensional data point generated after processing the real-time motion data and the real-time cognitive data within a single time window. Specifically, the observation vector is a vector composed of real-time motion data (such as joint angle information, gait information) and real-time cognitive data (such as attention information, decision speed information) after time alignment and normalization. For example, assuming that within a time window, the real-time motion data includes 3 joint angles and 2 gait features, and the real-time cognitive data includes 1 attention feature and 1 decision speed feature, then the observation vector can be expressed as ,in, is the real-time joint angle information, is real-time gait information, is real-time attention information, is the real-time decision speed information. In each first preset time window, the time-aligned and normalized real-time motion data and real-time cognitive data are combined into an observation vector.

[0135] The observation sequence is a sequence of observation vectors in multiple time windows arranged in chronological order, reflecting the changes in the user's physiological-behavioral patterns in a continuous time period. The joint observation sequence is expressed as ,in, For the The observation vector in multiple time windows is a joint observation vector of multiple time windows. By sliding the time window, the observation vectors in multiple time windows are arranged in chronological order to generate an observation sequence. The observation vector describes the state of the user at a certain moment, while the observation sequence describes the state change of the user over a period of time.

[0136] In step S206, the transition probability matrix is , used to describe the transition relationship between hidden states, the observation probability matrix is , which is used to describe the probability of generating a specific observation sequence under a specific hidden state. The transition probability matrix Elements Indicates that from the hidden state Move to hidden state The probability is calculated as:

[0137] ;

[0138] in, From the state Transfer to state The frequency of

[0139] The observation probability matrix is Elements Indicates hidden state In observing The probability of is calculated by the Gaussian distribution model:

[0140] ;

[0141] in, and They are The mean vector and covariance matrix of .

[0142] The Baum-Welch algorithm is used to train the hidden Markov model, and the optimization goal is to maximize the likelihood function of the observation sequence ,in are model parameters, is the probability distribution of the accident state; during the training process, the transition probability matrix is ​​updated iteratively and the observation probability matrix , until the likelihood function converges.

[0143] In step S207, the motion complexity curve is generated by calculating the transition frequency of the latent space and the entropy value of the joint observation sequence, and the calculation formula is:

[0144] ;

[0145] The cognitive load index time series is generated by calculating the duration of the latent space and the variance of the joint observation sequence, and the calculation formula is:

[0146] ;

[0147] in, For the The variance of the observation vector in the time window.

[0148] This embodiment achieves a refined joint analysis of user movement and cognitive ability through a physiological-behavioral coupling model construction method based on a hidden Markov model. The kernel density estimation method is used to perform non-parametric probability density estimation on sample movement information and sample cognitive information to generate joint probability distribution information, which effectively characterizes the dynamic correlation of multi-dimensional physiological-behavioral patterns. The joint probability distribution is mapped to the K-means clustering. The hidden space composed of hidden states is combined with the first preset time window that is dynamically adjusted to perform time alignment and normalization on the sample data, generate joint observation vectors and joint observation sequences, and solve the problems of time series asynchrony and dimensional difference of multi-source sensor data. Based on the Baum-Welch algorithm, the transition probability matrix and observation probability matrix of the hidden Markov model are optimized, and the observation probability is calculated through the Gaussian distribution model to maximize the likelihood function of the observation sequence, which significantly improves the model's ability to express complex behavior patterns. The motion complexity curve quantifies the stability of action control through the entropy value of the hidden state transition frequency, and the cognitive load index time series reflects the volatility of information processing through the mean of the variance of the observation vector. The two can work together to accurately identify the attenuation of motion coordination (such as abnormal joint trajectory) and cognitive overload risks (such as decision delay mutation). This method enhances the real-time and individual adaptability of physiological-behavioral pattern analysis through joint probability modeling and dynamic time window mechanism, and provides a reliable quantitative basis for ability assessment in dual-task scenarios.

[0149] See also Figure 3 In some embodiments, the motion feature vector is configured to be obtained by the following steps:

[0150] S301, segmenting the real-time motion data according to a second preset time window to obtain a plurality of first time series segments, each time series segment corresponding to a piece of real-time motion data within the second preset time window;

[0151] S302, extracting a first time domain feature from each first time series segment, where the first time domain feature includes a first mean, a first variance, a first maximum value, a first minimum value, and a first root mean square value;

[0152] S303, for each first time series segment, convert the time domain signal into a frequency domain signal by fast Fourier transform, and extract first frequency domain features, where the first frequency domain features include a first main frequency, a first spectrum energy, and a first spectrum entropy;

[0153] S304, extracting first spatiotemporal features for each first time series segment, where the first spatiotemporal features include spatial distribution of motion trajectories, spatial variations of motion speeds, and spatial variations of motion accelerations;

[0154] S305: Combine the first time domain feature, the first frequency domain feature, and the first space-time feature to generate a motion feature vector.

[0155] In step S302, a first time domain feature is extracted for each first time series segment. For example, for real-time joint angle information, a first mean value of each real-time joint angle information is calculated. and the first variance :

[0156] ;

[0157] ;

[0158] in, Indicates the first Real-time joint angle information at each moment, Indicates the length of the second preset time window.

[0159] In step S303, extract the first frequency domain feature, the first frequency domain feature includes the first main frequency, the first spectrum energy and the first spectrum entropy, for example, calculate the first spectrum energy of the real-time joint angle information :

[0160] ;

[0161] in, is the frequency domain signal frequency components, is the total number of frequency components.

[0162] In step S304, the first spatiotemporal feature is used to directly characterize the abnormal movement signs of the user in the spatial dimension, for example, to calculate the spatial distribution of the movement trajectory of the joint angle signal. :

[0163] ;

[0164] in, It's a time delay.

[0165] In step S305, the motion feature vector can be expressed as:

[0166] .

[0167] This embodiment achieves a refined representation of the user's motion state through a motion feature vector construction method that fuses multi-dimensional features. The real-time motion data is segmented based on the second preset time window, and the stability and amplitude fluctuation range of the joint motion are quantified in combination with the first time domain feature. The frequency domain features are extracted by fast Fourier transform, and the frequency offset and energy distribution anomaly of the periodic motion pattern can be identified. The spatiotemporal features are characterized by the autocorrelation function Calculate the temporal correlation of joint angles to capture spatial coordination attenuation (such as phase desynchronization of knee flexion and extension trajectories). Combine the first time domain feature, the first frequency domain feature, and the first spatiotemporal feature to generate a multi-dimensional motion feature vector, covering the time-frequency-spatial characteristics of the motion pattern, and significantly enhancing the detection sensitivity of subtle abnormal signs (such as early tremor and gait periodic disruption). This method overcomes the limitations of single-dimensional analysis by complementing multi-source features. For example, the time domain variance can often verify the physical meaning of the frequency domain main frequency offset, and the spatiotemporal distribution anomaly can explain the cause of the sudden change in the time domain mean, providing a high-resolution quantitative basis for the early identification of motor dysfunction.

[0168] In some embodiments, the cognitive feature vector is configured to be obtained by:

[0169] Performing a third preset time window segmentation on the real-time cognitive data to obtain a plurality of second time series segments, each second time series segment corresponding to a piece of real-time cognitive data within the third preset time window;

[0170] For each second time series segment, extract a second time domain feature, where the second time domain feature includes a second mean, a second variance, a second maximum value, and a second minimum value;

[0171] For each second time series segment, convert the time domain signal into a frequency domain signal by fast Fourier transform, and extract second frequency domain features, where the second frequency domain features include a second main frequency, a second spectrum energy, and a second spectrum entropy;

[0172] For each second time series segment, extract the second spatiotemporal features, where the second spatiotemporal features include spatial changes in attention distribution, spatial changes in decision speed, and spatial changes in cognitive load, and are used to characterize signs of cognitive decline of the user in the spatial dimension;

[0173] The second time domain feature, the second frequency domain feature and the second space-time feature are combined to generate a cognitive feature vector.

[0174] In this embodiment, for each second time series segment, a second time domain feature is extracted. For example, for real-time attention information, a second mean of the real-time attention information is calculated. and the second variance .

[0175] The second spatiotemporal feature, the temporal feature, is used to directly characterize the user's signs of cognitive decline in the spatial dimension.

[0176] The cognitive feature vector can be expressed as:

[0177] .

[0178] This embodiment realizes a refined quantitative analysis of the user's cognitive state through a cognitive feature vector construction method that integrates multi-dimensional features. The real-time cognitive data is segmented based on the third preset time window, and the stability of cognitive ability is quantified in combination with the second time domain feature. The second frequency domain feature is extracted by fast Fourier transform to identify abnormal periodic patterns of cognitive activities. For example, the decrease in the energy of the decision speed spectrum may reflect a disordered thinking rhythm, and the increase in the entropy of the attention spectrum indicates fragmented information processing. The second spatiotemporal feature is obtained by the autocorrelation function. Analyze the spatial correlation of cognitive parameters to capture the imbalance in attention resource allocation (such as abnormal spatial jumps of gaze points in visual tasks) or the spatial accumulation effect of decision delays. Combine the second time domain features, the second frequency domain features, and the second spatiotemporal features to generate a multi-dimensional cognitive feature vector, covering the time-frequency-spatial characteristics of cognitive functions, and significantly enhancing the detection sensitivity of early cognitive decline (such as attention drift and discretization of decision speed). This method distinguishes occasional distraction (instantaneous variance surge) from persistent cognitive decline (main frequency shift accompanied by continuous increase in spectral entropy) through complementary analysis of time domain fluctuation features and frequency domain energy features, and combines spatiotemporal features to reveal abnormal spatial allocation of cognitive resources (such as attention distraction patterns in multi-tasking scenarios), providing high-resolution quantitative basis for early identification of cognitive dysfunction.

[0179] The motion feature vector and cognitive feature vector are taken as the output of the feature extraction unit and transmitted to the feature fusion unit for subsequent abnormal state information generation.

[0180] In some embodiments, the motion feature vector, the cognitive feature vector, and the joint evaluation index are input into the feature fusion unit to obtain abnormal state information including:

[0181] Obtaining the user's rehabilitation information, including rehabilitation duration and rehabilitation progress, and initializing the weight distribution rules of the motion feature vector, cognitive feature vector, and joint evaluation index according to the rehabilitation information. The weight distribution rules are configured as early rehabilitation rules, mid-term rehabilitation rules, and late rehabilitation rules;

[0182] Perform feature alignment and normalization on motion feature vectors, cognitive feature vectors, and joint evaluation indicators;

[0183] Perform weighted fusion on the motion feature vector, cognitive feature vector and joint evaluation index after feature alignment and normalization to obtain a fused feature vector;

[0184] The fused feature vector is used to calculate the motion deviation index according to the motion complexity curve, and the motion deviation index includes a posture deviation coefficient, a joint disorder coefficient or a muscle contraction disorder coefficient;

[0185] The fused feature vector is used to calculate the cognitive bias index according to the cognitive load index time series. The cognitive bias index includes the user's slowness coefficient, misunderstanding coefficient, and attention distraction coefficient in the cognitive process.

[0186] Abnormal state information is generated according to the motor deviation index and the cognitive deviation index.

[0187] In this embodiment, the generation of abnormal state information is achieved through dynamic weight allocation and multimodal feature fusion. The weight allocation rules are initialized based on the user's rehabilitation information, where the early rehabilitation rules give higher weights to the motion feature vectors to strengthen basic action monitoring, the mid-term rehabilitation rules balance the weights of motion and cognitive features to reflect dual-task synergy, and the late rehabilitation rules focus on cognitive feature vectors and joint evaluation indicators to evaluate high-order functional recovery.

[0188] Feature alignment uses dynamic time warping (DTW) to eliminate the timing offset of motion and cognitive data, and normalization uses Z-score standardization to eliminate the dimensional differences of cross-modal features such as acceleration variance, joint range of motion, and response error rate.

[0189] The fused feature vector generated after weighted fusion dynamically calculates the posture deviation coefficient (such as the Hausdorff distance between the shoulder joint trajectory and the standard rehabilitation path), the joint obstacle coefficient (such as the deviation of the knee joint activity angle from the preset range) or the muscle contraction obstacle coefficient according to the motion complexity curve.

[0190] The cognitive bias index analyzes the attention distraction coefficient (such as abnormal frequency of gaze shifting) and misunderstanding coefficient (such as a sudden increase in the error rate of instruction response) and the attention distraction coefficient through the cognitive load index time series analysis.

[0191] Furthermore, the generation of abnormal status information adopts multi-level judgment logic, and the specific examples can be referred to as follows: if the posture deviation coefficient exceeds the threshold and the attention distraction coefficient continues to increase, it is marked as "motor-cognitive joint imbalance"; if the joint disorder coefficient and the misunderstanding coefficient exceed the standard at the same time in 5 consecutive time windows, then the error correction instructions such as "raise the left arm 10cm" are triggered, and the abnormal status mark and action correction parameters are output at the same time to realize closed-loop feedback control of the rehabilitation process.

[0192] This embodiment significantly improves the refinement and adaptability of rehabilitation assessment through phased weight allocation and multi-source feature fusion mechanism. The weight allocation rules are dynamically adjusted based on the user's rehabilitation time and progress. The early rehabilitation rules focus on motion feature vectors to monitor the reconstruction of basic motor functions. The mid-term rehabilitation rules balance the motion feature vectors and cognitive feature vectors to capture the coordination anomalies in dual-task execution. The late rehabilitation rules strengthen cognitive feature vectors and joint evaluation indicators to evaluate high-order cognitive recovery in complex environments. Dynamic time warping aligns motion data (acceleration variance time series) and cognitive data (response error rate time series) to eliminate the time asynchrony of cross-modal data acquisition; Z-score standardization unifies the dimensional differences of joint range of motion and cognitive load index to ensure the numerical comparability of weighted fusion. The fused feature vector is combined with the motion complexity curve to calculate the posture deviation coefficient, and the attention distraction coefficient is identified based on the cognitive load index time series. The multi-level decision logic jointly analyzes the motion deviation index and the cognitive deviation index to achieve real-time identification and closed-loop intervention of abnormal states, effectively improving the safety and personalized adaptation capabilities of rehabilitation training.

[0193] In some embodiments, inputting abnormal state information into a dynamic threshold model to detect abnormal behavior of a user, and generating a real-time feedback instruction through a preset feedback rule includes:

[0194] Initializing the initial threshold of the dynamic threshold model according to the rehabilitation information, the initial threshold includes a motor bias threshold and a cognitive bias threshold;

[0195] Dynamically adjust the motion deviation threshold and cognitive deviation threshold based on the user's real-time motion information and real-time cognitive data, including:

[0196] Dynamically adjusting the motion bias threshold according to the changing trend of the motion complexity curve, and dynamically adjusting the cognitive bias threshold according to the changing trend of the cognitive load index time series;

[0197] The movement deviation index is compared with the dynamically adjusted movement deviation threshold, and if the movement deviation index exceeds the movement deviation threshold, it is determined to be abnormal movement behavior;

[0198] Compare the cognitive bias index with the dynamically adjusted cognitive bias threshold. If the cognitive bias index exceeds the cognitive bias threshold, it is determined to be cognitive behavioral abnormality.

[0199] Generate test results based on motor behavior abnormalities and cognitive behavior abnormalities;

[0200] Generate preset feedback rules based on the test results;

[0201] The preset feedback rules include movement feedback rules and cognitive feedback rules;

[0202] The motion feedback rules are generated through the following steps:

[0203] generating movement adjustment instructions according to the type and degree of abnormal movement behavior, wherein the movement adjustment instructions include adjusting the intensity, frequency or posture requirements of the training task;

[0204] The cognitive feedback rules are generated through the following steps:

[0205] Generate cognitive adjustment instructions based on the type and degree of cognitive behavioral abnormalities, including adjusting the complexity, attention requirements or decision-making speed requirements of the training tasks;

[0206] Combine motor adjustment instructions and cognitive adjustment instructions to generate real-time feedback instructions.

[0207] In this embodiment, the dynamic threshold model initializes the motor deviation threshold and the cognitive deviation threshold through rehabilitation information. For example, a higher motor deviation threshold is set in the early rehabilitation stage to tolerate movement instability, and the cognitive deviation threshold sets an upper limit on the response error rate based on the initial cognitive load index.

[0208] During the dynamic adjustment process, the motion complexity curve updates the threshold in real time by analyzing the smoothness of the motion trajectory (such as the acceleration variance volatility): if the curve shows that the motion pattern tends to be stable, the motion deviation threshold is tightened frame by frame; the cognitive load index timing adjusts the threshold by monitoring the efficiency of attention allocation (such as the proportion of effective fixation points per unit time). If the cognitive load index continues to increase, the cognitive deviation threshold is relaxed synchronously to avoid excessive sensitivity and triggering false alarms.

[0209] When the detected movement deviation index (such as the knee joint movement angle deviates from the standard value by 18%) exceeds the dynamically adjusted movement deviation threshold (such as the movement deviation threshold is 15% at this time), it is judged as abnormal movement behavior (the type is marked as "joint angle exceeds the limit"); if the cognitive deviation index (such as a delay of 2.5 seconds for 3 consecutive responses) exceeds the dynamic cognitive deviation threshold (such as the cognitive deviation threshold is 2 seconds at this time), it is judged as cognitive behavioral abnormality (the type is "decision delay").

[0210] In the preset feedback rules, the movement feedback rule generates instructions according to the type and degree of abnormality: if the "joint angle exceeds the limit" and the degree is moderate, the voice prompt "Please reduce the knee bending angle by 5 degrees" will be triggered, and the training intensity will be reduced (such as reducing the resistance by 10%); the cognitive feedback rule generates instructions for "decision delay": reduce the number of digits in arithmetic problems from 3 to 2, and extend the response time requirement to 3 seconds. When abnormal movement behavior and cognitive behavior occur simultaneously (such as the shoulder joint lifting height is less than 12cm and the instructions are misunderstood twice in a row), the real-time feedback instructions will combine movement adjustment ("Raise the left arm to the 15cm mark") and cognitive intervention ("Please repeat the current training steps"), and output them synchronously through voice and interface highlight prompts to ensure the coordination and executability of error correction instructions.

[0211] This embodiment significantly improves the accuracy of abnormal detection of rehabilitation training and the timeliness of intervention through a dynamic threshold model and an adaptive feedback mechanism. The motion deviation threshold and cognitive deviation threshold initialized based on rehabilitation information can adapt to the needs of different rehabilitation stages. During the dynamic adjustment process, the motion complexity curve tightens the motion deviation threshold frame by frame through the smoothness analysis of the motion trajectory. At the same time, the cognitive load index timing dynamically relaxes the cognitive deviation threshold according to the change of attention allocation efficiency to avoid misjudgment caused by user fatigue or task overload. When the motion deviation index exceeds the dynamic motion deviation threshold, the motion feedback rule is triggered to generate voice prompts and adjust the intensity, frequency or posture requirements of the training task; if the cognitive deviation index exceeds the cognitive deviation threshold, the cognitive feedback rule is activated to adjust the complexity, attention requirements or decision-making speed requirements of the training task. For scenarios with concurrent abnormalities in motor behavior and cognitive behavior, the feedback instruction realizes the coordinated execution of multimodal error correction instructions by combining motion adjustment instructions and cognitive adjustment instructions, forming a real-time control chain from abnormal detection to closed-loop correction, effectively ensuring training safety and personalized adaptation capabilities.

[0212] See also Figure 4 In a second aspect, this embodiment further provides a motor and cognitive ability assessment system, which is applicable to the assessment method described in the first aspect, and the system includes:

[0213] The information acquisition module 111 is used to acquire the user's real-time motion information, the real-time motion information includes real-time motion data and real-time cognitive data, the real-time motion data includes real-time joint angle information and real-time gait information, and the real-time cognitive data includes real-time attention information and real-time decision speed information;

[0214] The logic processing module 112 is used to construct a physiological-behavioral coupling model, which is configured to be generated based on a hidden Markov model, input the real-time motion information into the physiological-behavioral coupling model to obtain a joint evaluation index, which includes a motion complexity curve and a cognitive load index time series; construct a hierarchical feature extraction model, which includes a feature extraction unit and a feature fusion unit, input the real-time motion information into the feature extraction unit to obtain a motion feature vector and a cognitive feature vector; input the motion feature vector, the cognitive feature vector and the joint evaluation index into the feature fusion unit to obtain abnormal state information, which includes motion abnormality and cognitive abnormality;

[0215] The abnormal feedback module 113 is used to input the abnormal state information into the dynamic threshold model to detect the abnormal behavior of the user, and generate real-time feedback instructions through preset feedback rules;

[0216] The instruction sending unit 114 is used to adjust the user's training task parameters in real time according to the real-time feedback instructions.

[0217] The method steps mentioned in the above system are consistent with the method steps described in the first aspect, and are not described in detail in this embodiment.

[0218] Furthermore, in some preferred embodiments, the evaluation system 11 also includes a miniaturized near-infrared brain imaging module, which uses fNIRS (functional near-infrared spectroscopy) technology to non-invasively collect prefrontal cortex blood oxygen signals through a head-mounted device to generate real-time blood oxygen concentration time series data to quantify cognitive load. This module is periodically activated during the execution of the training task (for example, 30 seconds every 10 minutes), identifies the level of cognitive resource consumption by detecting changes in hemoglobin concentration (such as an increase of 0.05 mmol / L in deoxyhemoglobin concentration), and synchronizes the data to the information acquisition module 111 as a supplementary input for real-time cognitive data (expanding the original real-time attention information and decision-making speed information). The physiological-behavioral coupling model of the logic processing module 112 integrates the blood oxygen concentration time series data (such as the area of ​​the prefrontal activation area exceeds 0.05 mmol / L) when calculating the cognitive load index time series. The system combines real-time decision-making speed information (such as response delay exceeding 1.8 seconds) to generate more accurate evidence for cognitive abnormality. The miniaturized design ensures that the device is lightweight and does not affect the freedom of head movement when worn. Its periodic detection data is also used to calibrate the cognitive bias threshold. For example, when the blood oxygen recovery rate is 20% lower than the baseline value, the dynamic threshold model relaxes the cognitive bias threshold by 5% to adapt to the user's current neurometabolic state.

[0219] The motor and cognitive ability assessment system 11 provided in this embodiment realizes all-round accurate monitoring and personalized adaptation of the rehabilitation training process through multi-dimensional data fusion and dynamic closed-loop control mechanism. The information acquisition module 111 synchronously collects real-time joint angle information, real-time gait information, real-time attention information and real-time decision-making speed information to construct a complete motor and cognitive behavior data set. In the logic processing module 112, the physiological-behavioral coupling model based on the hidden Markov model maps the above data into a motion complexity curve and a cognitive load index time series. The hierarchical feature extraction model generates a motion feature vector and a cognitive feature vector through a feature extraction unit, and then associates the joint evaluation index through a feature fusion unit to accurately identify motor abnormalities or cognitive abnormalities. The abnormal feedback module 113 uses a dynamic threshold model to determine behavioral deviations in real time, and generates targeted instructions through preset feedback rules. In addition, the miniaturized near-infrared brain imaging module periodically detects the blood oxygen signal of the prefrontal cortex through fNIRS technology, captures the changes in deoxyhemoglobin concentration to quantify cognitive resource consumption, and synchronizes its data to the information acquisition module 111 to supplement the real-time cognitive data, so that the physiological-behavioral coupling model integrates the blood oxygen concentration time series and decision speed when calculating the cognitive load index time series, significantly improving the neurophysiological basis for the judgment of cognitive abnormalities. The miniaturized design ensures that the freedom of head movement is not hindered, and the periodic blood oxygen data dynamically calibrates the cognitive bias threshold, forming a closed-loop adaptation from the neurometabolic state to the behavioral performance, and finally optimizes the training parameters in real time through the instruction sending unit 114 to achieve the goal of coordinated rehabilitation of motor ability and cognitive function.

[0220] See also Figure 5 In the third aspect, the present embodiment further provides a motor and cognitive ability assessment device, comprising an intelligent terminal 12, a motion acquisition device 13, a cognitive acquisition device 14, an audio playback device 15 and an auxiliary correction device 16, wherein the intelligent terminal 12 is equipped with the assessment system 11 described in the second aspect; the motion acquisition device 13 is electrically connected to the intelligent terminal 12, and the motion acquisition device 13 is used to collect real-time motion data; the cognitive acquisition device 14 is electrically connected to the intelligent terminal 12, and the cognitive acquisition device 14 is used to collect real-time cognitive data; the audio playback device 15 is electrically connected to the intelligent terminal 12, and the audio playback device 15 is used to provide audio guidance to the user according to the real-time feedback instructions of the intelligent terminal 12; the auxiliary correction device 16 is electrically connected to the intelligent terminal 12, and the auxiliary correction device 16 is used to correct the user's movements according to the real-time feedback instructions, so that the user's real-time movements meet the requirements of the training task.

[0221] The motor and cognitive ability assessment device 1 described in this embodiment realizes the synchronous quantitative assessment and training adaptation of motor and cognitive abilities in a home environment through multimodal interaction and adaptive feedback mechanism. The motion acquisition device 13 can use a wearable inertial sensor (such as a six-axis sensor built into a knee joint strap) to capture real-time joint angle information and gait parameters, and the data is transmitted to the smart terminal 12 via Bluetooth low energy consumption.

[0222] The cognitive acquisition device 14 can be a smart glasses with an integrated eye tracking module, which records the attention allocation characteristics by capturing the eye movement trajectory, and cooperates with a handheld reaction button device to synchronously collect the decision response time when completing the digital connection test, and record the real-time attention information and decision speed information.

[0223] The auxiliary correction device 16 can be a dynamic limit bracket for the knee joint, with a built-in angle sensor to monitor the flexion and extension range in real time. When the logic processing module 112 detects abnormal movement, the limit angle is dynamically adjusted by the stepper motor to force the user's movements to meet the training specifications. In addition, the auxiliary correction device 16 can also be an adjustable wrist resistance brace, which dynamically adjusts the rotation damping through a micro servo motor to force the user to maintain a standard grip posture.

[0224] Furthermore, the evaluation device 1 provided in this embodiment may also include an auxiliary feedback device, and the auxiliary feedback device includes a multi-degree-of-freedom mechanical feedback device and a tactile navigation belt.

[0225] The multi-degree-of-freedom mechanical feedback device includes a three-axis linkage robot arm with a built-in torque sensor to simulate the resistance of daily activities (such as simulating the action of lifting a pot and pouring water). Resistance curve), to test the user's ability to adjust their strategy to sudden changes in load during home training, for example, when the resistance step increases Joint torque compensation response time Second.

[0226] The tactile navigation belt is equipped with a circularly distributed vibration motor, which encodes direction instructions through vibration patterns (e.g., two consecutive right-side vibrations represent a 30° right turn), and evaluates the user's cognitive-motor coordination ability (e.g., direction correction delay) when the smart terminal 12 generates a virtual obstacle scene (e.g., a random path deviation prompt appears). Second).

[0227] Optionally, an odor release module is also included. The odor release module serves as a backup component and uses a sealed atomization chamber to store standard odor preparations (such as mint). The odor release module is only released when the attention shifting evaluation mode is activated. It is used to test the motor control stability when the user's attention is shifted.

[0228] Preferably, the mechanical feedback device adopts a foldable storage design, the weight of the tactile belt is controlled at 180g and the surface is covered with a non-slip silicone layer, and the odor module quickly replaces the preparation compartment through a magnetic interface. All devices are powered by a Type-C unified interface to ensure that elderly users can complete device assembly, charging and basic maintenance without assistance, and achieve safe and reliable motor-cognitive joint rehabilitation training through natural human-computer interaction.

[0229] The motor and cognitive ability assessment device 1 described in this embodiment captures joint angles and gait parameters in real time through the motion acquisition device 13, and simultaneously records attention and decision-making speed information in combination with the cognitive acquisition device 14 to form a multi-dimensional behavioral data chain. The auxiliary correction device 16 dynamically adjusts the rigid bracket limit angle based on the abnormal detection result of the logic processing module 112 to ensure the standardization of the action. The multi-degree-of-freedom mechanical feedback device simulates the daily resistance environment through a three-axis linkage mechanical arm to accurately test the user's ability to adjust the strategy to cope with sudden changes in load; the tactile navigation belt uses a ring vibration motor to encode direction instructions to quantify cognitive-motor coordination efficiency in a virtual obstacle scene. The odor release module releases standard odor stimulation through a sealed atomization chamber to evaluate the movement stability during attention transfer. All components adopt foldable storage, lightweight design and magnetic preparation chamber, and cooperate with the Type-C unified power supply interface to reduce the complexity of operation, so that elderly users can independently complete the assembly and maintenance of the equipment, and realize the closed-loop adaptation of motor function enhancement and cognitive ability improvement in the home environment, ensuring the safety and effectiveness of rehabilitation training.

[0230] In some embodiments, the smart terminal 12 is configured as at least one of a tablet, a computer, and a mobile phone; the motion acquisition device 13 is configured as at least one of a camera, a camcorder, an induction mat, a radar, and an infrared sensor; the cognitive acquisition device 14 is configured as at least one of a smart bracelet, a smart helmet, smart glasses, headphones, a camera, a camcorder, a voice interaction device, and a VR device.

[0231] In this embodiment, the sensing mat of the motion acquisition device 13 adopts a flexible piezoresistive material stacked structure for zoned pressure detection. Pressure sensing units are distributed on the surface of the sensing mat. The plantar pressure distribution is captured in real time through changes in the piezoresistor value, supporting dynamic center of gravity tracking and gait phase recognition.

[0232] The motion acquisition device 13 covers multi-dimensional motion capture scenes through a combination of cameras, video cameras, induction mats, radars and infrared sensors. The induction mat adopts a complementary design of a pressure-sensitive capacitor matrix floor and a flexible piezoresistive material yoga mat to achieve accurate analysis from macroscopic motion trajectories to microscopic pressure characteristics. The cognitive acquisition device 14 synchronously obtains user physiological data (such as heart rate, EEG signals), visual focus trajectories, voice commands and virtual environment interactive behaviors through the coordinated operation of smart bracelets, smart helmets, smart glasses, headphones, voice interaction devices and VR devices, and constructs a multi-dimensional cognitive state map. The deep integration of motion and cognitive data enables the smart terminal 12 (tablet, computer or mobile phone) to generate personalized training plans. For example, when the induction mat detects gait imbalance, the AR projection guidance intensity is adjusted in combination with the visual attention data of smart glasses; when the abnormal pressure distribution triggers an early warning, the corrective instructions are broadcast in real time through the voice interaction device. This embodiment ensures the synchronization accuracy of motion capture and the real-time feedback response through protocol compatibility and data spatiotemporal alignment mechanism between devices. At the same time, it relies on modular architecture to support flexible configuration of different rehabilitation stages or training scenarios, thereby improving the adaptability and systematicness of home rehabilitation training while ensuring user privacy and data security.

[0233] In some embodiments, the sensing floor mat includes a pressure-sensitive capacitor matrix floor and a flexible piezoresistive material yoga mat. The pressure-sensitive capacitor matrix floor is used to capture the user's gait distribution and center of gravity trajectory in real time, and the flexible piezoresistive material yoga mat is used to detect the user's pressure distribution in partitions.

[0234] In this embodiment, the zone pressure detection of the flexible piezoresistive material yoga mat is achieved by the following steps:

[0235] A multi-layer flexible piezoresistive film is embedded in the base of the yoga mat. An independent detection area is formed on the surface of each film by etching a microcircuit, dividing the mat surface into square grid units, and a parallel piezoresistor array is set in each unit.

[0236] When the user applies pressure, the piezoresistive material in the pressure area deforms, causing the resistance value of the varistor in the corresponding grid unit to change, and the resistance change is converted into a voltage signal through a constant current source circuit;

[0237] The signal acquisition module polls and scans each grid unit at a sampling frequency of 100 Hz, quantizes the voltage signal using an analog-to-digital converter, and calculates the pressure value of each partition in real time;

[0238] The data processing module performs spatial clustering of adjacent grid cells, identifies the boundaries of continuous pressure areas, and distinguishes the left / right sole or upper limb support area based on the preset human foot / palm geometry template;

[0239] The pressure center algorithm is used to calculate the pressure distribution weight of each partition (such as the pressure ratio of the right leg support area The partition pressure data is spatially and temporally aligned with the spatial coordinates of the AR projection marker to generate dynamic feedback instructions.

[0240] The collaborative work of the sensing floor mat and the AR projection marker can be understood as follows: when the user walks along a virtual straight line, the piezoresistive array detects the heel-forefoot pressure transfer trajectory in real time, triggering dynamic adjustment of the AR path, further enhancing the effectiveness of training guidance.

[0241] The pressure-sensitive capacitor matrix floor of this embodiment captures the user's gait distribution and center of gravity trajectory in real time through the change of capacitance value, forming macroscopic motion trajectory data; the flexible piezoresistive material yoga mat embeds multiple layers of flexible piezoresistive film in the base, and divides the mat surface into square grid units using microcircuit etching technology, and each unit is provided with a parallel piezoresistor array. When the user applies pressure, the deformation of the piezoresistive material in the pressure area causes the resistance value of the corresponding grid to change, and the resistance value change is converted into a voltage signal through a constant current source circuit. The signal acquisition module polls and scans each grid unit at a sampling frequency of 100Hz, and calculates the pressure value of each partition in real time after quantization by an analog-to-digital converter. The data processing module performs spatial clustering on adjacent grids, identifies the boundaries of continuous pressure areas in combination with the preset human foot / palm geometry template, accurately distinguishes the left / right sole or upper limb support area, and calculates the pressure distribution weight of each partition through the pressure center algorithm. When the user walks along a virtual straight line, the piezoresistive array of the flexible piezoresistive material yoga mat detects the pressure transfer trajectory from the heel to the forefoot in real time, triggering dynamic adjustment of the AR path, so that the virtual projection markers are offset in real time or the visual prompts are enhanced according to the actual pressure distribution, thereby optimizing the real-time and accuracy of the training guidance. This embodiment realizes cross-scale data fusion from global motion trajectory to local pressure distribution through the complementary functions of the pressure-sensitive capacitor matrix floor and the flexible piezoresistive material yoga mat, and forms a closed-loop training system in combination with the dynamic feedback of the AR projection, effectively improving the effectiveness of gait correction and balance training.

[0242] Furthermore, the following examples can be developed in combination with the above technical solutions:

[0243] The user stands on the pressure mat and faces the tablet to start the "balance training mode";

[0244] The AR virtual coach demonstrates the squat action, the action acquisition device 13 captures the user's joint points, and the cognitive acquisition device 14 monitors the acceleration;

[0245] If the knees are detected to bend inward, the cognitive acquisition device 14 prompts "Both feet are externally rotated 15°";

[0246] Cognitive task superposition: During squats, the tablet displays random numbers, and the user needs to read the second to last number aloud;

[0247] Response Delay If the number is not a second, the number of subsequent digits will be automatically simplified (for example, from 5 digits to 3 digits).

[0248] Generate a report after training:

[0249] Sports score: The rate of reaching the target range of joint motion (e.g., hip flexion of 90° accounts for 85%).

[0250] Cognitive score: average response speed (1.2 seconds / question), error rate (5%).

[0251] Different from the prior art, the above technical solution has the following beneficial effects:

[0252] The present invention realizes the coordinated dynamic evaluation of movement and cognitive ability by constructing a physiological-behavioral coupling model and a hierarchical feature extraction model. Through the joint evaluation index (motion complexity curve and cognitive load index time series) generated by the hidden Markov model, combined with the time domain, frequency domain and time-space multi-dimensional feature extraction, the coupling anomaly of movement compensation behavior and cognitive load imbalance can be accurately captured. Based on the weight distribution rules and adaptive threshold detection mechanism dynamically adjusted in the rehabilitation stage, it can adapt to the personalized needs of different rehabilitation cycles (early, middle and late stages), identify abnormal states such as posture deviation, joint disorders, and distraction in real time, and adjust the training intensity, complexity and decision-making speed requirements through dynamic feedback rules, effectively avoiding secondary injuries caused by the accumulation of wrong actions. At the same time, the joint probability distribution constructed by the kernel density estimation method and the hidden Markov model observation space can analyze the temporal correlation between physiological and behavioral patterns without the need for complex sensors, breaking through the dependence of traditional evaluation on professional equipment, providing real-time movement correction and cognitive training optimization capabilities for home rehabilitation, and significantly improving the safety and effectiveness of autonomous training.

[0253] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concept of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. A method for assessing motor and cognitive abilities, characterized in that: The method comprises: Acquire real-time motion information of the user, the real-time motion information including real-time motion data and real-time cognitive data, the real-time motion data including real-time joint angle information and real-time gait information, the real-time cognitive data including real-time attention information and real-time decision speed information; Constructing a physiological-behavioral coupling model, wherein the physiological-behavioral coupling model is configured to be generated based on a hidden Markov model, inputting the real-time motion information into the physiological-behavioral coupling model, and obtaining a joint evaluation index, wherein the joint evaluation index includes a motion complexity curve and a cognitive load index time series; Constructing a hierarchical feature extraction model, the hierarchical feature extraction model includes a feature extraction unit and a feature fusion unit, inputting the real-time motion information into the feature extraction unit to obtain a motion feature vector and a cognitive feature vector; Inputting the motion feature vector, the cognitive feature vector and the joint evaluation index into the feature fusion unit to obtain abnormal state information, wherein the abnormal state information includes motion abnormality and cognitive abnormality; Inputting the abnormal state information into a dynamic threshold model to detect abnormal behavior of the user, and generating real-time feedback instructions through preset feedback rules; According to the real-time feedback instruction, the user's training task parameters are adjusted in real time.

2. The method for evaluating motor and cognitive abilities as claimed in claim 1, characterized in that: A physiological-behavioral coupling model is constructed, wherein the physiological-behavioral coupling model is configured to generate based on a hidden Markov model and includes: Acquiring sample motion information, wherein the sample motion information includes sample motion data and sample cognitive data; Performing a kernel density estimation operation on the sample motion information to obtain joint probability distribution information, wherein the joint probability distribution information includes the physiological-behavioral pattern of the user at a specific time point; Mapping the joint probability distribution information to a hidden space of a hidden Markov model; and, defining the length of a first preset time window, and dividing the sample motion information into a plurality of first preset time windows in time sequence; Performing time alignment and normalization processing on sample motion information in each first preset time window to obtain multiple joint observation vectors; Arranging the multiple joint observation vectors in time order and generating a joint observation sequence to form the observation space of the hidden Markov model; Constructing a transition probability matrix and an observation probability matrix of a hidden Markov model according to the latent space and the observation space, and performing training to obtain the trained physiological-behavioral coupling model; The motion complexity curve is configured to be generated by calculating the transition frequency of the latent space and the entropy value of the joint observation sequence, and the cognitive load index time series is configured to be generated by calculating the duration of the latent space and the variance of the joint observation sequence.

3. The method for evaluating motor and cognitive abilities as claimed in claim 1, characterized in that: The motion feature vector is configured to be obtained by the following steps: Segmenting the real-time motion data according to the second preset time window to obtain a plurality of first time series segments, each time series segment corresponding to a piece of real-time motion data within the second preset time window; Extracting a first time domain feature for each first time series segment, wherein the first time domain feature includes a first mean, a first variance, a first maximum value, a first minimum value, and a first root mean square value; For each first time series segment, convert the time domain signal into a frequency domain signal by fast Fourier transform, and extract first frequency domain features, where the first frequency domain features include a first main frequency, a first spectrum energy, and a first spectrum entropy; For each first time series segment, extracting a first spatiotemporal feature, wherein the first spatiotemporal feature includes a spatial distribution of a motion trajectory, a spatial variation of a motion speed, and a spatial variation of a motion acceleration; The first time domain feature, the first frequency domain feature and the first space-time feature are combined to generate the motion feature vector.

4. The method for evaluating motor and cognitive abilities as claimed in claim 1, wherein: The cognitive feature vector is configured to be obtained by the following steps: Segmenting the real-time cognitive data into a third preset time window to obtain a plurality of second time series segments, each second time series segment corresponding to a piece of real-time cognitive data within the third preset time window; For each second time series segment, extract a second time domain feature, wherein the second time domain feature includes a second mean, a second variance, a second maximum value, and a second minimum value; For each second time series segment, convert the time domain signal into a frequency domain signal by fast Fourier transform, and extract second frequency domain features, where the second frequency domain features include a second main frequency, a second spectrum energy, and a second spectrum entropy; For each second time series segment, extracting a second spatiotemporal feature, wherein the second spatiotemporal feature includes a spatial change in attention distribution, a spatial change in decision speed, and a spatial change in cognitive load, and is used to characterize a sign of cognitive decline of the user in a spatial dimension; The second time domain feature, the second frequency domain feature and the second space-time feature are combined to generate the cognitive feature vector.

5. The method for evaluating motor and cognitive abilities as claimed in claim 1, characterized in that: The motion feature vector, the cognitive feature vector and the joint evaluation index are input into the feature fusion unit to obtain abnormal state information including: Acquire rehabilitation information of the user, the rehabilitation information including rehabilitation duration and rehabilitation progress, and initialize the weight allocation rule of the motion feature vector, the cognitive feature vector and the joint evaluation index according to the rehabilitation information, wherein the weight allocation rule is configured as an early rehabilitation rule, a mid-term rehabilitation rule and a late rehabilitation rule; Performing feature alignment and normalization processing on the motion feature vector, cognitive feature vector and joint evaluation index; Performing weighted fusion on the motion feature vector, cognitive feature vector and joint evaluation index after feature alignment and normalization to obtain a fused feature vector; Calculating a motion deviation index of the fused feature vector according to a motion complexity curve, wherein the motion deviation index includes a posture deviation coefficient, a joint disorder coefficient, or a muscle contraction disorder coefficient; Calculate the cognitive bias index of the fused feature vector according to the cognitive load index time series, where the cognitive bias index includes the user's slowness coefficient, misunderstanding coefficient, and attention distraction coefficient in the cognitive process; Abnormal state information is generated according to the motor deviation index and the cognitive deviation index.

6. The method for assessing motor and cognitive abilities as claimed in claim 5, characterized in that: Inputting the abnormal state information into the dynamic threshold model to detect abnormal behavior of the user, and generating real-time feedback instructions through preset feedback rules include: Initializing an initial threshold of a dynamic threshold model according to the rehabilitation information, wherein the initial threshold includes a motion bias threshold and a cognitive bias threshold; According to the real-time motion information and real-time cognitive data of the user, the motion deviation threshold and the cognitive deviation threshold are dynamically adjusted, including: Dynamically adjusting the motion deviation threshold according to the changing trend of the motion complexity curve, and dynamically adjusting the cognitive deviation threshold according to the changing trend of the cognitive load index time series; Comparing the movement deviation index with the dynamically adjusted movement deviation threshold, and determining that the movement behavior is abnormal if the movement deviation index exceeds the movement deviation threshold; Comparing the cognitive bias index with the dynamically adjusted cognitive bias threshold, and determining that the cognitive behavior is abnormal if the cognitive bias index exceeds the cognitive bias threshold; generating a test result according to the motor behavior abnormality and cognitive behavior abnormality; Generate a preset feedback rule according to the detection result; The preset feedback rules include movement feedback rules and cognitive feedback rules; The motion feedback rule is generated by the following steps: Generate a movement adjustment instruction according to the type and degree of the abnormal movement behavior, wherein the movement adjustment instruction includes adjusting the intensity, frequency or posture requirement of the training task; The cognitive feedback rules are generated by the following steps: generating cognitive adjustment instructions according to the type and degree of the cognitive behavioral abnormality, wherein the cognitive adjustment instructions include adjusting the complexity, attention requirements or decision-making speed requirements of the training task; The movement adjustment instruction and the cognitive adjustment instruction are combined to generate the real-time feedback instruction.

7. A motor and cognitive ability assessment system, characterized in that: The evaluation method according to any one of claims 1 to 6, wherein the system comprises: An information acquisition module, used to acquire the user's real-time motion information, wherein the real-time motion information includes real-time motion data and real-time cognitive data, wherein the real-time motion data includes real-time joint angle information and real-time gait information, and the real-time cognitive data includes real-time attention information and real-time decision speed information; A logic processing module is used to construct a physiological-behavioral coupling model, wherein the physiological-behavioral coupling model is configured to be generated based on a hidden Markov model, and the real-time motion information is input into the physiological-behavioral coupling model to obtain a joint evaluation index, wherein the joint evaluation index includes a motion complexity curve and a cognitive load index time series; a hierarchical feature extraction model is constructed, wherein the hierarchical feature extraction model includes a feature extraction unit and a feature fusion unit, and the real-time motion information is input into the feature extraction unit to obtain a motion feature vector and a cognitive feature vector; the motion feature vector, the cognitive feature vector and the joint evaluation index are input into the feature fusion unit to obtain abnormal state information, wherein the abnormal state information includes motion abnormality and cognitive abnormality; An abnormality feedback module, used to input the abnormal state information into a dynamic threshold model to detect abnormal behavior of the user, and generate real-time feedback instructions through preset feedback rules; The instruction sending unit is used to adjust the user's training task parameters in real time according to the real-time feedback instruction.

8. A motor and cognitive ability assessment device, characterized in that: include: An intelligent terminal, on which the evaluation system according to claim 7 is mounted; A motion acquisition device, electrically connected to the intelligent terminal, and used to acquire the real-time motion data; A cognitive acquisition device, electrically connected to the intelligent terminal, and used to collect the real-time cognitive data; An audio playback device, electrically connected to the smart terminal, and used to provide audio guidance to the user according to the real-time feedback instructions of the smart terminal; The auxiliary correction device is electrically connected to the intelligent terminal, and is used to correct the user's movements according to the real-time feedback instructions so that the user's real-time movements meet the requirements of the training task.

9. The motor and cognitive ability assessment device according to claim 8, characterized in that: The smart terminal is configured as at least one of a tablet, a computer, and a mobile phone; The motion acquisition device is configured as at least one of a camera, a video camera, an induction mat, a radar, and an infrared sensor; The cognitive acquisition device is configured as at least one of a smart bracelet, a smart helmet, smart glasses, headphones, a camera, a video camera, a voice interaction device, and a VR device.

10. The motor and cognitive ability assessment device according to claim 9, characterized in that: The induction floor mat includes a pressure-sensitive capacitor matrix floor and a flexible piezoresistive material yoga mat. The pressure-sensitive capacitor matrix floor is used to capture the user's gait distribution and center of gravity trajectory in real time, and the flexible piezoresistive material yoga mat is used to detect the user's pressure distribution in partitions.

Images