A stroke patient physical and mental assessment rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory

Abstract

The application belongs to the field of evaluating rehabilitation technology, and discloses a stroke patient mind and body evaluation and rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory, which comprises a device installation and data acquisition module, a data preprocessing module, a feature set construction module, an evaluation module, a maintenance module, a data storage management and transmission module. The application performs contactless data acquisition and automatic evaluation scoring and question answering on the whole body movement function and psychological condition of the stroke patient, facilitates the development of related work in the hospital and community environment, and greatly improves the accuracy of the evaluation result and the convenience of the later maintenance by using an artificial intelligence algorithm and an expert traditional Chinese medicine rehabilitation theory. A rehabilitation physiotherapist can further evaluate the function of the patient according to the score of each action and the total score, combined with the mind and body data, so as to formulate targeted intervention measures.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of evaluating rehabilitation, and particularly relates to a stroke patient physical and mental evaluation and rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory. BACKGROUND

[0002] Stroke, also known as apoplexy, is an acute brain injury caused by sudden interruption of blood supply to the brain or rupture of cerebral blood vessels. According to the type of stroke, it can be divided into ischemic stroke (accounting for the majority) and hemorrhagic stroke. Ischemic stroke is caused by blockage or narrowing of cerebral blood vessels, and hemorrhagic stroke is caused by rupture of cerebral blood vessels leading to blood infiltration into brain tissue. Stroke can cause great harm to the body and mind of the patient. The physical effects of stroke include: 1). Limb dysfunction: the most common physical effect of stroke is hemiplegia (unilateral limb weakness or paralysis) which usually loses the ability to control one side of the body, showing weakness in the arm, leg and difficulty in movement. 2). Balance and coordination problems: due to damage to the area of the brain that controls balance and movement coordination, patients may experience gait instability, dizziness and other problems, increasing the risk of falling. 3). Seizures: some stroke patients may experience seizures after the onset of the disease, which is caused by abnormal nerve activity after brain damage. The psychological effects of stroke include: 1). Depression and anxiety: common psychological reactions after stroke are depression and anxiety, especially for those who lose the ability to move and be independent. The loss of physical function and uncertainty about the future can make them feel helpless, isolated and even lose confidence in life. 2). Decline in cognitive function: stroke affects the cognitive areas of the brain, leading to memory loss, difficulty concentrating, confusion and decreased judgment. These problems can affect patients' daily lives, making it difficult for them to complete tasks they were previously proficient in. 3). Psychological stress and social adaptation difficulties: due to the decline in quality of life and role changes caused by stroke, patients often feel great psychological stress. Some people feel like a burden to their families and have difficulty readjusting to work and social life, which further exacerbates their psychological distress.

[0003] Psychophysical assessment helps to early detect and identify various psychophysical problems after stroke, such as motor disorders, sensory disorders, cognitive disorders, mental disorders, etc. Rehabilitation therapy can effectively help stroke patients recover or improve impaired functions, such as limb motor function, sensory function, cognitive function, etc., and improve the patients' ability of daily life and quality of life. Psychophysical assessment is the premise and basis of rehabilitation therapy. Only through comprehensive psychophysical assessment, can we develop a scientific and reasonable rehabilitation therapy program. At the same time, the effect of rehabilitation therapy also needs to be monitored and evaluated through psychophysical assessment. Therefore, we need to propose a combined and effective solution for the psychophysical assessment and rehabilitation of stroke patients on the basis of the existing medical system. Western medicine adopts a systematic and evidence-based approach to the psychophysical assessment and rehabilitation of stroke patients. For physical assessment, CT scanning and National Institute of Health stroke scale (NIHSS) neurological assessment are commonly used. For psychological assessment, Barthel index is used to assess the independence of patients' activities of daily living (ADL), such as dressing, eating, and toileting, and Fugl-Meyer assessment scale (FMA) is used to assess the recovery of motor function, especially the motor ability of upper and lower limbs. Western medicine adopts a cycle scheme of assessment, rehabilitation training, and re-assessment for the assessment of the physical and psychological status of stroke patients. Although there are relatively rigorous quantitative tools and standardized processes, there are still problems such as insufficient attention to mental health assessment, one-sidedness of overall health assessment, and insufficient personalized assessment. In addition, the rehabilitation content of western medicine mainly includes progressive resistance training, walking training, and joint range of motion exercises, and the intervention for psychological problems adopts the therapy of prescribing antidepressants. Western medicine usually emphasizes the treatment of acute stage of stroke, and the time window of rehabilitation stage is often limited in western medicine. Western medicine treatment also leads to drug dependence and side effects. The local treatment of western medicine overemphasizes physical function recovery and ignores psychological and overall adaptation, resulting in poor rehabilitation effect.

[0004] In contrast, traditional Chinese medicine (TCM) adopts a unique holistic approach and syndrome differentiation for the assessment of physical and psychological conditions in stroke patients. Through the combination of four diagnostic methods (inspection, auscultation and olfaction, interrogation, and palpation), TCM provides a comprehensive assessment of patients. Compared with Western medicine, TCM emphasizes individualization and holistic conditioning, focusing not only on specific symptoms but also on the overall state, emotions, and social environment of patients. TCM has a unique approach to stroke rehabilitation, focusing on holistic conditioning and individualized syndrome differentiation, which covers both physical and psychological conditioning. This includes acupuncture, massage, herbal conditioning, emotional conditioning, and qigong and tai chi, which emphasize holistic conditioning and individual differences, covering both physical function recovery and psychological health. Compared with the standardized and localized treatment of Western medicine, the treatment methods of TCM are more flexible and natural, with fewer side effects, and have significant advantages in long-term rehabilitation and psychological regulation. TCM emphasizes holistic observation and syndrome differentiation, focusing on regulating the balance of the body and blood, and pursuing health preservation while treating diseases. This 'preventive treatment' concept attracts many people who focus on long-term health and disease prevention. With the improvement of living standards, people's attention to health has increased, and their needs have become diversified. However, TCM lacks assessment tools and standards that are as rigorously validated and widely recognized as those of Western medicine in the assessment of stroke physical and psychological conditions, which leads to a higher degree of subjectivity in the assessment results. By comparing the theoretical methods of TCM and Western medicine for the assessment and rehabilitation of stroke, the TCM concept of 'holistic conditioning and preventive treatment' aligns with the modern trend of pursuing health management. We need a stroke assessment and rehabilitation system based on TCM rehabilitation theory, which integrates Western quantitative processes and the assessment-training-assessment cycle theory with traditional TCM methods, to avoid the shortcomings of unclear mechanisms and lack of objectivity.

[0005] With the advancement of technology, Artificial Intelligence (AI) and large language models (such as GPT-4) are playing an increasingly important role in the physical and psychological assessment and rehabilitation of stroke patients. Through machine learning and deep learning technologies, AI can analyze medical images such as CT and MRI scans, helping doctors diagnose stroke types, damaged brain regions, and the severity of hemorrhage or infarction more quickly and accurately. AI-driven rehabilitation robots can assist stroke patients with limb rehabilitation training, such as upper and lower limb motor training. The robots can adjust the intensity and method of training in real time according to the patient's specific rehabilitation progress. By analyzing the patient's facial expressions, tone of voice, and language content, AI can assess the patient's emotional state, including anxiety, depression, and stress responses. Large language models can analyze the patient's dialogue records or voice logs to identify emotional changes and provide corresponding psychological intervention suggestions. Artificial intelligence and large language models currently possess advantages such as high efficiency and accuracy, personalization and dynamic adjustment, comprehensive coverage, widespread accessibility, and low cost, which will greatly assist in the physical and mental assessment and rehabilitation of stroke patients. Combined with traditional Chinese medicine rehabilitation theories, more practical and effective rehabilitation can be achieved.

[0006] The functional assessment of body movement plays an important role in the process of rehabilitation therapy. Currently, for post-stroke motor dysfunction, Western medicine clinics mostly use scales for assessment, which can be roughly divided into the following categories: ① based on muscle changes ② based on changes in movement patterns ③ based on changes in upper limb function. Traditional Chinese medicine mainly uses TCM techniques such as observation, auscultation and interrogation. However, these methods have some limitations: 1) recording, reviewing and analyzing require a lot of manpower and time; 2) the assessment analysis is highly subjective. Therefore, a high-precision device combined with AI intelligent recognition algorithm is needed to collect real-time detailed data on the body (such as invention patent CN202410333606.0 "Stroke rehabilitation quality assessment system and assessment method", which collects patient body data through artificial intelligence, but lacks real-time and detailed capture of patient body data, so we need a high-precision acquisition scheme that can capture patient movement data in real time). At the same time, in the clinical and community rehabilitation environment, the commonly used machine vision analysis device and inertial and pressure sensors show the data singularity of the overall body movement function assessment results, i.e., the incompleteness of the evaluation (such as invention patent CN202210827083.6 "Multi-dimensional lower limb rehabilitation assessment device and method for stroke patients", which analyzes and evaluates lower limb movement, but lacks a full-body movement evaluation, so we need a scheme that collects all types of movement data from the patient's whole body rather than a single type). In addition, some studies have developed a stroke patient prognosis psychological rehabilitation evaluation system (such as invention patent CN202110792300.8 "Internet terminal automatic data collection-based psychological evaluation system for stroke patients", which uses "questionnaire scoring + expert intervention" to evaluate the patient's psychology, but lacks the intelligent degree of rehabilitation evaluation, so we need an intelligent psychological evaluation scheme, such as invention patent CN202210188719.7 "Positive psychological nursing intervention system for stroke patients", which analyzes the psychological trend through image and video, but lacks the initiative of the evaluation and treatment, so we need a system that gives an intelligent rehabilitation response after the psychological evaluation is completed), but there is still room for improvement: 1) doctors are still needed to be present for real-time question and answer evaluation, which is not convenient for data collection and assessment in the patient's home or community environment; 2) the evaluation scale is too subjective, lacking objective human data to verify the conclusion; 3) it does not deeply combine traditional Chinese medicine rehabilitation theory for system development, but only stays on the surface of the disease.Meanwhile, for the physical and mental assessment and rehabilitation of stroke patients, many studies have carried out the integration of algorithms and medical theories (such as the invention patent CN202311686157.X "a light visual posture assessment and non-contact emotion guidance rehabilitation system", which identifies human facial expressions to predict emotions through algorithm training, but the training method cannot form personalized prediction for each user, that is, the algorithm training is personalized, and we need an AI-based personalized training method), but still has the following problems: the rigidity and dogmatism of traditional training, which cannot be personalized, and cannot master the unique physical movement and psychological expression characteristics of each patient; the training process lacks parameter guidance based on TCM exercise methods, and some irrelevant data cannot be deleted, which lacks medical rigor.

[0007] Through the above analysis, the problems and defects of the prior art are:

[0008] (1) The rigidity and dogmatism of traditional training cannot be personalized, and cannot master the unique physical movement and psychological expression characteristics of each patient;

[0009] (2) The training process lacks parameter guidance based on TCM exercise methods, and some irrelevant data cannot be deleted, which lacks medical rigor. SUMMARY

[0010] In view of the problems existing in the prior art, the present application provides a stroke patient physical and mental assessment and rehabilitation integrated system based on TCM rehabilitation theory.

[0011] The present application is realized in that a stroke patient physical and mental assessment and rehabilitation integrated system based on TCM rehabilitation theory comprises:

[0012] a device installation and data acquisition module, a data preprocessing module, a feature set construction module, an assessment module, a maintenance module, a data storage management and transmission module;

[0013] The device installation and data acquisition module is connected with the data preprocessing module, the data storage management and transmission module, and is used to build an assessment and rehabilitation task environment and install devices, carry out corresponding assessment tasks according to the adopted physical assessment scale and psychological question and answer system, and identify the required data from the screening;

[0014] The data preprocessing module is connected with the device installation and data acquisition module, the feature set construction module, the assessment module, the data storage management and transmission module, and is used to preprocess the data collected above, including data filtering and segmentation, semantic recognition and sorting;

[0015] The feature set construction module is connected with the data preprocessing module and the evaluation module, and is used for calculating corresponding kinematic features, classifying corresponding psychological analysis features, and dynamically screening and constructing features with patient personalization.

[0016] The evaluation module is connected with the data preprocessing module, the feature set construction module, the maintenance module, and the data storage management and transmission module, and is used for calculating individual automated scores and total scores for each task of the kinematic feature set constructed by the above modules by using the selected evaluation scale and the trained model to analyze the physical condition; using a local large language model, intelligently analyzing the psychological condition of the patient based on a knowledge graph of traditional Chinese medicine theory; and simultaneously retrieving relevant evaluation results according to the personalized knowledge graph constructed for each patient in different conditions.

[0017] The maintenance module is connected with the evaluation module, and is used for maintaining the constructed evaluation model, updating the constructed feature set by accumulating motion data of new patients, and updating parameters of each evaluation model by cross-validation method.

[0018] The data storage management and transmission module is connected with the device installation and data acquisition module, the feature set construction module, and the evaluation module, and is used for saving the score results of the patient, making personalized rehabilitation evaluation reports according to the training scores by the rehabilitation therapist, and sending the report, training plan and training opinions to the patient through the module, so that the patient can freely check.

[0019] Further, the device installation and data acquisition module method is as follows:

[0020] A rehabilitation evaluation area is divided, and the required devices for testing are installed; the patient stands in the planned area, watches a video explanation of how to perform a rehabilitation evaluation action and psychological question and answer / questionnaire, and uses a computer and a 2K intelligent wide-angle USB camera device to collect whole body motion data, and the computer performs intelligent question and answer interaction between the patient and the computer;

[0021] Then, a customized motion data acquisition system is run in the computer to identify and screen the required key joint tracking data, and a psychological question and answer system based on a large language model is run to perform semantic processing on the collected data; then, data set construction is performed according to 10 motion tasks contained in the adopted stroke rehabilitation motor function scale;

[0022] The data is derived from recruited stroke patients of various degrees; all participants are evaluated by experienced senior rehabilitation therapists using the STREAM scale, and sign the relevant informed consent form; after all the evaluation tasks are completed, a total of 10 different motion data sets are generated; these data are accumulated in the corresponding database according to the task type for later maintenance work;

[0023] Psychological question and answer / questionnaire data are collected through a computer questionnaire system; the question and answer or questionnaire is designed by a professional therapist based on TCM rehabilitation theory.

[0024] Further, the data preprocessing module method is as follows:

[0025] A wide-angle USB camera device is used to collect real-time video sources for motor function evaluation and input the evaluation system;

[0026] First, the video source is subjected to Gaussian filtering processing, and the self-improved YOLO algorithm is used to intelligently screen feature data;

[0027] Then, for the real-time video captured by the wide-angle camera, the self-improved YOLO algorithm is used to extract the data of the required joints, intelligently analyze the patient's movement thinking, and further construct a personalized movement data graph; the computer intelligent psychological evaluation system is used to collect the patient's intelligent question and answer data, and according to the question and answer results, the data is arranged through operations such as abstract, keyword extraction, etc., and further a personalized psychological data graph is constructed.

[0028] Further, the feature set construction module method is as follows:

[0029] From the video data preprocessed in the above steps, relevant kinematic features are extracted, such as identifying 17 key points of each human body, and each human body target frame information contains six data: horizontal coordinates and vertical coordinates of the human body anchor point, width and height of the human body prediction frame, confidence and class confidence of the human body detection frame;

[0030] Three main joint angles are defined: elbow angle, shoulder angle, and leg angle; the elbow angle refers to the angle formed by the shoulder, elbow, and wrist in a two-dimensional plane; the shoulder angle refers to the angle formed by the elbow, shoulder, and hip in a two-dimensional plane; the leg angle refers to the angle formed by the hip, knee, and lower leg in a two-dimensional plane;

[0031] The angles of the elbow, shoulder, and leg are used for rehabilitation exercise scoring; the calculation formulas of the three types of angles are as follows:

[0032]

[0033] The shoulder, elbow joint, wrist, hip, knee, and lower leg are represented by W, M, P, D, T, and X respectively; the shoulder-elbow line, elbow-wrist line, shoulder-hip line, hip-knee line, and knee-lower leg line are represented by WM, MP, WD, DT, and TX respectively; θ1, θ2, and θ3 represent the angles of the shoulder-elbow line and elbow-wrist line, the elbow-shoulder line and shoulder-hip line, and the hip-knee line and knee-lower leg line respectively.

[0034] Further, the evaluation module method is as follows:

[0035] The neural network model is used as the core of the physical evaluation model; the neural network model includes three parts of input layer, hidden layer and output layer; the number of input layer neurons is the number of kinematic characteristics, and the number of output layer neurons corresponds to several different scores of each task of STREAM, and the evaluation model can be automatically scored by AI;

[0036] The motor function intelligent evaluation system is used for automatic scoring and summation of the patient's current exercise; first, the trained neural network model is used for automatic scoring according to the feature set constructed in step 3, then the final score of each task is summed to obtain the total score of the current evaluation, and the body evaluation content corresponding to the scale is output;

[0037] The knowledge graph and intelligent recommendation algorithm are used as the core of the psychological evaluation model; the knowledge graph constructs and stores a knowledge graph according to the disease knowledge of stroke in the fields of traditional Chinese medicine and western medicine and the traditional Chinese medicine exercise method, including semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological state and intervention scheme and the semantic relationship between them, and each semantic type has a corresponding description for explaining the concept and characteristics of the semantic type.

[0038] Further, the data storage management and transmission module method is as follows:

[0039] The rehabilitation physiotherapist logs in the integrated evaluation system to obtain patient information, and uses the calculated score, exercise data and psychological evaluation results as a reference basis to assess the physical and mental rehabilitation of the stroke patient, summarizes a personalized rehabilitation evaluation report, and sends the report and the diagnosis and treatment opinions of the training plan to the patient through the data storage management and transmission system for further targeted strengthening practice.

[0040] Another object of the present application is to provide a stroke patient physical and mental evaluation and rehabilitation integrated method based on traditional Chinese medicine rehabilitation theory, comprising:

[0041] Step 1, build an evaluation and rehabilitation task environment and install equipment through the device installation and data acquisition module, carry out the corresponding evaluation task according to the adopted physical evaluation scale and psychological questionnaire system, and identify the required data;

[0042] Step 2, the data collected above are preprocessed by the data preprocessing module, including data filtering and segmentation, semantic recognition and arrangement;

[0043] Step 3, calculate the corresponding kinematic characteristics, classify the corresponding psychological analysis characteristics, and dynamically screen the characteristics through the feature set construction module, and construct a feature set with patient individualization;

[0044] Step 4, analyze the physical condition by the evaluation module using the selected evaluation scale, by the trained model to calculate the single automated score and total score of each task for the kinematic feature set constructed by the above module; use the local large language model to intelligently analyze the psychological condition of the patient based on the knowledge graph of traditional Chinese medicine theory; and retrieve the relevant evaluation results according to the personalized knowledge graph constructed for each patient in different conditions;

[0045] Step 5, maintain the constructed evaluation model through the maintenance module, update the above constructed feature set by accumulating the motion data of new patients, and update the parameters of each evaluation model through cross-validation method;

[0046] Step 6, save the score results of the patients through the data storage management and transmission module, and the rehabilitation therapist makes a personalized rehabilitation evaluation report according to the training score, and sends the report, training plan and training suggestions to the patient through the module, and the patient can freely check.

[0047] Another object of the present application is to provide a computer device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to make the processor execute the steps of the stroke patient physical and mental evaluation and rehabilitation integrated method based on traditional Chinese medicine rehabilitation theory.

[0048] Another object of the present application is to provide a computer readable storage medium storing a computer program, the computer program being executed by a processor to make the processor execute the steps of the stroke patient physical and mental evaluation and rehabilitation integrated method based on traditional Chinese medicine rehabilitation theory.

[0049] Another object of the present application is to provide an information data processing terminal for realizing the stroke patient physical and mental evaluation and rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory.

[0050] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by the present application are analyzed from the following aspects:

[0051] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty in solving the problems, the technical solution to be protected by the present application is closely combined with the results and data in the research and development process, and the technical problems solved by the technical solution are analyzed in detail and deeply. Some creative technical effects brought about after solving the problem are described as follows:

[0052] The application provides a stroke patient physical and mental evaluation rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory to evaluate and recuperate stroke patients. Body, namely patient's body: real-time body data acquisition and analysis based on high-precision wide-angle cameras and deep learning algorithms, processing of redundant data to induce a unique patient database by using a self-developed improved YOLO algorithm, and operation function evaluation by using a multi-modal fusion artificial intelligence algorithm. Mind, namely patient's mind: quick psychological test and data acquisition based on traditional Chinese medicine theory and a large language model, formation of a patient's all-round psychological atlas by using a "traditional Chinese medicine theory + intelligent question and answer + AI analysis" three-in-one method, and judgment of the patient's psychological state by data analysis and abnormal feature screening and psychological health evaluation based on artificial intelligence. Finally, the system comprehensively evaluates the physical and mental data, combines the artificial intelligence algorithm, absorbs the advantages of traditional Chinese medicine rehabilitation theory while avoiding the defects of western medicine rehabilitation theory, proposes a comprehensive rehabilitation scheme with medical basis, intelligently selects Taijiquan, Baduanjin, Wufengxi and Yijinjing as rehabilitation means from traditional Chinese medicine exercise methods according to the principle of "harmonizing mind, breath and body", and helps patients recover nerve function and relieve anxiety by regulating body function, promoting blood circulation and strengthening muscle strength. Compared with similar technologies at home and abroad, the digital physical and mental evaluation intelligent processing platform has a wider application scene (applicable to family, community and hospital environment), more accurate and rapid output result, and cheaper hardware, and is more suitable for large-scale promotion.

[0053] Step 1: A wide-angle USB camera (preferably but not limited to DS-UVC-U64 Pro equipment) is used to capture and collect real-time motion data. The non-contact camera device is used as the hardware device for data collection, which avoids the defects of wearing discomfort, size incompatibility and inability of seriously ill stroke patients to wear the device compared with the traditional data collection device based on wearable sensors (such as full-body motion capture equipment). During data collection, the wide-angle camera has a large image collection range and high data accuracy, which is beneficial to accurately understand the motion state of the patient and ensure the effective performance of the motion function evaluation. On the other hand, it is simple to use and does not need to be debugged, and it can be used as soon as it is turned on, which avoids the pre-marking and calibration steps of similar hand and leg data collection systems, and further realizes and facilitates data collection in a community environment.

[0054] Steps 2 and 3: The application has a data preprocessing module with good performance and a patient personalized feature set construction module. The self-developed improved YOLO algorithm used in the application can accurately extract the specified coordinate data of the stroke patient's body, has high precision, is lightweight and real-time. On the other hand, the self-developed algorithm is used for dynamic feature extraction and selection, forming a personalized feature library of the patient, thereby further improving the accuracy of the evaluation.

[0055] Step 4 and step 5: Compared with the traditional stroke patient whole body motion capture and psychosomatic evaluation method, the intelligent evaluation system of the application has the advantages of low cost, good stability, strong objectivity, etc., and can be conveniently applied in hospitals and community environments. The physical evaluation model adopted is developed based on neural network and has excellent classification and pattern recognition ability. The psychological evaluation model adopted is based on expert guidance and traditional Chinese medicine database construction and has excellent case analysis and psychological problem solving ability. At the same time, a later maintenance module is constructed, and through the motion data of patients accumulated in actual use, the internal parameters of the model are updated regularly, the accuracy in actual use is improved, and the adaptability of the local population is continuously improved.

[0056] Step 6: The data storage, management and transmission system of the application can make the physiotherapist obtain the motion data of the patient for evaluation, improve the work efficiency, and reduce the expenditure of the patient and social rehabilitation service. The data storage, management and transmission can more conveniently manage the data of the patient, avoid the influence of data loss on the rehabilitation progress, and at the same time, the physiotherapist can use the system to feed back the rehabilitation evaluation report to the patient in time, increase the pertinence of the rehabilitation training, and improve the effectiveness of the rehabilitation effect.

[0057] The intelligent rehabilitation system for stroke patients based on the traditional Chinese medicine rehabilitation theory of the application is a set of rehabilitation equipment with low cost, good stability and rich use scenarios. The equipment is expected to be applied to different rehabilitation environments such as medical clinics and communities in the future, and is used for assisting the physical and mental rehabilitation treatment of stroke patients. The operation process of the system prototype is standardized, the output evaluation result is objectivized and standardized, and the system is focused on being convenient for the attending physician to view and use, so that one rehabilitation physiotherapist can serve multiple patients, improve the work efficiency and reduce the workload, and also correspondingly reduce the financial burden of the patient and the social rehabilitation service. The physiotherapist can also use the accumulated patient data to regularly update and maintain the evaluation model, and improve the performance of the evaluation system.

[0058] Compared with the prior art, the present application has three essential differences and advantages: 1) The present application is based on a high-precision recognition algorithm of a deep learning model and an attention mechanism, which has higher precision and faster response than general algorithms and can perform real-time identification. Based on the improved YOLO algorithm, a unique database of patients can also be formed; 2) The present application is based on AI+large language, which uses artificial intelligence algorithms and local large prediction models for intelligent automatic scoring, evaluation, and question and answer surveys, so as to obtain evaluation data more recognized by rehabilitation therapists, rather than evaluating the motor function and psychological status of patients through subjective judgment; 3) The present application proposes an evaluation rehabilitation system based on TCM exercise methods, which forms a knowledge graph of TCM exercise methods based on patient data accumulated in actual clinical practice, and forms a stroke patient evaluation rehabilitation system with medical basis and different from Western medicine diagnosis and treatment. The TCM exercise method physical and mental evaluation rehabilitation system is a product of the deep integration of traditional Chinese medicine wisdom and modern technology, and is a new innovation in the direction of TCM and AI combined evaluation rehabilitation.

[0059] In summary, the present application performs contactless data collection and automatic evaluation and scoring of question and answer for the whole body motor function and psychological status of stroke patients, facilitates the development of related work in hospital and community environments, and greatly improves the accuracy of evaluation results and the convenience of post-maintenance by using artificial intelligence algorithms and expert TCM rehabilitation theory. Rehabilitation therapists can further evaluate the function of patients according to the score of each action and the total score, combined with physical and mental data, so as to develop targeted intervention measures.

[0060] Secondly, as the auxiliary evidence of the present application, it is also reflected in the following important aspects:

[0061] (1) The expected income and commercial value of the technical scheme of the present application after transformation are:

[0062] The commercial value of the present application lies in its innovative integration of TCM rehabilitation theory and modern artificial intelligence technology, providing a comprehensive solution for the rehabilitation treatment of stroke patients. The system realizes real-time evaluation of the physical motor function of patients through high-precision cameras and deep learning algorithms, and accurately analyzes the psychological status of patients by combining TCM theory with large language models. This integrated evaluation method not only improves the scientificity and individualization of rehabilitation programs, but also broadens their application scenarios, making them suitable for various environments such as families, communities, and hospitals. In addition, the system has obvious advantages in hardware cost and operation convenience, making it easier to promote and popularize. It not only provides accurate and fast evaluation results, but also intelligently recommends suitable TCM exercise methods such as Taijiquan and Baduanjin based on evaluation data, helping patients recover physically and mentally.

[0063] (2) The technical scheme of the present application fills the technical gap in the industry at home and abroad:

[0064] Traditional Chinese medicine rehabilitation has received extensive attention and development in China, especially in some traditional Chinese medicine hospitals and departments such as traditional Chinese medicine departments and orthopedics departments of general hospitals. The ideas and technical methods of traditional Chinese medicine rehabilitation have been widely applied. However, the evaluation and treatment methods of traditional Chinese medicine rehabilitation are still relatively scattered, and there is a lack of a systematic and integrated physical and mental evaluation rehabilitation system. Western medicine has relatively mature techniques and methods in the field of rehabilitation, but often focuses on the functional recovery of a single organ or system, lacking the guidance of a holistic view. Although there is a certain understanding and interest in traditional Chinese medicine rehabilitation theory and methods abroad, due to the differences in cultural background and medical system, its application and development still face many challenges. The present invention can fill the technical gap in the field of systematic traditional Chinese medicine rehabilitation system and the field of integrated rehabilitation evaluation of stroke patients. The system can comprehensively and systematically evaluate the physical and mental state of individuals based on the holistic functional view of traditional Chinese medicine. This includes comprehensive evaluation of physical function, psychological state, and quality of life, thereby more accurately reflecting the individual's rehabilitation needs. Based on the results of physical and mental integrated evaluation, the system can develop personalized rehabilitation programs. These programs not only focus on the recovery of physical function, but also pay attention to the adjustment of psychological state and the improvement of quality of life, thereby achieving comprehensive rehabilitation of body and mind.

[0065] (3) The technical solution of the present invention solves the technical problems that people have been eager to solve but have always failed to succeed:

[0066] 1. Traditional physical and mental evaluation methods often focus on a single aspect of physical function or psychological state, lacking comprehensiveness and accuracy.

[0067] 2. The mutual influence and connection between body and mind are often ignored, leading to incomplete and in-depth evaluation results.

[0068] 3. There are many types of traditional Chinese medicine rehabilitation techniques, but they often lack systematicness and integration.

[0069] 4. Traditional rehabilitation process management often relies on manual recording and monitoring, lacking intelligence and automation. The physical and mental evaluation rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory can comprehensively and systematically evaluate the physical and mental state of individuals based on the holistic view and dialectical treatment principles of traditional Chinese medicine. The system can consider multiple aspects such as physical function, psychological state, living habits, environmental factors, etc., thereby more accurately reflecting the individual's rehabilitation needs and potential problems.

[0070] (4) The technical solution of the present invention overcomes technical bias:

[0071] Traditional Chinese medicine rehabilitation theory emphasizes concepts such as yin and yang balance, zang-fu and meridian, and qi and blood, which are considered as vague or difficult to quantify in modern scientific system, leading some people to doubt the scientific nature of traditional Chinese medicine rehabilitation theory. Through scientific methods and strategies such as scientific verification, interdisciplinary cooperation, standardized evaluation, intelligent evaluation, and personalized treatment, this system can overcome these technical biases.

[0072] Thirdly, the technical solution of the present application integrates the physical and mental assessment of stroke patients with the rehabilitation process based on traditional Chinese medicine rehabilitation theory, solving the following key problems existing in the prior art:

[0073] 1. Low efficiency of data collection and processing: In the prior art, rehabilitation assessment for stroke patients mainly relies on manual operation, and the efficiency of data collection and processing is low, and there are problems of incomplete or incorrect data. The present application realizes the automatic collection and intelligent filtering of evaluation data through the device installation and data collection module 1 and the data preprocessing module 2, greatly improving the efficiency and accuracy of data processing.

[0074] 2. Lack of personalized assessment and rehabilitation guidance: Traditional rehabilitation assessment methods usually use fixed scales, which cannot effectively assess and guide rehabilitation for individual differences of different patients. The present application dynamically constructs personalized feature sets according to the kinematics and psychological characteristics of patients through the feature set construction module 3, and intelligently provides personalized rehabilitation assessment and schemes for each patient by combining traditional Chinese medicine rehabilitation theory and knowledge graph, greatly enhancing the accuracy and applicability of the assessment.

[0075] 3. Evaluation results and rehabilitation scheme updating not timely: The evaluation results of traditional rehabilitation methods are usually difficult to update and adjust in time according to the rehabilitation progress of patients. The present application continuously optimizes and updates the evaluation model by using the new data collected continuously through the maintenance module 5, realizes the dynamic adjustment of the rehabilitation scheme, ensures that the rehabilitation process matches the actual condition of the patient, and thus improves the rehabilitation effect.

[0076] 4. Inconvenient communication between patients and therapists: In the prior art, rehabilitation assessment results usually need to be transmitted manually, and the interaction between patients and therapists is delayed and inconvenient. The present application saves and transmits the evaluation results and rehabilitation schemes in time through the data storage management and transmission module 6, so that patients can check their rehabilitation progress at any time and obtain personalized rehabilitation training suggestions, thereby greatly improving the rehabilitation efficiency and the participation of patients.

[0077] The technical solution of the present application solves the problems of low evaluation efficiency, lack of personalization, and delayed feedback in the prior art through automatic and intelligent data collection and processing, combined with personalized rehabilitation assessment and dynamic scheme updating, significantly improving the rehabilitation effect of stroke patients and the technical level of industrial application. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 is a structure block diagram of a stroke patient mind and body assessment and rehabilitation integrated system based on a traditional Chinese medicine rehabilitation theory provided by an embodiment of the present application.

[0079] Figure 2 is a flowchart of a stroke patient mind and body assessment and rehabilitation integrated method based on a traditional Chinese medicine rehabilitation theory provided by an embodiment of the present application.

[0080] Figure 3 is a technical roadmap provided by an embodiment of the present application.

[0081] Figure 4 is a system motion analysis schematic diagram provided by an embodiment of the present application.

[0082] Figure 5 is an assessment analysis system interactive interface schematic diagram provided by an embodiment of the present application.

[0083] Figure 1 Chinese: 1, device installation and data acquisition module; 2, data preprocessing module; 3, feature set construction module; 4, evaluation module; 5, maintenance module; 6, data storage management and transmission module. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0085] As shown in Figure 1 , an embodiment of the present application provides a stroke patient mind and body assessment and rehabilitation integrated system based on a traditional Chinese medicine rehabilitation theory, comprising:

[0086] Device installation and data acquisition module 1, data preprocessing module 2, feature set construction module 3, evaluation module 4, maintenance module 5, data storage management and transmission module 6.

[0087] Device installation and data acquisition module 1 is connected with data preprocessing module 2 and data storage management and transmission module 6, used to build an assessment and rehabilitation task environment and install devices, carry out corresponding assessment tasks according to the adopted physical assessment scale and psychological question and answer system, and identify the required data from screening;

[0088] Data preprocessing module 2 is connected with device installation and data acquisition module 1, feature set construction module 3, evaluation module 4 and data storage management and transmission module 6, used to preprocess the data collected above, including data filtering and segmentation, semantic recognition and sorting;

[0089] The feature set construction module 3 is connected with the data preprocessing module 2 and the evaluation module 4, and is used for calculating corresponding kinematic features, classifying corresponding psychological analysis features, and dynamically screening and constructing a feature set with patient individualization.

[0090] The evaluation module 4 is connected with the data preprocessing module 2, the feature set construction module 3, the maintenance module 5 and the data storage management and transmission module 6, and is used for analyzing the physical condition by using the selected evaluation scale, calculating the single automatic score and total score of each task of the kinematic feature set constructed by the above modules through the trained model, analyzing the psychological condition of the patient through the local large language model and the knowledge graph intelligent analysis based on the theory of traditional Chinese medicine, and searching the relevant evaluation results according to the individualized knowledge graph constructed according to the different conditions of each patient.

[0091] The maintenance module 5 is connected with the evaluation module 4, and is used for maintaining the constructed evaluation model, updating the constructed feature set by accumulating the motion data of new patients, and updating the parameters of each evaluation model through the cross-validation method.

[0092] The data storage management and transmission module 6 is connected with the device installation and data acquisition module 1, the feature set construction module 3 and the evaluation module 4, and is used for saving the score results of the patient, making a personalized rehabilitation evaluation report according to the training score by a rehabilitation physiotherapist, and sending the report, the training plan and the training opinion to the patient through the module, so that the patient can freely check.

[0093] The stroke patient physical and mental evaluation and rehabilitation integrated system based on the rehabilitation theory of traditional Chinese medicine provided by the embodiment of the application first builds a complete evaluation and rehabilitation task environment through the device installation and data acquisition module 1. The module installs the required equipment according to the used physical assessment scale and psychological question and answer system, and acquires the physical motion data and psychological assessment data of the patient. After the data is screened, the data that needs to be focused on is identified for analysis by the subsequent modules.

[0094] Subsequently, the data preprocessing module 2 processes the collected raw data. This process includes data filtering, data segmentation, and semantic recognition and arrangement, so as to generate a high-quality and analyzable data set. The processed data can be further used to construct the individualized feature set of the patient, and provide a basis for evaluation and analysis. The module closely cooperates with the subsequent feature set construction module and evaluation module to ensure the integrity and consistency of the data.

[0095] Next, the feature set construction module 3 calculates the corresponding kinematic features according to the processed data, and classifies and arranges the psychological analysis features. Through dynamic screening, the personalized feature set of the patient is constructed. These feature sets include the patient's motor ability, psychological state and other information, which are used by the evaluation module 4 for analysis and scoring. The evaluation module uses the trained model to automatically score each task of the kinematic feature set, and comprehensively calculates the patient's physical condition. At the same time, the module also uses the knowledge graph based on traditional Chinese medicine theory to intelligently analyze the patient's psychological condition, and outputs the evaluation results combined with the personalized knowledge graph.

[0096] Finally, the maintenance module 5 is responsible for continuously updating the evaluation model, and optimizing the model using the data of new patients. The evaluation results are saved by the data storage management and transmission module 6, and transmitted to the rehabilitation therapist. The therapist generates a personalized rehabilitation evaluation report according to the results, and sends the report and the training plan to the patient. The patient can freely check his own rehabilitation progress and suggestions through the module, thereby providing scientific basis and guidance for subsequent rehabilitation treatment.

[0097] The device installation and data acquisition module method provided by the embodiment of the application is as follows:

[0098] The rehabilitation evaluation area is divided, and the required test equipment is installed; the patient stands in the planned area, watches a video explanation of how to perform a rehabilitation evaluation action guidance and psychological question and answer / questionnaire, and uses a computer and a 2K intelligent wide-angle USB camera device to collect whole body motion data, and the computer performs intelligent question and answer interaction between the patient;

[0099] Then, a customized motion data acquisition system is run in the computer to identify and screen the required key joint tracking data, and a psychological question and answer system based on a large language model is run to perform semantic processing on the collected data; then, data set construction is performed according to 10 motion tasks contained in the adopted stroke rehabilitation motor function scale;

[0100] The data is derived from recruited stroke patients of various degrees; all participants are evaluated by experienced senior rehabilitation therapists using the STREAM scale, and sign the relevant informed consent form; after all the evaluation tasks are completed, a total of 10 different motion data sets are generated; these data are accumulated in the corresponding database according to the task type for later maintenance work;

[0101] The psychological question and answer / questionnaire data are collected through a computer questionnaire system; the question and answer or questionnaire questions are designed by professional doctors and therapists based on traditional Chinese medicine rehabilitation theory.

[0102] The data preprocessing module method provided by the embodiment of the application is as follows:

[0103] With the wide-angle USB camera device, real-time video sources are collected for motor function evaluation and input into the evaluation system;

[0104] First, the video source is subjected to Gaussian filtering, and the feature data is intelligently screened through the self-improved YOLO algorithm;

[0105] Then, for the real-time video shot by the wide-angle camera, the data of the corresponding required joints are extracted through the self-improved YOLO algorithm, the patient's movement thinking is intelligently analyzed, and then the personalized movement data graph is constructed; the computer intelligent psychological evaluation system is used to collect the patient's intelligent question and answer data, and the data are arranged through operations such as abstract and keyword extraction according to the question and answer results, and then the personalized psychological data graph is constructed.

[0106] The feature set construction module method provided by the embodiment of the application is as follows:

[0107] From the video data preprocessed in the above steps, relevant kinematic features are extracted, such as identifying 17 key points of each human body, and each human body target frame information contains six data: horizontal coordinates and vertical coordinates of the human body anchor point, width and height of the human body prediction frame, confidence and class confidence of the human body detection frame;

[0108] Three main joint angles are defined: elbow angle, shoulder angle, and leg angle; the elbow angle refers to the angle formed by the shoulder, elbow and wrist in the two-dimensional plane; the shoulder angle refers to the angle formed by the elbow, shoulder and hip in the two-dimensional plane; and the leg angle refers to the angle formed by the hip, knee and calf in the two-dimensional plane;

[0109] The angles of the elbow, shoulder and leg are used for rehabilitation exercise scoring; the calculation formulas of the three types of angles are as follows:

[0110]

[0111] The shoulder, elbow joint, wrist, hip, knee and calf are represented by W, M, P, D, T and X respectively; the shoulder-elbow line, elbow-wrist line, shoulder-hip line, hip-knee line and knee-calf line are represented by WM, MP, WD, DT and TX respectively; θ1, θ2 and θ3 represent the angles of the shoulder-elbow line and elbow-wrist line, elbow-shoulder line and shoulder-hip line, and hip-knee line and knee-calf line respectively.

[0112] The evaluation module method provided by the embodiment of the application is as follows:

[0113] The neural network model is used as the core of the physical evaluation model, the neural network model comprises three parts of an input layer, a hidden layer and an output layer, the number of input layer neurons is the number of kinematic characteristics, the number of output layer neurons corresponds to several different scores of each task of STREAM, and the evaluation model can be automatically scored by AI;

[0114] The current movement of the patient is automatically scored and summed by using the intelligent evaluation system of motor function; first, the trained neural network model is used to automatically score according to the feature set constructed in step 3, then the final score of each task is summed to obtain the total score of the current evaluation, and the physical evaluation content corresponding to the scale is output;

[0115] The knowledge graph and the intelligent recommendation algorithm are used as the core of the psychological evaluation model; the knowledge graph is constructed and stored according to the disease knowledge about stroke in the fields of traditional Chinese medicine and western medicine and the traditional Chinese medicine movement skill, includes semantic types of symptom classification, evaluation diagnosis, syndrome classification, psychological state, intervention scheme and semantic relationships therebetween, each semantic type has a corresponding description for explaining the concept and characteristics of the semantic type.

[0116] The data storage management and transmission module method provided by the embodiment of the application is as follows:

[0117] The rehabilitation physiotherapist logs in the integrated evaluation system to obtain patient information, takes the calculated score, movement data and psychological evaluation result as a reference basis to evaluate the physical and mental rehabilitation of the stroke patient, summarizes a personalized rehabilitation evaluation report, and remotely sends the report and the diagnosis and treatment opinions of the training plan to the patient through the data storage management and transmission system, so that the patient can further strengthen the targeted practice.

[0118] As shown in Figure 2 The embodiment of the application provides a physical and mental evaluation and rehabilitation integrated method for stroke patients based on traditional Chinese medicine rehabilitation theory, which comprises the following steps:

[0119] S101, the evaluation and rehabilitation task environment is built and the equipment is installed through the equipment installation and data acquisition module, the corresponding evaluation task is carried out according to the adopted physical evaluation scale and psychological question and answer system, and the required data is screened and identified;

[0120] S102, the data collected above is preprocessed through the data preprocessing module, including data filtering and segmentation, semantic recognition and arrangement;

[0121] S103, the corresponding kinematic characteristics are calculated, the corresponding psychological analysis characteristics are classified, and the characteristics are dynamically screened through the feature set construction module, and the feature set with the individualization of the patient is constructed;

[0122] S104, analyze the physical condition by using the selected evaluation scale through the evaluation module, calculating the single automation score and total score of each task of the kinematic feature set constructed by the above module through the trained model, analyzing the psychological condition of the patient through the local large language model by intelligently analyzing the knowledge graph based on the theory of traditional Chinese medicine, and constructing a personalized knowledge graph according to the different conditions of each patient to retrieve the relevant evaluation results;

[0123] S105, maintain the constructed evaluation model through the maintenance module, update the above constructed feature set by accumulating the motion data of new patients, and update the parameters of each evaluation model through cross-validation method;

[0124] S106, save the score results of the patient through the data storage management and transmission module, make a personalized rehabilitation evaluation report according to the training score by the rehabilitation therapist, and send the report, training plan and training opinion to the patient through the module, and the patient can freely check.

[0125] Another object of the present application is to provide a computer device comprising a memory and a processor, the memory storing a computer program, the computer program being executed by the processor to make the processor execute the steps of the stroke patient physical and mental evaluation and rehabilitation integrated method based on the traditional Chinese medicine rehabilitation theory.

[0126] Another object of the present application is to provide a computer readable storage medium storing a computer program, the computer program being executed by a processor to make the processor execute the steps of the stroke patient physical and mental evaluation and rehabilitation integrated method based on the traditional Chinese medicine rehabilitation theory.

[0127] Another object of the present application is to provide an information data processing terminal for realizing the stroke patient physical and mental evaluation and rehabilitation integrated system based on the traditional Chinese medicine rehabilitation theory.

[0128] The present application is embodied as follows:

[0129] The present application provides a personalized physical and mental health service system based on "integration of body and mind". Human skeleton detection technology, multi-source psychological analysis technology, multi-mode data fusion technology, intelligent digital evaluation technology, personalized configuration technology of TCM exercise method health resort scheme, portable body data acquisition, multi-mode psychological data acquisition, digital physical and mental evaluation, personalized health resort module configuration and other technical innovation research are adopted; the above technologies are integrated to develop a stroke patient personalized physical and mental health service system and prototype based on TCM exercise method.

[0130] Portable "shape" data collection: using peripheral equipment such as high-precision wide-angle cameras to conveniently and non-contactly collect the specified behavior and action of the patient during the physical function assessment, and obtain the real-time video of the related human motion; introduce a convolutional neural network human key point detection and tracking system based on target recognition algorithm and re-identification algorithm to realize high-precision real-time human skeleton coordinate extraction; with the aid of advanced artificial intelligence algorithms (such as attention mechanism, time series-based operation process correction method), the extracted human skeleton data is corrected in precision, and accurate "shape" data is provided for the next intelligent assessment.

[0131] Multi-mode "spirit" data collection: using the recall personality development scale based on the field of traditional Chinese medicine spirit and psychology to assess the patient's overall psychology, including courage, self, willpower, thinking style, interpersonal relationship, sex quotient, world view, life events, and upbringing; self-developed psychological data collection scheme based on llama large language model, through AI to collect and summarize the patient's psychological question and answer results, to provide accurate "spirit" data for the next intelligent assessment.

[0132] Intelligent digital physical and mental assessment: research the focus of each clinical assessment scale (such as the FMS scale focusing on whole body movement function changes, and the HAD scale focusing on the emotional changes of the subjects) and conduct corresponding characteristic value extraction algorithm research; develop an intelligent assessment model based on deep learning to realize the digital reconstruction of the clinical assessment scale, thereby improving the efficiency and objectivity of the assessment. Based on the self-developed algorithm, the physical movement function assessment results and the psychological data are intelligently collected, the local traditional Chinese medicine movement skill knowledge graph is read, the individualized assessment result output is realized, and the patient's individualized stroke atlas is constructed.

[0133] Individualized traditional Chinese medicine physical and mental recuperation and rehabilitation: combined with the intelligent digital physical and mental assessment results, using the individualized recommendation configuration algorithm based on deep learning and the traditional Chinese medicine movement skill knowledge graph, according to the specific evaluation results of the body, the traditional Chinese medicine movement skills covered in the knowledge graph such as "Shi Shizhong Nourishing Life Qigong" and "Jingang Sanbu Jiufa" are intelligently screened, and the corresponding rehabilitation scheme in the graph is analyzed and screened according to the specific evaluation results of the psychology, and the intelligent rehabilitation scheme is output; according to the principle of "harmony of regulating mind, regulating breath and regulating body", the individualized stroke rehabilitation atlas of the patient is realized.

[0134] The present application comprises six modules, and each module functions as follows:

[0135] Device installation and data collection module: build the evaluation and rehabilitation task environment and install the equipment, carry out the corresponding assessment task according to the physical assessment scale and psychological question and answer system used, and screen and identify the required data.

[0136] Data preprocessing module: preprocess the collected data, including data filtering and segmentation, semantic recognition and organization.

[0137] Feature set construction module: calculate the corresponding kinematic features, classify the corresponding psychological analysis features, and dynamically filter the features to construct a personalized feature set for the patient.

[0138] Evaluation module: use the selected evaluation scale to analyze the physical condition by calculating the individual automated score and total score of each task for the kinematic feature set constructed by the above modules; use the local large language model to analyze the psychological condition of the patient through intelligent analysis of the knowledge graph based on traditional Chinese medicine theory; and retrieve relevant evaluation results according to the personalized knowledge graph constructed for each patient.

[0139] Maintenance module: maintain the constructed evaluation model, update the constructed feature set by accumulating new patient motion data, and update the parameters of each evaluation model through cross-validation.

[0140] Data storage, management and transmission module: save the patient's score results, and the rehabilitation therapist makes a personalized rehabilitation evaluation report according to the training score, and sends the report, training plan and training suggestions to the patient through this module, which can be freely consulted by the patient.

[0141] As Figure 3 , the following further elaborates on each step:

[0142] Step 1 (device installation and data collection)

[0143] First, divide the rehabilitation evaluation area and install the required test equipment. The patient stands in the planned area, watches a video explaining how to perform the rehabilitation evaluation motion and psychological question / questionnaire, and uses a computer and a 2K intelligent wide-angle USB camera device to collect full-body motion data, and the computer conducts intelligent question and answer interaction between the patient. Then run the customized motion data collection system in the computer (such as Figure 5The key node tracking data required is identified and screened, and a large language model-based psychological question and answer system is run to perform semantic processing on the collected data. Then, a data set is constructed according to the 10 movement tasks contained in the Stroke Rehabilitation Assessment of Movement (STREAM) scale adopted (for example, the therapist can replace the selected assessment scale according to the STREAM assessment tasks). A wide-angle USB camera, a computer, a table, and a chair are necessary equipment for the experimental environment; a pencil, a can, and a baseball are used as assessment aids (the aids are replaced according to the selected scale). When collecting data, it is necessary to ensure that most of the subject's movements can be captured by the lens, and to realize the conversion between the world coordinate system and the camera coordinate system. The psychological assessment question and answer data are collected and screened by AI to ensure the reliability of the data. Therefore, the camera needs to be fixed in the right place and calibrated, and a tripod is used to raise the camera to ensure the angle of view. The final report can be sent to the patient via the Internet, and the patient can choose to view the assessment results freely in daily life or in the hospital for professional rehabilitation training.

[0144] The Stroke Rehabilitation Assessment of Movement (STREAM) scale is an assessment scale used to evaluate the recovery of voluntary movement and basic mobility in the supine, sitting, and standing positions after stroke. There are 30 sub-items, including 10 upper limb movements, 10 lower limb movements, and 10 basic movements. The upper and lower limb movements are rated on a three-point scale, with each item scored 0 (unable to complete) to 2 (normal completion); the basic movements are rated on a four-point scale, with each item scored 0 (unable to complete) to 3 (independent completion). The total score is 70. It is sensitive to changes in stroke patients' rehabilitation and has good inter-rater and intra-rater reliability and internal consistency.

[0145] The whole body movement data is captured by a wide-angle USB camera device developed by Hikvision, a 400 million pixel smart wide-angle USB camera with a fixed focal length of 2.8 mm, a built-in omnidirectional microphone with a pickup distance of 5 meters, a resolution of 2560x1440, support for wide dynamic range, a range of more than 120 dB, suitable for backlight environment monitoring, and support for automatic focusing. This device is the preferred device for high-precision input of patient movement assessment real-time video data.

[0146] Preferably, the motion dataset construction module is constructed. The data is derived from recruited patients with various degrees of stroke. All participants are assessed by experienced senior rehabilitation therapists with STREAM scale and sign the relevant informed consent. After all the assessment tasks are completed, a total of 10 different motion datasets will be generated. These data will be accumulated in the corresponding database according to the task category for later maintenance work.

[0147] The psychological question and answer / questionnaire data are collected through a computer questionnaire system. The question and answer or questionnaire is designed by a professional therapist based on the TCM rehabilitation theory to ensure the effectiveness of data collection.

[0148] Step 2 (data preprocessing)

[0149] The wide-angle USB camera device is used to collect real-time video sources for motion function evaluation and input into the evaluation system. First, the video source is processed by Gaussian filtering, and the feature data is intelligently selected by self-improved YOLO algorithm. Then, for the wide-angle camera to shoot real-time video, the data of the corresponding required joints are extracted by self-improved YOLO algorithm, and the patient's motion thinking is intelligently analyzed, and then the personalized motion data atlas is constructed. The computer intelligent psychological evaluation system is used to collect the intelligent question and answer data of the patient, and the data is arranged by intelligent summary, keyword extraction and other operations according to the question and answer results, and then the personalized psychological data atlas is constructed.

[0150] Step 3 (feature set construction)

[0151] Preferably, the relevant kinematic features are extracted from the video data preprocessed in the above step, such as identifying 17 key points of each human body. Each human body target box information contains six data: horizontal coordinates of human body anchor points, vertical coordinates, width and height of human body prediction box, confidence and class confidence of human body detection box. The inventor defines three main joint angles: elbow angle, shoulder angle, and leg angle. The elbow angle refers to the angle formed by the shoulder, elbow and wrist points in the two-dimensional plane; the shoulder angle is the angle formed by the elbow, shoulder and hip points in the two-dimensional plane; and the leg angle is the angle formed by the hip, knee and calf points in the two-dimensional plane. The inventor proposes to extract the calculation data according to the three types of angles (θ1, θ2, θ3) and the range of motion of the joints to assist in completing the subsequent evaluation. The inventor develops a customized intelligent evaluation system for motion function (such as shown in Figure 4 The system can complete feature extraction and subsequent motion evaluation.

[0152] Preferably, the angles of the elbow, shoulder and leg are used for rehabilitation motion scoring. The calculation formulas of the three types of angles are as follows:

[0153]

[0154] Geometric figures of three types of included angles, such as Figure 4 As shown. To simplify the diagram, the shoulder, elbow, wrist, hip, knee, and lower leg are represented by W, M, P, D, T, and X, respectively. The lines connecting the shoulder and elbow, elbow and wrist, shoulder and hip, hip and knee, and knee and lower leg are represented by WM, MP, WD, DT, and TX, respectively. θ1, θ2, and θ3 represent the angles between the shoulder-elbow and elbow-wrist lines, the elbow-shoulder and shoulder-hip lines, and the hip-knee and knee-lower leg lines, respectively.

[0155] Preferably, when constructing psychological feature values, the llama large language model is used to extract keywords involved in the local TCM knowledge graph. Based on the key information, the corresponding evaluation results and rehabilitation plans are matched. Keywords that appear multiple times during the intelligent collection process will be labeled, and finally a database with personalized psychological features of patients is obtained.

[0156] Step 4 (Evaluation)

[0157] Prioritizing the use of a neural network model as the core of the body assessment model, this model comprises three parts: an input layer, hidden layers, and an output layer. The number of neurons in the input layer corresponds to the number of kinematic features, while the number of neurons in the output layer corresponds to the different scoring categories for each STREAM task. This assessment model can automatically score using AI.

[0158] Prioritize the use of intelligent assessment systems for motor function ( Figure 4 The system automatically scores and sums the scores for each patient's exercise. First, it uses a trained neural network model (one model per exercise task) to automatically score based on the feature set constructed in step 3. Then, it sums the final scores for each task to obtain the total score for this assessment, and simultaneously outputs the corresponding physical assessment content of the scale.

[0159] Preferably, a knowledge graph and intelligent recommendation algorithm are used as the core of the psychological assessment model. The knowledge graph is constructed and stored based on disease knowledge of stroke in both traditional Chinese and Western medicine, as well as traditional Chinese medicine exercise methods. This knowledge graph includes semantic types of symptom classification, assessment and diagnosis, syndrome classification, psychological state, and intervention plans, along with their semantic relationships. Each semantic type has a corresponding description explaining its concept and characteristics. Data is processed by the intelligent psychological data acquisition system. Keyword extraction is performed based on Natural Language Processing (NLP). Text analysis is conducted on the user-input text data to extract entities, relationships, and semantics from the text, outputting keywords and a summary. These keywords are then matched against the local traditional Chinese medicine knowledge graph, and the final result is output through an intelligent recommendation algorithm.

[0160] Step 5 (Rehabilitation Plan)

[0161] Preferably, a knowledge graph and intelligent recommendation algorithm are used as the core of the comprehensive physical and mental rehabilitation program development model. Input to the motor assessment system yields the final evaluation results (e.g., based on scale scores, corresponding symptom and motor state descriptions) and the psychological assessment system yields the final evaluation results (based on keywords and psychological states predicted by the intelligent recommendation algorithm). Through semantic analysis and keyword summarization, corresponding treatment plans are matched with the local knowledge graph. The intelligent recommendation algorithm, combined with the constructed personalized physical and psychological graphs of the patient, ultimately outputs an integrated physical and mental rehabilitation program, which patients can freely view the report.

[0162] Step 6 (Maintenance)

[0163] Prior to improving the accuracy of the assessment model in actual use, the assessment system is configured to accumulate patients' motion data, and the parameters of each assessment model are periodically trained and updated using cross-validation.

[0164] Step 7 (Storage, Management, and Transmission)

[0165] Priority can be given to rehabilitation therapists who can log into an integrated assessment system (such as...) Figure 5 (As shown) Obtain patient information, calculate scores, exercise data, and psychological assessment results as references (the system only displays the total score; to view individual scores, you need to open the individual record) to evaluate the physical and mental rehabilitation of stroke patients, summarize a personalized rehabilitation assessment report, and remotely send the report and treatment opinions of the training plan to the patient through the data storage management and transmission system for the patient to further strengthen their exercises in a targeted manner.

[0166] The present application is suitable for the sequelae and prognosis of nervous system diseases such as stroke.

[0167] Table 1 STREAM stroke rehabilitation score details

[0168]

[0169] It should be noted that the embodiments of the present application can be realized by hardware, software, or a combination of software and hardware. The hardware part can be realized by special logic; the software part can be stored in a memory and executed by a proper instruction execution system, such as a microprocessor or a specially designed hardware. Those skilled in the art can understand that the above-mentioned devices and methods can be realized by computer executable instructions and / or included in processor control codes, for example, such codes are provided on a carrier medium, such as a magnetic disk, a CD or DVD-ROM, a programmable memory, such as a read-only memory (firmware), or a data carrier, such as an optical or electronic signal carrier. The devices of the present application and their modules can be realized by hardware circuits, such as very large scale integrated circuits or gate arrays, semiconductors, such as logic chips, transistors, or programmable hardware devices, such as field programmable gate arrays, programmable logic devices, etc., by software executed by various types of processors, or by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0170] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any modification, equivalent replacement, and improvement within the technical range disclosed by the present application, and within the spirit and principle of the present application, should be covered within the protection scope of the present application.

Claims

1. A stroke patient mind and body assessment rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory, characterized in that, Comprise: Device installation and data acquisition module, data preprocessing module, feature set construction module, evaluation module, maintenance module, data storage management and transmission module; Device installation and data acquisition module, connected with data preprocessing module, data storage management and transmission module, used to build evaluation and rehabilitation task environment and install equipment, according to the adopted physical assessment scale and psychological question and answer system to carry out corresponding assessment task and identify the required data from screening; Data preprocessing module, connected with device installation and data acquisition module, feature set construction module, evaluation module, data storage management and transmission module, used for preprocessing the data collected above, including data filtering and segmentation, semantic recognition and arrangement; Feature set construction module, connected with data preprocessing module, evaluation module, used to calculate corresponding kinematic characteristics, classify corresponding psychological analysis characteristics, and dynamically screen the characteristics to build a personalized feature set for patients; Evaluation module, connected with data preprocessing module, feature set construction module, maintenance module, data storage management and transmission module, used to use the selected evaluation scale to calculate the total score of each task by the trained model to analyze the physical condition; Use local large language model to analyze the psychological condition of patients through knowledge graph intelligent analysis based on traditional Chinese medicine theory; At the same time, according to the individualized knowledge graph constructed according to the different conditions of each patient, the relevant evaluation results are retrieved; Maintenance module, connected with evaluation module, used to maintain the constructed evaluation model, update the constructed feature set by accumulating the motion data of new patients, and update the parameters of each evaluation model through cross validation method; Data storage management and transmission module, connected with device installation and data acquisition module, feature set construction module, evaluation module, used to save the score results of patients, and the rehabilitation therapist makes individualized rehabilitation evaluation report according to the training score, and sends the report, training plan and training opinion to the patient through the module, and the patient can freely inquire.

2. The system according to claim 1, wherein the system is characterized in that, The device installation and data acquisition module method is as follows: Divide the rehabilitation evaluation area and install the required test equipment; The patient stands in the planned area, watches a video explanation of how to perform a rehabilitation evaluation action guidance and psychological question and answer / questionnaire, and uses a computer and a 2K intelligent wide-angle USB camera device to collect full body motion data, and the computer psychological question and answer system and the patient carry out intelligent question and answer interaction; Then run the customized motion data acquisition system in the computer to identify and screen the required key joint tracking data, and then run the psychological question and answer system based on large language model to perform semantic processing on the collected data; Then, according to the 10 motion tasks contained in the adopted stroke rehabilitation motor function scale, the data set is constructed; The data is derived from recruited patients with various degrees of stroke; all participants are assessed by experienced senior rehabilitation therapists with STREAM scale and sign the relevant informed consent; after all the evaluation tasks are completed, a total of 10 different motion data sets will be generated; these data will be accumulated in the corresponding database according to the task type for later maintenance work; Psychological question and answer / questionnaire data are collected through a computer questionnaire system; the questions and answers or questionnaire are designed by professional therapists based on TCM rehabilitation theory.

3. The system of claim 1, wherein the system is characterized in that, The data preprocessing module method is as follows: A wide-angle USB camera device is used to collect real-time video sources for motor function evaluation and input the evaluation system; Firstly, the video source is subjected to Gaussian filtering processing, and the self-improved YOLO algorithm is used for intelligent screening of feature data; Then, for the real-time video shot by the wide-angle camera, the self-improved YOLO algorithm is used to extract the data of the corresponding required joints, intelligently analyze the patient's movement thinking, and then construct a personalized motion data graph; the computer intelligent psychological evaluation system is used to collect the patient's intelligent question and answer data, and the data is arranged through intelligent summary, keyword extraction and other operations according to the question and answer results, and then a personalized psychological data graph is constructed.

4. The system according to claim 1, wherein the system is characterized in that, The feature set construction module method is as follows: The relevant kinematic features are extracted from the video data preprocessed in the above steps, and 17 key points of each human body are identified, and each human body target box information contains six data: horizontal coordinates and vertical coordinates of the human body anchor point, width and height of the human body prediction box, confidence and class confidence of the human body detection box; Three main joint angles are defined: elbow angle, shoulder angle and leg angle; the elbow angle refers to the angle formed by the shoulder, elbow and wrist in the two-dimensional plane; the shoulder angle refers to the angle formed by the elbow, shoulder and hip in the two-dimensional plane; the leg angle refers to the angle formed by the hip, knee and calf in the two-dimensional plane; The angles of the elbow, shoulder and leg are used for rehabilitation exercise scoring; the calculation formulas of the three types of angles are as follows: The shoulder, elbow joint, wrist, hip, knee and calf are represented by W, M, P, D, T and X respectively; the shoulder-elbow line, elbow-wrist line, shoulder-hip line, hip-knee line and knee-calf line are represented by WM, MP, WD, DT and TX respectively; θ1, θ2 and θ3 represent the angles of the shoulder-elbow line and elbow-wrist line, elbow-shoulder line and shoulder-hip line, and hip-knee line and knee-calf line respectively.

5. The system of claim 1, wherein the system is characterized in that, The evaluation module method is as follows: A neural network model is used as the core of the body evaluation model; the neural network model includes an input layer, a hidden layer and an output layer; the number of input layer neurons is the number of kinematic features, and the number of output layer neurons corresponds to several different scores of each task of STREAM; this evaluation model can be automatically scored by AI; The patient's exercise is automatically scored and summed by using the motor function intelligent evaluation system; first, the trained neural network model is used to automatically score according to the feature set constructed in step 3, then the final score of each task is summed to obtain the total score of this evaluation, and the body evaluation content corresponding to the scale is output; The knowledge graph and intelligent recommendation algorithm are used as the core of the psychological evaluation model; the knowledge graph is constructed and stored according to the disease knowledge of stroke in the field of traditional Chinese and western medicine and the traditional Chinese exercise method, including the semantic types of symptom classification, evaluation and diagnosis, syndrome classification, psychological state, intervention scheme and the semantic relationship between them, each semantic type has a corresponding description to explain the concept and characteristics of the semantic type.

6. The system of claim 1, wherein the system is characterized by: The data storage management and transmission module method is as follows: The rehabilitation therapist logs in the integrated evaluation system to obtain patient information, uses the calculated score, exercise data and psychological evaluation results as a reference to assess the physical and mental rehabilitation of the stroke patient, and summarizes a personalized rehabilitation evaluation report. The report and the treatment suggestions of the training plan are sent to the patient through the data storage management and transmission system for further targeted practice.

7. A stroke patient mind and body assessment and rehabilitation integrated system based on traditional Chinese medicine rehabilitation theory according to any one of claims 1-6, wherein, The stroke patient physical and mental evaluation and rehabilitation integrated method based on traditional Chinese medicine rehabilitation theory comprises: Step 1, build the evaluation and rehabilitation task environment and install the equipment through the equipment installation and data acquisition module, carry out the corresponding evaluation tasks according to the adopted physical evaluation scale and psychological question and answer system, and identify the required data; Step 2, pre-process the collected data through the data pre-processing module, including data filtering and segmentation, semantic recognition and organization; Step 3, calculate the corresponding kinematic features, classify the corresponding psychological analysis features, and dynamically screen the features to build a personalized feature set for the patient; Step 4, use the selected evaluation scale through the evaluation module, automatically score each task and calculate the total score of the kinematic feature set constructed by the above module through the trained model to analyze the physical condition; use the local large language model to intelligently analyze the psychological condition of the patient based on the knowledge graph of traditional Chinese medicine theory; and according to the personalized knowledge graph constructed for each patient, retrieve the related evaluation results; Step 5, maintain the constructed evaluation model through the maintenance module, update the constructed feature set by accumulating the exercise data of new patients, and update the parameters of each evaluation model through cross-validation method; Step 6, save the score results of the patient through the data storage management and transmission module, and the rehabilitation therapist makes a personalized rehabilitation evaluation report according to the training score, and sends the report, training plan and training suggestions to the patient through the module, which can be freely consulted by the patient.

8. A computer device, comprising: The computer device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the stroke patient physical and mental evaluation and rehabilitation integrated method based on traditional Chinese medicine rehabilitation theory.

9. A computer readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the stroke patient mind and body assessment and rehabilitation integration method based on traditional Chinese medicine rehabilitation theory according to claim 7.

10. An information data processing terminal, characterized by The information data processing terminal is used to implement the stroke patient mind and body assessment and rehabilitation integration system based on traditional Chinese medicine rehabilitation theory according to any one of claims 1-6.

Images