Smart management method and system for home exercise rehabilitation for patients with copd

Abstract

The application provides a home exercise rehabilitation intelligent management method and system for COPD patients, and relates to the fields of medical engineering and digital health technology.The method comprises the following steps: prompting the patient to fill in an evaluation scale; determining a pending rehabilitation exercise option; after receiving confirmation information, taking the pending rehabilitation exercise option as a rehabilitation exercise recommended option; selecting a rehabilitation exercise video; playing the rehabilitation exercise video, shooting an exercise video of the patient performing the rehabilitation exercise, collecting various physiological index data of the patient; determining an exercise suitability index; determining a new rehabilitation exercise recommended option; and determining a compliance report.According to the application, the exercise suitability index of the patient can be determined based on the physiological index and the rehabilitation exercise video, and then the new rehabilitation exercise recommended option and the compliance report are determined.The real-time monitoring of the suitability of the rehabilitation exercise for the patient can recommend personalized rehabilitation exercise for the patient, improve the accuracy of the rehabilitation exercise recommendation, and improve the compliance of the patient.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical engineering and digital health technology, and particularly relates to a home exercise rehabilitation intelligent management method and system for COPD patients. BACKGROUND

[0002] With the rapid development of network information technology, mobile medical treatment has gradually attracted attention in the management of COPD patients. The application forms of existing mobile medical devices mainly include remote medical information systems, APPs and WeChat. The remote medical system mainly provides computers, home exercise devices and pulmonary rehabilitation programs for patients to improve the prognosis of COPD patients. However, this method requires patients to be familiar with the use of computers, and the computer is large in size, difficult to carry and expensive in hardware, which has certain practical difficulties. Special APPs can collect, transmit, analyze and feedback user health information to provide high-quality and efficient health management for patients. In addition, the APP can provide personalized exercise training plans for patients, and dynamically adjust the training load by combining real-time exercise performance and subjective feedback with intelligent algorithms to achieve precise, safe and effective exercise intervention. However, mobile APPs have high development costs, long development cycles, require patients to download and use them from the corresponding application store, and occupy memory. At the same time, most COPD patients are older users, which will further reduce the enthusiasm of patients to use mobile medical APPs and affect their acceptance of mobile applications. Therefore, related technologies are difficult to monitor the appropriateness of rehabilitation exercise for patients in real time, and it is difficult to recommend personalized rehabilitation exercise for patients, resulting in low rehabilitation effect.

[0003] The information disclosed in the background section of this application is only intended to deepen the understanding of the general background of the application and should not be considered as recognition or in any form as implying that this information constitutes prior art known to those skilled in the art. SUMMARY

[0004] The present application provides a home exercise rehabilitation intelligent management method and system for COPD patients, which can solve the technical problems that related technologies are difficult to monitor the appropriateness of rehabilitation exercise for patients in real time, and it is difficult to recommend personalized rehabilitation exercise for patients.

[0005] According to a first aspect of the present application, a home exercise rehabilitation intelligent management method for COPD patients is provided, comprising:

[0006] At the beginning of the rehabilitation period, prompting the patient to fill in the evaluation scale at the patient end;

[0007] The platform end collects the evaluation scale, determines the pending rehabilitation exercise options according to the evaluation scale and the rehabilitation exercise type selection record of the patient in the historical rehabilitation period, and sends them to the medical end;

[0008] After the medical side confirms, the pending rehabilitation exercise option is sent to the patient side as a rehabilitation exercise recommendation option;

[0009] The patient is prompted to complete wearing the wearable monitoring device, and the patient is prompted to stand in front of the camera of the terminal device, and select a rehabilitation exercise video in the rehabilitation exercise recommendation option;

[0010] During the rehabilitation exercise, the rehabilitation exercise video is played, the movement video of the patient performing the rehabilitation exercise is captured by the camera, and the physiological index data of the patient is collected by the wearable monitoring device;

[0011] The platform side fuses the features of the movement video and the physiological index data through the movement monitoring model to obtain the action trend feature, and then determines the movement suitability index of the patient, and sends it to the medical side. The feature fusion includes splicing the feature information of the video frame of the movement video with the feature information of the physiological index data, performing time series analysis on the spliced feature information, and determining the movement suitability index;

[0012] After receiving the rehabilitation exercise adjustment scheme from the medical side, a new rehabilitation exercise recommendation option is determined;

[0013] At the end of the rehabilitation period, a compliance report is determined according to the historical record of the patient performing the rehabilitation exercise.

[0014] According to the present application, the platform side collects the evaluation scale, and determines the pending rehabilitation exercise option according to the evaluation scale and the rehabilitation exercise type selection record of the patient in the historical rehabilitation period, and sends it to the medical side, including:

[0015] Combine the questions in each category of the evaluation scale with the answers of the patient to obtain category statements;

[0016] Process each category statement through a natural language processing model to obtain category semantic feature information;

[0017] Process the description information of each rehabilitation exercise option through a natural language processing model to obtain description semantic feature information of each category of rehabilitation exercise option;

[0018] Process the description semantic feature information and the category semantic feature information of each rehabilitation exercise option through the first cross-attention mechanism to obtain the training adaptability description information of each rehabilitation exercise option;

[0019] Determine the training adaptability index of each rehabilitation exercise option according to the training adaptability description information;

[0020] According to the historical category information of the rehabilitation exercise option selected in the rehabilitation exercise type selection record of the patient in the historical rehabilitation period;

[0021] According to the historical category information and the training adaptability index, a pending rehabilitation exercise option is determined and sent to the medical care end.

[0022] According to the present application, the pending rehabilitation exercise option is determined and sent to the medical care end, comprising:

[0023] According to the training adaptability index, an adaptive rehabilitation exercise option is selected from the plurality of rehabilitation exercise options;

[0024] According to the historical category information, a historical selection proportion of each category is determined;

[0025] According to the category to which the adaptive rehabilitation exercise option belongs, the training adaptability index of the adaptive rehabilitation exercise option, and the historical selection proportion, a selection index of each adaptive rehabilitation exercise option is determined;

[0026] According to the selection index, the pending rehabilitation exercise option is selected from the adaptive rehabilitation exercise options and sent to the medical care end.

[0027] According to the present application, the selection index of each adaptive rehabilitation exercise option is determined, comprising:

[0028] According to the formula

[0029] ,

[0030] The selection index of the i-th adaptive rehabilitation exercise option is determined , wherein, is the training adaptability index of the i-th adaptive rehabilitation exercise option, is the historical selection proportion of the historical category information of the same category as the category to which the i-th adaptive rehabilitation exercise option belongs, and is a random weight, wherein, is the probability that 1 is , is the probability that 0 is , and i is a positive integer.

[0031] According to the present application, the platform end fuses the motion video and the physiological index data through the motion monitoring model to obtain the action trend feature, and further determines the motion adaptability index of the patient, comprising:

[0032] The motion video is sampled to obtain a plurality of motion video frames;

[0033] According to the time stamp of the motion video frame, a plurality of rehabilitation exercise video frames are selected from the plurality of video frames of the rehabilitation exercise video;

[0034] According to the timestamp of the motion video frame, target physiological indicator data is selected from the physiological indicator data at multiple time points;

[0035] First image feature information of the multiple partitions of the jth motion video frame is obtained, and second image feature information of the multiple partitions of the jth rehabilitation motion video frame is obtained;

[0036] The first image feature information and the second image feature information are processed through the second cross-attention mechanism of the motion monitoring model to obtain multiple fusion image feature information;

[0037] The multiple fusion image feature information is spliced and processed through the first full connection layer and the first activation layer to obtain the jth action feature information;

[0038] The jth action feature information and the j-1th action hidden feature information are processed through the motion timing sub-model of the motion monitoring model to obtain the jth action hidden feature information, wherein when j=1, the j-1th action hidden feature information is a zero vector, and the action hidden feature information is used to describe the action trend characteristics of the patient;

[0039] The physiological feature information of the jth set of target physiological indicator data is obtained through the data feature extraction level of the motion monitoring model;

[0040] The jth physiological feature information and the jth action hidden feature information are spliced to obtain the jth motion state information;

[0041] The jth motion state information and the j-1th state hidden feature information are processed through the state timing sub-model of the motion monitoring model to obtain the jth state hidden feature information, wherein when j=1, the j-1th state hidden feature information is a zero vector;

[0042] When j= The jth state hidden feature information is input into the second full connection layer and the second activation layer for processing to obtain the motion suitability index of the patient, wherein, is the serial number of the last motion video frame.

[0043] According to the present application, the training step of the motion monitoring model comprises:

[0044] Obtaining sample motion video frames, sample rehabilitation motion video frames and sample physiological indicator data;

[0045] The kth sample action hidden feature information is obtained through the kth sample motion video frame and the kth sample rehabilitation motion video frame, and the second cross-attention mechanism and the motion timing sub-model of the motion monitoring model;

[0046] Obtaining the state hidden feature information of the kth sample by the kth sample action hidden feature information and the kth set of sample physiological index data, and the state time sequence submodel of the motion monitoring model;

[0047] Obtaining the loss function of the motion monitoring model according to the sample action hidden feature information and the sample state hidden feature information;

[0048] Training the motion monitoring model according to the loss function of the motion monitoring model, and obtaining the trained motion monitoring model.

[0049] According to the application, the loss function of the motion monitoring model is obtained, comprising:

[0050] Determining the first action feature information of each sample motion video frame and the second action feature information of each sample rehabilitation motion video frame by the action recognition model;

[0051] Inputting the sample action hidden feature information into the third full connection layer and the third activation layer for processing, and obtaining the action state matching index of each sample motion video frame;

[0052] Determining the action state loss function according to the action state matching index, the first action feature information and the second action feature information;

[0053] Inputting the sample action hidden feature information into the fourth full connection layer and the fourth activation layer for processing, and obtaining the action trend matching index of each sample motion video frame;

[0054] Determining the action trend loss function according to the action trend matching index, the first action feature information and the second action feature information;

[0055] Inputting the last sample state hidden feature information into the second full connection layer and the second activation layer for processing, and obtaining the sample motion suitability index;

[0056] Obtaining the motion suitability loss function according to the sample motion suitability index and the labeled information of the sample physiological index data;

[0057] Obtaining the loss function of the motion monitoring model according to the action state loss function, the action trend loss function and the motion suitability loss function.

[0058] According to the application, the action trend loss function is determined, comprising:

[0059] According to the formula

[0060] ,

[0061] Determining the action trend loss function , wherein This refers to the first motion feature information of the k-th sample motion video frame. This refers to the second motion feature information of the t-th sample rehabilitation exercise video frame. The action trend matching index is the latent feature information of the action of the k-th sample. The preset number of comparisons is N, where N is the number of sample motion video frames, max is the function to take the maximum value, min is the function to take the minimum value, and k, N, and All are positive integers.

[0062] According to a second aspect of the present invention, a smart management system for home-based exercise rehabilitation for COPD patients is provided, comprising:

[0063] The assessment scale module prompts patients to fill out the assessment scale on the patient's end at the beginning of the rehabilitation cycle;

[0064] The module for pending rehabilitation exercise options involves the platform collecting assessment scales, determining pending rehabilitation exercise options based on the assessment scales and the patient's rehabilitation exercise type selection records in the historical rehabilitation cycle, and sending them to the medical staff.

[0065] The rehabilitation exercise recommendation option module, after confirmation by medical staff, sends the pending rehabilitation exercise options as rehabilitation exercise recommendations to the patient's end;

[0066] The rehabilitation exercise video module prompts the patient to put on the wearable monitoring device, and prompts the patient to stand in front of the camera of the terminal device and select the rehabilitation exercise video from the rehabilitation exercise recommendation options;

[0067] The data acquisition module plays the rehabilitation exercise video through the patient's terminal device during the rehabilitation exercise process, captures the rehabilitation exercise video of the patient through the camera, and collects various physiological indicator data of the patient through wearable monitoring devices.

[0068] The exercise suitability index module involves the platform fusing features from exercise videos and physiological index data using an exercise monitoring model to obtain movement trend features, thereby determining the patient's exercise suitability index and sending it to the medical staff. The feature fusion includes splicing the feature information of the video frames of the exercise video with the feature information of the physiological index data, and performing time-series analysis on the spliced ​​feature information to determine the exercise suitability index.

[0069] The new rehabilitation exercise recommendation option module determines the new rehabilitation exercise recommendation options after receiving the rehabilitation exercise adjustment plan from the medical staff.

[0070] The compliance reporting module determines compliance reports based on the patient's historical records of rehabilitation exercises at the end of the rehabilitation cycle.

[0071] By adopting the above technical solutions, the application can achieve the following technical effects:

[0072] According to the application, the patient can be prompted to fill in the evaluation scale at the beginning of the rehabilitation period, and the pending rehabilitation exercise option is determined. After receiving the confirmation information from the medical side, the pending rehabilitation exercise option is taken as the rehabilitation exercise recommended option, and then the rehabilitation exercise video is selected. During the process of starting the rehabilitation exercise of the patient, the exercise video of the patient performing the rehabilitation exercise is shot, and various physiological index data of the patient are collected, and then the exercise suitability index of the patient is determined, and the new rehabilitation exercise recommended option is determined, and then the compliance report is determined. The suitability of the rehabilitation exercise for the patient is monitored in real time, and personalized rehabilitation exercise is recommended for the patient, which improves the accuracy of the rehabilitation exercise recommendation and improves the patient compliance. And at the beginning of the rehabilitation period, the patient can be prompted to use the terminal device to fill in the evaluation scale at the patient end, to provide basic data for determining the pending rehabilitation exercise option. When determining the pending rehabilitation exercise option, the selection index of each adaptive rehabilitation exercise option is determined based on the collected evaluation scale and the rehabilitation exercise type selection record of the patient in the historical rehabilitation period, and then the pending rehabilitation exercise option is determined and sent to the medical side. When determining the selection index, the situation that the rehabilitation option with high adaptability to the patient but less selected by the patient in the past has a probability of being determined as the pending rehabilitation exercise option is considered, and the random weight is determined based on the training adaptability index of the adaptive rehabilitation exercise option and the historical selection proportion. The accuracy, objectivity and comprehensiveness of the determination of the selection index of each adaptive rehabilitation exercise option are improved, thereby improving the accuracy of the pending rehabilitation exercise option, significantly improving the relevance and training acceptability of the pending rehabilitation exercise option, and then improving the completion degree and compliance. When training the exercise monitoring model, the action state loss function, the action trend loss function and the exercise suitability loss function are obtained, and then the loss function of the exercise monitoring model is obtained. The situation that the patient's action in the sample exercise video frame may lag behind the standard action in the sample rehabilitation exercise video frame with the same timestamp is considered, the sample rehabilitation exercise video frame corresponding to the second action feature information with the maximum similarity to the first action feature information of the sample exercise video frame is taken as the sample rehabilitation exercise video frame corresponding to the sample exercise video frame, thereby determining the value of the accurate matching degree between the action trend of the patient in the sample exercise video frame and the standard action trend, and the consistency degree with the corresponding action trend matching index, and then determining the action trend loss function. And the action state loss function and the exercise suitability loss function are set, the accuracy of obtaining the sample action hidden feature information, the sample state hidden feature information and the sample exercise suitability index is improved, thereby assisting the training of the exercise monitoring model, improving the accuracy and pertinence of the training, and improving the performance of the training exercise monitoring model. Further, the exercise suitability index can be sent to the medical side, and after receiving the rehabilitation exercise adjustment scheme from the medical side, the new rehabilitation exercise recommended option is determined, and then the compliance report is determined at the end of the rehabilitation period.Real-time monitoring of the appropriateness of rehabilitation exercise for patients, and recommending personalized rehabilitation exercise for patients, improves the accuracy of rehabilitation exercise recommendation and improves patient compliance.

[0073] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present application. Other features and aspects of the present application will become more apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other embodiments according to these drawings without creative labor.

[0075] Figure 1 An exemplary flowchart of a home exercise rehabilitation intelligent management method for COPD patients according to an embodiment of the present application is shown;

[0076] Figure 2 An exemplary flowchart of determining a pending rehabilitation exercise option according to an embodiment of the present application is shown;

[0077] Figure 3 An exemplary flowchart of determining a patient's exercise appropriateness index according to an embodiment of the present application is shown;

[0078] Figure 4 An exemplary block diagram of a home exercise rehabilitation intelligent management system for COPD patients according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0079] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0080] The technical solutions of the present application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0081] Figure 1An exemplary flowchart of a home exercise rehabilitation intelligent management method for COPD patients according to an embodiment of the present application is shown, which comprises:

[0082] Step S1, at the beginning of the rehabilitation period, prompting the patient to fill in the evaluation scale at the patient end;

[0083] Step S2, the platform end collects the evaluation scale, determines the pending rehabilitation exercise options according to the evaluation scale and the rehabilitation exercise type selection record of the patient in the historical rehabilitation period, and sends them to the medical end;

[0084] Step S3, after confirmation at the medical end, the pending rehabilitation exercise options are sent to the patient end as rehabilitation exercise recommended options;

[0085] Step S4, prompting the patient to complete wearing the wearable monitoring device, prompting the patient to stand in front of the camera of the terminal device, and selecting a rehabilitation exercise video in the rehabilitation exercise recommended options;

[0086] Step S5, during the rehabilitation exercise, playing the rehabilitation exercise video, taking the exercise video of the patient performing the rehabilitation exercise through the camera, and collecting various physiological index data of the patient through the wearable monitoring device;

[0087] Step S6, the platform end fuses the features of the exercise video and the physiological index data through the exercise monitoring model, obtains the action trend feature, and further determines the exercise suitability index of the patient, and sends it to the medical end, wherein the feature fusion comprises splicing the feature information of the video frame of the exercise video with the feature information of the physiological index data, performing time series analysis on the spliced feature information, and determining the exercise suitability index;

[0088] Step S7, after receiving the rehabilitation exercise adjustment scheme from the medical end, determining new rehabilitation exercise recommended options;

[0089] Step S8, at the end of the rehabilitation period, determining the compliance report according to the historical record of the patient performing the rehabilitation exercise.

[0090] The home exercise rehabilitation intelligent management method and system for COPD patients according to the embodiments of the present application can prompt the patient to fill in the evaluation scale at the beginning of the rehabilitation period, and determine the pending rehabilitation exercise options. After receiving the confirmation information from the medical end, the pending rehabilitation exercise options are selected as rehabilitation exercise recommended options, and then a rehabilitation exercise video is selected. During the process of the patient starting the rehabilitation exercise, the exercise video of the patient performing the rehabilitation exercise is taken, and various physiological index data of the patient are collected, and then the exercise suitability index of the patient is determined, and new rehabilitation exercise recommended options are determined, and then a compliance report is determined. The rehabilitation exercise is monitored in real time for the suitability of the patient, and personalized rehabilitation exercise is recommended for the patient, which improves the accuracy of the rehabilitation exercise recommendation and improves the patient compliance.

[0091] According to an embodiment of the present application, in step S1, at the beginning of a rehabilitation period, the patient is prompted to fill in an assessment scale at the patient end using a terminal device. The rehabilitation period can be one week, two weeks, etc. At the beginning of the rehabilitation period, the patient is prompted to fill in an assessment scale at the patient end using a terminal device (e.g. a mobile phone, etc.). The assessment scale can include information such as the patient's height, weight, age, vital signs, 6-minute walk test, lung function test results, COPD assessment scale, patient dyspnea assessment scale, Brog scale, etc.

[0092] In an example, the patient terminal includes a plurality of first-level entries, second-level entries and third-level entries. In the first-level entry "Home", the second-level entries of "Search box", "Information push box", "Weather", "Consultation function" and "Patient exchange" are included. In the third-level entries, the search box supports fuzzy search, the information push box can push relevant content according to the patient's needs and interests, the weather can display weather data such as temperature, humidity and wind speed, and the patient and his family can decide the type of exercise training according to the daily weather information. In the consultation function, the user can choose to consult the nurse, doctor and rehabilitation physician about the disease and exercise related problems. In the patient exchange, the patient can exercise with the patient and share experience. In the first-level entry "Exercise module", the second-level entries of "Exercise assessment", "Exercise prescription", "Exercise type", "Exercise monitoring" and "Exercise diary" are included. In the third-level entries, the exercise assessment includes physiological, exercise capacity and symptom assessment, such as height, weight, age, vital signs, 6-minute walk test, lung function test results, COPD Assessment Test (CAT), Modified Medical Research Council (mMRC), Brog scale, etc. The exercise prescription can be matched to the corresponding individualized exercise prescription according to the assessment results, and different exercise types can be selected by the user's preference to choose the preferred exercise method, and finally realize the individualization of exercise intervention. The implementation of exercise prescription follows the FITT-VP principle, including frequency, intensity, time, exercise type, and total amount and progression. After the patient selects the corresponding exercise prescription, the key points of the exercise and safety instructions should be explained. The exercise type includes aerobic exercise (such as walking, jogging, swimming, up and down stairs, cycling and bodybuilding, etc.), resistance training (such as hand dynamometer, elastic band, sandbag, blowing ball, dumbbell, barbell, etc.), flexibility training, breathing training (such as pursed-lip breathing, abdominal breathing, abdominal breathing load method, etc.), TCM rehabilitation (such as Taijiquan, Baduanjin, Liuzi Jue, etc.). Exercise monitoring can record the patient's heart rate, blood oxygen saturation and other data during exercise through wearable devices. In the exercise diary, there is a daily check-in module, calorie consumption, exercise duration, walking distance, use of special drugs, and recording of special events. The background sets up video playback monitoring, and 80% of the set duration is considered to complete the daily exercise amount, and the daily check-in is displayed in the check-in calendar, and the corresponding points can be obtained. Through the points and visualized data presentation, the patient's exercise enthusiasm can be stimulated, and the positivity and compliance of exercise can be improved.

[0093] In the example, in the first-level entry "health education module" in the patient end, the second-level entries including "disease knowledge", "knowledge of lung function test", "knowledge of exercise rehabilitation", "quitting smoking", and "emotion management" and the like are included. In the third-level entry, in the disease knowledge, the physiological characteristics of the respiratory system of the elderly (text and image, voice), understanding of COPD, risk factors of COPD (text and image, voice), common manifestations of COPD (text and image, voice), how the patient can simply judge that he or she may have COPD (text and image, voice), how to know that he or she has COPD (text, video, voice), hazards of COPD (text, video, voice), common complications of COPD (text, video, voice), identification and processing of acute exacerbation of COPD (text and image, voice), common preventive measures and levels of COPD (text and image, voice) and the like are included. In the knowledge of lung function test, common tests of COPD (text and image, voice), lung function test (text and image, voice), benefits of lung function test (text and image, voice), matters needing attention of lung function test (text and image, voice) and the like are included. The knowledge of exercise rehabilitation can also be displayed in the form of text and image, video, voice and the like. In the quitting smoking, the importance of quitting smoking, common methods of quitting smoking (text and image, voice) and the like are included. In the emotion management, how the COPD patient carries out the family psychological nursing (text and image, voice) and the like are included. In the first-level entry "my" in the patient end, the second-level entries including "personal information", "exercise record", "my collection", "opinion feedback", "timely reminder", "emergency assistance" and "system setting" and the like are included. In the third-level entry, the personal information includes general demographic information (for example, gender, birth year and month, height, weight, current residence, marital status, main caregiver, educational level, family per capita monthly income, medical payment method and the like) and disease-related information (for example, disease duration, number of acute exacerbations, severity of dyspnea, smoking, whether to carry out home exercise rehabilitation, type of exercise rehabilitation, comorbidity and the like). In the exercise record, the assessment time, score, result and suggestion of the self-evaluation questionnaire of the patient in the recent 7 days or one month can be displayed, and a trend chart or other simple statistical graph is used for visual display. In the my collection, the patient can add the content in the health education module by clicking the "heart-shaped mark" set in the text and image or video. The opinion feedback can be used for the patient to feedback the abnormal problems in the use process. The timely reminder can be used for the patient and his or her family to set the exercise reminding time according to their own exercise habits. In the emergency assistance, a red button can be set, and the patient can click to send a help signal. In the system setting, the security setting (for example, password reset, account cancellation, privacy agreement and the like), personalized setting (for example, font, avatar and the like), punch card points and version update are included.

[0094] In this way, the patient can be prompted to use the terminal device to fill in the evaluation scale in the patient end at the beginning of the rehabilitation period, so as to provide basic data for determining the pending rehabilitation exercise options.

[0095] Figure 2 An exemplary flowchart of determining a pending rehabilitation exercise option according to an embodiment of the present application is shown.

[0096] According to an embodiment of the present application, the platform side collects the evaluation scale, and determines a pending rehabilitation exercise option according to the evaluation scale and the patient's rehabilitation exercise type selection record in the historical rehabilitation period, and sends it to the medical side, including: combining the question in each category of the evaluation scale with the answer of the patient, obtaining a category statement; processing each category statement through a natural language processing model to obtain category semantic feature information; processing the description information of each rehabilitation exercise option through a natural language processing model to obtain description semantic feature information of each category of rehabilitation exercise option; processing the description semantic feature information and the category semantic feature information of each rehabilitation exercise option through a first cross-attention mechanism to obtain training adaptability description information of each rehabilitation exercise option; determining the training adaptability index of each rehabilitation exercise option according to the training adaptability description information; determining the historical category information of the rehabilitation exercise option selected in the patient's rehabilitation exercise type selection record in the historical rehabilitation period; determining the pending rehabilitation exercise option according to the historical category information and the training adaptability index, and sending it to the medical side.

[0097] According to an embodiment of the present application, the question in each category of the evaluation scale is combined with the answer of the patient to obtain a category statement. For example, the category of a certain evaluation scale is six-minute walk test length, and the answer of the patient is 400 meters. The six-minute walk test length is 400 meters, which is the category statement.

[0098] According to an embodiment of the present application, each category statement is processed through a natural language processing model (such as a natural language processing deep model such as BERT) to obtain the category semantic feature information, which can describe the patient's current rehabilitation condition according to the patient's answer. The number of category semantic feature information is the same as the number of categories in the evaluation scale.

[0099] According to an embodiment of the present application, similar to obtaining category semantic feature information, the description information of each rehabilitation exercise option is processed through a natural language processing model to obtain description semantic feature information of each category of rehabilitation exercise option, which can describe the characteristics of each rehabilitation exercise option. The description information of each rehabilitation exercise option can include the type, difficulty and applicable patient of the rehabilitation exercise, for example, the description information of Taijiquan is low-difficulty traditional Chinese medicine rehabilitation, which is suitable for patients with poor rehabilitation condition.

[0100] According to an embodiment of the present application, the description semantic feature information and the category semantic feature information of each rehabilitation exercise option are processed by a first cross-attention mechanism to obtain training adaptability description information of each rehabilitation exercise option. The Q matrix in the first cross-attention mechanism is a query matrix. Multiplying the description semantic feature information by the query matrix can project the description semantic feature information into an attention space more suitable for calculation, thereby obtaining a transformed vector, which is a query vector. The K matrix in the first cross-attention mechanism is a key-value matrix. Multiplying the category semantic feature information by the key-value matrix can project the category semantic feature information into an attention space more suitable for calculation, thereby obtaining a transformed vector, which is a key-value vector. Based on the same processing manner, a key-value vector corresponding to each category semantic feature information can be obtained. Since the query matrix and the key-value matrix have the same shape, the dimensions of the query vector and the key-value vector are also the same, and then the dot product operation of the query vector and the key-value vector can be performed. Further, multiplying the query vector by the key-value vector corresponding to each category semantic feature information (i.e., dot product operation), dividing by the square root of the dimension of the key-value vector, and then performing operation by a softmax activation function can obtain a related weight of each category semantic feature information, which can describe the relevance between each category semantic feature information and the description semantic feature information, i.e., the relevance between the current rehabilitation condition of the patient and the rehabilitation exercise option. The greater the related weight, the higher the relevance between the current rehabilitation condition of the patient and the rehabilitation exercise option, and the more suitable the rehabilitation exercise for the patient. Of course, the query matrix and the key-value matrix are learnable weight matrices, and the parameters of the query matrix and the key-value matrix can be determined by training.

[0101] According to an embodiment of the present application, multiplying each category semantic feature information by a weight matrix of the first cross-attention mechanism (i.e., the V matrix of the first cross-attention mechanism) can obtain a weight vector corresponding to each category semantic feature information. The weight matrix is a learnable weight matrix, and the parameters of the weight matrix can be determined by training. Further, weighted sum of a plurality of weight vectors by the related weight corresponding to each category semantic feature information can obtain a new vector, which is the training adaptability description information, which can fuse the features of the rehabilitation condition of the patient related to the rehabilitation exercise option and describe the features of the rehabilitation exercise option, for determining whether the rehabilitation exercise option matches the rehabilitation condition of the patient, and improving the accuracy and pertinence of the feature expression of the rehabilitation exercise option. Based on the same processing manner, the training adaptability description information of each rehabilitation exercise option can be obtained.

[0102] According to an embodiment of the present application, the training adaptability index of each rehabilitation exercise option is determined according to the training adaptability description information. The training adaptability description information corresponding to each rehabilitation exercise option is input into a full connection layer and an activation layer (for example, a Sigmoid activation function) for processing, so as to map the training adaptability description information of each rehabilitation exercise option to the probability (for example, 0.6, 0.8, etc.) of the adaptation of each rehabilitation exercise option to the patient, that is, the training adaptability index, which can describe the degree of adaptation of each rehabilitation exercise option to the patient. The higher the training adaptability index is, the more suitable the corresponding rehabilitation exercise option is to the patient.

[0103] According to an embodiment of the present application, the historical category information to which the rehabilitation exercise option selected by the patient in the historical rehabilitation cycle is determined according to the rehabilitation exercise type selection record. The category to which the rehabilitation exercise option selected by the patient in the historical rehabilitation cycle is the historical category information, for example, the category of lifting dumbbells selected by the patient in the rehabilitation exercise type selection record in a certain historical rehabilitation cycle is resistance training, and the resistance training is the historical category information corresponding to the historical rehabilitation cycle.

[0104] According to an embodiment of the present application, the pending rehabilitation exercise option is determined according to the historical category information and the training adaptability index and sent to the medical side, including: selecting the adaptive rehabilitation exercise option from the plurality of rehabilitation exercise options according to the training adaptability index; determining the historical selection proportion of each category according to the historical category information; determining the selection index of each adaptive rehabilitation exercise option according to the category to which the adaptive rehabilitation exercise option belongs, the training adaptability index of the adaptive rehabilitation exercise option and the historical selection proportion; and selecting the pending rehabilitation exercise option from the adaptive rehabilitation exercise options according to the selection index and sending it to the medical side.

[0105] According to an embodiment of the present application, the adaptive rehabilitation exercise option is selected from the plurality of rehabilitation exercise options according to the training adaptability index. The training adaptability index threshold (for example, 0.8, 0.85, etc.) can be set, and when the training adaptability index of the rehabilitation exercise option is greater than the training adaptability index threshold, it can be considered that the rehabilitation exercise option is more suitable for the current rehabilitation condition of the patient, that is, the rehabilitation exercise option can be determined as the adaptive rehabilitation exercise option. In theory, there should be at least one adaptive rehabilitation exercise option for each category, for example, Tai Chi in traditional Chinese rehabilitation is an adaptive rehabilitation exercise option, lifting barbells in resistance training is an adaptive rehabilitation exercise option, and slow running in aerobic exercise is an adaptive rehabilitation exercise option.

[0106] According to an embodiment of the present application, according to the historical category information, the proportion of historical selection of each category is determined. For example, 6 historical category information is determined, including 2 aerobic exercises, and the proportion of historical selection of aerobic exercise is 2 / 6, that is, 1 / 3.

[0107] According to an embodiment of the present application, according to the category to which the adaptive rehabilitation exercise option belongs, the training adaptability index of the adaptive rehabilitation exercise option and the proportion of historical selection, the selection index of each adaptive rehabilitation exercise option is determined, including: determining the selection index of the i th adaptive rehabilitation exercise option according to formula (1) ,

[0108] (1),

[0109] wherein, is the training adaptability index of the i th adaptive rehabilitation exercise option, is the proportion of historical selection of the historical category information same as the category to which the i th adaptive rehabilitation exercise option belongs, and is a random weight, wherein, is the probability that 1 is , is the probability that 0 is , , i is a positive integer.

[0110] According to an embodiment of the present application, in formula (1), is the probability that 1 is , The greater, The smaller, The greater, The smaller the probability that 1 is, The greater the probability that 0 is. Since Therefore, The smaller the probability that 1 is, The greater the probability that 1 is, that is, the greater the product of the training adaptability index of the i th adaptive rehabilitation exercise option and the proportion of historical selection of the historical category information same as the category to which the adaptive rehabilitation exercise option belongs, The greater the probability that 0 is, The greater the probability that the probability that the selection index of the i-th adaptive rehabilitation exercise option is maintained as the training adaptability index is greater, in other words, the training adaptability index of the i-th adaptive rehabilitation exercise option can be greater (i.e., the adaptive rehabilitation exercise option has a higher degree of adaptation to the patient), or the historical selection proportion of the same category historical category information of the i-th adaptive rehabilitation exercise option is greater (i.e., the patient selects the category to which the adaptive rehabilitation exercise option belongs more frequently), and thus, the greater the probability that the i-th adaptive rehabilitation exercise option is given a greater selection index, the greater the probability that the i-th adaptive rehabilitation exercise option is selected as the pending rehabilitation exercise option, and the more likely the i-th adaptive rehabilitation exercise option is selected as the pending rehabilitation exercise option.

[0111] According to an embodiment of the present application, in formula (1), The smaller, The greater, The smaller, The greater the probability that the probability that the probability that the selection index of the i-th adaptive rehabilitation exercise option is maintained as the training adaptability index is greater, The greater the probability that the probability that the probability that the selection index of the i-th adaptive rehabilitation exercise option is maintained as the training adaptability index is greater, Thus, The greater the probability that the probability that the probability that the selection index of the i-th adaptive rehabilitation exercise option is maintained as the training adaptability index is greater, The greater the probability that the probability that the probability that the selection index of the i-th adaptive rehabilitation exercise option is maintained as the training adaptability index is greater, The probability of the product of the training adaptability index and the historical selection proportion of the historical category information of the same category as the adaptive rehabilitation exercise option being greater. Since the training adaptability index of the i-th adaptive rehabilitation exercise option and the historical selection proportion of the historical category information of the same category as the adaptive rehabilitation exercise option are both values less than 1, the smaller the product of the training adaptability index of the i-th adaptive rehabilitation exercise option and the historical selection proportion of the historical category information of the same category as the adaptive rehabilitation exercise option, the greater the probability of the training adaptability index of the i-th adaptive rehabilitation exercise option being reduced by the historical selection proportion of the historical category information of the same category as the adaptive rehabilitation exercise option. In other words, the training adaptability index of the i-th adaptive rehabilitation exercise option can be smaller (i.e., the adaptive rehabilitation exercise option has a lower degree of adaptation to the patient), or the historical selection proportion of the historical category information of the same category as the adaptive rehabilitation exercise option is small (i.e., the patient selects the category of the adaptive rehabilitation exercise option less frequently), so the probability of the i-th adaptive rehabilitation exercise option being assigned a smaller selection index is greater, and the probability of being selected as the pending rehabilitation exercise option is smaller, and it is less likely to be selected as the pending rehabilitation exercise option. And through the above processing mode, the rehabilitation option that has a higher degree of adaptation to the patient but is selected less frequently by the patient can have a probability of being determined as the pending rehabilitation exercise option. For example, the training adaptability index of a certain adaptive rehabilitation exercise option is high (i.e., the adaptive rehabilitation exercise option has a high degree of adaptation to the patient), but the historical selection proportion of the historical category information of the same category as the adaptive rehabilitation exercise option is low (i.e., the patient selects the category of the adaptive rehabilitation exercise option less frequently), and the selection index of the adaptive rehabilitation exercise option is still maintained as the training adaptability index, so the adaptive rehabilitation exercise option has a probability of being assigned a higher selection index, and thus the adaptive rehabilitation exercise option has a probability of being selected as the pending rehabilitation exercise option.

[0112] According to an embodiment of the present application, the pending rehabilitation exercise option is selected from the adaptive rehabilitation exercise options according to the selection index and sent to the medical side. The adaptive rehabilitation exercise option with the largest selection index can be determined as the pending rehabilitation exercise option, and the above pending rehabilitation exercise option is sent to the medical side, so that the doctor can evaluate whether the pending rehabilitation exercise can be used for the rehabilitation exercise of the patient. After being sent to the medical side, the doctor who evaluates whether the pending rehabilitation exercise can be used for the rehabilitation exercise of the patient is unique, so that the doctor can have a more comprehensive understanding of the rehabilitation progress of the patient. Of course, one doctor can receive information of multiple patients at the same time.

[0113] In the example, the medical care end includes a plurality of first-level entries, second-level entries and third-level entries. The first-level entry "patient management" includes second-level entries such as "user list", and in the third-level entries, the user list includes basic information, exercise assessment, exercise compliance report, exercise prescription, exercise record, pre-warning value report (for example, data anomaly, compliance anomaly, completion anomaly, assessment anomaly, other anomaly, etc.). The first-level entry "education module" includes second-level entries such as "knowledge education" and "skill training", and in the third-level entries, the knowledge education includes disease knowledge training, exercise rehabilitation knowledge training, guideline and consensus reading, etc., and the skill training includes guideline and consensus reading, exercise training method training, first aid skill training, etc. In the first-level entry "support module", second-level entries such as "medical care exchange", "doctor-patient exchange" and "nurse-patient exchange" are included, and in the third-level entries, the above-mentioned second-level entries all support voice, text and pictures. In the first-level entry "personal center", second-level entries such as "personal information", "reminder function", "my collection" and "system settings" are included, and in the third-level entries, the personal information includes gender, age, education level, professional title and other information, the reminder function can set an exercise viewing reminder function according to the personal working time, the my collection includes the content added to the my collection by clicking the "heart-shaped logo" set in the text or video of the education module, and the system settings include security settings such as password reset, account cancellation, privacy agreement, etc.

[0114] In this way, based on the collected evaluation scale and the patient's rehabilitation exercise type selection record in the historical rehabilitation period, the selection index of each adaptive rehabilitation exercise option can be determined, and then the pending rehabilitation exercise option is determined and sent to the medical care end. When determining the selection index, the situation that the rehabilitation option with higher adaptability to the patient but less selection times by the patient has a probability of being determined as the pending rehabilitation exercise option is considered, and the random weight is determined based on the training adaptability index of the adaptive rehabilitation exercise option and the historical selection proportion. The accuracy, objectivity and comprehensiveness of the determination of the selection index of each adaptive rehabilitation exercise option are improved, thereby improving the accuracy of the pending rehabilitation exercise option, significantly improving the relevance and training acceptability of the pending rehabilitation exercise option, and further improving the completion and compliance.

[0115] According to the embodiment of the application, in step S3, after receiving the confirmation information of the medical care end, the pending rehabilitation exercise option is sent to the patient end as a rehabilitation exercise recommendation option. After receiving the confirmation information of the medical care end, it can be considered that the doctor confirms that the pending rehabilitation exercise option can be used for the patient's rehabilitation exercise, and therefore the pending rehabilitation exercise option can be sent to the patient end as a rehabilitation exercise recommendation option for the patient's rehabilitation exercise.

[0116] According to the embodiment of the present application, in step S4, before the patient starts the rehabilitation exercise, the patient is prompted to wear the wearable monitoring device, and is prompted to stand in front of the camera of the terminal device, and select a rehabilitation exercise video in the rehabilitation exercise recommendation option. Before the patient starts the rehabilitation exercise, the patient can be prompted to wear the wearable monitoring device (for example, a wearable blood oxygen monitor, a smart watch, etc.) through voice, text and other ways, and is prompted to stand in front of the camera of the terminal device, and select a rehabilitation exercise video in the rehabilitation exercise recommendation option. For example, the rehabilitation exercise recommendation option is Taijiquan, and the patient can select a Taijiquan video of different difficulty as the rehabilitation exercise video.

[0117] According to the embodiment of the present application, in step S5, during the process that the patient starts the rehabilitation exercise, the rehabilitation exercise video is played on the patient end of the terminal device, the exercise video of the patient performing the rehabilitation exercise is shot by the camera, and the physiological index data of the patient is collected by the wearable monitoring device. The rehabilitation exercise video played on the patient end of the terminal device can be used to assist the patient to perform the rehabilitation exercise, and the standardization of the rehabilitation exercise of the patient is improved by imitating the video to perform the exercise. The exercise video of the patient performing the rehabilitation exercise shot by the camera can be used to monitor whether the action of the patient performing the rehabilitation exercise is standard. The physiological index data (for example, systolic pressure, diastolic pressure, heart rate, blood oxygen, etc.) of the patient collected by the wearable monitoring device can be used to monitor the physical state of the patient, and then judge whether the patient is suitable for the current rehabilitation exercise.

[0118] Figure 3 An exemplary flow chart for determining the exercise suitability index of the patient according to the embodiment of the present application is shown.

[0119] According to an embodiment of the present application, in step S6, the platform side fuses features of the motion video and the physiological index data through the motion monitoring model, obtains action trend features, and further determines the motion suitability index of the patient, including: sampling the motion video to obtain a plurality of motion video frames; selecting a plurality of rehabilitation motion video frames from the plurality of video frames of the rehabilitation motion video according to the time stamps of the motion video frames; selecting target physiological index data from the physiological index data at a plurality of time points according to the time stamps of the motion video frames; obtaining first image feature information of a plurality of partitions of a jth motion video frame, and obtaining second image feature information of a plurality of partitions of a jth rehabilitation motion video frame; processing the first image feature information and the second image feature information through a second cross-attention mechanism of the motion monitoring model to obtain a plurality of fused image feature information; splicing the plurality of fused image feature information and processing through a first full connection layer and a first activation layer to obtain jth action feature information; processing the jth action feature information and (j-1)th action hidden feature information through a motion time sequence submodel of the motion monitoring model to obtain jth action hidden feature information, wherein when j=1, the (j-1)th action hidden feature information is a zero vector, and the action hidden feature information is used to describe the action trend features of the patient; obtaining physiological feature information of the jth set of target physiological index data through a data feature extraction layer of the motion monitoring model; splicing the jth physiological feature information and the jth action hidden feature information to obtain jth motion state information; processing the jth motion state information and (j-1)th state hidden feature information through a state time sequence submodel of the motion monitoring model to obtain jth state hidden feature information, wherein when j=1, the (j-1)th state hidden feature information is a zero vector; when j , inputting the jth state hidden feature information into a second full connection layer and a second activation layer for processing to obtain the motion suitability index of the patient, wherein, is the serial number of the last motion video frame.

[0120] According to an embodiment of the present application, the motion video is sampled to obtain a plurality of motion video frames. For example, sampling is performed every 1 second or 0.5 seconds to obtain a motion video frame. Further, according to the time stamps of the motion video frames, rehabilitation motion video frames with the same time stamps as those of the above-mentioned motion video frames are selected from the plurality of video frames of the rehabilitation motion video. Similarly, according to the time stamps of the motion video frames, physiological index data of the patient at the same time as the time stamps of the above-mentioned motion video frames is collected from the physiological index data at a plurality of time points, i.e., the target physiological index data. The number of rehabilitation motion video frames, the number of motion video frames, and the number of groups of target physiological indexes (each group including a plurality of physiological indexes) are the same, and the above-mentioned data can be continuously processed.

[0121] According to an embodiment of the present application, the first image feature information of the plurality of partitions of the jth exercise video frame is obtained, and the second image feature information of the plurality of partitions of the jth rehabilitation exercise video frame is obtained. For example, each exercise video frame can be equally divided into nine partitions. Further, the images of the plurality of partitions of the jth exercise video frame are processed by a convolutional neural network model (for example, a CNN model or the like), and the first image feature information corresponding to each partition is obtained, which can describe the features of the images of each partition in the jth exercise video frame. Similarly, the second image feature information of each partition of the jth rehabilitation exercise video frame can be obtained. The number of the first image feature information and the second image feature information is the same as the number of the partitions.

[0122] According to an embodiment of the present application, similar to obtaining the training adaptability description information of each rehabilitation exercise option, the first image feature information and the second image feature information are processed by the second cross-attention mechanism of the exercise monitoring model, and a plurality of fusion image feature information is obtained, which can fuse the features of the second image feature information related to the first image feature information, describe the features of the first image feature information, that is, fuse the features of the rehabilitation exercise video (that is, the standard action) related to the exercise video (that is, the action made by the patient during the exercise process), describe the features of the exercise video, and improve the accuracy and comprehensiveness of the feature expression of the exercise video. Based on the same processing manner, the fusion image feature information corresponding to each first image feature information can be obtained. Further, the plurality of fusion image feature information is spliced and processed by the first full connection layer and the first activation layer (for example, a ReLU activation function), and the jth action feature information is obtained, which can fuse the features of the standard action in the rehabilitation exercise video related to the action made by the patient during the exercise process, and describe the features of the action of the patient in the jth exercise video frame. Based on the same processing manner, the action feature information corresponding to each exercise video frame can be obtained.

[0123] According to the embodiment of the present application, since the patient performs the exercise by imitating the rehabilitation exercise video, the patient's action in the exercise video frame may lag behind the standard action in the rehabilitation exercise video frame with the same timestamp, for example, the patient's action in the exercise video frame with the current timestamp is similar to the standard action in the rehabilitation exercise video frame with the previous timestamp, and therefore, the jth action feature information and the (j-1)th action hidden feature information can be processed by the exercise timing sub-model (for example, an LSTM recurrent neural network model) of the exercise monitoring model, so as to obtain the jth action hidden feature information. The (j-1)th action hidden feature information is a vector output by the exercise timing sub-model after the (j-1)th action feature information and the (j-2)th action hidden feature information are input into the exercise timing sub-model. When j=1 (i.e., the first exercise video frame), the (j-1)th action hidden feature information is a zero vector. In the process of obtaining the jth action hidden feature information, each action hidden feature information obtained combines the previous action hidden feature information, and therefore, in the process of obtaining the jth action hidden feature information, the jth action hidden feature information is compared with the previous action hidden feature information, and the trend of the action performed by the patient in the exercise process is considered, so that the jth action hidden feature information can accurately describe the feature of the action of the patient in the jth exercise video frame and the feature of the action trend of the patient, and therefore, the action hidden feature information is used to describe the action trend feature of the patient. Therefore, the consistency of the action of the patient in the exercise video with the standard action in the rehabilitation exercise video can be compared based on the action trend.

[0124] According to the embodiment of the present application, the jth set of physiological index data is processed by the data feature extraction level (for example, including a fully connected layer and an activation layer, and the activation layer is a feature extraction level processed by using a ReLU activation function) of the exercise monitoring model, so as to obtain the physiological feature information of the jth set of target physiological index data, which can describe the feature of the body state of the patient in the jth exercise video frame. Further, the jth physiological feature information (feature information of the physiological index data) and the jth action hidden feature information (feature information of the video frame of the exercise video) are spliced (i.e., the feature fusion of the exercise video and the physiological index data is performed), so as to obtain the jth exercise state information, which can comprehensively describe the exercise state of the patient in the jth exercise video frame from the aspects of the action trend and the body state. Based on the same processing manner, the exercise state information corresponding to each exercise video frame can be obtained.

[0125] According to an embodiment of the present application, the motion suitability index is determined by performing time series analysis on the spliced feature information. Similarly to obtaining the jth action hidden feature information, the jth motion state information (i.e., the spliced feature information) and the (j-1)th state hidden feature information are processed (i.e., time series analysis) by the state time series sub-model of the motion monitoring model, so as to obtain the jth state hidden feature information. When j=1, the (j-1)th state hidden feature information is a zero vector. The above process of obtaining the state hidden feature information is iteratively performed, and when j= When j=1, the (j-1)th state hidden feature information is a zero vector. The above process of obtaining the state hidden feature information is iteratively performed, and when j=

[0126] In the example, the platform end includes a plurality of first-level entries and second-level entries. In the first-level entry "user and permission", the second-level entries such as "account, role, organization" are included, and in the account, role, organization, a plurality of functions and points are included, for example, OAuth2.1, which can allow users to securely access the system with a unified account (for example, a unified authentication account of a hospital, a third-party login account), without directly giving the password to the platform. OIDC can not only "authorize", but also "authenticate", for example, confirming who the logged-in user is, and the basic identity information (for example, name, employee number, role, etc.) of the user. In RBAC, patients can query their own prescriptions, educational content, and upload exercise diaries, medical staff can query and manage patient data, issue and adjust prescriptions, and view warning reports, administrators can manage user accounts, data reports, permissions, etc. In the first-level entry "evaluation engine", the second-level entries such as "scale, rule, version" are included, and in the scale, rule, version, a plurality of functions and points are included, for example, CAT, mMRC, 6MWT, etc. Scale configuration, question bank version management: scoring and boundary value (can be stratified according to population). In the first-level entry "prescription engine", the second-level entries such as "template library, rule library, advanced strategy" are included, and in the template library, rule library, advanced strategy, a plurality of functions and points are included, for example, FITT-VP template platform built-in "exercise prescription template library" with the six dimensions as the core, after inputting the evaluation results (i.e., CAT, mMRC, 6-minute walk, etc.), the system can apply the template to generate a personalized exercise prescription of frequency-intensity-duration-type-total-advanced. Rule tree, DSL, that is, the platform can provide a configurable rule language to describe the mapping relationship between evaluation indicators and prescription parameters (i.e., frequency, intensity, duration, type, and advancement), and medical staff or researchers do not need to modify the code, but write rules in the DSL file, and the system can generate or adjust the exercise prescription. After the platform automatically generates the prescription, it needs to be reviewed and confirmed by medical staff, and only after the review is passed can it be issued to the patient. Advancement and rollback, that is, prescription release and withdrawal. In the first-level entry "user portrait and recommendation", the second-level entries such as "portrait calculation, content recommendation" are included, and in the portrait calculation, content recommendation, a plurality of functions and points are included, for example, risk, preference, adherence, content tagging and similarity, A / B, for example, there are two sets of content under the same topic in the education module: one is text + illustration (A version), and the other is video + voice explanation (B version). The system can randomly assign patients to different groups and collect click-through rate, completion rate, exercise adherence, etc. Data. The platform can select the better version according to the results and optimize the content library in the long term. Multi-arm tiger machine, that is, when the platform wants to push health education content or exercise prescription, it does not need to be completely random, but can use the "multi-arm tiger machine" algorithm for dynamic allocation, for example, there are three ways to teach respiratory training: text, short video, and voice. The system will randomly recommend at the beginning, and gradually allocate more resources to the form with the best feedback effect.This way, new possibilities can be continuously explored, while ensuring that most patients receive the "best possible" content.

[0127] In an example, in the first-level entry "Early Warning and Notification", the second-level entries such as "Rule Engine, Routing" are included, and in the Rule Engine, Routing, a plurality of functions and points are included, such as early warning level and threshold management; event routing; multi-channel (e.g., in-station, SMS, push); etc. In the first-level entry "Data Governance", the second-level entries such as "Indicator Warehouse, Quality Control" are included, and in the Indicator Warehouse, Quality Control, a plurality of functions and points are included, such as evaluation / prescription / journal / warning fact table; ETL, i.e., the platform will periodically extract data from the patient side, the medical side, and the management side, clean and unify (e.g., de-duplication, unit conversion, format unification, outlier processing, etc.), and then load it into the data warehouse on the platform side as the input of the prescription engine, the early warning engine, and the report. Verification, i.e., check if the patient's birth date is reasonable; check if the total score of the CAT scale falls within the legal range of 0-40 points; check the hash value when uploading files to ensure that they have not been tampered with or lost during transmission; de-identification, i.e., replace the patient's name with a random number, such as P001, P002; only show the first 6 digits and the last 2 digits of the ID number, with stars in between; only keep the address to the city level, without showing the specific house number. Audit, i.e., who exported which exercise adherence report at what time; which doctor modified the patient's prescription and which parameters were modified; which administrator adjusted the permissions. In the first-level entry "Content Management", the second-level entries such as "CMS and Knowledge Base" are included, and in the CMS and Knowledge Base, a plurality of functions and points are included, such as content storage and classification: disease knowledge, exercise rehabilitation, smoking cessation guidance, emotion management, etc., organized by topic tree and tags. Version and timed release: content can have version management and support timed push, e.g., one COPD education article per day. Visual editing: backend personnel do not need to write code, they can just write documents to produce text / video content. Precise delivery: combined with the "Portrait and Recommendation" module, the materials in CMS can be accurately pushed according to the patient's portrait.

[0128] In an example, in the first-level entry "incentive mechanism", the second-level entries such as "points" are included, and in the points, a plurality of functions and points are included, for example, when the daily exercise duration reaches 0%, the basic points are still given, the points are gradually increased with the increase of the completion degree, the visual punch-in calendar is matched to form an incentive closed loop, and the exercise enthusiasm of the patient is stimulated. The platform has a built-in points and exercise ranking function, and the patient can obtain points by completing daily exercise punch-in, learning or sharing mutual aid of educational content. The points can be used to exchange virtual medals, ranking display or offline rewards, combined with the ranking incentive, to enhance the long-term participation and exercise enthusiasm of the patient. In the first-level entry "multi-type exercise support", a plurality of functions and points are included, for example, the prescription and training library covers aerobic exercise, resistance training, flexibility training, breathing training (for example, pursed-lip breathing, diaphragmatic breathing, load training, etc.), and traditional Chinese medicine rehabilitation methods (for example, Tai Chi, Eight Brocade, Six Word Dharma), realizing diversified intervention selection. In the first-level entry "wearable device exercise monitoring", a plurality of functions and points are included, for example, the platform supports connecting common wearable devices, and records key indicators such as heart rate and blood oxygen saturation in real time during exercise to ensure training safety and prescription availability. In the first-level entry "operation and audit", the second-level entries such as "log, board" are included, and in the log, board, a plurality of functions and points are included, for example, access and operation log; abnormal index board; compliance audit export. In the first-level entry "community and mutual aid", the second-level entries such as "sick friend exchange" are included, and in the sick friend exchange, a plurality of functions and points are included, for example, providing mutual exchange, exercise punch-in and experience sharing between patients, supporting text, picture and like interaction. In the first-level entry "API gateway", the second-level entries such as "unified access, flow limiting" are included, and in the unified access, flow limiting, a plurality of functions and points are included, for example, REST / GraphQL; JWT, that is, after the user logs in, the system issues a JWT containing user identity, role, expiration time and other information, and then the user only needs to bring this token every time the interface is accessed, and the platform side verifies the signature to confirm the identity without repeatedly querying the database. Flow limiting, that is, a user can submit a maximum of 10 assessment questionnaires per minute, and more will be rejected or delayed. Fuse, that is, if the "exercise prescription generation" module fails continuously, the fuse will temporarily stop calling and directly return a "system busy" prompt instead of freezing all user requests. Gray release, that is, an "interactive video exercise" is added to the education module, which is only opened to 5% of patient users first, and if the data and feedback are good, it will be gradually expanded to all people.

[0129] According to an embodiment of the present application, the training step of the motion monitoring model comprises: obtaining sample motion video frames, sample rehabilitation motion video frames and sample physiological index data; obtaining the kth sample action hidden feature information through the kth sample motion video frame and the kth sample rehabilitation motion video frame, and the second cross-attention mechanism and the motion timing sub-model of the motion monitoring model; obtaining the kth sample state hidden feature information through the kth sample action hidden feature information and the kth set of sample physiological index data, and the state timing sub-model of the motion monitoring model; obtaining the loss function of the motion monitoring model according to each sample action hidden feature information and sample state hidden feature information; training the motion monitoring model according to the loss function of the motion monitoring model to obtain the trained motion monitoring model.

[0130] According to an embodiment of the present application, similar to obtaining motion video frames, rehabilitation motion video frames and physiological index data, sample motion video frames, sample rehabilitation motion video frames and sample physiological index data are obtained. Of course, the number of sample motion video frames, the number of sample rehabilitation motion video frames and the number of sample physiological index data groups are the same. Further, similar to obtaining the jth action hidden feature information and the jth state hidden feature information, the kth sample action hidden feature information is obtained through the kth sample motion video frame and the kth sample rehabilitation motion video frame, and the second cross-attention mechanism and the motion timing sub-model of the motion monitoring model, and then the kth sample state hidden feature information is obtained through the kth sample action hidden feature information and the kth set of sample physiological index data, and the state timing sub-model of the motion monitoring model.

[0131] According to an embodiment of the present application, the loss function of the motion monitoring model is obtained according to the respective sample action hidden feature information and sample state hidden feature information, comprising: determining first action feature information of each sample motion video frame and second action feature information of each sample rehabilitation motion video frame through an action recognition model; inputting the respective sample action hidden feature information into a third fully connected layer and a third activation layer for processing to obtain an action state matching index of each sample motion video frame; determining an action state loss function according to the action state matching index and the first action feature information and the second action feature information; inputting the respective sample action hidden feature information into a fourth fully connected layer and a fourth activation layer for processing to obtain an action trend matching index of each sample motion video frame; determining an action trend loss function according to the action trend matching index and the first action feature information and the second action feature information; inputting the last sample state hidden feature information into a second fully connected layer and a second activation layer for processing to obtain a sample motion suitability index; obtaining a motion suitability loss function according to the sample motion suitability index and the labeled information of the sample physiological index data; and obtaining the loss function of the motion monitoring model according to the action state loss function, the action trend loss function and the motion suitability loss function.

[0132] According to an embodiment of the present application, each sample motion video frame and each sample rehabilitation motion video frame is respectively input into a trained action recognition model (for example, an I3D, 3DResNet or the like), so that the first action feature information of each sample motion video frame and the second action feature information of each sample rehabilitation motion video frame are obtained, wherein the first action feature information can describe the characteristics of the action of the patient in the sample motion video frame, and the second action feature information can describe the characteristics of the standard action in the sample rehabilitation motion video frame. Since the action of the patient in the sample motion video frame with the same timestamp may be lagged relative to the standard action in the sample rehabilitation motion video frame, the first action feature information of the sample motion video frame at a certain timestamp may have a higher similarity with the second action feature information of the sample rehabilitation motion video frame at the last timestamp.

[0133] According to an embodiment of the present application, the respective sample action hidden feature information is input into a third fully connected layer and a third activation layer (for example, a Sigmoid activation function) for processing, so that each sample action hidden feature information can be mapped into probability information (for example, 0.6, 0.7 or the like) of the matching between the action of the patient in each sample motion video frame and the standard action, i.e., the action state matching index, which can describe the similarity between the action of the patient and the standard action.

[0134] According to an embodiment of the present application, similarity (for example, cosine similarity, etc.) between the first action feature information of each sample motion video frame and the second action feature information of the sample rehabilitation motion video frame with the same timestamp is determined. Since the first action feature information and the second action feature information are obtained based on the trained action recognition model, the similarity can accurately describe the real similarity between the action of the patient in each sample motion video frame and the standard action in the sample rehabilitation motion video frame with the same timestamp. The action state matching index corresponding to each sample motion video frame is the difference between the similarity of the first action feature information of the same sample motion video frame and the second action feature information of the sample rehabilitation motion video frame with the same timestamp. The action state loss function corresponding to each sample motion video frame is obtained by summing the action state loss functions of the sample motion video frames. In the training process, the action state loss function is reduced, and the action state matching index is close to the real similarity between the action of the patient and the standard action, which can improve the accuracy of the action state matching index and the accuracy of the acquisition of the sample action hidden feature information. The sample action hidden feature information can accurately describe the characteristics of the action of the patient in each sample motion video frame to assist the training of the motion monitoring model.

[0135] According to an embodiment of the present application, since the sample action hidden feature information not only includes the characteristics of the action of the patient in each motion video frame, but also carries the characteristics of the action trend of the patient, the sample action hidden feature information is input into the fourth fully connected layer and the fourth activation layer (for example, Sigmoid activation function) for processing. The sample action hidden feature information is mapped to the probability information of the matching between the action trend of the patient in each sample motion video frame and the standard action trend, that is, the action trend matching index.

[0136] According to an embodiment of the present application, the action trend loss function is determined according to the action trend matching index, the first action feature information and the second action feature information, including: determining the action trend loss function according to formula (2) ,

[0137] (2),

[0138] wherein, is the first action feature information of the kth sample motion video frame, is the second action feature information of the tth sample rehabilitation motion video frame, is the action trend matching index of the kth sample action hidden feature information, N is the number of sample motion video frames, max is a maximum function, min is a minimum function, k, N and are positive integers.

[0139] According to an embodiment of the application, in formula (2), represents the maximum value of the similarity (for example, cosine similarity, etc.) between the first action feature information of the kth sample motion video frame and the second action feature information of the sample rehabilitation motion video frame corresponding to the kth sample motion video frame. Since the patient performs motion by imitating the sample rehabilitation motion video, the patient's action in the sample motion video frame may be lagged behind the standard action in the sample rehabilitation motion video frame with the same time stamp, and therefore, the maximum value of the similarity between the first action feature information of the kth sample motion video frame and the second action feature information of the sample rehabilitation motion video frame corresponding to each current time stamp and the previous time stamp can be determined as the matching degree between the first action feature information of the kth sample motion video frame and the second action feature information of the corresponding sample rehabilitation motion video frame (i.e., the sample rehabilitation motion video frame that the patient is imitating in the kth sample motion video frame), which can describe the matching degree between the action trend of the patient in the kth sample motion video frame and the standard action trend. is a preset value, which can be considered as the number of video frames required to determine the sample rehabilitation motion video frame corresponding to the kth sample motion video frame.

[0140] According to an embodiment of the application, in formula (2), represents the minimum value of the matching degree between the first action feature information of the kth sample motion video frame and the second action feature information of the corresponding sample rehabilitation motion video frame, and the action trend matching index of the kth sample action hidden feature information, and the maximum value of the matching degree between the first action feature information of the kth sample motion video frame and the second action feature information of the corresponding sample rehabilitation motion video frame, and the action trend matching index of the kth sample action hidden feature information. Further, the ratio of the minimum value and the maximum value can be obtained, which can be considered as a value describing the accurate matching degree between the action trend of the patient in the kth sample motion video frame and the standard action trend, the consistency degree of the action trend matching index of the kth sample action hidden feature information (which can also describe the matching degree between the action trend of the patient in the kth sample motion video frame and the standard action trend), and further the action trend loss function of the kth sample motion video frame can be obtained. In the training process, the value of can be made close to 1, and the value of The action trend matching index of the kth sample motion video frame is close to 0, and the accuracy of matching the action trend of the patient in the kth sample motion video frame to the standard action trend is close to 100%, so that the accuracy of the action trend matching index in describing the matching degree of the action trend of the patient in the kth sample motion video frame to the standard action trend is improved, and then the action hidden feature information can accurately describe the characteristics of the action trend of the patient. Further, the sum of the action trend loss functions of each sample motion video frame is obtained, that is, the action trend loss function.

[0141] According to an embodiment of the present application, similar to the acquisition of the motion suitability index, the last sample state hidden feature information is input into the second fully connected layer and the second activation layer for processing, so that the sample motion suitability index is obtained, and the sample physiological index data is artificially labeled (when a doctor determines that a patient is suitable for the current rehabilitation exercise according to the sample physiological index data, the sample physiological index data can be labeled as 1, and if not suitable, it is labeled as 0). Since the sample motion suitability index can also describe the probability of the patient adapting to the current rehabilitation exercise, the cross-entropy loss function based on the sample motion suitability index and the labeled information of the sample physiological index data is determined, that is, the motion suitability loss function. In the training process, the motion suitability loss function is reduced, and then the sample motion suitability index is close to the labeled information of the sample physiological index data, so that the accuracy of the determination of the sample motion suitability index is improved, and then the accuracy of the acquisition of each sample state hidden feature information is improved, so that the sample state hidden feature information can accurately describe the characteristics of the change trend of the patient's motion state, to assist the training of the motion monitoring model.

[0142] According to an embodiment of the present application, the loss function of the motion monitoring model is obtained according to the action state loss function, the action trend loss function and the motion suitability loss function. The action state loss function, the action trend loss function and the motion suitability loss function are summed and processed, so that the loss function of the motion monitoring model is obtained.

[0143] According to an embodiment of the present application, the motion monitoring model is trained according to the loss function of the motion monitoring model, and a trained motion monitoring model is obtained. Through the loss function of the motion monitoring model, and by using the gradient descent method to adjust the parameters of the motion monitoring model, the motion monitoring model is trained, and after multiple training (that is, after training using multiple action state matching indexes, action trend matching indexes, first action feature information, second action feature information, sample motion suitability indexes and labeled information of sample physiological index data), the trained motion monitoring model can be obtained.

[0144] In this way, the action state loss function, the action trend loss function and the motion suitability loss function can be obtained when training the motion monitoring model, and then the loss function of the motion monitoring model is obtained. Considering that the action of the patient in the sample motion video frame may be lagged compared to the standard action in the sample rehabilitation motion video frame with the same timestamp, the sample rehabilitation motion video frame corresponding to the second action feature information with the highest similarity to the first action feature information of the sample motion video frame is determined as the sample rehabilitation motion video frame corresponding to the sample motion video frame, so as to determine the value of the degree of accurate matching between the action trend of the patient in the sample motion video frame and the standard action trend and the degree of consistency with the corresponding action trend matching index, and then determine the action trend loss function. The action state loss function and the motion suitability loss function are set, the accuracy of obtaining the sample action hidden feature information, the sample state hidden feature information and the sample motion suitability index is improved, the training of the motion monitoring model is assisted, the accuracy and pertinence of the training are improved, and the performance of the trained motion monitoring model is improved.

[0145] According to an embodiment of the present application, in step S7, the motion suitability index is sent to the medical side, and after receiving the rehabilitation motion adjustment scheme of the medical side, a new rehabilitation motion recommendation option is determined. For example, the doctor thinks that the motion suitability index is too low, and the current rehabilitation motion is not suitable for the patient. According to the current rehabilitation condition of the patient and the motion suitability index of the patient performing the current rehabilitation motion, a new rehabilitation motion is determined, which is the new rehabilitation motion recommendation option. For example, the current rehabilitation motion is too difficult for the patient, resulting in too fast heart rate and too high blood pressure of the patient. Therefore, the difficulty can be reduced, and a new rehabilitation motion option is recommended, for example, the above steps are re-executed, or the doctor manually recommends.

[0146] According to an embodiment of the present application, in step S8, at the end of the rehabilitation period, a compliance report is determined according to the historical record of the patient performing the rehabilitation motion. The compliance report can include information such as the number of attendance days of the patient in the rehabilitation period and the completion degree of the rehabilitation motion. For example, the compliance report shows that the patient performed five rehabilitation motions in the last rehabilitation period, and the completion rate of each rehabilitation motion was 100%. Then, according to the compliance report, the patient's active rehabilitation motion can be supervised.

[0147] In the example, in the home exercise rehabilitation intelligent management method for COPD patients, a management end is further included, and the management end includes a plurality of first-level entries, second-level entries and third-level entries. In the first-level entry "user management", the second-level entries such as "user permission management", "data management", "file uploading" and "information feedback" are included. In the third-level entries, the user permission management includes functions such as registration, login, logout and association binding of the account of medical staff or users. The data management includes exercise assessment data, exercise record data, early warning values (for example, data anomaly, completion anomaly, etc.), has data new creation, modification and deletion operations, and the data report includes user total number, data anomaly statistics, etc. The file uploading includes a function of regularly sending articles related to COPD in the form of combination of text and images, and increases the readability of the articles. The information feedback includes a function of accepting feedback from patients and medical staff.

[0148] In the example, the core algorithm includes: evaluation to prescription mapping: standardize CAT, mMRC, 6MWT (or walking self-test), age, weight, comorbidity, weather, vital sign values, etc. into a feature vector x, generate individual prescription P(x) through prescription rule base R and parameterized template T (FITT-VP: frequency F, intensity I, time T, type T, total amount V, and progress P). Progress strategy: according to the completion C, exercise intensity RPE and abnormal events E in the past 7 days, make ±5-15% fine tuning of F, I and T according to threshold or multi-arm tiger machine strategy. For example, most of the time (for example, 90%) recommend the best set of content (i.e. high click rate and high compliance), and a small part of the time (for example, 10%) randomly recommend other candidate content, and observe the effect. Education content linkage: match the knowledge base content K based on the user portrait (i.e. risk, preference, compliance, etc.), push according to the theme, sub-theme and material (for example, text and image, audio and video, voice, etc.) tree, and record the click rate and completion rate for A and B optimization. Completion and compliance calculation: completion = (proportion of achievement of prescription duration or target step number / power) × weight + (key interaction item completion) × weight; for the device-free scene, the combination of video completion ≥80% + check-in + self-assessment questionnaire is adopted for judgment; abnormal events E include: number of non-compliance, long time without login, symptom self-evaluation deterioration, etc. Early warning and notification and incentive: the strategy engine maps E to early warning levels (for example, data anomaly, compliance anomaly, completion anomaly, evaluation anomaly, etc.), and routes to remind the medical staff end and the patient end.

[0149] In an example, the weak network and availability design includes offline priority: key pages and materials (e.g., prescription and education content in the past 7 days) are cached locally; breakpoint continuation: upload evaluation and diary adopts chunk submission, i.e., chunk size is configurable, when the patient submits the exercise diary or evaluation questionnaire in a weak network environment, the system will cut the data into small pieces (e.g., 100 KB per piece), and automatically retry the failure; lightweight reporting: preferentially upload structured summaries (e.g., completion degree, abnormal code, milestone, etc.), and delay synchronization of material playing details.

[0150] In an example, the data and interface include user: store login account, role, affiliated organization, etc.; patient: associated with the user, including basic information, disease history, comorbidity, etc.; evaluation: save scale results and timestamps, version numbers of CAT, mMRC, 6-minute walk, etc.; exercise prescription: save frequency, intensity, duration, type, total amount, and advanced strategy according to the FITT-VP structure; exercise session: record start time, end time, completion degree, and abnormal event number of each training; exercise diary: punch card, subjective effort (RPE), special medication and events, etc.; warning: warning record and processing loop for abnormal compliance, abnormal completion degree, and abnormal evaluation; knowledge content: disease knowledge, lung function test, exercise rehabilitation, smoking cessation, emotion management, etc.; recommendation record: content or prescription fine-tuning suggestions pushed to the patient and feedback effect; audit log: key operation and data access traces, supporting compliance audit. Interface (Chinese path) submits evaluation: submit CAT, mMRC, 6-minute walk, etc., and returns scores and suggestions; generates prescription: inputs evaluation features, and returns individualized FITT-VP prescription; obtains education recommendation: returns text, video, audio, and voice lists according to the profile; reports warning: reports data abnormal code, and triggers routing and notification by the platform end. Security and permission: full-site encrypted transmission; access control based on role and minimum authorization; key data desensitization and operation traces.

[0151] In this way, the exercise suitability index can be sent to the medical end, and after receiving the rehabilitation exercise adjustment scheme of the medical end, a new rehabilitation exercise recommendation option is determined, and then at the end of the rehabilitation period, an adherence report is determined. Real-time monitoring of the suitability of rehabilitation exercise for patients, and recommending personalized rehabilitation exercise for patients, improves the accuracy of rehabilitation exercise recommendation and improves patient adherence.

[0152] The home exercise rehabilitation intelligent management method and system for COPD patients according to the embodiments of the present application can prompt the patient to fill in the evaluation scale and determine the pending rehabilitation exercise option at the beginning of the rehabilitation period. After receiving the confirmation information from the medical side, the pending rehabilitation exercise option is taken as the rehabilitation exercise recommended option, and then the rehabilitation exercise video is selected. During the process of the patient starting the rehabilitation exercise, the patient's exercise video is shot, and the patient's various physiological index data are collected, and then the patient's exercise suitability index is determined, and the new rehabilitation exercise recommended option is determined, and then the compliance report is determined. The suitability of the rehabilitation exercise for the patient is monitored in real time, and personalized rehabilitation exercise is recommended for the patient, improving the accuracy of the rehabilitation exercise recommendation and the patient's compliance. At the beginning of the rehabilitation period, the patient can be prompted to fill in the evaluation scale on the patient side using the terminal device, providing basic data for determining the pending rehabilitation exercise option. When determining the pending rehabilitation exercise option, the selection index of each adaptive rehabilitation exercise option is determined based on the collected evaluation scale and the patient's rehabilitation exercise type selection record in the historical rehabilitation period, and then the pending rehabilitation exercise option is determined and sent to the medical side. When determining the selection index, the situation that the rehabilitation option with higher adaptability to the patient but less selection times by the patient needs to be determined as the pending rehabilitation exercise option is considered, and the random weight is determined based on the training adaptability index of the adaptive rehabilitation exercise option and the historical selection proportion. The accuracy, objectivity and comprehensiveness of the determination of the selection index of each adaptive rehabilitation exercise option are improved, thereby improving the accuracy of the pending rehabilitation exercise option, significantly improving the relevance and training acceptability of the pending rehabilitation exercise option, and then improving the completion and compliance. When training the exercise monitoring model, the action state loss function, the action trend loss function and the exercise suitability loss function are obtained, and then the loss function of the exercise monitoring model is obtained. The situation that the patient's action in the sample exercise video frame may lag behind the standard action in the sample rehabilitation exercise video frame with the same timestamp is considered, the sample rehabilitation exercise video frame corresponding to the second action feature information with the maximum similarity to the first action feature information of the sample exercise video frame is taken as the sample rehabilitation exercise video frame corresponding to the sample exercise video frame, thereby determining the value of the accurate matching degree between the action trend of the patient in the sample exercise video frame and the standard action trend and the consistency degree with the corresponding action trend matching index, and then determining the action trend loss function. The action state loss function and the exercise suitability loss function are set, and the accuracy of obtaining the sample action hidden feature information, the sample state hidden feature information and the sample exercise suitability index is improved, thereby assisting the training of the exercise monitoring model, improving the accuracy and pertinence of the training, and improving the performance of the training exercise monitoring model. Further, the exercise suitability index can be sent to the medical side, and after receiving the rehabilitation exercise adjustment scheme from the medical side, the new rehabilitation exercise recommended option is determined, and then the compliance report is determined at the end of the rehabilitation period.Real-time monitoring of the appropriateness of rehabilitation exercise for patients, and recommending personalized rehabilitation exercise for patients, improve the accuracy of rehabilitation exercise recommendation, and improve patient compliance.

[0153] Figure 4 An example of a block diagram of a home exercise rehabilitation intelligent management system for COPD patients according to an embodiment of the present application is shown, the system comprises:

[0154] An evaluation scale module prompts the patient to fill in the evaluation scale at the patient end at the beginning of the rehabilitation period;

[0155] A pending rehabilitation exercise option module collects the evaluation scale at the platform end, determines the pending rehabilitation exercise option according to the evaluation scale and the rehabilitation exercise type selection record of the patient in the historical rehabilitation period, and sends it to the medical end;

[0156] A rehabilitation exercise recommendation option module sends the pending rehabilitation exercise option as a rehabilitation exercise recommendation option to the patient end after confirmation at the medical end;

[0157] A rehabilitation exercise video module prompts the patient to complete wearing the wearable monitoring device, prompts the patient to stand in front of the camera of the terminal device, and selects a rehabilitation exercise video in the rehabilitation exercise recommendation option;

[0158] A data acquisition module plays the rehabilitation exercise video through the patient end of the terminal device during the rehabilitation exercise, shoots the exercise video of the patient performing rehabilitation exercise through the camera, and acquires the physiological index data of the patient through the wearable monitoring device;

[0159] A movement appropriateness index module acquires the movement appropriateness index of the patient through the feature fusion of the movement video and the physiological index data by the movement monitoring model at the platform end, and sends it to the medical end, the feature fusion includes splicing the feature information of the video frame of the movement video with the feature information of the physiological index data, performing time sequence analysis through the spliced feature information, and determining the movement appropriateness index;

[0160] A new rehabilitation exercise recommendation option module determines a new rehabilitation exercise recommendation option after receiving the rehabilitation exercise adjustment scheme of the medical end;

[0161] An adherence report module determines the adherence report according to the historical record of the patient performing rehabilitation exercise at the end of the rehabilitation period.

[0162] The present application can be a method, device, system and / or computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon for performing various aspects of the present application.

[0163] Those skilled in the art should understand that the above description and the embodiments of the present application shown in the drawings are only used as examples and do not limit the present application. The purpose of the present application has been fully and effectively achieved. The functional and structural principles of the present application have been demonstrated and described in the embodiments, and the embodiments of the present application can be modified or changed in any way without departing from the principles.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A smart management method for home-based exercise rehabilitation for COPD patients, characterized in that, include: At the start of the rehabilitation cycle, patients are prompted to complete an assessment questionnaire on their devices. The platform collects assessment scales, selects and records the types of rehabilitation exercises the patient has performed in the historical rehabilitation cycle based on the assessment scales, determines the pending rehabilitation exercise options, and sends them to the medical staff. After confirmation by the medical staff, the pending rehabilitation exercise options will be sent to the patient as recommended rehabilitation exercise options. The system prompts the patient to put on the wearable monitoring device, stand in front of the camera on the terminal device, and select a rehabilitation exercise video from the recommended rehabilitation exercise options. During the rehabilitation exercise, rehabilitation exercise videos are played, and videos of patients performing rehabilitation exercises are captured by a camera. Various physiological data of patients are collected through wearable monitoring devices. The platform uses a motion monitoring model to perform feature fusion on motion videos and physiological indicator data to obtain motion trend features, thereby determining the patient's exercise suitability indicators and sending them to the medical staff. The feature fusion includes splicing the feature information of the video frames of the motion video with the feature information of the physiological indicator data, and performing time-series analysis on the spliced ​​feature information to determine the exercise suitability indicators. After receiving the rehabilitation exercise adjustment plan from the medical staff, determine the new recommended rehabilitation exercise options; At the end of the rehabilitation period, adherence reports are determined based on the patient's history of rehabilitation exercises. The platform uses a motion monitoring model to fuse features from motion videos and physiological data to obtain movement trend characteristics, thereby determining the patient's exercise suitability indicators, including: The motion video is sampled to obtain multiple motion video frames; Based on the timestamps of the motion video frames, select multiple rehabilitation exercise video frames from multiple video frames in the rehabilitation exercise video. Based on the timestamps of motion video frames, select the target physiological indicator data from physiological indicator data at multiple time points; Obtain the first image feature information of multiple partitions of the j-th motion video frame, and obtain the second image feature information of multiple partitions of the j-th rehabilitation motion video frame; The second cross-attention mechanism of the motion monitoring model is used to process the first image feature information and the second image feature information to obtain multiple fused image feature information. Multiple fused image feature information are stitched together and processed through a first fully connected layer and a first activation layer to obtain the j-th action feature information; The motion temporal sub-model of the motion monitoring model is used to process the j-th motion feature information and the (j-1)-th motion latent feature information to obtain the j-th motion latent feature information. When j=1, the (j-1)-th motion latent feature information is a zero vector. The motion latent feature information is used to describe the patient's motion trend characteristics. Physiological feature information of the j-th group of target physiological index data is obtained through the data feature extraction hierarchy of the motion monitoring model. The j-th physiological feature information and the j-th action latent feature information are concatenated to obtain the j-th motion state information; The state-temporal sub-model of the motion monitoring model is used to process the j-th motion state information and the (j-1)-th state latent feature information to obtain the j-th state latent feature information. When j=1, the (j-1)-th state latent feature information is a vector of all zeros. In j= At that time, the latent feature information of the j-th state is input into the second fully connected layer and the second activation layer for processing to obtain the patient's exercise suitability index, where, This is the sequence number of the last moving video frame.

2. The intelligent home-based exercise rehabilitation management method for COPD patients according to claim 1, characterized in that, The platform collects assessment scales, and based on these scales and the patient's records of rehabilitation exercise types selected during historical rehabilitation cycles, determines potential rehabilitation exercise options and sends them to the healthcare provider, including: The questions in each category of the assessment scale are combined with the patient's answers to obtain category statements; By using a natural language processing model, each category statement is processed to obtain category semantic feature information; By using a natural language processing model, the descriptive information of each rehabilitation exercise option is processed to obtain the descriptive semantic feature information of each category of rehabilitation exercise option. The descriptive semantic features and category semantic features of each rehabilitation exercise option are processed through the first cross-attention mechanism to obtain the training adaptability description information of each rehabilitation exercise option. Based on the training fit description information, determine the training fit index for each rehabilitation exercise option; Based on the patient's history of rehabilitation exercise selection records, the historical category information of the rehabilitation exercise options selected; Based on the historical category information and the training suitability index, the pending rehabilitation exercise options are determined and sent to the medical staff.

3. The intelligent home-based exercise rehabilitation management method for COPD patients according to claim 2, characterized in that, Based on the historical category information and the training suitability indicators, pending rehabilitation exercise options are determined and sent to the medical staff, including: Based on the training suitability index, select the appropriate rehabilitation exercise option from multiple rehabilitation exercise options; Based on the historical category information, determine the historical selection percentage for each category; Based on the category to which the appropriate rehabilitation exercise option belongs, the training suitability index of the appropriate rehabilitation exercise option, and the historical selection ratio, determine the selection index for each appropriate rehabilitation exercise option. Based on the selected indicators, select the pending rehabilitation exercise option from the suitable rehabilitation exercise options and send it to the medical staff.

4. The intelligent home-based exercise rehabilitation management method for COPD patients according to claim 3, characterized in that, Based on the category to which the appropriate rehabilitation exercise option belongs, the training suitability index of the appropriate rehabilitation exercise option, and the historical selection ratio, the selection index for each appropriate rehabilitation exercise option is determined, including: According to the formula ， Determine the selection criteria for the i-th suitable rehabilitation exercise option ,in, For the i-th suitable rehabilitation exercise option, The percentage of historical selections for the same category as the i-th adaptive rehabilitation exercise option. and For random weights, where, The probability of being 1 is , The probability of being 0 is , , where i is a positive integer.

5. The intelligent home-based exercise rehabilitation management method for COPD patients according to claim 1, characterized in that, The training steps of the motion monitoring model include: Acquire sample motion video frames, sample rehabilitation motion video frames, and sample physiological index data; The latent feature information of the k-th sample action is obtained by using the k-th sample motion video frame and the k-th sample rehabilitation motion video frame, as well as the second cross-attention mechanism and motion temporal sub-model of the motion monitoring model. The state latent feature information of the kth sample is obtained by using the action latent feature information of the kth sample, the physiological index data of the kth group of samples, and the state temporal sub-model of the motion monitoring model. Based on the latent feature information of each sample's action and the latent feature information of the sample's state, the loss function of the motion monitoring model is obtained. The motion monitoring model is trained based on its loss function to obtain the trained motion monitoring model.

6. The intelligent home-based exercise rehabilitation management method for COPD patients according to claim 5, characterized in that, Based on the latent feature information of each sample's action and state, the loss function of the motion monitoring model is obtained, including: Using a motion recognition model, the first motion feature information of each sample motion video frame and the second motion feature information of each sample rehabilitation motion video frame are determined. The latent feature information of each sample action is input into the third fully connected layer and the third activation layer for processing to obtain the action state matching index of each sample motion video frame. Based on the action state matching index, as well as the first action feature information and the second action feature information, determine the action state loss function; The latent feature information of each sample action is input into the fourth fully connected layer and the fourth activation layer for processing to obtain the action trend matching index of each sample motion video frame. Based on the action trend matching index, as well as the first action feature information and the second action feature information, determine the action trend loss function; The latent feature information of the last sample state is input into the second fully connected layer and the second activation layer for processing to obtain the sample motion suitability index. Based on the annotation information of the sample exercise fitness index and sample physiological index data, the exercise fitness loss function is obtained; The loss function of the motion monitoring model is obtained based on the motion state loss function, motion trend loss function, and motion suitability loss function.

7. The intelligent home-based exercise rehabilitation management method for COPD patients according to claim 6, characterized in that, Based on the action trend matching index, as well as the first action feature information and the second action feature information, the action trend loss function is determined, including: According to the formula ， Determine the action trend loss function ,in, This refers to the first motion feature information of the k-th sample motion video frame. This refers to the second motion feature information of the t-th sample rehabilitation exercise video frame. The action trend matching index is the latent feature information of the action of the k-th sample. The preset number of comparisons is N, where N is the number of sample motion video frames, max is the function to take the maximum value, min is the function to take the minimum value, and k, N, and All are positive integers.

8. A smart home-based exercise rehabilitation management system for COPD patients, used to perform the method as described in any one of claims 1-7, characterized in that, include: The assessment scale module prompts patients to fill out the assessment scale on the patient's end at the beginning of the rehabilitation cycle; The module for pending rehabilitation exercise options involves the platform collecting assessment scales, determining pending rehabilitation exercise options based on the assessment scales and the patient's rehabilitation exercise type selection records in the historical rehabilitation cycle, and sending them to the medical staff. The rehabilitation exercise recommendation option module, after confirmation by medical staff, sends the pending rehabilitation exercise options as rehabilitation exercise recommendations to the patient's end; The rehabilitation exercise video module prompts the patient to put on the wearable monitoring device, and prompts the patient to stand in front of the camera of the terminal device and select the rehabilitation exercise video from the rehabilitation exercise recommendation options; The data acquisition module plays the rehabilitation exercise video through the patient's terminal device during the rehabilitation exercise process, captures the rehabilitation exercise video of the patient through the camera, and collects various physiological indicator data of the patient through wearable monitoring devices. The exercise suitability index module involves the platform fusing features from exercise videos and physiological index data using an exercise monitoring model to obtain movement trend features, thereby determining the patient's exercise suitability index and sending it to the medical staff. The feature fusion includes splicing the feature information of the video frames of the exercise video with the feature information of the physiological index data, and performing time-series analysis on the spliced ​​feature information to determine the exercise suitability index. The new rehabilitation exercise recommendation option module determines the new rehabilitation exercise recommendation options after receiving the rehabilitation exercise adjustment plan from the medical staff. The compliance reporting module determines compliance reports based on the patient's historical records of rehabilitation exercises at the end of the rehabilitation cycle.

Images