Motion recognition method and device for rehabilitation training of patients with knee osteoarthritis

Abstract

The application discloses a kind of action recognition method and equipment for rehabilitation training of knee osteoarthritis patient, belong to action recognition and auxiliary rehabilitation technical field.Recognition method includes first image after being handled by the multiple continuation modules of front and rear stacking, each continuation module sequentially outputs first information set;After being handled by corresponding intermediate mechanism, two adjacent first information sets obtain corresponding second information set, carry out semi-dispersed feature purification to multiple second information sets, obtain second image;Second image is handled using human key point detection model, and the position information of patient human key point is extracted and obtained;The action of patient is identified based on position information.The application can pass through multi-level feature association in the process of handling and obtaining second image, better establish the spatial correlation between long-distance key points, greatly improve the accuracy of subsequent key point detection, and then improve the accuracy of action recognition.

Description

Technical Field

[0001] This invention belongs to the field of motion recognition and assisted rehabilitation technology, specifically relating to a motion recognition method and device for rehabilitation training of patients with knee osteoarthritis. Background Technology

[0002] Knee osteoarthritis (KOA) is a chronic joint disease characterized by degeneration of the knee cartilage and bone hyperplasia. It is one of the most common osteoarthritis diseases among middle-aged and elderly people. This disease causes knee pain, stiffness, and limited mobility, severely affecting basic daily living abilities such as walking and climbing stairs, reducing quality of life, and placing a heavy medical and care burden on families and society.

[0003] Rehabilitation training is a core method for non-surgical treatment of knee osteoarthritis and a crucial step in postoperative functional recovery. Scientific and standardized rehabilitation training can improve knee joint mobility, strengthen lower limb muscles, correct abnormal gait, slow disease progression, and reduce the risk of disability. However, due to the large number of patients, the long rehabilitation period, and the extremely high requirements for the standardization of training movements, traditional rehabilitation models are no longer sufficient to meet the needs of large-scale, personalized, and long-term rehabilitation. Summary of the Invention

[0004] In view of this, the present invention provides a motion recognition method and device for rehabilitation training of patients with knee osteoarthritis, so as to better meet the needs of large-scale, personalized and long-term rehabilitation training.

[0005] To achieve the above objectives, the solution adopted by this invention is: a motion recognition method for rehabilitation training of patients with knee osteoarthritis, comprising the following steps:

[0006] Acquire the patient's first image;

[0007] After the first image is processed by multiple continuation modules stacked one after the other, each continuation module outputs a first information set in sequence;

[0008] After being processed by the corresponding intermediary mechanism, two adjacent first information sets are used to obtain corresponding second information sets. The intermediary mechanism is used to perform intermediary dual adjustment on the two adjacent first information sets.

[0009] Semi-distributed feature purification is performed on multiple sets of the second information to obtain a second image;

[0010] The second image is processed using a human key point detection model to extract the location information of the patient's human key points;

[0011] The patient's actions are identified based on the location information.

[0012] The aforementioned continuation module, intermediary mechanism, and feature purification internal operator were all trained before processing the first image. Similarly, the aforementioned human keypoint detection model was also trained before processing the second image. In practical applications, images taken by patients often contain various factors that affect the extraction of human keypoint location information, such as backlighting, shadows, motion blur caused by limb movements, and lens trailing. To address these issues, this invention first performs targeted optimization processing on the original first image to generate a second image. Based on the second image, the location information of human keypoints is extracted, improving the accuracy of keypoint location recognition.

[0013] In existing image optimization techniques, each feature extraction module operates relatively independently, resulting in low inter-layer correlation of the obtained information. This makes it impossible to fully depict the linkage relationship between multiple human key points (e.g., hip-knee-ankle), and the optimized image has limited effect on improving the extraction of key point position information. This application, after obtaining multiple first information sets, first performs intermediate dual adjustment on two adjacent first information sets, and then performs semi-distributed feature purification on all second information sets. This breaks down multi-level feature associations, better establishes spatial relationships between long-distance key points, significantly improves the accuracy of subsequent key point detection, and thus improves the accuracy of action recognition.

[0014] As an exemplary implementation, the connection continuation module processing procedure includes:

[0015] The first renovation agency processes the information to be processed to obtain the first renovation information, the third renovation information, and the fourth renovation information.

[0016] The second modification agency processes the first and third modification information to obtain the fifth, sixth, and seventh modification information.

[0017] The fourth modification information and the seventh modification information are coupled to obtain dual information;

[0018] The first information set is obtained by coupling the fifth modification information with the sixth modification information.

[0019] As an exemplary implementation, the first modification mechanism processing includes:

[0020] The information to be processed is processed by the first convolutional layer and the first LeakyReLU layer in series to obtain the first modification information;

[0021] The information to be processed is processed by the second convolutional layer and the first tanh layer in series to obtain the second modification information;

[0022] The second modification information and the information to be processed are subtracted element by element and then processed by the second tanh layer to obtain the third modification information;

[0023] The third modification information is multiplied element-wise by the first modification information to obtain the fourth modification information.

[0024] As an exemplary implementation, the second modification mechanism processing includes:

[0025] The first modification information is processed by the third convolutional layer and the third tanh layer in series to obtain the fifth modification information;

[0026] The third modification information is processed by the fourth convolutional layer and the second LeakyReLU layer to obtain the sixth modification information;

[0027] The seventh modification information is obtained by subtracting the fifth modification information from the sixth modification information element by element.

[0028] As an exemplary implementation, two adjacent first information sets are processed by corresponding intermediary institutions to obtain corresponding second information sets, including:

[0029] Obtain two sets of first information adjacent to the intermediary agency, the former set of first information is denoted as the former set of first information, and the latter set of first information is denoted as the latter set of first information; obtain the dual information in the corresponding continuation module;

[0030] The previous information set is processed by a first dilated convolutional layer and a first ReLU layer in series to obtain the first adjustment information;

[0031] The first adjustment information is multiplied element-wise with the second information set and then processed by a series of fifth convolutional layers and second ReLU layers to obtain the second adjustment information;

[0032] The first information set and the second information set are subtracted element by element, and then processed by a second dilated convolutional layer and a third ReLU layer in series to obtain the third adjustment information;

[0033] The second information set is obtained by coupling the dual information, the second adjustment information, and the third adjustment information.

[0034] As an exemplary implementation, the fourth modification information and the seventh modification information are concatenated and then coupled through a sixth convolutional layer and a fourth ReLU layer in series.

[0035] Or / and, the fifth modification information and the sixth modification information are concatenated and then coupled through a series of seventh convolutional layers and fifth ReLU layers;

[0036] Alternatively, the dual information, the second adjustment information, and the third adjustment information are added element by element and then coupled through the sixth ReLU layer.

[0037] As an exemplary implementation, semi-distributed feature purification is performed on multiple sets of the second information, including:

[0038] The second information sets are integrated by averaging the features across all channels to obtain multiple first purification information sets; the multiple first purification information sets are coupled to obtain second purification information sets.

[0039] The second information sets are integrated with full-channel feature maximization to obtain multiple third purification information; the multiple third purification information are coupled to obtain fourth purification information.

[0040] Multiple sets of the second information are concatenated and then processed by the first dimensionality reduction module to obtain the fifth purified information. The first dimensionality reduction module is used to reduce the dimensionality of the channel. The fifth purified information is adjusted according to the second purified information to obtain the sixth purified information.

[0041] The sixth purification information is processed by the second dimensionality reduction module to obtain the seventh purification information. The second dimensionality reduction module is used to reduce the channel dimension. The seventh purification information is adjusted according to the fourth purification information to obtain the second image.

[0042] As an exemplary implementation, the coupling of multiple first purification information includes: the multiple first purification information are added element by element and then processed through a first sigmoid layer to obtain the second purification information;

[0043] Or / and, multiple third purification information are coupled, including: after multiple third purification information are spliced ​​together, they are processed through an eighth convolutional layer and a second sigmoid layer to obtain the fourth purification information;

[0044] Or / and, adjusting the fifth purification information according to the second purification information, including weighting the fifth purification information according to the second purification information;

[0045] Or / and, adjusting the seventh purification information according to the fourth purification information, including weighting the seventh purification information according to the fourth purification information.

[0046] As an exemplary implementation, the first dimensionality reduction module has a ninth convolutional layer and a seventh ReLU layer connected in series;

[0047] The second dimensionality reduction module has a cascaded tenth convolutional layer and an eighth ReLU layer.

[0048] The present invention also provides a motion recognition device for rehabilitation training of patients with knee osteoarthritis, including a processor and a memory, wherein the memory stores a computer program, and the processor loads the computer program to execute the method described above.

[0049] The beneficial effects of this invention are:

[0050] Traditional knee osteoarthritis rehabilitation training primarily relies on on-site guidance and assessment by therapists. Therapists judge the standardization of movements by visually observing the patient's actions and combining this with subjective experience, lacking objective and quantifiable assessment standards. Manual observation demands a high level of personal experience and is prone to omissions, leading to highly subjective assessment results that fail to accurately reflect the patient's rehabilitation status and cannot provide reliable data support for adjusting training programs. Furthermore, most knee osteoarthritis patients require long-term home rehabilitation, but the lack of real-time professional guidance in the home setting can lead to problems such as improper movement and inappropriate training intensity. This not only fails to achieve rehabilitation effects but may also exacerbate cartilage wear and trigger acute pain due to incorrect movements. Simultaneously, the completion and quality of home training are difficult to track effectively, making it difficult for therapists to promptly grasp the patient's progress, resulting in delayed adjustments to the rehabilitation program and generally low patient adherence. This invention employs image-based rehabilitation training movement recognition technology. It eliminates the need for patients to wear sensors, straps, or other wearable devices, achieving movement capture and analysis solely through a regular camera. It offers advantages such as being contactless, low-cost, and easy to operate, meeting the rehabilitation needs of various scenarios including home, community, and hospital settings. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the visualization process of obtaining a second image from a first image processing step according to the present invention. Detailed Implementation

[0052] The embodiments of the present invention will now be described with reference to the accompanying drawings. The terminology used in the embodiments section is for illustrative purposes only and is not intended to limit the scope of the invention.

[0053] This embodiment provides a motion recognition method for rehabilitation training of patients with knee osteoarthritis, including the following steps:

[0054] Step 1: Use a smartphone, tablet, or other terminal device with a camera to acquire video images of the patient. Extract single frames from the continuous video stream at a certain frequency to obtain the first image. The first image can be an RGB color digital image, which is a three-dimensional matrix with dimensions of H×W×3, where H is the image height in pixels, W is the image width in pixels, and 3 indicates that it has three channels (R, G, and B channels).

[0055] Step Two, as follows Figure 1As shown, after the first image is processed by multiple stacked continuation modules, each continuation module sequentially outputs a first information set. The number of continuation modules is typically greater than or equal to three; for example, in this embodiment, the number of continuation modules is set to four. In adjacent continuation modules, the output port of the preceding continuation module is directly connected to the input port of the following continuation module, realizing the layer-by-layer transmission of feature information. Assuming the information input to the continuation modules is denoted as the information to be processed, then the information to be processed received by the input port of the first continuation module is the first image, and after processing, the first continuation module outputs the first first information set. The information to be processed received by the input port of the second continuation module is this first first information set. And so on, the Nth continuation module outputs the Nth first information set, and the information to be processed received by the input port of the (N+1)th continuation module is this Nth first information set.

[0056] Specifically, in this embodiment, the connection module includes a first modification mechanism and a second modification mechanism. For a single connection module, the internal data processing procedure includes:

[0057] The first modification mechanism processes the information to be processed, obtaining first modification information, third modification information, and fourth modification information; the second modification mechanism processes the first and third modification information, obtaining fifth, sixth, and seventh modification information; the fourth and seventh modification information are coupled to obtain dual information; the fifth and sixth modification information are coupled to obtain a first information set. Each of the aforementioned modification information, dual information, and first information set is a three-dimensional digital matrix.

[0058] In this application, there are no particular restrictions on the coupling method between the fourth and seventh modification information, nor are there any particular restrictions on the coupling method between the fifth and sixth modification information, as long as the coupling of the two feature information can be achieved. As an example, in some implementations, the fourth and seventh modification information are concatenated along the channel dimension and then coupled by a series of sixth convolutional layers and fourth ReLU layers; the fifth and sixth modification information are concatenated along the channel dimension and then coupled by a series of seventh convolutional layers and fifth ReLU layers.

[0059] In some embodiments, the first modification mechanism processing includes:

[0060] The first modified information is obtained by processing the information to be processed through the first convolutional layer and the first LeakyReLU layer; the second modified information is obtained by processing the information to be processed through the second convolutional layer and the first tanh layer; the second modified information is processed by subtracting the corresponding element from the information to be processed element by element (e.g., subtracting the corresponding element from the information to be processed from the element in the second modified information) and then processed by the second tanh layer; the third modified information is multiplied by the first modified information element by element to obtain the fourth modified information.

[0061] In some embodiments, the second modification mechanism process includes:

[0062] The first modification information is processed by the third convolutional layer and the third tanh layer to obtain the fifth modification information; the third modification information is processed by the fourth convolutional layer and the second LeakyReLU layer to obtain the sixth modification information; the fifth modification information and the sixth modification information are subtracted element by element (for example, the element in the fifth modification information is subtracted from the corresponding element in the sixth modification information) to obtain the seventh modification information.

[0063] By employing the aforementioned algorithm, the reconnection mechanism extracts residual information (third and seventh modification information) during the calculation of the first information set, enabling it to identify and suppress interference from abrupt changes in light and shadow. Furthermore, through multi-branch decoupling, it isolates noise from effective features, facilitating the distinction between true limb edges and false edges generated by motion blur.

[0064] Step 3: After processing by the corresponding intermediary agencies, two adjacent first information sets obtain corresponding second information sets. The intermediary agencies are used to perform intermediary dual adjustments on the two adjacent first information sets. For example, assuming M continuation modules are set up, then M-1 intermediary agencies are needed. These M-1 intermediary agencies have a one-to-one correspondence with the subsequent M-1 continuation modules (according to the direction of feature information transmission from front to back, the Mth continuation module corresponds to the M-1th intermediary agency, and the mth continuation module corresponds to the m-1th intermediary agency). Two adjacent first information sets refer to the first information sets output by two adjacent continuation modules. For example, the two adjacent first information sets processed by the first intermediary agency are the first first information set (output by the first continuation module) and the second first information set (output by the second continuation module), and the two adjacent first information sets processed by the Nth intermediary agency are the Nth first information set (output by the Nth continuation module) and the N+1th first information set (output by the N+1th continuation module).

[0065] Specifically, in some implementations, the internal data processing procedures for a single intermediary institution include:

[0066] Obtain two first information sets adjacent to the intermediary agency, the former first information set is denoted as the previous information set, and the latter first information set is denoted as the latter information set; obtain the dual information in the corresponding continuation module (specifically, the (m-1)th intermediary agency obtains the dual information generated in the mth continuation module);

[0067] The previous set of information is processed by the first dilated convolutional layer and the first ReLU layer in series to obtain the first adjustment information;

[0068] The first adjustment information is multiplied element-wise with the second information set, and then processed by the fifth convolutional layer and the second ReLU layer in a series to obtain the second adjustment information;

[0069] The first set of information is subtracted from the second set of information element by element (e.g., the element in the second set of information is subtracted from the corresponding element in the first set of information), and then processed by the second dilated convolutional layer and the third ReLU layer in series to obtain the third adjustment information.

[0070] The second information set is obtained by coupling the dual information, the second adjustment information, and the third adjustment information.

[0071] The intermediary mechanism employs the aforementioned algorithm to achieve cross-layer joint filtering of transient noise, ensuring the stability of features related to key human body points. It also utilizes dilated convolution to expand the receptive field, which helps correct spatial breaks in limbs caused by blurring. Furthermore, the intermediary mechanism introduces dual information as a noise correction benchmark, constraining the feature domains of two layers, which helps unify the multi-layer lighting and shadow domains, further improving the quality of the second image.

[0072] In this application, there are no particular restrictions on the coupling method of the dual information, the second adjustment information, and the third adjustment information, as long as the coupling of the two feature information can be achieved. As an example, in some embodiments, the dual information, the second adjustment information, and the third adjustment information are added element-wise and then processed by a sixth ReLU layer to achieve coupling. Each of the aforementioned adjustment information and second information sets is a three-dimensional digital matrix. There are no particular restrictions on the dilatancy of the two dilated convolutional layers. As an example, in some embodiments, the kernel size of both the first and second dilated convolutional layers is 3×3, and the dilatancy is 2.

[0073] Step 4: Perform semi-distributed feature purification on all the second information sets to obtain the second image. In some implementations, the semi-distributed feature purification process includes:

[0074] The various sets of second information are integrated using full-channel feature averaging to obtain the corresponding first purified information. Full-channel feature averaging, also known as full-channel feature value average pooling, is achieved by calculating the average of all channel feature values ​​at the same spatial location along the channel dimension using Global Average Pooling (GAP). All the first purified information is coupled to obtain the second purified information. As a non-limiting example, in some implementations, all the first purified information is added element-wise and then processed through a first sigmoid layer to achieve coupling and obtain the second purified information.

[0075] The third purified information is obtained by performing full-channel feature maximization integration on each set of second information. Full-channel feature maximization integration, also known as full-channel feature value max pooling, is achieved by extracting the maximum value of all channel features at the same spatial location along the channel dimension using Global MaxPooling (GMP). All the third purified information is coupled to obtain the fourth purified information. As a non-limiting example, in some implementations, all the third purified information is concatenated along the channel dimension and then processed through an eighth convolutional layer and a second sigmoid layer to achieve coupling and obtain the fourth purified information.

[0076] After all the second information sets are concatenated along the channel dimension, they are processed by the first dimensionality reduction module to obtain the fifth purified information. The first dimensionality reduction module is used to reduce the dimensionality of the channel dimension, so that the number of channels in the fifth purified information is less than the number of channels in the information obtained by concatenating all the second information sets along the channel dimension. For example, in some implementations, the first dimensionality reduction module has a cascaded ninth convolutional layer and a seventh ReLU layer. Assuming that the number of channels in each second information set is G (G is usually greater than 10), after concatenating N second information sets, the number of information channels obtained is G×N. After processing by the first dimensionality reduction module, the number of channels in the fifth purified information can be G.

[0077] The fifth purification information is adjusted based on the second purification information to obtain the sixth purification information. In some embodiments, the second purification information has one channel. The second purification information can be first broadcast along the channel dimension to expand the number of channels, making the data size of the broadcast second purification information exactly equal to the data size of the fifth purification information. Then, the corresponding elements of the second and fifth purification information are multiplied together to perform weighted processing on the fifth purification information based on the second purification information, thus obtaining the sixth purification information.

[0078] The sixth purified information is processed by the second dimensionality reduction module to obtain the seventh purified information. The second dimensionality reduction module is used to reduce the channel dimension so that the number of channels in the seventh purified information is 3. In some embodiments, the second dimensionality reduction module has a cascaded tenth convolutional layer and an eighth ReLU layer.

[0079] The seventh purification information is adjusted based on the fourth purification information to obtain the second image. In some embodiments, the data size of the fourth purification information is exactly the same as the data size of the seventh purification information, and the corresponding elements of the fourth purification information and the seventh purification information are multiplied together to achieve weighted processing of the seventh purification information based on the fourth purification information to obtain the second image.

[0080] The purified information and the second image described above are both three-dimensional digital matrices. By employing the aforementioned semi-distributed feature purification, the gradient attenuation of key point image features caused by inverse compensation for blurring is achieved, offsetting edge weakening caused by ghosting and shadows, and improving the clarity of human body edge contours in the second image.

[0081] Step 5: Process the second image using a human keypoint detection model to extract the location information of the patient's human keypoints. This application does not impose any particular limitation on the human keypoint detection model; any existing or well-known model capable of performing this function can be used, such as MediaPipe Pose, AlphaPose, RTMPose, ViTPose, or OpenPose. If necessary, these human keypoint detection models can be retrained using a self-built dataset tailored to the rehabilitation training scenario. The output information of the aforementioned human keypoint detection model includes the type of human keypoint (e.g., knee joint, ankle joint, etc.) and the pixel coordinates (location information) of each keypoint in the second image.

[0082] Step 6: Identify the patient's actions based on location information.

[0083] As one implementation method, multiple first images can be obtained by sequentially capturing image frames from real-time acquired video footage at a certain frequency and in a time sequence. The method provided by this invention is then used to generate corresponding second images based on the first images. A set of human body key point location information is then extracted from each second image. Based on the location information (e.g., pixel coordinates) of each set of human body key points, the required target data is calculated to complete action recognition.

[0084] For example, in some embodiments, the knee flexion angle is identified based on the positional information of the hip, knee, and ankle on the same side of the patient. Specifically, video footage of the patient during rehabilitation training is captured from a certain angle (e.g., 45°) tilted from the patient's front. Image frames are extracted from the video footage at a certain frequency (e.g., 2 times / second) as the first image. The second image is obtained by processing the video footage according to the method provided by this invention. Based on the positional information (pixel coordinates) of the hip, knee, and ankle on the same side of the patient, the knee flexion angle of the lower limb movement is calculated and identified in real time. When the flexion angle exceeds a preset threshold, prompts such as voice and pop-up windows can be triggered in real time to remind the patient to correct the movement and avoid aggravating the condition due to incorrect training methods. Tests show that the method provided by this invention (identifying the movement after detecting human key points based on the second image) significantly improves the accuracy of knee flexion angle recognition compared to directly identifying the movement after detecting human key points based on the first image.

[0085] For example, in other embodiments, when a patient is performing straight leg raise rehabilitation training, the patient lies flat, and video images are captured from the patient's side. Image frames are extracted from the video images at a certain frequency (e.g., 4 times / second) as the first image. The second image is obtained by processing the video images according to the method provided in this invention. Then, based on the pixel coordinates of the hip, knee, and ankle key points on the same side of the patient in the second image, the angle formed by the hip, knee, and ankle is calculated as the target data to complete motion recognition. When the angle is >170°, the patient's lower limb is considered to be almost fully extended, meeting the requirements of straight leg raise rehabilitation training. When the angle is <150°, it is determined that there is knee flexion compensation, which does not meet the standard. The patient can be reminded via voice or pop-up window that the leg is not fully extended or that the knee should not bend. Additionally, calculating the required target data may also include: calculating the vertical distance from the ankle to the straight line between the hip and shoulder based on the pixel coordinates of the shoulder, hip, and ankle key points on the same side of the patient, thereby achieving motion recognition and obtaining the vertical height of the ankle during the patient's straight leg raise rehabilitation training in real time. Within one cycle of raising and lowering the ankle, when the maximum leg raise height is detected to be lower than the preset minimum value, the device will remind the patient via voice or pop-up window to further increase the leg raise height, and can provide real-time feedback on the difference between the maximum leg raise height and the preset minimum height; when the maximum leg raise height is detected to be greater than the preset maximum value, the device will remind the patient to lower the leg raise height, and can provide real-time feedback on the amount by which the maximum leg raise height exceeds the preset maximum height, assisting the patient in correcting the error.

[0086] The extracted key human body location information is arranged chronologically and stored digitally in a storage device. Doctors can remotely view and analyze the relevant data through software, thereby accurately grasping the patient's training status, correcting the patient's incorrect training methods, and remotely guiding the patient to carry out targeted professional rehabilitation training at home based on the patient's condition.

[0087] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. Those skilled in the art, under the guidance of the present invention, can make various similar representations without departing from the spirit and claims of the present invention, and such modifications all fall within the protection scope of the present invention.

Claims

1. A motion recognition method for rehabilitation training of patients with knee osteoarthritis, characterized in that, Includes the following steps: Acquire the patient's first image; After the first image is processed by multiple continuation modules stacked one after the other, each continuation module outputs a first information set in sequence; After being processed by the corresponding intermediary mechanism, two adjacent first information sets are used to obtain corresponding second information sets. The intermediary mechanism is used to perform intermediary dual adjustment on the two adjacent first information sets. Semi-distributed feature purification is performed on multiple sets of the second information to obtain a second image; The second image is processed using a human key point detection model to extract the location information of the patient's human key points; The patient's actions are identified based on the location information.

2. The method according to claim 1, characterized in that: The connection continuation module processing procedure includes: The first renovation agency processes the information to be processed to obtain the first renovation information, the third renovation information, and the fourth renovation information. The second modification agency processes the first and third modification information to obtain the fifth, sixth, and seventh modification information. The fourth modification information and the seventh modification information are coupled to obtain dual information; The first information set is obtained by coupling the fifth modification information with the sixth modification information.

3. The method according to claim 2, characterized in that: The first modification mechanism's processing procedure includes: The information to be processed is processed by the first convolutional layer and the first LeakyReLU layer in series to obtain the first modification information; The information to be processed is processed by the second convolutional layer and the first tanh layer in series to obtain the second modification information; The second modification information and the information to be processed are subtracted element by element and then processed by the second tanh layer to obtain the third modification information; The third modification information is multiplied element-wise by the first modification information to obtain the fourth modification information.

4. The method according to claim 2, characterized in that: The second modification mechanism's processing includes: The first modification information is processed by the third convolutional layer and the third tanh layer in series to obtain the fifth modification information; The third modification information is processed by the fourth convolutional layer and the second LeakyReLU layer to obtain the sixth modification information; The seventh modification information is obtained by subtracting the fifth modification information from the sixth modification information element by element.

5. The method according to claim 2, 3, or 4, characterized in that: After two adjacent sets of the first information are processed by their respective intermediary agencies, a corresponding set of the second information is obtained, including: Obtain two sets of first information adjacent to the intermediary agency, the former set of first information is denoted as the former set of first information, and the latter set of first information is denoted as the latter set of first information; obtain the dual information in the corresponding continuation module; The previous information set is processed by a first dilated convolutional layer and a first ReLU layer in series to obtain the first adjustment information; The first adjustment information is multiplied element-wise with the second information set and then processed by a series of fifth convolutional layers and second ReLU layers to obtain the second adjustment information; The first information set and the second information set are subtracted element by element, and then processed by a second dilated convolutional layer and a third ReLU layer in series to obtain the third adjustment information; The second information set is obtained by coupling the dual information, the second adjustment information, and the third adjustment information.

6. The method according to claim 5, characterized in that: The fourth modification information and the seventh modification information are concatenated and then coupled through a sixth convolutional layer and a fourth ReLU layer in series. Or / and, the fifth modification information and the sixth modification information are concatenated and then coupled through a series of seventh convolutional layers and fifth ReLU layers; Alternatively, the dual information, the second adjustment information, and the third adjustment information are added element by element and then coupled through the sixth ReLU layer.

7. The method according to claim 1, characterized in that: Semi-distributed feature purification is performed on multiple sets of the second information, including: The second information sets are integrated by averaging the features across all channels to obtain multiple first purification information sets; the multiple first purification information sets are coupled to obtain second purification information sets. The second information sets are integrated with full-channel feature maximization to obtain multiple third purification information; the multiple third purification information are coupled to obtain fourth purification information. Multiple sets of the second information are concatenated and then processed by the first dimensionality reduction module to obtain the fifth purified information. The first dimensionality reduction module is used to reduce the dimensionality of the channel. The fifth purified information is adjusted according to the second purified information to obtain the sixth purified information. The sixth purification information is processed by the second dimensionality reduction module to obtain the seventh purification information. The second dimensionality reduction module is used to reduce the channel dimension. The seventh purification information is adjusted according to the fourth purification information to obtain the second image.

8. The method according to claim 7, characterized in that: The coupling of multiple first purification information includes: the multiple first purification information are added element by element and then processed by a first sigmoid layer to obtain the second purification information; Or / and, multiple third purification information are coupled, including: after multiple third purification information are spliced ​​together, they are processed through an eighth convolutional layer and a second sigmoid layer to obtain the fourth purification information; Or / and, adjusting the fifth purification information according to the second purification information, including weighting the fifth purification information according to the second purification information; Or / and, adjusting the seventh purification information according to the fourth purification information, including weighting the seventh purification information according to the fourth purification information.

9. The method according to claim 7, characterized in that: The first dimensionality reduction module has a ninth convolutional layer and a seventh ReLU layer connected in series; The second dimensionality reduction module has a cascaded tenth convolutional layer and an eighth ReLU layer.

10. A motion recognition device for rehabilitation training of patients with knee osteoarthritis, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor loading the computer program to perform the method as described in any one of claims 1-9.

Images