Patient gait recognition algorithm based on improved Lite-HRNet model

Abstract

The invention provides a patient gait recognition algorithm based on an improved Lite-HRNet model, and provides a lightweight high-resolution human body posture estimation network applied to human body gait recognition and detection for the problems of limited computing resources and low accuracy of the lightweight human body posture estimation network. Specifically, videos are randomly selected from a patient gait video data set for video data enhancement, the videos after data enhancement are obtained, the videos are input into the improved Lite-HRNet model frame by frame for training, and the trained improved Lite-HRNet model is obtained; the method for improving the Lite-HRNet model comprises the following steps: firstly, introducing phantom convolution and a CBAM attention mechanism into a second convolution in a trunk stage to carry out image feature extraction; secondly, introducing phantom convolution and a decoupling attention mechanism into all branches of the model to serve as a basic residual model; finally, a continuous human body key point sequence is obtained through output of the GC module in up-sampling of the fourth stage; and inputting the continuous human body key point sequence into the C3D network to obtain a gait classification result.

Description

Technical Field

[0001] The present invention relates to the technical field of human key point detection, and in particular to a patient gait recognition algorithm based on an improved Lite-HRNet model. Background Art

[0002] Lumbar disc herniation is a common and epidemic disease among low back pain. Prevention and large-scale screening of lumbar disc herniation are of great significance to promoting the development of medical and health care and improving people's quality of life and health level.

[0003] At present, most hospitals use traditional image detection methods, which require manual labeling by imaging experts. Facing tens of thousands of medical image labeling, a huge workload is required, and its accuracy is limited by doctors' subjectivity, huge differences in cognition, and fatigue.

[0004] In recent years, with the continuous development of human posture estimation networks, certain results have been achieved in gait recognition of patients with lumbar disc herniation: compared with traditional time-consuming and labor-intensive manual labeling, machine learning-based methods have the characteristics of high degree of automation and easy scalability.

[0005] However, existing human posture estimation algorithms are difficult to combine with gait recognition algorithms for patients with lumbar disc herniation. Although researchers use deeper and more complex networks to estimate human posture, the increase in the number of neural network parameters slows down the operation, which is not conducive to practical applications in mobile or embedded devices in hospitals. Existing lightweight human posture estimation networks are not accurate, and there are problems such as misjudgment or omission of joints during the detection process.

[0006] Therefore, how to keep the detection accuracy of the human posture estimation network basically unchanged, reduce the number of network parameters and computational complexity, realize the combination of lightweight human posture estimation network and gait recognition algorithm, and apply it in hospital equipment is a problem that technical personnel in this field need to solve. Summary of the invention

[0007] In view of the limitations and challenges of existing solutions, the present invention proposes a patient gait recognition algorithm based on an improved Lite-HRNet model. In view of the limited computing resources of mobile or embedded devices in hospitals and the low accuracy of existing lightweight human posture estimation networks, the present method proposes a lightweight and high-resolution human posture estimation network for human gait recognition detection. Specifically, a video is randomly selected from a patient gait video data set for video data enhancement to obtain a data-enhanced video, which is input frame by frame into an improved Lite-HRNet model for training to obtain a trained improved Lite-HRNet model; the improved Lite-HRNet model includes: firstly, the second convolution in the trunk stage introduces phantom convolution and CBAM attention mechanism for image feature extraction; secondly, all branches of the model introduce phantom convolution and decoupled attention mechanism as the basic residual model; finally, the GC module is used in the upsampling of the fourth stage to output a continuous sequence of human key points; the continuous sequence of human key points is input into the C3D network to obtain the result of gait classification. Specifically, this method proposes a lightweight gait recognition and detection algorithm to assist doctors in preventing and screening lumbar disc herniation. It combines a lightweight human posture estimation and detection network with a C3D video analysis network, and finally combines the lightweight gait recognition algorithm with the treatment of lumbar disc herniation in medicine. This method better solves the problem of doctors' prevention and screening of lumbar disc herniation, helps doctors better diagnose and treat patients, and realizes its application on mobile or embedded devices with limited computing resources in hospitals.

[0008] A patient gait recognition algorithm based on an improved Lite-HRNet model of the present invention comprises the following steps:

[0009] S1: Obtain a patient gait video dataset, randomly select N videos from the patient gait video dataset to perform video data enhancement, and obtain a data-enhanced video;

[0010] S2: Input the data-enhanced video frame by frame into the improved Lite-HRNet model for training, and obtain the trained improved Lite-HRNet model; the improved Lite-HRNet model includes: firstly, introducing phantom convolution and CBAM attention mechanism in the second convolution of the trunk stage to extract image features; secondly, introducing phantom convolution and decoupled attention mechanism as the basic residual model in all branches of the model; finally, using GC module output in the upsampling of the fourth stage to obtain a continuous sequence of human key points;

[0011] S3: The continuous human key point sequence is processed and converted into the input format of the C3D network, and then the key point sequence is input into the C3D network to obtain the patient gait classification result.

[0012] Furthermore, in step S1, N videos are randomly selected from the patient gait video dataset for video data enhancement, and the video data enhancement includes:

[0013] S21: mirror flip, randomly rotate and scale the video;

[0014] S22: extract 16 frames from each video after mirror flipping, random rotation and scaling, and process the video sequence into image data required for subsequent use.

[0015] Furthermore, in the step S2, the second convolution in the trunk stage introduces phantom convolution and CBAM attention mechanism to extract image features, which specifically includes the following steps:

[0016] S31: The data-enhanced image is input into the first convolution of the backbone stage with a convolution kernel size of 3×3 and a channel number of 32, and a feature map Feature-T0 is obtained after a downsampling operation, and the feature map size is 1 / 2 of the original image;

[0017] S32: The feature map Feature-T0 is input into the bottleneck module designed in the second stage of the backbone. The module is composed of two phantom convolutions in series. A deep convolution with a step size of 2 is introduced into the module to reduce the size of the feature map to half of the input. The jump connection is also downsampled to achieve Add operation alignment. After the deep convolution, the CBAM attention mechanism is introduced to enhance the spatial and channel information, and the feature map Feature-T1 is output. Its feature map size is 1 / 4 of the original image and the number of channels is 32.

[0018] Furthermore, in step S2, phantom convolution and decoupled attention mechanism are introduced as the basic residual structure in all branches of the model, which specifically includes the following steps:

[0019] S41: The feature map Feature-T1 is used as a high-resolution main branch, the size of the main branch feature map is 1 / 4 of the original image, and the main branch is downsampled by two times to obtain a feature map secondary branch Feature-Down1, whose feature map size is 1 / 8 of the original image;

[0020] S42: the feature map Feature-T1 and the feature map Feature-Down1 are passed through the basic structure of the second stage Stage2 in parallel, and phantom convolution and decoupled attention mechanism are introduced in the basic structures of Stage2, Stage3 and Stage4 to replace the first point-by-point convolution of the original shuffle module, and SE attention mechanism is introduced to replace the second point-by-point convolution of the original shuffle module;

[0021] S43: Fusing the feature graph Feature-T1 and the feature graph Feature-Down1 in the transition structure of Stage2, downsampling the feature graph Feature-T1 by two times and fusing it with the feature graph Feature-Down1 to obtain the feature graph Feature-Down2, upsampling the feature graph Feature-Down1 by two times and fusing it with the feature graph Feature-T1 to obtain the feature graph Feature-T2, and downsampling the feature graph Feature-Down1 by two times to obtain a feature graph sub-branch Feature-X1;

[0022] S44: The feature maps Feature-T2, Feature-Down2 and Feature-X1 are passed through the third stage Stage3 basic structure in parallel, and feature fusion is performed in the Stage3 transition structure. The feature map Feature-T2 is downsampled twice, Feature-X1 is upsampled twice, and the features are fused with the feature map Feature-Down2 to obtain the feature map Feature-Down3. The feature map Feature-Down2 is upsampled twice, Feature-X1 is upsampled four times, and the features are fused with the feature map Feature-T2 to obtain the feature map Feature-T3. The feature map Feature-T2 is downsampled twice, Feature-Down2 is downsampled four times, and the features are fused with the feature map Feature-X1 to obtain the feature map Feature-X2. At the same time, Feature-X1 is downsampled twice to obtain the feature map sub-branch Feature-M1.

[0023] S45: The feature maps Feature-T3, Feature-Down3, Feature-X2 and Feature-M1 are passed through the fourth stage Stage4 basic structure in parallel, and feature fusion is performed in the Stage4 transition structure. The feature map Feature-T3 is downsampled twice, Feature-X2 is upsampled twice, and Feature-M1 is upsampled four times, and the feature map Feature-Down3 is feature fused to obtain the feature map Feature-Down4. The feature map Feature-T3 is downsampled four times, Feature-Down3 is downsampled twice, and Feature-M1 is upsampled four times. The feature map Feature-M1 is upsampled twice and fused with the feature map Feature-X2 to obtain the feature map Feature-X3, the feature map Feature-T3 is downsampled eight times, Feature-Down3 is downsampled four times, Feature-X2 is downsampled twice and fused with the feature map Feature-M1 to obtain the feature map Feature-M2, the feature map Feature-M1 is upsampled eight times, Feature-X2 is upsampled four times, Feature-Down3 is upsampled twice and fused with the feature map Feature-T3 to obtain the feature map Feature-T4.

[0024] Furthermore, in step S2, the upsampling in the fourth stage uses the GC module output to obtain a continuous sequence of human key points, which specifically includes the following steps:

[0025] S51: The feature map Feature-T4 is not processed, and the feature maps Feature-Down4, Feature-X3, and Feature-M2 are upsampled by a convolution layer with a convolution kernel size of 1×1 and a BN layer, and then amplified by n times by Upsample, and finally the upsampled result is obtained by the GC module;

[0026] S52: Upsample the feature map Feature-M2 by 8 times to obtain Feature-Up1, upsample the feature map Feature-X3 by 4 times to obtain Feature-Up2, upsample the feature map Feature-Down4 by 2 times to obtain Feature-Up3, add and fuse the feature maps Feature-Up1, Feature-Up2 and Feature-Up3 with the feature layer Feature-T4 retaining the highest resolution, and finally obtain a continuous sequence of human key points through a Relu function and a convolution layer with a convolution kernel size of 1×1 and 17 convolution kernels.

[0027] Furthermore, in step S3, the continuous human key point sequence is processed into an input format of a C3D network, and then the key point sequence is input into the C3D network to obtain a patient gait classification result, which specifically includes the following steps:

[0028] S61: inputting the 16 frames of image data of the video data into the improved Lite-HRNet in sequence, and outputting 16 sequences input into the C3D network;

[0029] S62: Process the key point sequence into the input format of the C3D network, the number of frames taken out of the video clip is 16, the image resolution is resized to 128×172, and then randomly cropped to 112×112;

[0030] S63: The C3D network consists of eight convolutional layers, five pooling layers, two fully connected layers and a SoftMax classifier. The convolution kernels of all layers are 3×3×3 in size. The convolutional layer is followed by a maximum pooling layer. The two connection layers of the network convert the feature map into the patient gait classification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the patient gait recognition algorithm based on the improved Lite-HRNet model.

[0032] Figure 2 Improved Lite-HRNet model structure diagram

[0033] Figure 3 Schematic diagram of the bottleneck module in the improved Lite-HRNet model

[0034] Figure 4 This is the internal structure diagram of the phantom convolution in the bottleneck module

[0035] Figure 5 The internal structure diagram of the CBAM attention mechanism in the bottleneck module

[0036] Figure 6 Schematic diagram of the block module in the improved Lite-HRNet model

[0037] Figure 7 Schematic diagram of the upsampling GC module in the improved Lite-HRNet model

[0038] Figure 8 This is the internal structure diagram of the GC module

[0039] Fig. 9 Internal structure diagram of the C3D network model DETAILED DESCRIPTION

[0040] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below with reference to examples and drawings.

[0041] Refer to the attached Figure 1 As shown, the present invention proposes a patient gait recognition algorithm based on an improved Lite-HRNet model, comprising the following steps:

[0042] S1: Obtain a patient gait video dataset, randomly select N videos from the patient gait video dataset to perform video data enhancement, and obtain a data-enhanced video;

[0043] S2: Input the data-enhanced video frame by frame into the improved Lite-HRNet model for training, and obtain the trained improved Lite-HRNet model; the improved Lite-HRNet model includes: firstly, introducing phantom convolution and CBAM attention mechanism in the second convolution of the trunk stage to extract image features; secondly, introducing phantom convolution and decoupled attention mechanism as the basic residual model in all branches of the model; finally, using GC module output in the upsampling of the fourth stage to obtain a continuous sequence of human key points;

[0044] S3: The continuous human key point sequence is processed and converted into the input format of the C3D network, and then the key point sequence is input into the C3D network to obtain the patient gait classification result.

[0045] The following is a detailed description of each of the above steps:

[0046] In the above step S1, firstly, a patient gait video dataset is obtained, and data enhancement is performed on the dataset, wherein the data enhancement includes: mirror flipping, random rotation and scale scaling of the video; 16 frames are extracted from each video after data enhancement, the human body detection frame is expanded to a fixed aspect ratio of 4:3, and then the frame is cropped from the image, and each image is resized to 256×192 or 384×288. Each image undergoes a series of data enhancement operations, including random flipping, random rotation [-45°, 45°] and scale scaling [0.8, 1.2], and then is converted into a key point sequence, which is divided into a training set and a test set in a ratio of 7:3.

[0047] In the above step S2, the data-enhanced video is input frame by frame into the improved Lite-HRNet model for training to obtain the trained improved Lite-HRNet model; in the embodiment of the present invention, the improved Lite-HRNet model includes: 1. In the second convolution in the trunk stage, phantom convolution and CBAM attention mechanism are introduced to extract image features; 2. All branches of the model are introduced with phantom convolution and decoupled attention mechanism as the basic residual model; 3. In the upsampling of the fourth stage, the GC module output is used to obtain a continuous sequence of human key points.

[0048] As attached Figure 2 As shown in the figure, in the second convolution of the stem stage, phantom convolution and CBAM attention mechanism are introduced to extract image features, which includes the following steps:

[0049] S1: The data-enhanced image is input into the first convolution of the stem stage (stem) with a kernel size of 3×3 and a channel number of 32, and a feature map Feature-T0 is obtained after downsampling operation. The feature map size is 1 / 2 of the original image, and the number of output channels is 32;

[0050] S2: Input the feature map Feature-T0 into the bottleneck module designed in the second trunk stage, such as Figure 3 As shown, in the embodiment of the present invention, in the designed bottleneck module, it is mainly composed of a ghost convolution (GhostModule) module, a deep convolution DWConv (DepthwiseConvolution) and a lightweight channel and spatial attention module (ConvolutionalBlock AttentionModule, CBAM); wherein, the structures of the ghost convolution module and the CBAM attention mechanism module are respectively as shown Figure 4 and Figure 5 As shown in the figure, the designed bottleneck module is composed of two phantom convolutions in series. A deep convolution with a step size of 2 is introduced in the module to reduce the size of the feature map to half of the input. The skip connection is also downsampled to achieve Add operation alignment. The CBAM attention mechanism is introduced after the deep convolution to enhance the spatial and channel information, and the feature map Feature-T1 is output. Its feature map size is 1 / 4 of the original image and the number of channels is 32.

[0051] In the embodiment of the present invention, two phantom convolutions are introduced in the bottleneck module to replace two point-by-point convolutions. The first part of the phantom convolution compresses the channel through convolution and outputs the basis and fewer feature maps. The second part obtains more feature maps through linear transformation. The linear transformation uses the grouped convolution operation to obtain the feature map and the feature map obtained in the first part is combined into the final output. The computational complexity of ordinary convolution is: n·h·w·c·k·k, where n and c are the number of convolution kernels and the number of channels respectively, h and w are the height and width of the feature map respectively, and k is the size of the convolution kernel; the computational complexity of phantom convolution is in are a small number of feature maps in the first part, is the number of linear operations in the second part, d is the kernel size of each linear operation, and the theoretical speedup ratio of introducing phantom convolution to optimize ordinary convolution is shown as follows:

[0052]

[0053] The parameter compression ratio can also be calculated as:

[0054]

[0055] Through the above calculations, we can know that using Ghost convolution can reduce the number of parameters by s times and save s times the training time.

[0056] Furthermore, after the feature map undergoes phantom convolution, it is further subjected to depthwise convolution with a stride of 2 to reduce the size of the feature map to half of the input. The skip connection is also downsampled to achieve alignment of the fusion operation.

[0057] Furthermore, through the CBAM attention mechanism, which includes two sub-modules, the channel attention module (CAM) and the spatial attention module (SAM), which operate on the channel and space respectively, the output feature map is fused with the jump connection to obtain the feature map Feature-T1.

[0058] As attached Figure 2 As shown in the figure, all branches in the stage of the model introduce phantom convolution and decoupled attention mechanism as the basic residual model, as shown in the attached figure. Figure 6As shown in the figure, the designed block module adopts the basic architecture of shuffleblock. First, channel splitting (channelsplit) divides the image channel into two parts, the left side is used for residual connection, and the right side is used for feature extraction. The ghost convolution (1×1Ghost) is introduced to replace the first point-by-point convolution (1×1PWConv). At the same time, the decoupled attention module (DFC) is introduced as a parallel branch of the ghost convolution, and the SE attention mechanism is introduced to replace the second point-by-point convolution. Finally, the left and right branches are combined (concatenation) to realize feature fusion through channel shuffle operation (channelshuffle). Stage2, Stage3 and Stage4 of the model include the following steps:

[0059] S1: The feature map Feature-T1 is used as a high-resolution main branch, and the size of the main branch feature map is 1 / 4 of the original image. At the same time, the main branch is downsampled by two times to obtain a feature map secondary branch Feature-Down1, and its feature map size is 1 / 8 of the original image;

[0060] S2: Fusing the feature map Feature-T1 and the feature map Feature-Down1 in the transition structure of Stage2, downsampling the feature map Feature-T1 by two times and fusing it with the feature map Feature-Down1 to obtain the feature map Feature-Down2, upsampling the feature map Feature-Down1 by two times and fusing it with the feature map Feature-T1 to obtain the feature map Feature-T2, and downsampling the feature map Feature-Down1 by two times to obtain a feature map sub-branch Feature-X1;

[0061] S3: The feature maps Feature-T2, Feature-Down2 and Feature-X1 are passed through the third stage Stage3 basic structure in parallel, and feature fusion is performed in the Stage3 transition structure. The feature map Feature-T2 is downsampled twice, Feature-X1 is upsampled twice, and the features are fused with the feature map Feature-Down2 to obtain the feature map Feature-Down3. The feature map Feature-Down2 is upsampled twice, Feature-X1 is upsampled four times, and the features are fused with the feature map Feature-T2 to obtain the feature map Feature-T3. The feature map Feature-T2 is downsampled twice, Feature-Down2 is downsampled four times, and the features are fused with the feature map Feature-X1 to obtain the feature map Feature-X2. At the same time, Feature-X1 is downsampled twice to obtain the feature map sub-branch Feature-M1.

[0062] S4: The feature maps Feature-T3, Feature-Down3, Feature-X2 and Feature-M1 are passed through the fourth stage Stage4 basic structure in parallel, and feature fusion is performed in the Stage4 transition structure. The feature map Feature-T3 is downsampled twice, Feature-X2 is upsampled twice, and Feature-M1 is upsampled four times, and the feature map Feature-Down3 is fused to obtain the feature map Feature-Down4. The feature map Feature-T3 is downsampled four times, Feature-Down3 is downsampled twice, and Feature-M1 is upsampled four times. The feature map Feature-M1 is upsampled twice and fused with the feature map Feature-X2 to obtain the feature map Feature-X3, the feature map Feature-T3 is downsampled eight times, Feature-Down3 is downsampled four times, Feature-X2 is downsampled twice and fused with the feature map Feature-M1 to obtain the feature map Feature-M2, the feature map Feature-M1 is upsampled eight times, Feature-X2 is upsampled four times, Feature-Down3 is upsampled twice and fused with the feature map Feature-T3 to obtain the feature map Feature-T4.

[0063] As attached Figure 2 As shown in the figure, the GC module is used in the upsampling after the fourth stage (stage4), as shown in the attached Figure 7 As shown in Figure 1, h, w and c represent the height, width and number of channels of the feature map respectively, and n is the upsampling multiple of the feature map (n=2, 4, 8). The input feature map first passes through a convolution layer with a convolution kernel size of 1×1 (1×1Conv) and a BN layer, then is directly amplified by n times through Upsample, and finally passes through the global context attention GC module (Global Context Blcok). The internal structure of the GC module is shown in the attached figure. Figure 8 As shown, the final output is a continuous sequence of human key points. The upsampling fusion after the fourth stage includes the following steps:

[0064] S1: The feature map Feature-T4 is not processed, and the feature maps Feature-Down4, Feature-X3 and Feature-M2 are upsampled by 2 times, 4 times and 8 times respectively;

[0065] S2: Upsample the feature map Feature-M2 by 8 times to obtain Feature-Up1, upsample the feature map Feature-X3 by 4 times to obtain Feature-Up2, upsample the feature map Feature-Down4 by 2 times to obtain Feature-Up3, add and fuse the feature maps Feature-Up1, Feature-Up2 and Feature-Up3 with the feature layer Feature-T4 retaining the highest resolution, and finally obtain a continuous sequence of human key points through the Relu function and a convolution layer with a convolution kernel size of 1×1 and 17 convolution kernels.

[0066] Furthermore, the continuous human key point sequence is processed into the input format of the C3D network, and then the key point sequence is input into the C3D network, as shown in the attached figure. Fig. 9 As shown in the figure, the C3D network consists of eight convolution layers, five pooling layers, two fully connected layers and one classifier layer. The convolution kernels of all layers are 3×3×3 in size. The two connection layers of the network convert the feature map into the patient gait classification result to obtain the patient gait classification result, which specifically includes the following steps:

[0067] S1: Input the 16 frames of image data of the video data into the improved Lite-HRNet in sequence, and output 16 sequences input into the C3D network;

[0068] S2: Process the key point sequence into the input format of the C3D network, the number of frames taken out of the video clip is 16, the image resolution is resized to 128×172, and then randomly cropped to 112×112;

[0069] S3: The processed key point data is input into the C3D network. Through multiple convolutional layers and pooling layers, the feature information is gradually extracted to obtain a feature vector of fixed length, which is finally mapped into the gait classification result through the fully connected layer.

[0070] Through the above-mentioned embodiment description, those skilled in the art can know that the present invention provides a patient gait recognition algorithm based on the improved Lite-HRNet model. First, in the second convolution of the trunk stage, phantom convolution and CBAM attention mechanism are introduced to extract image features, so as to capture the spatial and channel information of the feature map while keeping the model calculation amount unchanged; secondly, all branches of the model introduce phantom convolution and decoupled attention mechanism as the basic residual model, and phantom convolution is used to replace the original point-by-point convolution. At the same time, SE attention mechanism is integrated to strengthen the channel information within the branch and make up for the spatial information loss caused by phantom convolution; finally, in the upsampling of the fourth stage, the GC module output is used to obtain a continuous sequence of human key points, which is used for the transition layer to enhance the perception of feature maps of different resolutions and make full use of feature information of different resolutions; the continuous sequence of human key points after the improved Lite-HRNet is input into the C3D network to obtain the result of gait classification.

[0071] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A patient gait recognition algorithm based on the improved Lite-HRNet model, characterized by The following steps are involved: S1: Obtain a patient gait video dataset, randomly select N videos from the patient gait video dataset to perform video data enhancement, and obtain a data-enhanced video; S2: Input the data-enhanced video frame by frame into the improved Lite-HRNet model for training, and obtain the trained improved Lite-HRNet model; the improved Lite-HRNet model includes: firstly, introducing phantom convolution and CBAM attention mechanism in the second convolution of the trunk stage to extract image features; secondly, introducing phantom convolution and decoupled attention mechanism as the basic residual model in all branches of the model; finally, using GC module output in the upsampling of the fourth stage to obtain a continuous sequence of human key points; S3: The continuous human key point sequence is processed and converted into the input format of the C3D network, and then the key point sequence is input into the C3D network to obtain the patient gait classification result.

2. The patient gait recognition algorithm based on the improved Lite-HRNet model according to claim 1, characterized in that: In the step S1, N videos are randomly selected from the patient gait video dataset for video data enhancement, and the video data enhancement includes: S21: mirror flip, randomly rotate and scale the video; S22: extract 16 frames from each video after mirror flipping, random rotation and scaling, and process the video sequence into image data required for subsequent use.

3. The patient gait recognition algorithm based on the improved Lite-HRNet model according to claim 1, characterized in that: In step S2, the second convolution in the trunk stage introduces phantom convolution and CBAM attention mechanism to extract image features, which specifically includes the following steps: S31: The data-enhanced image is input into the first convolution layer of the backbone stage with a convolution kernel size of 3×3 and a step size of 2, and is downsampled by two times to obtain a feature map Feature-T0; S32: The feature map Feature-T0 is input into the second downsampling module of the backbone stage. The module is composed of two phantom convolutions in series. A deep convolution with a step size of 2 is introduced into the module to reduce the size of the feature map to half of the input. The jump connection is also downsampled to achieve Add operation alignment. After the deep convolution, the CBAM attention mechanism is introduced to enhance the spatial and channel information, and the feature map Feature-T1 is output.

4. The patient gait recognition algorithm based on the improved Lite-HRNet model according to claim 1, characterized in that: In step S2, phantom convolution and decoupled attention mechanism are introduced in all branches of the model as the basic residual model, which specifically includes the following steps: S41: taking the feature map Feature-T1 as a high-resolution main branch, and downsampling the main branch by two times to obtain a feature map secondary branch Feature-Down1; S42: passing the feature map Feature-T1 and the feature map Feature-Down1 through the second stage Stage2 in parallel, introducing phantom convolution and decoupled attention mechanism to replace the first point-by-point convolution of the shuffle module in the basic structure of Stage2, and introducing SE attention mechanism to replace the second point-by-point convolution; S43: performing feature fusion on the feature map Feature-T1 and the feature map Feature-Down1 in the transition structure of Stage2 to obtain fused feature maps Feature-T2 and Feature-Down2, and downsampling the feature map Feature-Down1 by two times to obtain a feature map sub-branch Feature-X1; S44: The feature maps Feature-T2, Feature-Down2 and Feature-X1 are passed through the third stage Stage3 basic structure in parallel, and feature fusion is performed in the Stage3 transition structure to obtain fused feature maps Feature-T3, Feature-Down3 and Feature-X2, and Feature-X1 is downsampled twice to obtain a feature map sub-branch Feature-M1; S45: The feature maps Feature-T3, Feature-Down3, Feature-X2 and Feature-M1 are passed through the fourth stage Stage4 basic structure in parallel, and feature fusion is performed in the Stage4 transition structure to obtain fused feature maps Feature-T4, Feature-Down4, Feature-X3 and Feature-M2.

5. The patient gait recognition algorithm based on the improved Lite-HRNet model according to claim 1, characterized in that: In step S2, the upsampling in the fourth stage uses the GC module output to obtain a continuous sequence of human key points, which specifically includes the following steps: S51: The feature map Feature-T4 is not processed, and the feature maps Feature-Down4, Feature-X3, and Feature-M2 are upsampled by a convolution layer with a convolution kernel size of 1×1 and a BN layer, and then amplified by n times by Upsample, and finally the upsampled result is obtained by the GC module; S52: Upsample the feature map Feature-M2 by 8 times to obtain Feature-Up1, upsample the feature map Feature-X3 by 4 times to obtain Feature-Up2, upsample the feature map Feature-Down4 by 2 times to obtain Feature-Up3, add and fuse the feature maps Feature-Up1, Feature-Up2 and Feature-Up3 with the feature layer Feature-T4 retaining the highest resolution, and finally obtain a continuous sequence of human key points through a Relu function and a convolution layer with a convolution kernel size of 1×1 and 17 convolution kernels.

6. The patient gait recognition algorithm based on the improved Lite-HRNet model according to claim 1, characterized in that: In the step S3, the continuous human key point sequence is processed into the input format of the C3D network, and then the key point sequence is input into the C3D network to obtain the patient gait classification result, which specifically includes the following steps: S61: inputting the 16 frames of image data of the video data into the improved Lite-HRNet in sequence, and outputting 16 sequences input into the C3D network; S62: Process the key point sequence into the input format of the C3D network, the number of frames taken out of the video clip is 16, the image resolution is resized to 128×172, and then randomly cropped to 112×112; S63: The C3D network consists of eight convolutional layers, five pooling layers, two fully connected layers and a SoftMax classifier. The convolution kernels of all layers are 3×3×3 in size. The convolutional layer is followed by a maximum pooling layer. The two connection layers of the network convert the feature map into the patient gait classification result.

Images