A method and system for gait analysis in pediatric rehabilitation

Abstract

This invention relates to the field of artificial intelligence technology, specifically to a gait analysis method and system for pediatric rehabilitation. The method involves analyzing data collected by plantar pressure sensors to obtain the gait cycles of the left and right legs, and determining the start time of multiple stages corresponding to each stage in each gait cycle for each leg. Based on the start time of each stage, multiple frames corresponding to the stage are identified from videos on both sides. These frames corresponding to the start point of each stage are converted into multiple grayscale images. Based on the multiple grayscale images corresponding to each side of the video and a joint recognition model, the position of each joint on each side is obtained. The positions of the joints identified from both sides are processed to obtain the position of each joint. The position sequence of each joint within each gait cycle is obtained according to the order of the stages in the gait cycle. The analysis results are obtained by processing the multiple position sequences corresponding to multiple gait cycles. This invention improves the accuracy of gait analysis.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to a gait analysis method and system for children's rehabilitation. Background Technology

[0002] As part of rehabilitation medicine, pediatric rehabilitation primarily assists children with functional impairments. Through specialized training and therapy, these children can learn or relearn basic motor skills and daily living skills, improving their self-care abilities. This helps alleviate or overcome limitations caused by disability or illness, improving their quality of life; it also helps children better integrate into society and participate in school and community activities. Gait analysis is a scientific method for assessing an individual's gait pattern and plays a crucial role in pediatric rehabilitation training, especially in the diagnosis of psychomotor developmental disorders such as cerebral palsy, autism, ADHD, and Tourette syndrome, as well as in the monitoring of central nervous system infections. In the early stages of rehabilitation, analyzing a child's gait helps diagnose specific problems, such as gait abnormalities, insufficient muscle strength, or coordination issues, which can help doctors develop more targeted rehabilitation plans. During the rehabilitation process, gait analysis helps doctors assess the effectiveness of rehabilitation.

[0003] Gait acquisition and analysis primarily rely on wearable devices and video images. Wearable device-based methods require wearing the equipment, which can be uncomfortable, especially for children, making it difficult for them to accurately mimic their natural walking patterns. Video analysis-based methods overcome these issues, but accuracy remains a challenge. Summary of the Invention

[0004] To minimize interference with children and maximize the accuracy of the analysis, in a first aspect, the present invention provides a gait analysis method for children's rehabilitation, the method comprising the following steps:

[0005] Data collected by multiple plantar pressure sensors is analyzed to obtain the gait cycle of the left and right legs during walking, and to obtain multiple stages of each leg's gait cycle, as well as the start time of each stage; at the same time, videos of both sides of the child's walking process are collected, and multiple frames corresponding to the stage start point are found from each side's video.

[0006] The multiple frames corresponding to the starting point of each stage are converted into multiple grayscale images. The position of each joint on each side is obtained based on the multiple grayscale images corresponding to each side video and the joint recognition model. The positions of the joints recognized from both sides are processed to obtain the position of each joint. The position sequence of each joint in each gait cycle is obtained according to the order of the stages in the gait cycle.

[0007] The analysis results are obtained by processing multiple position sequences corresponding to multiple gait cycles.

[0008] Preferably, the step of analyzing data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking specifically involves:

[0009] A sequence of typical plantar pressure distribution maps for each stage of the left and right legs is pre-constructed;

[0010] Data collected from multiple plantar pressure sensors is converted into plantar pressure distribution maps to be matched. These maps are arranged in chronological order to form a sequence of plantar pressure distribution maps to be matched. A sliding window method is used to find plantar pressure distribution map subsequences in the sequence of plantar pressure distribution maps to be matched that have a matching degree greater than a threshold with the typical plantar pressure distribution map sequence. The starting point of the subsequence is marked as the corresponding stage starting point.

[0011] This allows us to obtain the gait cycles of the left and right legs.

[0012] Preferably, obtaining the position of each joint on each side based on the plurality of grayscale images corresponding to each side's video and the joint recognition model specifically involves:

[0013] The feature map of each grayscale image is obtained by two-dimensional convolution. The feature map corresponding to the grayscale image is added to the first feature map sequence according to the time of the grayscale image in the video. The second feature map sequence of the multiple grayscale image sequences is obtained by three-dimensional convolution.

[0014] The first feature map sequence is divided into multiple sub-sequences according to the depth of the convolution kernel of the three-dimensional convolution, and the sub-sequences are inserted into the second feature map sequence in sequence to obtain the feature map fusion result;

[0015] The feature map fusion result is input into the corresponding joint recognition model to identify the position coordinates of the joint.

[0016] Preferably, the training of the joint recognition model includes two stages, specifically:

[0017] In the first stage, the joint recognition model is trained using the multiple grayscale images corresponding to each side of the video.

[0018] In the second stage, for the left joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the left video to obtain the left joint recognition model; for the right joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the right video to obtain the right joint recognition model.

[0019] Preferably, the step of processing the positions of the joints identified from both sides to obtain the position of each joint specifically involves:

[0020] Determine whether the joint set identified by the left joint recognition model and the joint set identified by the right joint recognition model have the same joint. If they do, take the average of the two positions as the position of the same joint.

[0021] Preferably, the step of processing multiple position sequences corresponding to multiple gait cycles to obtain analysis results specifically involves:

[0022] The position sequence of multiple gait cycles is fitted according to joints and stages to obtain the final position of each joint in one gait cycle;

[0023] The position of each fitted joint in one gait cycle is compared with the position of each fitted joint in one gait cycle in the previous fitting, and the comparison results are plotted on a graph.

[0024] In another aspect, the present invention also provides a gait analysis system for children's rehabilitation, the system comprising the following modules:

[0025] The gait phase acquisition module is used to analyze data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking, and to obtain multiple phases of each leg's gait cycle, as well as the start time of each phase; at the same time, it collects videos of both sides of the child's walking process and finds multiple frames corresponding to the start of each phase from the videos of each side.

[0026] The joint position acquisition module is used to convert multiple frames corresponding to the starting point of each stage into multiple grayscale images, obtain the position of each joint on each side based on the multiple grayscale images corresponding to each side video and the joint recognition model, process the positions of the joints identified from both sides to obtain the position of each joint, and obtain the position sequence of each joint in each gait cycle according to the order of the stages in the gait cycle.

[0027] The analysis module is used to process multiple position sequences corresponding to multiple gait cycles to obtain analysis results.

[0028] Preferably, the step of analyzing data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking specifically involves:

[0029] A sequence of typical plantar pressure distribution maps for each stage of the left and right legs is pre-constructed;

[0030] Data collected from multiple plantar pressure sensors is converted into plantar pressure distribution maps to be matched. These maps are arranged in chronological order to form a sequence of plantar pressure distribution maps to be matched. A sliding window method is used to find plantar pressure distribution map subsequences in the sequence of plantar pressure distribution maps to be matched that have a matching degree greater than a threshold with the typical plantar pressure distribution map sequence. The starting point of the subsequence is marked as the corresponding stage starting point.

[0031] This allows us to obtain the gait cycles of the left and right legs.

[0032] Preferably, obtaining the position of each joint on each side based on the plurality of grayscale images corresponding to each side's video and the joint recognition model specifically involves:

[0033] The feature map of each grayscale image is obtained by two-dimensional convolution. The feature map corresponding to the grayscale image is added to the first feature map sequence according to the time of the grayscale image in the video. The second feature map sequence of the multiple grayscale image sequences is obtained by three-dimensional convolution.

[0034] The first feature map sequence is divided into multiple sub-sequences according to the depth of the convolution kernel of the three-dimensional convolution, and the sub-sequences are inserted into the second feature map sequence in sequence to obtain the feature map fusion result;

[0035] The feature map fusion result is input into the corresponding joint recognition model to identify the position coordinates of the joint.

[0036] Preferably, the training of the joint recognition model includes two stages, specifically:

[0037] In the first stage, the joint recognition model is trained using the multiple grayscale images corresponding to each side of the video.

[0038] In the second stage, for the left joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the left video to obtain the left joint recognition model; for the right joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the right video to obtain the right joint recognition model.

[0039] Preferably, the step of processing the positions of the joints identified from both sides to obtain the position of each joint specifically involves:

[0040] Determine whether the joint set identified by the left joint recognition model and the joint set identified by the right joint recognition model have the same joint. If they do, take the average of the two positions as the position of the same joint.

[0041] Preferably, the step of processing multiple position sequences corresponding to multiple gait cycles to obtain analysis results specifically involves:

[0042] The position sequence of multiple gait cycles is fitted according to joints and stages to obtain the final position of each joint in one gait cycle;

[0043] The position of each fitted joint in one gait cycle is compared with the position of each fitted joint in one gait cycle in the previous fitting, and the comparison results are plotted on a graph.

[0044] In addition, the present invention also provides a computer program product that implements the method described above when executed by a processor.

[0045] This invention uses a plantar pressure sensor combined with a joint recognition model to accurately identify the position of each gait cycle stage node, and compares the obtained position with the previous position to obtain the comparison result in rehabilitation training, which effectively improves the accuracy of gait analysis. Attached Figure Description

[0046] Figure 1 This is a flowchart of Example 1;

[0047] Figure 2 This is a diagram showing the distribution of plantar pressure.

[0048] Figure 3 This is a schematic diagram of feature fusion. Detailed Implementation

[0049] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this invention, if the collection of personal privacy data is involved, such as facial recognition or mobile phone usage information, prior consent will be obtained, including but not limited to verbal reminders, posters, and mobile phone reminders; if there are conflicts with laws and regulations, production or use will be within the scope permitted by laws and regulations.

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Example 1: This invention provides a gait analysis method for children's rehabilitation, such as... Figure 1 As shown, the method includes the following steps:

[0052] S1. Analyze the data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking, and obtain multiple stages of each leg's gait cycle, as well as the start time of each stage; at the same time, collect videos of both sides of the child during walking, and find multiple frames corresponding to the stage start from each side's video.

[0053] Compared to sensors strapped to the ankle, plantar pressure sensors, which collect pressure information from the sole of the foot (e.g., plantar pressure insoles), are more readily accepted by children and have less impact on their walking. This invention uses plantar pressure sensors to acquire pressure information from a child's foot, thereby obtaining each stage of the gait cycle and the start time of each stage.

[0054] The gait cycle refers to the entire process from the moment a foot makes contact with the ground until the next time the same foot makes contact with the ground. A gait cycle is divided into multiple phases, such as initial contact, flat foot phase, mid-stance, and heel lift. During each phase, the joints in the body undergo changes, such as the knee, wrist, ankle, elbow, and hip joints. Analyzing these joints can help determine changes in a child's gait, which is of great significance for assessing rehabilitation training.

[0055] The insole contains numerous pressure sensors that continuously collect pressure data. Analysis of this data reveals various stages of the gait. In one specific embodiment, the analysis of data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking specifically involves:

[0056] A sequence of typical plantar pressure distribution maps for each stage of the left and right legs is pre-constructed;

[0057] Foot pressure distribution maps were pre-collected during the walking process of children undergoing different rehabilitation treatments. Figure 2 A plantar pressure distribution map is shown. During walking, pressure sensors continuously collect pressure data. For example, if the pressure sensor collects data at a cycle of 0.2 seconds, multiple plantar pressure distribution maps will be obtained for each child. These maps are arranged in chronological order to form a sequence of plantar pressure distribution maps to be matched.

[0058] The pressure distribution map sequence is segmented to obtain a subsequence corresponding to each stage. When there are multiple children, one stage corresponds to multiple subsequences. For example, if there are 10 children, the flat foot stage will correspond to 10 subsequences. Multiple subsequences are used for each stage because different diseases cause children to walk in different postures or ways.

[0059] When determining the stage of the child to be analyzed, it is preferable to determine the typical plantar pressure distribution map sequence based on the condition. For example, there are 10 sub-sequences for the flat stage of the foot, and epilepsy corresponds to 3 of them. In the analysis of children with epilepsy, it is preferable to use these three sub-sequences. These three sub-sequences are the typical plantar pressure distribution map sequences.

[0060] Using a sliding window approach, a subsequence of plantar pressure distribution map that matches the typical plantar pressure distribution map sequence with a matching degree greater than a threshold is found in the sequence of plantar pressure distribution maps to be matched, and the starting point of the subsequence is marked as the corresponding stage starting point;

[0061] After acquiring the plantar pressure distribution (PPD) sequence of the child to be analyzed, a sliding window approach is used to find the PPD subsequence that best matches each stage. That is, a typical PPD sequence is slid across the PPD sequence of the child to be analyzed to find the PPD subsequence that best matches the typical PPD sequence. Since the typical PPD sequence corresponds to a stage, the best-matching PPD subsequence corresponds to that stage. It is important to note that the above process only finds the corresponding stage within a single gait cycle. When there are multiple gait cycles, this method can find the stages contained in each gait cycle. There are several ways to determine the gait cycle, such as based on time. A gait cycle has a preset time, and the PPD sequence to be matched is divided into multiple sequences based on this preset time. Each sequence is then matched using a matching degree greater than a threshold. In a more detailed embodiment, if determining the gait cycle of a leg requires simultaneously matching the plantar pressure distribution map sequences corresponding to two legs using the above matching method, that is, if the matching degree of the plantar pressure distribution map sequence to be matched for the first leg and the first typical plantar pressure distribution map sequence is greater than a threshold, and the matching degree of the plantar pressure distribution map sequence to be matched for the other leg and the first typical plantar pressure distribution map sequence is greater than a threshold, and the time difference between the two is less than a preset value, then it is considered that the current gait cycle is in a certain stage.

[0062] In a preferred embodiment, the matching calculation method is to calculate the similarity of each distribution map pair in the two sequences, and then accumulate all the similarities in the sequences; the higher the similarity, the better the match. The distribution map pair includes two distribution maps, one located in a typical plantar pressure distribution map sequence and the other located in the plantar pressure distribution map sequence to be matched.

[0063] This allows us to obtain the gait cycles of the left and right legs.

[0064] A complete gait cycle refers to the entire process from the moment a foot touches the ground until the next time the same foot touches the ground. Since a single-sided camera would obscure the other side, this invention uses dual-sided cameras for filming and extracts the gait cycles of the left and right legs separately.

[0065] S2, convert multiple frames corresponding to the starting point of each stage into multiple grayscale images, obtain the position of each joint on each side based on the multiple grayscale images corresponding to each side video and the joint recognition model, process the positions of the joints recognized from both sides to obtain the position of each joint; obtain the position sequence of each joint in each gait cycle according to the order of the stages in the gait cycle;

[0066] The starting point of each stage is a specific moment. Centered on this moment, several frames are extracted from the left and right sides of the starting point to obtain multiple frames corresponding to each stage's starting point. Using dual cameras will yield two frame sequences; for example, the flat-foot phase of a gait cycle corresponds to two frame sequences. These two frame sequences are converted to grayscale images. Then, based on the multiple grayscale images corresponding to each side of the video and the joint recognition model, the position of each joint on each side is obtained. For example, if there are three joints to analyze, the positions of one, two, or three joints are identified from the frame sequence captured by the left camera, and the positions of one, two, or three joints are identified from the frame sequence captured by the right camera. The position of each joint is then obtained from the positions of the joints identified on both sides, thus obtaining the position of each joint at, for example, the starting moment of the flat-foot phase. The position of each joint can also be obtained for the next stage, such as the starting moment of the mid-stability phase, thus obtaining the position sequence of each joint. For example, if we are analyzing three joints A, B, and C, and the gait cycle is divided into 6 phases, then there are 6 positions for joint A, 6 positions for joint B, and 6 positions for joint C. This forms the position sequence corresponding to each joint in each gait cycle.

[0067] When a stage starts with multiple frames, the accuracy of joint recognition can be improved, but the complexity also increases. In order to obtain the spatial and temporal information of joints in multiple frames, in a preferred embodiment, the position of each joint on each side is obtained based on the multiple grayscale images corresponding to each side of the video and the joint recognition model, specifically as follows:

[0068] The feature map of each grayscale image is obtained by two-dimensional convolution. The feature map corresponding to the grayscale image is added to the first feature map sequence according to the time of the grayscale image in the video. The second feature map sequence of the multiple grayscale image sequences is obtained by three-dimensional convolution.

[0069] The first feature map sequence is divided into multiple sub-sequences according to the depth of the convolution kernel of the three-dimensional convolution, and the sub-sequences are inserted into the second feature map sequence in sequence to obtain the feature map fusion result;

[0070] 3D convolution can obtain not only spatial information but also temporal information. Fusing the features of 3D and 2D convolution helps to extract more information. The stride in the depth of the 3D convolution is the depth of the 3D convolution kernel. For example, if a grayscale image has 12 features, and the 3D convolution kernel is 3×3×3 with a stride of 3, then grayscale images 1, 2, and 3 will generate the first feature map through the 3D convolution kernel; grayscale images 4, 5, and 6 will generate the second feature map; grayscale images 7, 8, and 9 will generate the third feature map; and grayscale images 10, 11, and 12 will generate the fourth feature map. These four feature maps constitute the second feature map sequence.

[0071] Then, the first feature map sequence is divided into 4 groups. The first group is inserted after the first feature map, the second group is inserted after the second feature map, and so on. Figure 3 As shown.

[0072] Preferably, the length and width of the convolution kernel of the two-dimensional convolution are the same as the length and width of the convolution kernel of the three-dimensional convolution, and the stride in length and width is the same, which can ensure that the feature map of the two-dimensional convolution and the feature map of the three-dimensional convolution are the same size.

[0073] The feature map fusion result is input into the corresponding joint recognition model to identify the position coordinates of the joint.

[0074] If the grayscale image comes from the left camera, the feature map fusion result is input into the left joint recognition model; otherwise, it is input into the right joint recognition model to obtain the key position coordinates. The joint recognition model can be any existing joint recognition model or instance detection model; there is no specific limitation. In another alternative embodiment, the videos captured by both cameras use the same joint recognition model to identify joints and their positions.

[0075] Because the left and right cameras capture different content, in a more detailed embodiment, the training of the joint recognition model includes two phases, specifically:

[0076] In the first stage, the joint recognition model is trained using the multiple grayscale images corresponding to each side of the video.

[0077] In the second stage, for the left joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the left video to obtain the left joint recognition model; for the right joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the right video to obtain the right joint recognition model.

[0078] Some joints can be identified from the videos on both sides, while others cannot be identified due to occlusion. In a more specific embodiment, the position of each joint is obtained by processing the positions of the joints identified from both sides, specifically as follows:

[0079] Determine whether the joint set identified by the left joint recognition model and the joint set identified by the right joint recognition model have the same joint. If they do, take the average of the two positions as the position of the same joint.

[0080] Take the intersection of the joint set identified by the left key recognition model and the joint set identified by the right key recognition model. If the intersection is not empty, that is, the joint is identified in both models, then take the average of the two as the position of the same joint.

[0081] Specifically, the position sequence corresponding to each joint in each gait cycle is obtained. In a specific embodiment, the position sequence corresponding to each joint point in each gait cycle of each leg is obtained. For example, analyzing the gait cycle of the left leg yields the position sequence corresponding to each joint point in each gait cycle of the left leg, and analyzing the gait cycle of the right leg yields the position sequence corresponding to each joint point in each gait cycle of the right leg.

[0082] S3 processes multiple position sequences corresponding to multiple gait cycles to obtain analysis results.

[0083] A gait cycle consists of multiple joints, each joint corresponding to multiple positions, and each position corresponding to a phase. For example, if we are analyzing three joints A, B, and C, and the gait cycle is divided into 6 phases, then there are 6 positions for joint A, 6 positions for joint B, and 6 positions for joint C. This constitutes the position sequence corresponding to each joint in each gait cycle. The data corresponding to such a gait cycle is ([p11, p12, p13, p14, p15, p16], [p21, p22, p23, p24, p25, p26], [p31, p32, p33, p34, p35, p36], [p41, p42, p43, p44, p45, p46]), where p11 represents the position of the first phase of the first joint, and p21 represents the position of the first phase of the second joint.

[0084] If there are five gait cycles, then five of the above sets will be obtained.

[0085] In one specific embodiment, the process of processing multiple position sequences corresponding to multiple gait cycles to obtain analysis results specifically includes:

[0086] The position sequence of multiple gait cycles is fitted according to joints and stages to obtain the final position of each joint in one gait cycle;

[0087] In a gait cycle, a joint has a position sequence, where the position sequence is the sequence of positions of the joint within a gait cycle. When there are multiple joints, a gait cycle includes multiple position sequences for each joint, and when there are multiple gait cycles, each joint corresponds to multiple position sequences. Fitting the position sequences of the same joint in different gait cycles yields the position of each joint in a gait cycle. Fitting refers to fitting the positions of the same joint in the same phase of different gait cycles. There are various fitting methods, including but not limited to averaging.

[0088] The position of each fitted joint in one gait cycle is compared with the position of each fitted joint in one gait cycle in the previous fitting, and the comparison results are plotted on a graph.

[0089] The position of each joint in the previous fitting in a gait cycle refers to the joint position in a gait cycle obtained in the same way in the previous rehabilitation training or before the rehabilitation training. As those skilled in the art will obviously know, it is a comparison of the same joints, and then the comparison results are plotted. The comparison results include, but are not limited to, changes in joint movement distance or elevation distance.

[0090] In another embodiment, the process of processing multiple position sequences corresponding to multiple gait cycles to obtain the analysis results specifically includes:

[0091] When analyzing the joints corresponding to the gait cycle of the left leg, the position of the joint in the same stage of different gait cycles of the left leg is averaged to obtain the position of the joint in that stage, and then the position sequence of the same joint in one gait cycle is obtained. The value of the position sequence is the position obtained by averaging. The position sequence corresponding to each joint is compared with the position sequence corresponding to the same joint in the gait cycle of the left leg of a normal child.

[0092] Table 1 shows the position of the same joint at different stages of different gait cycles when analyzing the gait cycle of the left leg. The fourth row shows the average of the positions of this joint at the same stage of different gait cycles. It should be noted that Table 1 is only an illustration for a single joint. When there are multiple joints, multiple results as shown in Table 1 will be obtained.

[0093] Table 1

[0094] Phase 1 Phase 2 Phase 3 Phase 4 Gait cycle 1 1,1 1,2 2,2 2,3 Gait cycle 2 1,2 2，2 2,2 2.4 Gait cycle 1,1.5 1.5,2 2.2 2,3.5

[0095] Similarly, when analyzing the joints corresponding to the gait cycle of the right leg, the position of the joint in the same stage of different gait cycles of the left leg is averaged to obtain the position of the joint in that stage, thereby obtaining the position sequence of the same joint in a gait cycle. The value of the position sequence is the position obtained by averaging. The position sequence corresponding to each joint is compared with the position sequence corresponding to the same joint in the gait cycle of the right leg of a normal child.

[0096] In another aspect, the present invention also provides a gait analysis system for children's rehabilitation, the system comprising the following modules:

[0097] The gait phase acquisition module is used to analyze data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking, and to obtain multiple phases of each leg's gait cycle, as well as the start time of each phase; at the same time, it collects videos of both sides of the child's walking process and finds multiple frames corresponding to the start of each phase from the videos of each side.

[0098] The joint position acquisition module is used to convert multiple frames corresponding to the starting point of each stage into multiple grayscale images, obtain the position of each joint on each side based on the multiple grayscale images corresponding to each side video and the joint recognition model, process the positions of the joints recognized from both sides to obtain the position of each joint, obtain the position sequence of each joint in each gait cycle according to the order of the stages in the gait cycle, and then obtain the position sequence corresponding to each joint in each gait cycle.

[0099] The analysis module is used to process multiple position sequences corresponding to multiple gait cycles to obtain analysis results.

[0100] Preferably, the step of analyzing data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking specifically involves:

[0101] A sequence of typical plantar pressure distribution maps for each stage of the left and right legs is pre-constructed;

[0102] Data collected from multiple plantar pressure sensors is converted into plantar pressure distribution maps to be matched. These maps are arranged in chronological order to form a sequence of plantar pressure distribution maps to be matched. A sliding window method is used to find plantar pressure distribution map subsequences in the sequence of plantar pressure distribution maps to be matched that have a matching degree greater than a threshold with the typical plantar pressure distribution map sequence. The starting point of the subsequence is marked as the corresponding stage starting point.

[0103] This allows us to obtain the gait cycles of the left and right legs.

[0104] Preferably, obtaining the position of each joint on each side based on the plurality of grayscale images corresponding to each side's video and the joint recognition model specifically involves:

[0105] The feature map of each grayscale image is obtained by two-dimensional convolution. The feature map corresponding to the grayscale image is added to the first feature map sequence according to the time of the grayscale image in the video. The second feature map sequence of the multiple grayscale image sequences is obtained by three-dimensional convolution.

[0106] The first feature map sequence is divided into multiple sub-sequences according to the depth of the convolution kernel of the three-dimensional convolution, and the sub-sequences are inserted into the second feature map sequence in sequence to obtain the feature map fusion result;

[0107] The feature map fusion result is input into the corresponding joint recognition model to identify the position coordinates of the joint.

[0108] Preferably, the training of the joint recognition model includes two stages, specifically:

[0109] In the first stage, the joint recognition model is trained using the multiple grayscale images corresponding to each side of the video.

[0110] In the second stage, for the left joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the left video to obtain the left joint recognition model; for the right joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the right video to obtain the right joint recognition model.

[0111] Preferably, the step of processing the positions of the joints identified from both sides to obtain the position of each joint specifically involves:

[0112] Determine whether the joint set identified by the left joint recognition model and the joint set identified by the right joint recognition model have the same joint. If they do, take the average of the two positions as the position of the same joint.

[0113] Preferably, the step of processing multiple position sequences corresponding to multiple gait cycles to obtain the analysis results specifically involves:

[0114] The position sequence of multiple gait cycles is fitted according to joints and stages to obtain the final position of each joint in one gait cycle;

[0115] The position of each fitted joint in one gait cycle is compared with the position of each fitted joint in one gait cycle in the previous fitting, and the comparison results are plotted on a graph.

[0116] In addition, the present invention also provides a computer program product that implements the method described above when executed by a processor.

[0117] Finally, the present invention also provides a computer-readable storage medium on which a computer program is stored, the computer program implementing the method described above when executed by a processor.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or in terms of their contribution to the prior art, can be embodied in the form of a computer product. This invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Other embodiments may also be used. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gait analysis method for children's rehabilitation, characterized in that, The method includes the following steps: Data collected by multiple plantar pressure sensors is analyzed to obtain the gait cycle of the left and right legs during walking, and to obtain multiple stages of each leg's gait cycle, as well as the start time of each stage; at the same time, videos of both sides of the child's walking process are collected, and multiple frames corresponding to the stage start point are found from each side's video. The multiple frames corresponding to the starting point of each stage are converted into multiple grayscale images. The position of each joint on each side is obtained based on the multiple grayscale images corresponding to each side video and the joint recognition model. The positions of the joints recognized from both sides are processed to obtain the position of each joint. The position sequence of each joint in each gait cycle is obtained according to the order of the stages in the gait cycle. The analysis results are obtained by processing multiple position sequences corresponding to multiple gait cycles; The method for obtaining the gait cycle of the left and right legs during walking specifically involves: pre-constructing a sequence of typical plantar pressure distribution maps for each stage of the left and right legs; converting data collected from multiple plantar pressure sensors into plantar pressure distribution maps to be matched, constructing a sequence of plantar pressure distribution maps to be matched in chronological order, and using a sliding window approach to find plantar pressure distribution map sub-sequences in the sequence of plantar pressure distribution maps to be matched that have a matching degree greater than a threshold with the typical plantar pressure distribution map sequence, marking the starting point of the sub-sequence as the starting point of the corresponding segment; thereby obtaining the gait cycle of the left and right legs. The specific steps for obtaining the position of each joint on each side are as follows: A feature map of each grayscale image is obtained using two-dimensional convolution; the feature map corresponding to the grayscale image is added to a first feature map sequence according to the time in the video; a second feature map sequence of multiple grayscale image sequences is obtained using three-dimensional convolution; the first feature map sequence is divided into multiple sub-sequences according to the depth of the convolution kernel of the three-dimensional convolution; the sub-sequences are inserted sequentially into the second feature map sequence to obtain a feature map fusion result; the feature map fusion result is input into the corresponding joint recognition model to identify the position coordinates of the joint. The process of processing the positions of the joints identified from both sides to obtain the position of each joint is as follows: determine whether there are any joints in the joint set identified by the left joint recognition model and the joint set identified by the right joint recognition model. If so, take the average of the two positions as the position of the joints in the same way. The analysis results are obtained as follows: multiple position sequences of multiple gait cycles are fitted according to joints and stages to obtain the final position of each joint in a gait cycle; the fitted position of each joint in a gait cycle is compared with the position of each joint in a gait cycle of the previous fitting, and the comparison results are plotted on the graph.

2. The method as described in claim 1, characterized in that, The training of the joint recognition model includes two stages, specifically: In the first stage, the joint recognition model is trained using the multiple grayscale images corresponding to each side of the video. In the second stage, for the left joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the left video to obtain the left joint recognition model; for the right joint recognition model, the joint recognition model is retrained using the multiple grayscale images corresponding to the right video to obtain the right joint recognition model.

3. A gait analysis system for children's rehabilitation, characterized in that, The system includes the following modules: The gait phase acquisition module is used to analyze the data collected by multiple plantar pressure sensors to obtain the gait cycle of the left and right legs during walking, and to obtain multiple phases of each leg's gait cycle, as well as the start time of each phase. Simultaneously, videos of the child walking are collected from both sides, and multiple frames corresponding to the starting point of each stage are found from each side video. The joint position acquisition module is used to convert multiple frames corresponding to each stage start point into multiple grayscale images, obtain the position of each joint on each side based on the multiple grayscale images corresponding to each side video and the joint recognition model, and process the positions of the joints recognized from both sides to obtain the position of each joint. The position sequence of each joint within each gait cycle is obtained according to the order of the stages in the gait cycle; The analysis module is used to process multiple position sequences corresponding to multiple gait cycles to obtain analysis results; The method for obtaining the gait cycle of the left and right legs during walking specifically involves: pre-constructing a sequence of typical plantar pressure distribution maps for each stage of the left and right legs; converting data collected from multiple plantar pressure sensors into plantar pressure distribution maps to be matched, constructing a sequence of plantar pressure distribution maps to be matched in chronological order, and using a sliding window approach to find plantar pressure distribution map sub-sequences in the sequence of plantar pressure distribution maps to be matched that have a matching degree greater than a threshold with the typical plantar pressure distribution map sequence, marking the starting point of the sub-sequence as the corresponding stage starting point; thereby obtaining the gait cycle of the left and right legs. The specific steps for obtaining the position of each joint on each side are as follows: A feature map of each grayscale image is obtained using two-dimensional convolution; the feature map corresponding to the grayscale image is added to a first feature map sequence according to the time in the video; a second feature map sequence of multiple grayscale image sequences is obtained using three-dimensional convolution; the first feature map sequence is divided into multiple sub-sequences according to the depth of the convolution kernel of the three-dimensional convolution; the sub-sequences are inserted sequentially into the second feature map sequence to obtain a feature map fusion result; the feature map fusion result is input into the corresponding joint recognition model to identify the position coordinates of the joint. The process of processing the positions of the joints identified from both sides to obtain the position of each joint is as follows: determine whether there are any joints in the joint set identified by the left joint recognition model and the joint set identified by the right joint recognition model. If so, take the average of the two positions as the position of the joints in the same way. The analysis results are obtained as follows: multiple position sequences of multiple gait cycles are fitted according to joints and stages to obtain the final position of each joint in a gait cycle; the fitted position of each joint in a gait cycle is compared with the position of each joint in a gait cycle of the previous fitting, and the comparison results are plotted on the graph.

4. A computer program product, characterized in that, The computer program product, when executed by a processor, implements the method as described in any one of claims 1-2.

Images