A non-contact gait cycle feature analysis system, method and application thereof

Abstract

The present invention discloses a contactless gait cycle feature analysis system, method and application thereof in screening of children with autism, including a contactless gait acquisition device and a feature analysis subsystem, wherein the gait acquisition device includes an acquisition path and a camera, which is used to collect videos and foot pressure of the person to be tested along the acquisition path; the feature analysis subsystem includes a natural gait acquisition module, an adaptive gait cycle detection module, a gait cycle standardization module, a gait feature extraction module and an abnormality assessment module; it is used to extract skeleton data of human movement according to the collected video and pressure value, and calibrate the data to perform gait cycle recognition and segmentation, and obtain gait features related to autism after height standardization, and realize disease assessment based on a machine learning algorithm. The present invention is simple to operate during use, has little restriction on the site, and is fast to collect, and can effectively assess the probability of autism through gait, and has important clinical application prospects.

Description

Technical Field

[0001] The present invention relates to the field of image processing and intelligent medical technology, and in particular to a contactless gait cycle feature analysis system and method and application thereof in autism screening. Background Art

[0002] Gait is the pattern of movement of various parts of the body when a person walks. It can reflect a person's physical development, emotional state, nervous system function, and muscle activity. Gait analysis is of great significance in the fields of medicine, rehabilitation, and sports science. However, most of the current general gait analysis systems rely on wearable devices, which will affect the natural gait acquisition of the test subject. A few non-contact analysis systems have strict requirements on the gait acquisition process and require manual annotation to capture useful segments. It is difficult to automatically identify the gait cycle from the natural gait video, resulting in a relatively single feature acquisition and limited information. Therefore, a non-contact adaptive gait cycle feature analysis method is desired.

[0003] Autism spectrum disorder is a neurodevelopmental disease characterized by social interaction and communication disorders, as well as restrictive, repetitive interests and behavior patterns. Clinically, it is found that the gait pattern of autistic children is atypical, manifested by unstable walking and uncoordinated posture. This is to a certain extent related to the special development of their nervous system, which leads to difficulties in motor control and coordination, further affecting the patient's daily life and social interactions. Therefore, specific gait patterns are an important clue for identifying autism. However, gait characteristics are often ignored because they are not easy to detect in the early stages, and there is currently no effective gait feature extraction and recognition method for autistic children, which makes gait analysis difficult to apply in the field of autism screening. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to propose a contactless gait cycle feature analysis system and method and its application in screening of children with autism, so as to solve the limitations of existing gait analysis methods and application scenarios.

[0005] The first aspect of the present invention is to disclose a contactless gait cycle feature analysis system, comprising a contactless gait acquisition device and a feature analysis subsystem, wherein the gait acquisition device and the feature analysis subsystem are connected via WIFI; the contactless gait acquisition device comprises a collection path and a camera, wherein positioning points are respectively arranged at both ends of the collection path for the person to be tested to determine the round-trip position; the camera is used to collect a video of the person to be tested going back and forth along the collection path, and a wireless transmission device 6 is arranged on the camera for connecting to the feature analysis subsystem via WIFI;

[0006] The feature analysis subsystem includes a natural gait acquisition module, an adaptive gait cycle detection module, a gait cycle standardization module, a gait feature extraction module and an abnormality assessment module;

[0007] The natural gait collection module uses the contactless gait collection device to collect the natural lateral walking gait data of the person to be tested contactlessly throughout the whole process;

[0008] The adaptive gait cycle detection module is used to extract and automatically calibrate the skeleton sequence from the gait data collected by the natural gait collection module, and further detect and segment the complete gait cycle;

[0009] The gait cycle standardization module is used to perform noise reduction and centralization using a gait cycle standardization algorithm according to the same-direction gait cycle obtained by the adaptive gait cycle detection module, and to perform height standardization processing by integrating height difference factors, so as to obtain a height-standardized gait cycle joint point sequence integrating height factors;

[0010] The gait feature extraction module is used to extract gait features related to autism based on the height-standardized gait cycle joint point sequence obtained by the gait cycle standardization module, wherein the gait features include the duration, step length, step speed, joint angle, center of gravity stability and contralateral coordination of several gait cycles of the subject to be tested;

[0011] The abnormality assessment module is used to extract autism-related gait features based on the gait features of several gait cycles of the subject to be tested obtained by the gait feature extraction module.

[0012] Furthermore, the collection trail includes a pressure sensor layer and an anti-skid layer from bottom to top, the pressure sensor layer includes a plurality of pressure sensors arranged in a dot matrix, and the total area of ​​the pressure sensors covers ≥95% of the collection trail, and is used for real-time feedback of pressure information on the surface of the trail, the anti-skid layer is located on the top of the pressure sensor layer, and positioning points are provided on the upper surface, the positioning points are located at both ends of the anti-skid layer, and the color of the positioning points is sufficiently distinguishable from the surface color of the anti-skid layer, so as to facilitate the analysis of the start and end point positioning; and the collection trail is also provided with a wireless transmission device, and the collection trail and the camera transmit the collected data to the feature analysis subsystem in a WIFI manner through the wireless transmission device; and the collection trail and the camera realize the time synchronization of the collected data of the camera and the pressure sensor in a Bluetooth manner through the wireless transmission device, and serve as a switch for simultaneous start / end of collection, so that the collection of the two starts and ends at the same time, so that the two can collect data corresponding to each other in time;

[0013] Moreover, the gait feature in the gait feature extraction module also includes foot pressure.

[0014] Optionally, the collection trail is only provided with an anti-slip layer, and positioning points and nameplates are provided on the upper surface of the anti-slip layer, the positioning points are located at both ends of the anti-slip layer, and the colors of the positioning points are sufficient to be distinguished from the surface color of the anti-slip layer, so as to facilitate the analysis of the starting and ending point positioning; the nameplate is printed with information on the length, width, and distance between the collection trails; and the gait features in the gait feature extraction module include the duration, step length, step speed, joint angle, center of gravity stability and contralateral coordination of several gait cycles of the object to be tested.

[0015] Furthermore, the camera is 1.5-4 m away from the collection trail, and the camera viewing angle is adjusted to ensure that the edge of the trail can be fully captured.

[0016] Furthermore, the automatic calibration in the adaptive gait cycle detection module refers to automatically calibrating the skeleton features by utilizing the perspective and rotation principles in image processing according to the pixel length occupied by the positioning point and the angle with the horizontal line of the picture.

[0017] A second aspect of the present invention is to disclose a contactless gait cycle feature analysis method, comprising:

[0018] Step 1: Use a camera to collect multiple videos of the person to be tested going back and forth between the positioning points at both ends of the collection trail, wherein the video includes multiple gait cycles; use the pressure sensor of the pressure sensor layer to collect the pressure value of the trail;

[0019] Step 2: Use motion detection and tracking technology based on computer vision and signal processing technology to extract skeleton data during human movement; use the principles of perspective and distortion in image processing to automatically calibrate skeleton features; perform gait cycle recognition and segmentation based on the calibrated skeleton features, exclude the gait cycle of turning at the corner from the gait cycle in step 1, and obtain a complete gait cycle in the same direction;

[0020] Step 3: Use the gait cycle standardization algorithm to calculate the complete same-direction gait cycle obtained in step 2 to obtain the height-standardized gait cycle;

[0021] Step 4: According to the multiple standard gait cycles of the subject to be tested obtained in step 3, each gait cycle is calculated to obtain gait characteristics, wherein the gait characteristics include duration, foot pressure, step length, step speed, joint angle, center of gravity stability, contralateral coordination and other characteristics;

[0022] Step 5: The gait features obtained in step 4 are combined with the height features and input into the pre-trained machine learning model for probability prediction, extracting autism-related gait features to achieve disease assessment.

[0023] Furthermore, the gait cycle normalization algorithm in step 3 includes:

[0024] Traverse the horizontal and vertical coordinates of all joint points to calculate the Z score, where the Z score refers to the difference between a single data point and the overall data mean, identify the frame at that moment corresponding to the joint point whose Z score exceeds a preset threshold as a singular frame, and remove the singular frame;

[0025] Perform Gaussian smoothing on all relevant nodes;

[0026] The processed joint points are centered, and the gait is decomposed into the movement of the center of gravity of the human body and the movement of the limbs. The center of gravity of the human body is calculated as the coordinate origin, and the height feature is normalized to convert the height of all the images into a uniform level, thereby obtaining a standardized gait cycle joint point sequence.

[0027] Furthermore, the center of gravity of the human body is calculated based on the calculated motion sequence of the center of gravity of the skeleton in each gait cycle combined with the weight distribution of the human body.

[0028] Furthermore, the acquisition of gait features in step 4 includes:

[0029] The duration feature is the time corresponding to the gait cycle;

[0030] The foot pressure feature is the average value of the pressure change of the footsteps on the collection path during the gait cycle, and the pressure value comes from the data collected by the pressure sensor of the pressure sensor layer of the collection path;

[0031] The step length feature refers to the average length of each step taken during the gait cycle, specifically the average value of the maximum lateral distance of the horizontal coordinate movement of the left heel and the right heel calculated in combination with the joint position;

[0032] The gait speed feature is the speed of progress during the gait cycle, which is calculated based on the ratio of step length to cycle duration;

[0033] Joint angle characteristics include the maximum range of motion of the left and right upper arms, the maximum range of motion of the thighs, and the head tilt angle;

[0034] The maximum range of left and right upper arm movement refers to the average of the maximum angles between the left and right elbows and the left and right shoulders during the gait cycle; the maximum range of thigh movement refers to the average of the maximum angles between the left and right knees and the left and right hips during the gait cycle; the head inclination angle refers to the maximum value of the angle between the line connecting the nose and the neck and the horizontal plane during the gait cycle;

[0035] The head tilt feature is the maximum value of the angle between the line connecting the nose and neck and the horizontal plane during the gait cycle;

[0036] The center of gravity stability characteristic is the standard deviation of the body's center of gravity in the y-axis direction during the gait cycle;

[0037] The contralateral synergy feature refers to the average value of the correlation of the horizontal coordinate displacement of the contralateral hand and foot, where the correlation is obtained using the Pearson correlation coefficient function.

[0038] The third aspect of the present invention is to disclose an application of a contactless gait cycle characteristic analysis system in autism screening.

[0039] A fourth aspect of the present invention is to disclose a contactless gait cycle feature analysis device, the analysis device comprising a processor, and the processor is used to process the contactless gait cycle feature analysis method.

[0040] The beneficial effects of the present invention are:

[0041] 1. The present invention proposes a contactless gait collection device. Compared with the existing contact gait collection equipment, the present invention does not require wearing sensors, thus avoiding the interference of wearable devices on user activities. Compared with the existing contactless devices, the present invention only requires a monocular camera and a trail device to complete the collection. In addition, through special positioning points, trail sensors and adaptive calibration algorithms, there is no need to manually position the camera angle and measure the height separately before gait data collection, thus solving the gait analysis problems caused by changes in shooting angles and individual differences in height development among children.

[0042] 2. The present invention provides a set of non-contact gait cycle feature analysis algorithms. Based on the above device, the method automatically calibrates from the collected natural gait data and adaptively extracts a complete standardized gait cycle, and further extracts the multi-dimensional gait features of the subject such as foot pressure, pace, step length, joint angle, contralateral synergy based on gait cycle analysis, and implements disease assessment based on machine learning algorithms to solve the subjective and rough limitations of existing behavioral observation methods.

[0043] 3. The present invention provides the application of the above method in autism screening, which is simple to operate, has little restriction on the site, and is fast to collect. It can effectively analyze the development of children and their coordinated motor ability through gait, and then evaluate the probability of autism. Therefore, it has important clinical application prospects and is expected to bring substantial benefits to autistic patients and their families. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the structure of the contactless gait acquisition device provided in Example 1;

[0045] Figure 2 This is a flow chart of the feature analysis subsystem described in Example 1 of the present invention;

[0046] Figure 3 Schematic diagram of skeleton feature point distribution and gait cycle involved in Example 1 of the present invention; wherein, Figure 3A shows the whole body skeleton drawn on the left, and the joint points corresponding to each coordinate point on the right; Figure 3 B shows the Figure 3 A visualizes the time-varying sequence produced by Figure 3 C is a schematic diagram after automatic calibration of skeleton features;

[0047] Figure 4 This is a visualization diagram of gait feature extraction involved in step 4 of Example 1 of the present invention.

[0048] in:

[0049] 1: Pressure sensor layer 2: Anti-slip layer 3: Logo positioning layer

[0050] 4: Nameplate 5: Camera 6: Wireless transmission device DETAILED DESCRIPTION

[0051] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The described specific embodiments are only used to explain the present invention and are not intended to limit the present invention.

[0052] Example 1

[0053] A contactless gait cycle feature analysis system, which is used for autism screening, includes a contactless gait acquisition device and a feature analysis subsystem, wherein the gait acquisition device and the feature analysis subsystem are connected via WIFI. The feature analysis subsystem includes a natural gait acquisition module, an adaptive gait cycle detection module, a gait cycle standardization module, a gait feature extraction module, and an abnormality assessment module. Each module is coupled in a front-to-back order to realize functions in sequence.

[0054] like Figure 1 As shown, the contactless gait collection device includes a collection path and a camera, and the collection path and the camera are both provided with a wireless transmission device 6, and the collection path and the camera are connected by Bluetooth through the wireless transmission device 6, as a switch for simultaneous start / end of collection, so that the collection of the two starts and ends at the same time, so that the two can collect data corresponding to each other in time. Moreover, the collection path and the camera transmit the collected data to the feature analysis subsystem by WIFI through the wireless transmission device 6. The wireless transmission device 6 realizes the time synchronization of the collected data of the camera and the pressure sensor by Bluetooth, and transmits the collected data to the feature analysis subsystem by WIFI.

[0055] The collection trail is laid on the ground to be measured, with a width of about 50-70cm and a length of about 8-10m. It has three layers from bottom to top, namely: pressure sensor layer 1, anti-skid layer 2 and mark positioning layer 3. The pressure sensor layer 1 is a plurality of flat pressure sensors fixed on the base, and the pressure sensors are arranged side by side in a dot matrix. The total area of ​​the pressure sensors has a coverage rate of ≥95% on the collection trail, a range of ≥70Kg, a thickness of ≤6mm, and a response time of ≤50ms. It is used to record the pressure information on the surface of the trail in real time and save it as a time series in csv format. This embodiment uses the GML670 micro gravity sensor of Glus. The anti-skid layer 2 is made of rubber, with a rough surface, a thickness of ≤5mm, a surface color of yellow-green, and a 3-4cm white border around it. The anti-skid layer is pasted on the upper surface of the pressure sensor layer. The mark positioning layer 3 includes positioning points and nameplates 4, which are pasted on the upper surface of the anti-skid layer 2. The positioning points are located at both ends of the longitudinal direction of the anti-skid layer 2, and are displayed in different patterns respectively. like Figure 1 As shown, the pattern of the positioning points includes a triangular and a square pattern located at both ends of the anti-slip layer 2. The area of ​​the triangular and square patterns is about 400-900 cm 2 , the color is black, which is used for subsequent analysis of the starting and ending point positioning. The nameplate 4 displays the device specification information, including the length and width of the collection trail, and the distance between the positioning points. The camera 5 is a high-frame rate and high-resolution camera. The camera 5 is placed vertically relative to the collection trail and is 1.5-4m away from the collection trail. The sampling rate is ≥100Hz, the resolution is ≥1024×768 pixels, and the focal length is 10-20mm adjustable. This embodiment uses a Sony ZV-E10 camera and a Sony SELP1020G zoom lens. When in use, the gait collection device is placed in an open space indoors or outdoors with a flat ground and bright light. The camera angle of view of the camera 5 is adjusted to ensure that the edge of the trail can be fully captured, and the horizontal and pitch angles of the camera are centered as much as possible, without precision restrictions. The camera 5 needs to be equipped with a tripod 40-100cm from the ground with an adjustable height. A wireless transmission device 6 is also provided on the camera 5 and the pressure sensor layer 1. The wireless transmission device 6 realizes time synchronization of the collected data of the camera and the pressure sensor through Bluetooth, and transmits the collected data to the feature analysis subsystem through WIFI.

[0056] Figure 2 The structural flow chart of the feature analysis subsystem is shown, which includes a natural gait acquisition module, an adaptive gait cycle detection module, a gait cycle standardization module, a gait feature extraction module and an abnormality assessment module.

[0057] The natural gait collection module utilizes the gait collection device to collect the child's natural lateral walking gait data in a contactless manner throughout the entire process.

[0058] The adaptive gait cycle detection module is used to extract and automatically calibrate the skeleton sequence from the gait data collected by the natural gait collection module, and further detect and segment the complete gait cycle.

[0059] The gait cycle standardization module is used to perform noise reduction and centralization using a gait cycle standardization algorithm based on the same-direction gait cycle obtained by the adaptive gait cycle detection module, and to perform height standardization by integrating height difference factors to obtain a height-standardized gait cycle joint point sequence integrating height factors.

[0060] The gait feature extraction module is used to extract gait features related to autism based on the height-standardized gait cycle joint point sequence obtained by the gait cycle standardization module. The gait features include the duration of several gait cycles of the subject to be tested, foot pressure, step length, step speed, joint angle, center of gravity stability and contralateral coordination.

[0061] Among them, the duration feature is the time corresponding to the gait cycle, in seconds.

[0062] The foot pressure feature is the average value of the pressure change of the footsteps on the collection path during the gait cycle, and the pressure value comes from the data collected by the pressure sensor of the pressure sensor layer 1 of the collection path.

[0063] The step length feature is the average of the maximum lateral distance of the horizontal coordinate movement of the left and right heels during the gait cycle, calculated in combination with the joint positions. The gait speed feature is the speed of progress during the gait cycle, calculated based on the ratio of step length to cycle duration.

[0064] The joint angle characteristics include the maximum range of motion of the left and right upper arms, the maximum range of motion of the thighs and the head tilt angle.

[0065] The maximum amplitude of left and right upper arm movements refers to the average of the maximum angles between the forward and backward swinging of left and right elbows and the left and right shoulders during the gait cycle; the maximum amplitude of thigh movement refers to the average of the forward and backward swinging of left and right knees and the maximum angles between left and right hips during the gait cycle; the head inclination angle refers to the maximum value of the angle between the line connecting the nose and neck and the horizontal plane during the gait cycle.

[0066] The center of gravity stability characteristic is the standard deviation of the body's center of gravity in the y-axis direction during the gait cycle.

[0067] The contralateral synergy feature refers to the average value of the correlation of the horizontal coordinate displacement of the contralateral hand and foot, where the correlation is obtained using the Pearson correlation coefficient function.

[0068] The abnormality assessment module is used to extract autism-related gait features based on the gait features of several gait cycles of the subject to be tested obtained by the gait feature extraction module.

[0069] A non-contact gait cycle feature analysis method, comprising:

[0070] Step 1: Before use, the operator checks the integrity of the gait collection device 1, especially the positioning point, adjusts the camera distance and shooting angle, makes the camera 5 1.5-4m away from the collection trail, and adjusts the camera angle of the camera 5 to ensure that the edge of the trail can be fully captured;

[0071] After the test is correct, the operator guides the child to stand at the positioning point on one side of the collection trail (such as standing on Figure 1 The operator will inform the collection process, after the object to be measured is confirmed, irrelevant personnel will leave the scene, and the operator will turn on the camera and trail collection device to start recording, and prompt to start;

[0072] After the start command is issued, the subject to be tested will go back and forth 2-4 times between the two ends of the tracking point within the range of the collection trail. To ensure the quality of subsequent analysis, the subject must reach the tracking point on one side of the trail each time before turning around, and can walk naturally. At least 15 complete gait cycles are collected, and the data is saved in AVI format, with a sampling rate of 100Hz and a resolution of 1920×1080 pixels. Among them, the gait cycle refers to the process of completing a complete walking action, including the time from one foot landing to the same foot landing again. Here, the first step is the right foot. The measurement and analysis of the gait cycle can provide key information about individual walking movements, including gait stability, gait rhythm, and gait abnormalities, which is of great significance for evaluating individual motor ability and disease diagnosis. A complete gait cycle refers to one step for each left and right foot, and the turning points on both sides during the step are not counted in the gait cycle count. The actual collection process takes 3-6 minutes, which has the advantages of short time and simple tasks, and is very friendly to the actual scenario of large-scale screening of children.

[0073] After the subject to be tested completes the gait task, the operator ends the recording, and the gait video captured by the camera 5 and the sensor values ​​transmitted by the pressure sensor layer 1 are transmitted to the host computer via the wireless transmission device 6, and the collection is completed.

[0074] According to the existing clinical data collection experience, some autistic children appear unfamiliar and emotional. If this happens, the operator can ask the parents to cooperate appropriately and guide the children to do preparatory exercises, and complete the data collection after the children have adapted and their emotions have stabilized.

[0075] Step 2:

[0076] S201: The collected gait video is used to use the Pose LandmarkDetection posture estimation function provided by Google's open source MediaPipe library to extract the coordinates of 33 joints of the human body, including important joints such as hands, arms, upper body, hips, legs and feet. The coordinates of each joint point include horizontal and vertical coordinates, depth of field and confidence information. All joint points are drawn into a full body skeleton according to the actual topological structure, such as Figure 3 A. The time-varying sequence of the horizontal and vertical coordinates of the joint points extracted from each video is retained as a matrix of specification (N, 33, 2), where 33 is the number of joint points, 2 is the horizontal and vertical coordinates, and N is the total number of sampling points. The visualization result is as follows Figure 3 As shown in B.

[0077] S202: Automatically calibrate the skeleton features using the perspective and distortion principles in image processing. Since the camera 5 may have a horizontal offset during the shooting process, this causes the following Figure 3 The skeleton coordinate angle shown in B is offset, which causes the illusion that the person is not moving horizontally. In addition, due to the perspective of the camera when shooting, the scale of each data is different, and the person in the same data will be larger in the middle than at the edge, so the length and distance information cannot be directly determined from the image. Therefore, the positioning point of the gait acquisition device 1 is used to solve the above two key problems. First, from the original video image, the yellow-green color of the surface of the anti-slip layer 2 and its white edge are locked according to the RGB color value. Then, the target detection technology is used in this target area to lock the geometric area with the RGB color value of black (0,0,0), that is, to find the positioning points at both ends of the acquisition path. According to the pixel length occupied by the positioning point and the angle with the horizontal line of the picture, the perspective and rotation principles in image processing can be used to automatically calibrate the skeleton features. After calibration, Figure 3 C. To adapt to various application scenarios, the trail is allowed to have different sizes. Before calibration, the parameter a of the actual distance between the trail positioning points in the nameplate 4 needs to be checked. For the actual distance a between the positioning points in the trail nameplate 4, the number of pixels b between the positioning points in the picture, and the angle α between the endpoint connection line and the horizontal line, that is, the skeleton sequence x with the extracted specification of (N, 33, 2) and the skeleton sequence x′ after automatic calibration are:

[0078]

[0079] S203: Complete gait cycle recognition and segmentation from the calibrated skeleton sequence. A gait cycle refers to the process of completing a complete walking action. Since the gait video data collected in step 1 has a high degree of freedom and is not marked, the following algorithm is used to solve this problem. First, combined with Figure 3A calculates the position coordinates of the right foot center r(n) and the left foot center l(n), and calculates the distance change sequence d(n) between the centers of both feet on the x-axis. When d(n)>0, it means that the right foot is to the right of the left foot at time n, as shown in formula (3):

[0080]

[0081] Among them, mark i,x (n) represents the x-coordinate of the ith joint point in the nth frame. Then, find the extreme point z of the sequence d(n) i , the extreme point is the point where the absolute value distance between the left and right feet is the largest, representing the moment when both feet step out and touch the ground at the same time. However, a complete gait cycle includes two steps, which will contain three extreme points. Walking to the right will contain two minimum values ​​and one maximum value, while walking to the left will contain the opposite. Therefore, further screening is required. Here, firstly, according to formula (4), s(n) is used to determine the walking direction at time n. Among them, mark 0,x is the horizontal coordinate of the nose coordinate point, joint mark 11 and mark 12 The midpoint is the coordinate of the neck coordinate point, and sign is a sign operation. When the s(n) value is 1, it means the direction is to the right, and -1 means to the left.

[0082]

[0083] Here, for all extreme points z i Screening is performed. For time n, when s(n) is 1, the minimum point is retained, and when it is -1, the maximum point is retained. The extreme point after screening is z′ i , where i = 0, 1, ..., M. In practical terms, z′ i represents the moment when the right foot touches the ground, that is, the start or end point of the gait cycle, and M is the number of times the right foot touches the ground. Therefore, the gait cycle T i The frame number interval can be expressed as:

[0084] T i =[z′ i ,z′ i+1 ] (5)

[0085] Where i = 0, 1, ..., M-1. Since only the same-direction gait features need to be considered, gait cycles that contain turns at corners need to be excluded. Specifically, the walking directions s(n) in the gait cycle are judged one by one. If s(n) in i cycles have different signs, there is a turn in the cycle, and the i-th cycle is deleted. In addition, due to the problem of incorrect recognition of the left and right limbs of the skeleton and frame drops in MediaPipe, the cycles that do not contain 2 maximum values ​​and 1 minimum value or 2 minimum values ​​and 1 maximum value in the gait cycle are excluded. In addition, gait cycles with more than 30% frame drops are excluded, that is, for gait cycle T i , must satisfy formula (6):

[0086] z′ i+1 -z′ i ≥(1-30%)*fps*[frame2time(z′ i+1 )-frame2time(z′ i )] (6)

[0087] Among them, fps is the image sampling rate during acquisition, which is 100 Hz here, and frame2time(n) is the frame-to-time conversion function, which represents the time corresponding to the nth moment in seconds.

[0088] Finally, N complete gait cycles T (i=1, 2, ..., N) in the same direction are obtained through calculation.

[0089] Step 3: When drawing the complete same-direction gait cycle obtained in step 2, some of the markers will drift, and the person will shrink as he moves away from the center of the picture. Therefore, the gait cycle standardization algorithm is used to calculate the gait cycle obtained in step 2. The specific method is as follows:

[0090] Traverse the horizontal and vertical coordinates of all joint points to calculate the Z score, which refers to the difference between a single data point and the overall data mean. Identify the frame corresponding to the joint point whose Z score exceeds the threshold (set to 3 here) as a singular frame, and remove the singular frame. Usually, these values ​​are caused by the misalignment of skeleton recognition at a certain moment. Since the sampling rate of the present invention is relatively high, removing singular values ​​will not affect subsequent analysis. On the contrary, retaining them will cause abnormal fluctuations in local data. Afterwards, Gaussian smoothing is performed on all relevant nodes. This step is crucial to improving the quality of gait data because it can effectively reduce the noise introduced by measurement errors, environmental interference or other random factors, ensure the smoothness and continuity of key point data, and provide a more accurate and reliable data basis for subsequent gait feature extraction and analysis. For the position coordinate mark of the i-th joint point i,x and mark i,y , the Gaussian smoothing process is as follows:

[0091]

[0092] Among them, j is the radius of the Gaussian kernel, which determines the range of joint points used for smoothing. It is usually 1-5, and here it is 3. σ is the standard deviation of the Gaussian kernel, which determines the degree of weighting. The larger the standard deviation, the smaller the weight of the joint points farther away from the center point. It can be adjusted between 0.5 and 2, and here it is 1.

[0093] Then the centering operation is implemented. The role of centering is to decompose the gait into the movement of the center of gravity of the human body and the movement of the limbs, and then the gait features can be extracted separately. The specific method is to calculate the movement sequence g(n) of the center of gravity of the skeleton in each gait cycle. Combined with the weight distribution of the human body, the center of the hip is used as the center of gravity. The center of gravity position of the nth frame is calculated by the average of joint point 23 and joint point 24:

[0094]

[0095] Furthermore, the center of gravity g is used as the coordinate origin to calculate the relative positions of the 33 joint points, thereby achieving centralization:

[0096] mark′ i,x (n) = mark i,x (n)-g x (n) (10)

[0097] mark′ i,y (n) = mark i,y (n)-g y (n) (11)

[0098] Further eliminate the impact of height differences on gait features. For the same sample, due to the distortion of the camera perspective, the height of a person will shrink as he moves away from the center of the picture, and for different samples, there are large differences in height between individuals. This difference in height will affect gait features such as stride and interfere with the recognition results. For this reason, the height features are normalized here. The principle is to convert the height of all individuals to a uniform level. The specific approach is: First, calculate the sum of the distances of the head, upper body, and lower body joints in each frame to estimate the height, and use the length of the frame as the scaling factor α(n) corresponding to the nth frame:

[0099]

[0100] Among them, distance(mark i ,mark j ) represents the distance between the i-th and j-th joint points.

[0101] Afterwards, the coordinates of each joint point in all frames are scaled according to the scaling factor α(n):

[0102]

[0103] Thus, a standardized gait cycle joint point sequence is obtained.

[0104] Step 4: Based on the clinical research conducted in the hospital, some autism-specific gait characteristics have been found, including duration, foot pressure, step length, gait speed, joint angle, center of gravity stability, contralateral coordination, etc. Based on the several standard gait cycles of each subject obtained in step 3, feature extraction operations are performed on each gait cycle. Figure 4 As shown, the following is a detailed introduction to the gait feature extraction method used:

[0105] Duration feature: the duration feature f T is the time corresponding to the gait cycle, in seconds. The starting frame n1 and the ending frame n2 in this cycle are converted to the starting time t by the frame number time conversion function frame2time start and the end time t end , and further calculate the duration feature f T :

[0106] f T =t end -t start =frame2time(n2)-frame2time(n1)

[0107] Foot pressure: Foot pressure characteristics f stress is the average value of the change in the pressure of the footsteps on the trail during the gait cycle. From the pressure sensor data collected by the trail pressure sensor layer 1, find the time interval (t start ,t end ), calculate the absolute value of its gradient to represent the pressure change, and further calculate the average value as the foot pressure feature f stress , assuming (t start ,t end ) is M:

[0108]

[0109] Step length: step length characteristic f stride Refers to the average length of each step during the gait cycle. stride The horizontal coordinate marks of the left heel (joint point 29) and the right heel (joint point 30) 29,x and mark 30,x Average value of maximum horizontal distance moved:

[0110]

[0111] Pace: Pace characteristic f speed Refers to the forward speed during the gait cycle, in normalized distance per second. stride and cycle length f T , the pace f can be further calculated speed :

[0112]

[0113] Joint angle: The joint angle includes the maximum range of motion of the left and right upper arms. arm , the maximum range of thigh movement f leg and head inclination angle f head Among them, the maximum amplitude of the left and right upper arm movements is f arm Refers to the average of the maximum angles between the left and right elbows (joint points 13 and 14) and the left and right shoulders (joint points 11 and 12) during the gait cycle, in degrees. It can be calculated using the trigonometric formula:

[0114]

[0115] Maximum range of motion of the thigh leg Refers to the average of the maximum angles between the left and right knees (joint points 25 and 26) and the left and right hips (joint points 23 and 24) during the gait cycle, in degrees. It can be calculated using the trigonometric formula:

[0116]

[0117] Head tilt angle f haad Refers to the maximum value of the angle between the line connecting the nose (joint point 0) and the neck (midpoint of joint points 11 and 12) and the horizontal plane during the gait cycle, in degrees. It can be calculated using the trigonometric formula:

[0118]

[0119] Center of gravity stability: Center of gravity stability characteristics f stability is the standard deviation of the body center of gravity g(n) in the y-axis direction during the gait cycle. According to the center of gravity sequence g(n) obtained in the steps, the center of gravity stability characteristic f is calculated stability , for g(n) with N sampling points:

[0120]

[0121] Contralateral synergy: Contralateral synergy characteristicsf coordIt refers to the average value of the correlation of the horizontal coordinate displacement of the contralateral limbs (i.e., right wrist joint point 16 and left ankle joint point 27, left wrist joint point 15 and right ankle joint point 28). Here, the correlation is calculated using the Pearson correlation coefficient function:

[0122]

[0123] All autism-related gait features can be calculated based on the feature extraction algorithm. It is worth noting that the gait cycle sequence used here has been standardized for height in the previous module, and the obtained gait features have been integrated with the height factor. The algorithm has been tested and can effectively avoid the impact of children's height differences on gait data.

[0124] Step 5: The gait features obtained in step 4 are combined with the height features and input into the pre-trained machine learning model for probability prediction to extract autism-related gait features.

[0125] According to the duration, foot pressure, step length, step speed, upper arm amplitude, thigh amplitude, head inclination, center of gravity stability, and contralateral coordination characteristics of several gait cycles of the test subject extracted in step 4, the average of all height-standardized gait cycle characteristics is calculated, and then the height features are combined into 10-dimensional features, which are input into the pre-trained machine learning model for probability prediction. Here, Random Forest is used as the classifier, and the output is the probability value P of the test subject suffering from autism:

[0126] P = RandomForest(f T ,f stress ,f stride ,f arm ,f leg ,f head ,f speed ,f stability ,f coord ,fHeight) (15)

[0127] The model output probability value P ranges from [0,1]. The closer it is to 1, the higher the probability of illness. Random forest is an integrated learning technology that can perform classification, regression or other tasks by constructing multiple decision trees, and the output mode (classification or regression) depends on the output results of most decision trees. The random forest model performs well in processing multidimensional data, and is particularly suitable for the multidimensional features extracted from the gait cycle in this study. At present, this model has been effectively verified on a clinical data set of children with autism. Based on a large-scale sample of about 200 people, the recognition rate after 10-fold cross-validation reached 90%, which is close to the level of clinical physicians.

[0128] The present invention discloses a contactless gait cycle characteristic analysis method and its application in autism screening. The method has simple operation, little restriction on the site, and fast collection. It can effectively test the development and coordinated movement ability of the test subject through gait analysis, and then evaluate the probability of autism. Due to the shortage of pediatric psychologist resources in my country, the present invention can greatly assist clinical work, provide an objective and effective behavior quantification method, and can also be extended to multiple occasions such as families and schools to screen potential autism patients on a large scale. Therefore, it has important clinical application prospects and is expected to bring substantial benefits to autistic patients and their families.

[0129] Example 2

[0130] This embodiment is similar to Embodiment 1. Only the differences between this embodiment and Embodiment 1 are described below, and the same parts are not repeated here.

[0131] A contactless gait collection device includes a collection path and a camera; the collection path is laid on the ground to be measured and has only an anti-slip layer 2. Since the pressure sensor will bring a certain accuracy error, the pressure sensor layer 1 is not provided in this embodiment. At this time, the anti-slip layer is used as the bottom layer, and the positioning point and the nameplate 4 are printed on the upper surface of the anti-slip layer 2.

[0132] Since the pressure sensor will significantly increase the cost of the device, this embodiment realizes abnormality assessment without setting the pressure sensor layer 1, but it will bring some accuracy loss. At this time, the anti-slip layer serves as the bottom layer, and the wireless transmission device 6 is not set in the collection trail. The collected data is transmitted to the feature analysis subsystem by the camera via WIFI. Compared with Example 1, the input of the abnormality assessment module in step five in Example 2 is the duration, step length, pace, upper arm amplitude, thigh amplitude, head inclination, center of gravity stability, and contralateral coordination characteristics of several gait cycles of the object to be tested extracted in step four. The characteristics of all height-standardized gait cycles are averaged, and then the height characteristics are fused to form 9-dimensional features. The random forest model used is:

[0133] P = RandomForest(f T ,f stride ,f arm ,f leg ,f head ,f speed ,f stability ,f coord ,f 身高 )

[0134] The output is the probability value P that the subject to be tested suffers from autism.

[0135] The model has also been effectively verified on a clinical data set of children with autism, with a recognition rate of 87%. The loss of accuracy is within an acceptable range, but the significant reduction in cost can enhance the application value of the invention.

[0136] Although the preferred embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many forms without departing from the scope of protection of the present invention and the claims. As long as it involves the use of optical frequency comb dispersion interference multi-optical path difference resolution capability and a multi-degree-of-freedom measurement method of a measurement target composed of a polygonal conical mirror, these all fall within the scope of protection of the present invention.

Claims

1. A non-contact gait cycle characteristic analysis system, characterized in that The invention comprises a contactless gait collection device and a feature analysis subsystem, wherein the gait collection device and the feature analysis subsystem are connected via WIFI; the contactless gait collection device comprises a collection path and a camera, wherein positioning points are respectively arranged at both ends of the collection path for the person to be tested to determine the round-trip position; the camera is used to collect a video of the person to be tested going back and forth along the collection path; and the camera is provided with a wireless transmission device (6) for connecting to the feature subsystem via WIFI; The feature analysis subsystem includes a natural gait acquisition module, an adaptive gait cycle detection module, a gait cycle standardization module, a gait feature extraction module and an abnormality assessment module; The natural gait collection module uses the contactless gait collection device to collect the natural lateral walking gait data of the person to be tested contactlessly throughout the whole process; The adaptive gait cycle detection module is used to extract and automatically calibrate the skeleton sequence from the gait data collected by the natural gait collection module, and further detect and segment the complete gait cycle; Specifically, it includes: determining the moment when the left and right feet touch the ground at the same time based on the distance change sequence of the centers of the two feet on the x-axis; then, according to the walking direction, excluding gait features containing corner turns, excluding features of incorrect recognition of the left and right limbs of the skeleton, and excluding gait cycles with more than 30% frame loss, to obtain gait cycles in the same direction; The gait cycle standardization module is used to perform noise reduction and centralization using a gait cycle standardization algorithm according to the same-direction gait cycle obtained by the adaptive gait cycle detection module, and to perform height standardization processing by integrating height difference factors, so as to obtain a height-standardized gait cycle joint point sequence integrating height factors; The gait feature extraction module is used to extract gait features related to autism according to the height-standardized gait cycle joint point sequence obtained by the gait cycle standardization module, wherein the gait features include the duration, step length, step speed, joint angle, center of gravity stability, foot pressure and contralateral coordination of several gait cycles of the subject to be tested; The abnormality assessment module is used to extract autism-related gait features based on the gait features of several gait cycles of the subject to be tested obtained by the gait feature extraction module; The collection path comprises, from bottom to top, a pressure sensor layer (1) and an anti-skid layer (2); the pressure sensor layer (1) comprises a plurality of pressure sensors arranged in a dot matrix, and the total area of ​​the pressure sensors covers ≥95% of the collection path, and is used for real-time feedback of pressure information on the surface of the path; the anti-skid layer is located on the top of the pressure sensor layer (1), and positioning points are arranged on the upper surface; the positioning points are located at both ends of the anti-skid layer, and the color of the positioning points is sufficiently distinguishable from the color of the surface of the anti-skid layer, so as to facilitate the analysis of the start and end point positioning; and the collection path is also provided with a wireless transmission device (6); the collection path and the camera transmit the collected data to the feature analysis subsystem in a WIFI manner through the wireless transmission device (6); and the collection path and the camera achieve time synchronization of the collected data of the camera and the pressure sensor in a Bluetooth manner through the wireless transmission device (6), so that the collection of the two starts and ends at the same time, so that the two can collect data in a time-corresponding manner.

2. The contactless gait cycle characteristic analysis system according to claim 1, characterized in that: The collection trail is only provided with an anti-slip layer (2), and positioning points and a nameplate (4) are provided on the upper surface of the anti-slip layer (2). The positioning points are located at both ends of the anti-slip layer, and the color of the positioning points is sufficiently distinguishable from the surface color of the anti-slip layer (2), so as to facilitate the analysis of the starting and ending point positioning; the nameplate (4) is printed with information on the length and width of the collection trail and the distance between the positioning points.

3. The contactless gait cycle characteristic analysis system according to claim 1, characterized in that: The automatic calibration in the adaptive gait cycle detection module refers to automatically calibrating the skeleton features according to the pixel length occupied by the positioning point and the angle with the horizontal line of the picture, using the perspective and rotation principles in image processing.

4. A non-contact gait cycle feature analysis method, characterized in that include: Step 1: using a camera (5) to collect multiple videos of the person to be tested going back and forth between the positioning points at both ends within the collection path, wherein the video includes multiple gait cycles; using the pressure sensor of the pressure sensor layer (1) to collect the pressure value of the path; Step 2: Use motion detection and tracking technology based on computer vision and signal processing technology to extract skeleton data during human movement; use the principles of perspective and distortion in image processing to automatically calibrate skeleton features; perform gait cycle recognition and segmentation based on the calibrated skeleton features, exclude the gait cycle of turning at the corner from the gait cycle in step 1, and obtain a complete gait cycle in the same direction; Step 3: Use the gait cycle standardization algorithm to calculate the complete same-direction gait cycle obtained in step 2 to obtain the height-standardized gait cycle; Step 4: According to the multiple standard gait cycles of the subject to be tested obtained in step 3, each gait cycle is calculated to obtain gait characteristics, wherein the gait characteristics include duration, foot pressure, step length, step speed, joint angle, center of gravity stability, contralateral coordination and other characteristics; Step 5: The gait features obtained in step 4 are integrated with the height features and input into the pre-trained machine learning model for probability prediction to extract autism-related gait features; Among them, in step 2, gait cycle recognition and segmentation are performed according to the calibrated skeleton features, and the gait cycle of turning at the corner is excluded from the gait cycle of step 1 to obtain a complete gait cycle in the same direction, which specifically includes: According to the calibrated skeleton sequence, the position coordinates of the right foot center r(n) and the left foot center l(n) are calculated, and the distance change sequence d(n) between the centers of both feet on the x-axis is calculated. When d(n)>0, it means that the right foot is to the right of the left foot at time n, as shown in the following formula: Among them, mark i,x (n) represents the x coordinate of the ith joint point in the nth frame; then, find the extreme point z of the sequence d(n) i , the extreme point is the point where the absolute distance between the left and right feet is the largest, representing the moment when both feet step out and touch the ground at the same time; According to formula (2), the walking direction at time n is determined by s(n); where mark 0,x is the horizontal coordinate of the nose coordinate point, joint mark 11 and mark 12 The midpoint is the coordinate of the neck coordinate point, and sign is the sign operation. When the s(n) value is 1, it means the direction is right, and -1 means the direction is left; Then, for all extreme points z i Screening is performed. For time n, when s(n) is 1, the minimum point is retained, and when it is -1, the maximum point is retained. The extreme point after screening is z′ i , where i = 0, 1, ..., M; Gait cycle T i The frame number interval is expressed as: T i =[z′ i ,z′ i+1 ] (3) Where i = 0, 1, ..., M-1; Eliminate gait cycles that contain turns at corners: determine the walking directions s(n) in the gait cycles one by one. If s(n) in i cycles have different signs, there is a turn in the cycle, and the i-th cycle is deleted. Eliminate the phenomenon of incorrect recognition of the left and right limbs of the skeleton and frame drops: delete the cycles that do not contain 2 maximum values ​​and 1 minimum value or 2 minimum values ​​and 1 maximum value in the gait cycle; Exclude frames that exceed 30% of the gait cycle, that is, for the gait cycle T i , must satisfy formula (4): With' i+1 -With' i ≥(1-30%)*fps*[frame2time(z′ i+1 )-frame2time(z′ i )] (4) Among them, fps is the image sampling rate during acquisition, frame2time(n) is the frame-time conversion function, which represents the time corresponding to the nth moment in seconds; Finally, N complete gait cycles T in the same direction are calculated. i (i=1,2,…,N).

5. The contactless gait cycle feature analysis method according to claim 4, characterized in that: The gait cycle normalization algorithm in step 3 includes: Traverse the horizontal and vertical coordinates of all joint points to calculate the Z score, where the Z score refers to the difference between a single data point and the overall data mean, identify the frame at that moment corresponding to the joint point whose Z score exceeds a preset threshold as a singular frame, and remove the singular frame; Perform Gaussian smoothing on all relevant nodes; The processed joint points are centered, and the gait is decomposed into the movement of the center of gravity of the human body and the movement of the limbs. The center of gravity of the human body is calculated as the coordinate origin, and the height feature is normalized to convert the height of all the images into a uniform level, thereby obtaining a standardized gait cycle joint point sequence.

6. The contactless gait cycle feature analysis method according to claim 5, characterized in that: The center of gravity of the human body is calculated based on the calculated motion sequence of the center of gravity of the skeleton in each gait cycle combined with the weight distribution of the human body.

7. The contactless gait cycle feature analysis method according to claim 4, characterized in that: Acquisition of gait features in step 4 include: The duration feature is the time corresponding to the gait cycle; The foot pressure feature is the average value of the pressure change of the footsteps on the collection path during the gait cycle, and the pressure value comes from the data collected by the pressure sensor of the pressure sensor layer (1) of the collection path; The stride length feature refers to the average length of each step taken during the gait cycle, specifically the average value of the maximum lateral distance of the horizontal coordinate movement of the left heel and the right heel calculated in combination with the joint position; the gait speed feature refers to the forward speed during the gait cycle, which is calculated based on the ratio of the stride length to the cycle length; Joint angle characteristics include the maximum range of motion of the left and right upper arms, the maximum range of motion of the thighs, and the head tilt angle; The maximum range of left and right upper arm movement refers to the average of the maximum angles between the left and right elbows and the left and right shoulders during the gait cycle; the maximum range of thigh movement refers to the average of the maximum angles between the left and right knees and the left and right hips during the gait cycle; the head inclination angle refers to the maximum value of the angle between the line connecting the nose and the neck and the horizontal plane during the gait cycle; The center of gravity stability characteristic is the standard deviation of the body's center of gravity in the y-axis direction during the gait cycle; The contralateral synergy feature refers to the average value of the correlation of the horizontal coordinate displacement of the contralateral hand and foot, where the correlation is obtained using the Pearson correlation coefficient function.

8. Application of the contactless gait cycle characteristic analysis system according to any one of claims 1 to 3 in autism screening.

9. A non-contact gait cycle characteristic analysis device, characterized in that: The analysis device comprises a processor, and the processor is used to process the contactless gait cycle feature analysis method according to any one of claims 4 to 7.

Images