Method for assisting early judgment of teenager idiopathic scoliosis based on walking video

Abstract

The invention discloses a method for assisting early judgment of adolescent idiopathic scoliosis based on a walking video. The method comprises the steps of obtaining two-dimensional key point coordinates of an AIS patient; deriving three-dimensional key point coordinates; calculating a corresponding power spectrum index; and obtaining the disease degree of the AIS patient. The method has the advantages that the AIS patient can conveniently monitor the development condition of the own illness state in real time; an AlphaPose system is adopted, and meanwhile, a DLT method is introduced, so that the capability of estimating coordinates of three-dimensional key points can be enhanced; the MAnE algorithm and the MGE algorithm are adopted, the walking characteristics of the AIS patient can be described from the two dimensions of time and space, and the characteristics of abnormal gaits can be obtained more clearly; a computer technology is applied to data acquisition, mathematics is applied to feature calculation, a doctor can be effectively assisted in AIS early diagnosis, a corresponding treatment scheme is formulated, and therefore the monitoring effect and safety can be improved, and the life quality of an AIS patient can be improved.

Description

Technical Field

[0001] The present invention belongs to the cross-technical field of mathematics, computer and medicine, and particularly relates to a method for early diagnosis of adolescent idiopathic scoliosis based on walking video assistance. Background Art

[0002] In recent years, with the development of artificial intelligence, computer vision technology has been widely used in the fields of disease diagnosis, classification and prediction due to its excellent ability to extract information from images. For example, deep learning analysis can automatically classify eye diseases and predict cardiovascular risks through patient medical history and blood samples. In addition, some studies have found that computer vision can be used to detect image features or key points. Influenced by these methods, some scholars extract patients' joints as key points through Kinect sensors and determine spinal posture deformities through factors such as shoulder height difference, pelvic height difference and leg length mismatch. Entropy measurement can characterize the rate of change of information in dynamic systems and has been widely used to quantify some biological signals, such as gene networks, protein interaction networks and organisms. In recent years, entropy methods have been effectively applied to analyze gait instability in neurological diseases.

[0003] Among all types of scoliosis, adolescent idiopathic scoliosis (AIS) is a typical orthopedic problem that mostly occurs between the ages of 10 and 18. The degree of scoliosis worsens as bones grow, causing disability due to back pain and cardiopulmonary damage, so early diagnosis of scoliosis can reduce surgical risks, but it remains challenging. Many professional organizations, such as the Scoliosis Research Society and the North American Pediatric Orthopedic Association, believe that early detection can increase the chances of effective non-surgical treatment. At present, the diagnostic methods of AIS can be mainly divided into two categories: physical examination methods and radiological methods, but due to some uncertain factors, there is still no better early diagnosis plan.

[0004] The Chinese invention patent application with application number 201711396072.2 discloses a new molecular marker associated with adolescent idiopathic scoliosis, which can be used for diagnosis in the early stages of the disease.

[0005] The Chinese invention patent application with application number 202211112412.5 discloses a PUMC classification method for adolescent idiopathic scoliosis, which uses a pre-trained vertebral semantic segmentation model to process each spinal image and automatically perform PUMC classification.

[0006] The Chinese invention patent application with application number 202310719244.4 discloses a method for predicting the effect of adolescent scoliosis correction surgery, which can predict the correction effect of AIS surgery with high accuracy, thereby assisting clinical physicians in preoperative planning of surgical plans.

[0007] The Chinese invention patent application with application number 202311293757.X discloses a method for surgical segment selection for adolescent idiopathic scoliosis based on deep learning assistance. The entire process from inputting X-ray images to calculating confidence does not require human intervention.

[0008] With the rapid development of computer vision technology, early diagnosis of scoliosis is expected to change from manual judgment to machine processing. As an auxiliary means of X-ray film, it can realize early diagnosis of AIS patients while minimizing the complexity of detection, reducing the radiation damage suffered by adolescents, assisting doctors in making judgments and reducing the inconvenience of treatment for patients. However, no effective method has been found yet. Summary of the invention

[0009] In order to solve the above problems, the purpose of the present invention is to provide a method for early diagnosis of adolescent idiopathic scoliosis based on walking video.

[0010] In order to achieve the above object, the method for early diagnosis of adolescent idiopathic scoliosis based on walking video provided by the present invention comprises the following steps performed in sequence:

[0011] 1) Collect walking videos of AIS patients and mark the joint points in each frame of the image, then select multiple key points from the joint points and obtain the two-dimensional key point coordinates of the AIS patients;

[0012] 2) using a direct linear transformation method to derive the three-dimensional key point coordinates from the two-dimensional key point coordinates of the AIS patient obtained in step 1);

[0013] 3) According to the three-dimensional key point coordinates of the AIS patient obtained in step 2), the sliding angle entropy and the sliding edge weighted graph entropy of the key point sequence of the AIS patient when walking are calculated, and then the sliding angle entropy and the sliding edge weighted graph entropy are fused using the canonical correlation analysis method, and the corresponding power spectrum index is calculated according to the fused value;

[0014] 4) The power spectrum index of each AIS patient obtained in step 3) is sorted and classified, and compared with the Cobb angle results of X-ray examination to obtain the severity of the AIS patient.

[0015] In step 1), the method of collecting the walking video of the AIS patient and marking the joint points in each frame image, then selecting multiple key points from the joint points, and obtaining the two-dimensional key point coordinates of the AIS patient is:

[0016] A binocular camera was used to shoot walking videos of AIS patients facing the camera and with their backs to the camera. The joints in each frame were annotated using the AlphaPose system. A total of 14 joints, including the left shoulder, neck, right shoulder, left elbow, right elbow, left hip, right hip, left knee, right knee, left ankle, right ankle, left elbow, right elbow and hip, were selected as key points 1-14, and the two-dimensional key point coordinates of AIS patients were obtained.

[0017] The AlphaPose system includes five modules: a data loading module, a detection module, a data conversion module, a posture detection module and a post-processing module; wherein the data loading module loads the information in the input image, video or camera into a first-in-first-out queue; then the detection module uses a human body detector to detect the human body area, and stores the detected human body area in a first-in-first-out queue; then the data conversion module performs non-maximum suppression on the detected human body area to exclude the human body area with low detection accuracy, and crops and resizes the retained human body area to facilitate subsequent posture estimation; then the posture estimation module uses a posture estimator to estimate the human body posture, and uses a tracker to track the human body across frames; finally, the post-processing module uses posture non-maximum to suppress and exclude key points with low detection accuracy, and visualizes the retained key points and saves the two-dimensional key point coordinates.

[0018] In step 2), the method of using the direct linear transformation method to derive the three-dimensional key point coordinates from the two-dimensional key point coordinates of the AIS patient obtained in step 1) is:

[0019] Firstly, the 2D key point coordinates of AIS patients were processed using a Butterworth low-pass filter to remove noise and outliers. Then, the 3D key point coordinates were derived from the 2D key point coordinates using a direct linear transformation method based on the intrinsic and extrinsic parameters of the camera to describe the relationship between the key points on the image plane and the key points in the 3D space.

[0020] In step 3), the sliding angle entropy and sliding edge weighted graph entropy of the key point sequence of the AIS patient when walking are calculated according to the three-dimensional key point coordinates of the AIS patient obtained in step 2), and then the sliding angle entropy and sliding edge weighted graph entropy are fused using the canonical correlation analysis method. The method for calculating the corresponding power spectrum index according to the fused value is:

[0021] Suppose the key point sequence composed of each key point in the human skeleton graph is J = {j 1,j 2 ,…,j n}, key point j i The corresponding coordinates are (x i ,y i ,z i ),i=1,…,n;

[0022] The calculation method of the sliding angle entropy is as follows:

[0023] (1) Based on the three-dimensional key point coordinates of the AIS patient obtained in step 2), the difference in joint angles on the left and right sides of the AIS patient in each frame of the image is calculated, and all the above differences constitute an angle difference dynamic network The formula is as follows:

[0024]

[0025] In the formula, represents the difference between the left shoulder joint angle 1 and the right shoulder joint angle 4, represents the difference between the left shoulder and spine joint angle 2 and the right shoulder and spine joint angle 3; t represents time;

[0026] Calculate the difference of each joint angle between two consecutive frames of images, and form an inter-frame joint angle difference network based on all the above differences The formula is as follows:

[0027]

[0028] In the formula, Represents the difference between two adjacent frames of the left shoulder joint angle 1, Represents the difference between two adjacent frames of images of the left shoulder and spine joint angle 2, Represents the difference between two adjacent frames of the right shoulder and spine joint angle 3, represents the difference of the right shoulder joint angle 4, and s represents the number of frames;

[0029] (2) Based on the above angle difference dynamic network Establish the angle difference dynamic network diagram signal f t (n), based on the inter-frame joint angle difference network Establish the inter-frame joint angle difference network graph signal h s (n), the formula is as follows;

[0030] f t (n) = {f t (1),…,f t (N)}

[0031] h s (n) = {h s (1),…,hs (N)}

[0032] (3) The above angle difference dynamic network graph signal f is converted by using the Cartesian product of the graph signal t (n) and the inter-frame joint angle difference network signal h s (n) are combined to form the graph signal y(n) based on the joint angle, and the formula is as follows:

[0033] y(n)=f t (n)*h s (n)={y(1),…,y(N)}

[0034] (4) Use phase space reconstruction to divide the above joint angle-based graph signal y(n) into multiple sliding windows to generate an embedding vector Set the embedding dimension to 2≤m∈N, then the embedding vector The formula is as follows:

[0035]

[0036] (5) Calculate the embedding vector of each sliding window The sliding angle entropy is as follows:

[0037]

[0038] In the formula, θ i Represents each joint angle value in the joint angle-based graph signal y(n);

[0039] The calculation method of the sliding edge weighted graph entropy is as follows:

[0040] (1) The line connecting key point 2 and key point 1 is taken as edge 1, the line connecting key point 1 and key point 6 is taken as edge 2, the line connecting key point 6 and key point 10 is taken as edge 3, the line connecting key point 10 and key point 14 is taken as edge 4, the line connecting key point 2 and key point 3 is taken as edge 5, the line connecting key point 3 and key point 7 is taken as edge 6, the line connecting key point 7 and key point 11 is taken as edge 7, and the line connecting key point 11 and key point 14 is taken as edge 8. Then, the length of each of the above edges is calculated respectively. Then, the length differences between edge 1 and edge 5, edge 2 and edge 6, edge 3 and edge 7, and edge 4 and edge 8 are calculated respectively. All the above length differences constitute a distance difference dynamic network DD. i , the formula is as follows:

[0041]

[0042] In the formula, represents the key point set, represents the edge set, w t represents the edge weight set;

[0043] Connect the same key point between two adjacent frames, and then calculate the length of each line. All the above lengths constitute the inter-frame joint distance network InD s , the formula is as follows:

[0044]

[0045] In the formula, represents the key point set, In represents the edge set, s represents the edge weight set;

[0046] (2) Using the Cartesian product of the graph signal, the distance difference dynamic network DD i and inter-frame joint distance network InD s Combined together, a dynamic network DG based on joint distance is constructed t ;

[0047] DG t =DD t *InD s

[0048] (4) According to the above joint distance-based dynamic network DG t Establish graph signal d(n);

[0049] (4) Use phase space reconstruction to divide the above graph signal d(n) into multiple sliding windows to generate an embedding vector Set the embedding dimension to 2≤m∈N, and embedding vector The formula is as follows:

[0050]

[0051] (5) Calculate the embedding vector of each sliding window The sliding edge weighted graph entropy of :

[0052]

[0053] In the formula,

[0054] Finally, the above sliding angle entropy and sliding edge weighted graph entropy are fused using the canonical correlation analysis method, and the corresponding power spectrum index α is calculated based on the fused values.

[0055] In step 4), the power spectrum index of each AIS patient obtained in step 3) is sorted and classified, and compared with the Cobb angle result of X-ray examination to obtain the degree of disease of the AIS patient:

[0056] The power spectrum index α of each AIS patient obtained in step 3) above is sorted and divided into three categories: mild, moderate and severe, and compared with the Cobb angle results of X-ray examination, and the classification accuracy of each type of AIS patient is calculated, thereby obtaining the severity of the AIS patient.

[0057] The method for early diagnosis of adolescent idiopathic scoliosis based on walking video provided by the present invention has the following beneficial effects:

[0058] 1. The data collection method of shooting AIS patients walking videos is simple and convenient, not limited by time and space, and can reduce radiation damage, making it easier for AIS patients to monitor the development of their own disease in real time. Therefore, it provides a portable means of real-time monitoring of AIS degree;

[0059] 2. The AlphaPose system is used, and the DLT method is introduced to enhance the ability to estimate the coordinates of three-dimensional key points, solving the problems of skeleton tilt and lack of complete depth information in deep learning methods;

[0060] 3. The MAnE and MGE algorithms can be used to describe the walking characteristics of AIS patients from two dimensions: time and space, and to more clearly obtain the characteristics of abnormal gait;

[0061] 4. Applying computer technology to data acquisition and mathematics to feature calculation can effectively assist doctors in early diagnosis of AIS and develop corresponding treatment plans, thereby improving the effectiveness and safety of monitoring and improving the quality of life of AIS patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 Schematic diagram of the AlphaPose system structure in the present invention.

[0063] Figure 2 Schematic diagram of key points of AIS patients selected in the present invention.

[0064] Figure 3 This is a schematic diagram of the angle difference dynamic network in the present invention.

[0065] Figure 4 This is a schematic diagram of the network of joint angle differences between frames in the present invention.

[0066] Figure 5 This is a schematic diagram of the distance difference dynamic network in the present invention.

[0067] Figure 6 This is a schematic diagram of the inter-frame joint distance network in the present invention. DETAILED DESCRIPTION

[0068] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] The method for early diagnosis of adolescent idiopathic scoliosis based on walking video provided by the present invention comprises the following steps performed in sequence:

[0070] 1) Collect walking videos of AIS patients and mark the joint points in each frame of the image, then select multiple key points from the joint points and obtain the two-dimensional key point coordinates of the AIS patients;

[0071] A binocular camera was used to shoot walking videos of AIS patients facing the camera and with their backs to the camera. Figure 1 The AlphaPose system shown in the figure marks the joints in each frame of the image, and then selects 14 joints including the left shoulder, neck, right shoulder, left elbow, right elbow, left hip, right hip, left knee, right knee, left ankle, right ankle, left elbow, right elbow and hip as key points 1-14, and obtains the two-dimensional key point coordinates of the AIS patient;

[0072] The AlphaPose system includes five modules: a data loading module, a detection module, a data conversion module, a posture detection module and a post-processing module; wherein the data loading module loads the information in the input image, video or camera into a first-in-first-out queue; then the detection module uses a human body detector to detect the human body area, and stores the detected human body area in a first-in-first-out queue; then the data conversion module performs non-maximum suppression on the detected human body area to exclude the human body area with low detection accuracy, and crops and resizes the retained human body area to facilitate subsequent posture estimation; then the posture estimation module uses a posture estimator to estimate the human body posture, and uses a tracker to track the human body across frames; finally, the post-processing module uses posture non-maximum to suppress and exclude key points with low detection accuracy, and visualizes the retained key points and saves the two-dimensional key point coordinates.

[0073] 2) using the direct linear transformation (DLT) method to derive the three-dimensional key point coordinates from the two-dimensional key point coordinates of the AIS patient obtained in step 1);

[0074] 3D key point detection of the human body refers to estimating the 3D coordinates of various parts of the human body from a single or multiple frame image. Due to the characteristics of camera imaging, points in the real world need to be transformed into points in the image through coordinate system conversion. The DLT method is a 3D key point detection method for the human body based on the principle of projective geometry. It uses a linear model to describe the relationship between key points on the image plane and key points in 3D space.

[0075] Firstly, the 2D key point coordinates of AIS patients were processed using a Butterworth low-pass filter to remove noise and outliers. Then, the direct linear transformation (DLT) method was used to describe the relationship between the key points on the image plane and the key points in the 3D space through the intrinsic and extrinsic parameters of the camera, and the 3D key point coordinates were derived from the 2D key point coordinates.

[0076] 3) According to the three-dimensional key point coordinates of the AIS patient obtained in step 2), the moving angle entropy (MAnE) and the moving edge-weighted graph entropy (MGE) of the key point sequence of the AIS patient when walking are calculated, and then the above sliding angle entropy and moving edge-weighted graph entropy are fused using the canonical correlation analysis method, and the corresponding power spectrum index is calculated according to the fused value;

[0077] Suppose the key point sequence composed of each key point in the human skeleton graph is J = {j 1 ,j 2 ,…,j n}, key point j i The corresponding coordinates are (x i ,y i ,z i ),i=1,…,n;

[0078] The calculation method of the sliding angle entropy is as follows:

[0079] (1) According to the three-dimensional key point coordinates of the AIS patient obtained in step 2), the difference of the joint angles on the left and right sides of the AIS patient in each frame of the image is calculated, and all the above differences are used to form the following Figure 3 Angle difference dynamic network shown The formula is as follows:

[0080]

[0081] In the formula, represents the difference between the left shoulder joint angle 1 and the right shoulder joint angle 4, represents the difference between the left shoulder and spine joint angle 2 and the right shoulder and spine joint angle 3; t represents time;

[0082] Calculate the difference of each joint angle between two consecutive frames of images, and construct the following equation from all the above differences: Figure 4 The inter-frame joint angle difference network shown The formula is as follows:

[0083]

[0084] In the formula, Represents the difference between two adjacent frames of the left shoulder joint angle 1, Represents the difference between two adjacent frames of images of the left shoulder and spine joint angle 2, Represents the difference between two adjacent frames of the right shoulder and spine joint angle 3, represents the difference of the right shoulder joint angle 4, and s represents the number of frames;

[0085] (2) Based on the above angle difference dynamic network Establish the angle difference dynamic network diagram signal f t (n), based on the inter-frame joint angle difference network Establish the inter-frame joint angle difference network graph signal h s (n), the formula is as follows;

[0086] f t (n) = {f t (1),…,f t (N)}

[0087] h s (n) = {h s (1),…,h s (N)}

[0088] (3) The above angle difference dynamic network graph signal f is converted by using the Cartesian product of the graph signal t (n) and the inter-frame joint angle difference network signal h s (n) are combined to form the graph signal y(n) based on the joint angle, and the formula is as follows:

[0089] y(n)=f t (n)*h s (n)={y(1),…,y(N)}

[0090] (4) Use phase space reconstruction to divide the above joint angle-based graph signal y(n) into multiple sliding windows to generate an embedding vector Set the embedding dimension to 2≤m∈N, then the embedding vector The formula is as follows:

[0091]

[0092] (5) Calculate the embedding vector of each sliding window The sliding angle entropy is as follows:

[0093]

[0094] In the formula, θ i Represents each joint angle value in the joint angle-based graph signal y(n);

[0095] The calculation method of the sliding edge weighted graph entropy is as follows:

[0096] (1) Figure 5 As shown, the line connecting key point 2 and key point 1 is used as edge 1, the line connecting key point 1 and key point 6 is used as edge 2, the line connecting key point 6 and key point 10 is used as edge 3, the line connecting key point 10 and key point 14 is used as edge 4, the line connecting key point 2 and key point 3 is used as edge 5, the line connecting key point 3 and key point 7 is used as edge 6, the line connecting key point 7 and key point 11 is used as edge 7, and the line connecting key point 11 and key point 14 is used as edge 8, and then the length of each of the above edges is calculated respectively, and then the length differences between edge 1 and edge 5, edge 2 and edge 6, edge 3 and edge 7, edge 4 and edge 8 are calculated respectively, and all the above length differences constitute a distance difference dynamic network DD i , the formula is as follows:

[0097]

[0098] In the formula, represents the key point set, represents the edge set, w t represents the edge weight set;

[0099] like Figure 6 As shown in FIG. 1 , the same key point between two adjacent frames is connected, and then the length of each connection line is calculated respectively (for example, the key point 1 at t=1 and the key point 1 at t=2 are connected to calculate the length of the connection line). The above-mentioned lengths constitute the inter-frame joint distance network InD s , the formula is as follows:

[0100]

[0101] In the formula, represents the key point set, In represents the edge set, s represents the edge weight set;

[0102] (2) Using the Cartesian product of the graph signal, the distance difference dynamic network DD i and inter-frame joint distance network InD s Combined together, a dynamic network DG based on joint distance is constructed t ;

[0103] DG t =DD t *InD s

[0104] (5) According to the above joint distance-based dynamic network DG t Establish graph signal d(n);

[0105] (4) Use phase space reconstruction to divide the above graph signal d(n) into multiple sliding windows to generate an embedding vector Set the embedding dimension to 2≤m∈N, and embedding vector The formula is as follows:

[0106]

[0107] (5) Calculate the embedding vector of each sliding window The sliding edge weighted graph entropy of :

[0108]

[0109] In the formula,

[0110] Finally, the above sliding angle entropy and sliding edge weighted graph entropy are fused using the canonical correlation analysis method, and the corresponding power spectrum index α is calculated based on the fused values.

[0111] 4) The power spectrum index of each AIS patient obtained in step 3) is sorted and classified, and compared with the Cobb angle results of X-ray examination to obtain the severity of the AIS patient.

[0112] The power spectrum index α of each AIS patient obtained in step 3) above is sorted and divided into three categories: mild, moderate and severe, and compared with the Cobb angle results of X-ray examination, and the classification accuracy of each type of AIS patient is calculated, thereby obtaining the severity of the AIS patient.

Claims

1. A method for early diagnosis of adolescent idiopathic scoliosis based on walking video, characterized in that: The method for early diagnosis of adolescent idiopathic scoliosis based on walking video comprises the following steps performed in sequence: 1) Collect walking videos of AIS patients and mark the joint points in each frame of the image, then select multiple key points from the joint points and obtain the two-dimensional key point coordinates of the AIS patients; 2) using a direct linear transformation method to derive the three-dimensional key point coordinates from the two-dimensional key point coordinates of the AIS patient obtained in step 1); 3) According to the three-dimensional key point coordinates of the AIS patient obtained in step 2), the sliding angle entropy and the sliding edge weighted graph entropy of the key point sequence of the AIS patient when walking are calculated, and then the sliding angle entropy and the sliding edge weighted graph entropy are fused using the canonical correlation analysis method, and the corresponding power spectrum index is calculated according to the fused value; 4) The power spectrum index of each AIS patient obtained in step 3) is sorted and classified, and compared with the Cobb angle results of X-ray examination to obtain the severity of the AIS patient.

2. The method for early diagnosis of adolescent idiopathic scoliosis based on walking video according to claim 1 is characterized in that: In step 1), the method of collecting the walking video of the AIS patient and marking the joint points in each frame image, then selecting multiple key points from the joint points, and obtaining the two-dimensional key point coordinates of the AIS patient is: A binocular camera was used to shoot walking videos of AIS patients facing the camera and with their backs to the camera. The joints in each frame were annotated using the AlphaPose system. A total of 14 joints, including the left shoulder, neck, right shoulder, left elbow, right elbow, left hip, right hip, left knee, right knee, left ankle, right ankle, left elbow, right elbow and hip, were selected as key points 1-14, and the two-dimensional key point coordinates of AIS patients were obtained. The AlphaPose system includes five modules: a data loading module, a detection module, a data conversion module, a posture detection module and a post-processing module; wherein the data loading module loads the information in the input image, video or camera into a first-in-first-out queue; then the detection module uses a human body detector to detect the human body area, and stores the detected human body area in a first-in-first-out queue; then the data conversion module performs non-maximum suppression on the detected human body area to exclude the human body area with low detection accuracy, and crops and resizes the retained human body area to facilitate subsequent posture estimation; then the posture estimation module uses a posture estimator to estimate the human body posture, and uses a tracker to track the human body across frames; finally, the post-processing module uses posture non-maximum to suppress and exclude key points with low detection accuracy, and visualizes the retained key points and saves the two-dimensional key point coordinates.

3. The method for early diagnosis of adolescent idiopathic scoliosis based on walking video according to claim 1 is characterized in that: In step 2), the method of using the direct linear transformation method to derive the three-dimensional key point coordinates from the two-dimensional key point coordinates of the AIS patient obtained in step 1) is: Firstly, the 2D key point coordinates of AIS patients were processed using a Butterworth low-pass filter to remove noise and outliers. Then, the 3D key point coordinates were derived from the 2D key point coordinates using a direct linear transformation method based on the intrinsic and extrinsic parameters of the camera to describe the relationship between the key points on the image plane and the key points in the 3D space.

4. The method for early diagnosis of adolescent idiopathic scoliosis based on walking video according to claim 1, characterized in that: In step 3), the sliding angle entropy and sliding edge weighted graph entropy of the key point sequence of the AIS patient when walking are calculated according to the three-dimensional key point coordinates of the AIS patient obtained in step 2), and then the sliding angle entropy and sliding edge weighted graph entropy are fused using the canonical correlation analysis method. The method for calculating the corresponding power spectrum index according to the fused value is: Suppose the key point sequence composed of each key point in the human skeleton graph is J = {j1, j2, ..., j n }, key point j i The corresponding coordinates are (x i ,y i ,z i ),i=1,…,n; The calculation method of the sliding angle entropy is as follows: (1) Based on the three-dimensional key point coordinates of the AIS patient obtained in step 2), the difference in joint angles on the left and right sides of the AIS patient in each frame of the image is calculated, and all the above differences constitute an angle difference dynamic network The formula is as follows: In the formula, represents the difference between the left shoulder joint angle 1 and the right shoulder joint angle 4, represents the difference between the left shoulder and spine joint angle 2 and the right shoulder and spine joint angle 3; t represents time; Calculate the difference of each joint angle between two consecutive frames of images, and form an inter-frame joint angle difference network based on all the above differences The formula is as follows: In the formula, Represents the difference between two adjacent frames of the left shoulder joint angle 1, Represents the difference between two adjacent frames of images of the left shoulder and spine joint angle 2, Represents the difference between two adjacent frames of the right shoulder and spine joint angle 3, represents the difference of the right shoulder joint angle 4, and s represents the number of frames; (2) Based on the above angle difference dynamic network Establish the angle difference dynamic network diagram signal f t (n), based on the inter-frame joint angle difference network Establish the inter-frame joint angle difference network graph signal h s (n), the formula is as follows; f t (n)={f t (1),…,f t (N)} h s (n)={h s (1),…,h s (N)} (3) The above angle difference dynamic network graph signal f is converted by using the Cartesian product of the graph signal t (n) and the inter-frame joint angle difference network signal h s (n) are combined to form the graph signal y(n) based on the joint angle, and the formula is as follows: y(n)=f t (n)*h s (n)={y(1),…,y(N)} (4) Use phase space reconstruction to divide the above joint angle-based graph signal y(n) into multiple sliding windows to generate an embedding vector Set the embedding dimension to 2≤m∈N, then the embedding vector The formula is as follows: (5) Calculate the embedding vector of each sliding window The sliding angle entropy is as follows: In the formula, θ i Represents each joint angle value in the joint angle-based graph signal y(n); The calculation method of the sliding edge weighted graph entropy is as follows: (1) The line connecting key point 2 and key point 1 is taken as edge 1, the line connecting key point 1 and key point 6 is taken as edge 2, the line connecting key point 6 and key point 10 is taken as edge 3, the line connecting key point 10 and key point 14 is taken as edge 4, the line connecting key point 2 and key point 3 is taken as edge 5, the line connecting key point 3 and key point 7 is taken as edge 6, the line connecting key point 7 and key point 11 is taken as edge 7, and the line connecting key point 11 and key point 14 is taken as edge 8. Then, the length of each of the above edges is calculated respectively. Then, the length differences between edge 1 and edge 5, edge 2 and edge 6, edge 3 and edge 7, and edge 4 and edge 8 are calculated respectively. All the above length differences constitute a distance difference dynamic network DD. i , the formula is as follows: In the formula, represents the key point set, represents the edge set, w t represents the edge weight set; Connect the same key point between two adjacent frames, and then calculate the length of each line. All the above lengths constitute the inter-frame joint distance network InD s , the formula is as follows: In the formula, represents the key point set, In represents the edge set, s represents the edge weight set; (2) Using the Cartesian product of the graph signal, the distance difference dynamic network DD i and inter-frame joint distance network InD s Combined together, a dynamic network DG based on joint distance is constructed t ; DG t =DD t *InD s (3) According to the above joint distance-based dynamic network DG t Establish graph signal d(n); (4) Use phase space reconstruction to divide the above graph signal d(n) into multiple sliding windows to generate an embedding vector Set the embedding dimension to 2≤m∈N, and embedding vector The formula is as follows: (5) Calculate the embedding vector of each sliding window The sliding edge weighted graph entropy of : In the formula, Finally, the above sliding angle entropy and sliding edge weighted graph entropy are fused using the canonical correlation analysis method, and the corresponding power spectrum index α is calculated based on the fused values.

5. The method for early diagnosis of adolescent idiopathic scoliosis based on walking video according to claim 1, characterized in that: In step 4), the power spectrum index of each AIS patient obtained in step 3) is sorted and classified, and compared with the Cobb angle result of X-ray examination to obtain the degree of disease of the AIS patient: The power spectrum index α of each AIS patient obtained in step 3) above is sorted and divided into three categories: mild, moderate and severe, and compared with the Cobb angle results of X-ray examination, and the classification accuracy of each type of AIS patient is calculated, thereby obtaining the severity of the AIS patient.

Images