Gait evaluation method, system and device, electronic equipment, storage medium and computer program product

Abstract

The invention discloses a gait evaluation method, system and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps that image data are acquired, and the image data comprise data of movement of a to-be-evaluated person in a detection area; performing feature extraction on the image data to obtain human body features of the to-be-evaluated person; based on a local covariance matrix, carrying out feature extraction on the human body features to obtain motion features of the to-be-evaluated person; and inputting the motion features into a first model to obtain a gait assessment result of the to-be-assessed person. According to the scheme provided by the invention, firstly, manual participation is not needed, full-automatic gait evaluation is realized, and the accuracy is high; secondly, the image data of the to-be-assessed person is convenient to collect, and the scheme is convenient to implement; and thirdly, the data volume of the motion features extracted by utilizing the local covariance is small, so that the calculation power and time cost of model analysis by utilizing the motion features are relatively low, and the evaluation efficiency is high.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a gait assessment method, system, apparatus, related equipment, storage medium, and computer program product. Background Art

[0002] In related technologies, two approaches are usually used to evaluate children's gait. One approach is for experienced medical staff to judge the child's gait by observing the child's walking, that is, to judge the child's gait manually. The other approach is to obtain the child's surface electromyography information and use a filter group to identify the child's gait through the child's surface electromyography information.

[0003] However, the accuracy of manual judgment is difficult to guarantee and requires a lot of human resources. At the same time, it is difficult to collect children's electromyographic information, making the plan difficult to implement. Summary of the Invention

[0004] To solve related technical problems, embodiments of the present application provide a gait assessment method, system, device, electronic device, storage medium, and computer program product.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present invention provides a gait assessment method, comprising:

[0007] Acquiring image data, wherein the image data includes data of the person to be evaluated moving in the detection area;

[0008] Performing feature extraction on the image data to obtain the body features of the person to be evaluated;

[0009] Extracting the human body features based on the local covariance matrix to obtain the motion features of the person to be evaluated;

[0010] The motion characteristics are input into the first model to obtain a gait evaluation result of the person to be evaluated.

[0011] In the above solution, the loss function of the first model is associated with normal gait, abnormal gait and the evaluation results output by the first model.

[0012] In the above solution, the feature extraction of the human body features based on the local covariance matrix includes:

[0013] constructing a first matrix using the body characteristics of the person to be evaluated;

[0014] constructing a second matrix using the first matrix, wherein the second matrix includes a local covariance matrix;

[0015] The second matrix is ​​used to obtain motion features of the person to be evaluated, where the motion features represent the motion range of the person to be evaluated.

[0016] In the above solution, the use of the second matrix to obtain the motion characteristics of the person to be evaluated includes:

[0017] Reconstructing a weight matrix using the second matrix;

[0018] The weight matrix is ​​used to determine the motion characteristics of the person to be evaluated from the human body characteristics.

[0019] In the above solution, the feature extraction of the image data to obtain the body features of the person to be evaluated includes:

[0020] Using a difference method and the image data, segmenting the image region to obtain image region data containing the person to be evaluated;

[0021] A second model is used to perform feature extraction on image region data containing the person to be evaluated to obtain body features of the person to be evaluated. The second model is used to detect key points of body posture. The second model includes a feature pyramid network (FPN).

[0022] In the above solution, the feature extraction of the human body features based on the local covariance matrix includes:

[0023] performing noise removal processing on the human body features;

[0024] Based on the local covariance matrix, feature extraction is performed on the human body features after noise removal.

[0025] In the above solution, the detection area includes a circular area.

[0026] In the above solution, the acquiring of image data includes:

[0027] Acquiring the image data through at least two cameras;

[0028] The at least two cameras are arranged around the detection area.

[0029] The present application also provides a gait assessment system, comprising:

[0030] an acquisition device for acquiring image data, wherein the image data includes data of the person to be evaluated moving in the detection area;

[0031] The gait assessment device is used to obtain the image data; perform feature extraction on the image data to obtain the human body features of the person to be assessed; perform feature extraction on the human body features based on a local covariance matrix to obtain the motion features of the person to be assessed; and input the motion features into a first model to obtain a gait assessment result of the person to be assessed.

[0032] In the above system, the collection device includes at least two cameras, and the two cameras are arranged around the detection area to form a ring-shaped collection point.

[0033] The present application also provides a gait assessment device, comprising:

[0034] an acquisition module, configured to acquire image data, wherein the image data includes data of the person to be assessed moving in the detection area;

[0035] A human body feature extraction module, configured to extract features from the image data to obtain the human body features of the person to be evaluated;

[0036] A motion feature extraction module, configured to extract the human body features to obtain the motion features of the person to be evaluated;

[0037] The output module is used to input the motion features into the first model to obtain the gait assessment result of the person to be assessed.

[0038] An embodiment of the present application further provides an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor,

[0039] The processor is configured to execute the steps of any of the above methods when running the computer program.

[0040] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0041] An embodiment of the present application further provides a computer program product, comprising a computer program, which implements the steps of any of the above methods when executed by a processor.

[0042] The gait assessment method, system, apparatus, related equipment, storage medium and computer program product provided in the embodiments of the present application obtain image data, wherein the image data includes data of the person to be assessed moving in the detection area; perform feature extraction on the image data to obtain the body features of the person to be assessed; perform feature extraction on the body features based on the local covariance matrix to obtain the motion features of the person to be assessed; input the motion features into a first model to obtain the gait assessment result of the person to be assessed. The solution provided in the embodiments of the present application obtains image data of the person to be assessed moving, extracts the body features in the image data, further uses the local covariance matrix to extract the motion features in the body features, uses the model to analyze the motion features, and obtains the gait assessment result of the person to be assessed. In this way, firstly, no human intervention is required, and gait assessment is fully automatic with high accuracy; secondly, the image data of the person to be assessed is convenient to collect and the solution is easy to implement; thirdly, the amount of motion feature data obtained by local covariance extraction is small, so the computing power and time cost of model analysis using motion features are low, and the assessment efficiency is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the gait assessment method according to an embodiment of the present application;

[0044] Figure 2 A flowchart illustrating an example gait assessment method for application in this application;

[0045] Figure 3 This is a schematic diagram of the structure of the gait assessment system according to an embodiment of the present application;

[0046] Figure 4 This is a schematic structural diagram of a gait assessment device according to an embodiment of the present application;

[0047] Figure 5 This is a schematic diagram of the electronic device structure of an embodiment of the present application. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Currently, hospitals and medical systems are required to assess children's motor health. Related technologies involve having children walk in person and then having experienced medical staff score them. Alternatively, they use a video of the child's movements, extract key frames from the video, and then have experts remotely perform manual judgments to determine whether the child's gait is normal. Because the gait assessment process requires human participation, the results may not accurately reflect the child's actual movement status and movement type, and it consumes a lot of manpower and computing resources.

[0050] In related technologies, in order to reduce manual participation in the gait assessment process, a gait assessment method for children with cerebral palsy based on a filter group is proposed. Specifically, the child can wear an electromyography acquisition device to obtain the child's electromyography information (which can also be understood as the original signal of surface electromyography) through the electromyography acquisition device; then the electromyography information is filtered and feature extracted through the filter group, and the gait assessment result is obtained by classifying the extracted features.

[0051] However, in actual use, children are afraid of electromyographic connections and are not easy to cooperate with, and the solutions in related technologies have major application defects.

[0052] Based on this, in various embodiments of the present application, image data of the person to be evaluated is obtained, human features in the image data are extracted, and motion features in the human features are further extracted using the local covariance matrix. The motion features are analyzed using a model to obtain a gait evaluation result of the person to be evaluated. In this way, on the one hand, no human participation is required, and gait evaluation is achieved fully automatically with high accuracy; on the other hand, the image data of the person to be evaluated is convenient to collect, and the solution is easy to implement; on the third hand, the amount of motion feature data obtained by using local covariance extraction is small. In this way, the computing power and time cost of using motion features for model analysis are low, and the evaluation efficiency is high.

[0053] The present application embodiment provides a gait assessment method, such as Figure 1 As shown, the method includes:

[0054] Step 101: Acquire image data, wherein the image data includes data of the person to be evaluated moving in the detection area;

[0055] Step 102: extracting features from the image data to obtain the body features of the person to be evaluated;

[0056] Step 103: extracting the human body features based on the local covariance matrix to obtain the motion features of the person to be evaluated;

[0057] Step 104: Input the motion features into the first model to obtain a gait assessment result of the person to be assessed.

[0058] Here, in actual application, the method of the embodiment of the present application is applied to electronic devices, such as smart phones, tablet computers, large-screen devices, etc. As long as it is a device that can implement the solution of the embodiment of the present application, the embodiment of the present application does not limit this.

[0059] In step 101, the electronic device may collect the image data for the detection area through a collection device. The collection device may specifically be a camera. The embodiment of the present application does not limit the method of obtaining the image data.

[0060] The detection area may include an area where the person to be evaluated performs exercise. Specifically, in one embodiment, the detection area may include a circular area.

[0061] That is, the person to be tested moves within the circular area where image acquisition is performed, so that the acquisition device can obtain image data of the person to be tested. The size of the circular area can be specifically set according to the actual situation of the person to be tested. For example, the size of the circular area can be adjusted according to the age and body shape of the child to be tested. For example, the adjustment method may include: pre-setting a standard age and body shape, and isometrically enlarging or reducing the size of the child to be tested based on the size relationship between the age and body shape of the child to be tested and the set standard value. This embodiment of the application is not limited to this.

[0062] In actual application, cameras may be arranged around the circular area, and the cameras may be used to collect motion data of the person to be evaluated within the circular area, thereby obtaining the image data.

[0063] Based on this, in one embodiment, the specific implementation of step 101 may include:

[0064] Acquiring the image data through at least two cameras;

[0065] The at least two cameras are arranged around the detection area.

[0066] Here, in actual application, the at least two cameras can be set (also understood as deployed) according to actual needs to ensure that image data from multiple angles can be obtained. By performing subsequent steps to analyze and process the image data from multiple angles, a more accurate gait assessment result of the person to be assessed can be obtained. Specifically, the at least two cameras can be set at the edge of the circular area. When the number of cameras set is greater than or equal to 3, the at least three cameras set can form a ring-shaped collection point. In this way, the at least three cameras can be used to obtain image data from multiple angles. The spacing between the at least three cameras can be set according to actual needs, for example, equidistantly, and this embodiment of the application is not limited to this.

[0067] In actual application, after obtaining the image data, the image area of ​​the person to be evaluated in the image data (which can also be understood as the foreground image or foreground part) can be extracted, thereby eliminating the part irrelevant to the person to be evaluated (which can also be understood as the background image or background part). In this way, only the image area of ​​the person to be evaluated in the image data is extracted, and there is no need to consider the part irrelevant to the person to be evaluated. More accurate feature extraction results can be obtained, while reducing the amount of calculation, and the human body characteristics of the person to be evaluated can be obtained more quickly and effectively.

[0068] Based on this, in one embodiment, the specific implementation of step 102 may include:

[0069] Using a difference method and the image data, segmenting the image region to obtain image region data containing the person to be evaluated;

[0070] The second model is used to perform feature extraction on the image region data containing the person to be evaluated to obtain the body features of the person to be evaluated. The second model is used to detect key points of human posture, and the second model includes FPN.

[0071] In practical applications, the difference method may specifically include an inter-frame difference method (also referred to as a frame difference method or an inter-frame difference algorithm) and a background difference method (also referred to as a background difference method or a background difference algorithm).

[0072] Specifically, the image area can be segmented using the inter-frame difference method and the image data, which can specifically include: calculating the difference between two adjacent frames of image in the image data at each pixel point, and using the difference and a preset threshold to determine a differential image, thereby achieving segmentation of the image area.

[0073] For example, assuming that N frames of images are obtained during the execution of step 101 (where N is an integer greater than or equal to 2), the image region is segmented using the inter-frame difference method. Specifically, the method may include: first determining a difference image (also referred to as an inter-frame difference image or a frame difference image) using the N frames of images and formula (1):

[0074]

[0075] Among them, BW(x,y) i represents the pixel value of the differential image corresponding to the i-th frame image at the coordinate (x, y) (where i can be an integer greater than or equal to 2 and less than or equal to N, where N represents the number of frames contained in the image data), P0 represents the initial null value (i.e., a preset fixed value, specifically 0), and f i (x,y) represents the pixel value of the i-th frame image at the coordinate (x,y), f i-1(x,y) represents the pixel value at coordinates (x,y) of the i-1th frame image, and T represents a preset threshold value, which can be set according to actual needs. In other words, if the difference between the same pixel value in the i-th frame image and the i-1th frame image is greater than or equal to the preset threshold, then the pixel value in the differential image corresponding to the i-th frame image is set to "1" (which can also be understood as setting the grayscale level to "255"); if the difference between the same pixel value in the i-th frame image and the i-1th frame image is less than the preset threshold, then the pixel value in the differential image corresponding to the i-th frame image is set to the initial null value. In this way, a binary image can be obtained, and the portion of the binary image consisting of pixels with a value of "1" is used as the image region data containing the object to be evaluated, thereby achieving image region segmentation.

[0076] In addition, the image area can be segmented using background difference method and image data.

[0077] For example, assuming that N frames of images are obtained during the execution of step 101 (where N is an integer greater than or equal to 2), the image region is segmented using the background difference method, which may specifically include: first using the N frames of images and formula (2) to determine the background sequence image:

[0078]

[0079] Among them, B i (x, y) represents the pixel value of the background sequence image corresponding to the i-th frame image at the coordinate (x, y) (wherein, the value of i can be an integer greater than or equal to 2 and less than or equal to N, where N represents the number of frames contained in the image data), P0 represents the initial null value (i.e., a preset fixed value, specifically 0), and f i (x,y) represents the pixel value of the i-th frame image at the coordinate (x,y), f i-1 (x,y) represents the pixel value of the i-1th frame image at the coordinates (x,y), and T represents the preset threshold value, which can be set according to actual needs. That is, if the difference between the same pixel value in the i-th frame image and the i-1th frame image is greater than the preset threshold value, then in the background sequence image corresponding to the i-th frame image, the pixel value (also known as the pixel value of the moving foreground part) is set to the initial null value; if the difference between the same pixel value in the i-th frame image and the i-1th frame image is less than or equal to the preset threshold value, then in the background sequence image corresponding to the i-th frame image, the pixel value (also known as the pixel value of the background part) is set to the corresponding pixel value in the i-th frame image.

[0080] After determining the background sequence image, for each pixel with coordinates (x, y), count the number of times the pixel value is not the initial null value. The number can be represented by m(x, y). In this way, the background image can be determined using formula (3) using the number and the background sequence image:

[0081]

[0082] Among them, B(x,y) represents the pixel value of the background image at the coordinate (x,y), m(x,y) represents the number of times the pixel value at the coordinate (x,y) is not the initial null value, N represents the number of frames contained in the image data, and B i (x, y) represents the pixel value at coordinates (x, y) of the background sequence image corresponding to the i-th frame (where i can be an integer greater than or equal to 2 and less than or equal to N). Thus, for each pixel, the average value of the pixel values ​​in the background sequence image when the pixel value is not null is calculated, and the average value is used as the pixel value in the background image phase to obtain the background image.

[0083] After obtaining the background image, each frame image and the background image can be used to segment the image region using formula (4):

[0084] D i (x,y)=|f i (x,y)-B(x,y)| (4)

[0085] Wherein, represents the segmented image (also understood as background difference) after the image region is segmented for the i-th frame image (wherein, the value of i can be an integer greater than or equal to 2 and less than or equal to N, where N represents the number of frames contained in the image data), f i (x,y) represents the pixel value of the i-th frame at coordinates (x,y), and B(x,y) represents the pixel value of the background image at coordinates (x,y). By calculating the difference between each pixel value in the i-th frame and the background image, the resulting segmented image shows that the pixel values ​​of the background portion remain almost unchanged between the two images and are close to 0. However, the pixel values ​​of the foreground portion (i.e., the image region corresponding to the subject to be evaluated) differ significantly from those in the background image and are not close to 0.

[0086] Furthermore, the segmented image can be subjected to image binarization processing, which specifically may include: setting a preset threshold value based on actual needs, and if a pixel value in the segmented image is greater than the preset threshold value, setting the pixel value to "1" (which can also be understood as the grayscale level being set to "255"); if a pixel value in the segmented image is less than or equal to the preset threshold value, setting the pixel value to "0" (which can also be understood as the grayscale level being set to "0"), thereby obtaining a binary image (also referred to as a target binary image), wherein the portion of the binary image composed of pixels with a pixel value of "1" is the image area data containing the person to be evaluated, and in this way, the image area can be segmented.

[0087] As described above, by segmenting the image area, image area data containing the person to be evaluated is obtained, so that feature extraction can be performed on the image area data containing the person to be evaluated, and the image area that does not contain the person to be evaluated (which can also be understood as the background area) is excluded, which can reduce the difficulty of feature extraction, reduce the time and computing power overhead of feature extraction, and at the same time, make the feature extraction results more accurate.

[0088] In actual application, the second model is used to extract features from the image region data containing the person to be evaluated to obtain the body features of the person to be evaluated, wherein the body features of the person to be evaluated may specifically include features of the head, shoulders, torso, and legs of the person to be evaluated.

[0089] In actual application, the second model can specifically include a human posture key point detection model based on FPN. The second model can be used to extract complex high-order features in the image data, and upsample the extracted high-order features, and then combine the upsampling results with the underlying features. The human posture key points in the image data (i.e., the human features) are obtained by analyzing the combined data.

[0090] Specifically, the key points of the human body posture may include the left eye, right eye, left ear, right ear, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left waist, right waist, left knee, right knee, left ankle, right ankle, etc. of the person to be evaluated.

[0091] For example, vector R = [x1, x2, ..., x18] and vector G = [y1, y2, ..., y18] can be defined, where each item in vector R represents the horizontal coordinate of a pixel of a human posture key point in the image region data containing the person to be evaluated, and each item in vector G represents the vertical coordinate of a pixel of a human posture key point in the image region data containing the person to be evaluated.

[0092] The positioning of head features can include left eye, right eye, left ear, right ear, and nose, corresponding to pixel coordinates [x1, x2, x3, x4, x5], [y1, y2, y3, y4, y5];

[0093] The positioning of shoulder and torso features may include neck, left shoulder, right shoulder, left waist, and right waist, corresponding to pixel coordinates [x6, x7, x8, x13, x14], [y6, y7, y8, y13, y14];

[0094] The positioning of leg features may include the left knee, right knee, left ankle, and right ankle, corresponding to pixel coordinates [x15, x16, x17, x18], [y15, y16, y17, y18];

[0095] The positioning of hand features may include the left elbow, right elbow, left wrist, and right wrist, corresponding to pixel coordinates [x9, x10, x11, x12], [y9, y10, y11, y12].

[0096] When the human body posture key point detection model is used for feature extraction, the human body posture key point detection model can be used to first determine the feature area corresponding to the torso of the person to be evaluated in the image area data. Based on the feature area corresponding to the torso (which can also be called feature position or feature positioning), feature area positioning is performed around the feature area corresponding to the torso (which can also be understood as determining the position of the feature area, which can also be called positioning search), thereby determining all the feature areas of the person to be evaluated. For example, after determining the feature area corresponding to the torso, the feature area corresponding to the head can be searched and determined around the torso (the specific method can include performing feature matching calculations, etc.), and then based on the feature areas corresponding to the head and torso, the feature areas corresponding to the remaining body parts (such as hands, legs, etc.) can be obtained. Specifically, the position of the opposite extension of the line connecting the torso and the head can be defined as the feature area corresponding to the legs. In this way, feature extraction can be quickly achieved, and the coordinates (which can also be understood as positions) of all human body posture key points (i.e., human features) can be obtained.

[0097] In practical applications, in order to make the extracted human features more accurate, in one embodiment, noise removal processing may be performed on the human features.

[0098] Specifically, the extracted coordinate information of the key points of the human body posture may be subjected to normal distribution noise removal processing to obtain the human body features after the noise removal processing.

[0099] In actual application, when executing step 103, feature extraction can be performed on the human body features after noise removal based on the local covariance matrix. In this way, features can be identified more accurately, thereby reducing feature extraction errors.

[0100] In practical applications, after obtaining the human features corresponding to each frame of image (specifically, each image region containing the subject to be evaluated), the relationship between the human features and time (specifically, represented by the number of frames) can be further determined, that is, the motion trajectory of the human features. For example, the coordinate position of the subject's leg features in each frame of image can be used to determine the motion trajectory of the subject's legs during the time corresponding to the image data (which can also be understood as the motion range, specifically including the coordinate change trend of key points of the human posture). The motion trajectory of the subject's legs can then be used to evaluate the subject's gait and obtain the corresponding gait assessment results.

[0101] In actual application, human body features can be extracted to obtain corresponding motion features (also called motion information features). The motion features represent the coordinate change trend of key points of human posture. Compared with human body features, the amount of data is small. In this way, the computing power and time cost in the subsequent evaluation process can be reduced, and the evaluation efficiency can be improved.

[0102] Specifically, in one embodiment, the specific implementation of step 103 may include:

[0103] constructing a first matrix using the body characteristics of the person to be evaluated;

[0104] constructing a second matrix using the first matrix, wherein the second matrix includes a local covariance matrix;

[0105] The second matrix is ​​used to obtain motion features of the person to be evaluated, where the motion features represent the motion range of the person to be evaluated.

[0106] Here, in actual application, a corresponding first matrix can be constructed for each human feature of the person to be evaluated, and each item in the first matrix can specifically include the coordinates of the human feature and the corresponding frame number. Exemplarily, the data format of any item in the first matrix can be (x, y, f), where f represents the frame number corresponding to the item, x represents the horizontal coordinate of the human feature in frame f, and y represents the vertical coordinate of the human feature in frame f.

[0107] At this time, since each data item in the first matrix contains three dimensions: horizontal coordinate, vertical coordinate and frame number, when extracting motion features from a large amount of data (which can also be understood as extracting the motion range of the human body features corresponding to the first matrix), the data of the first matrix can be processed by dimensionality reduction, thereby reducing the computing power and time required for feature extraction.

[0108] Specifically, for the first matrix, each data in the first matrix is ​​represented by T. It is assumed that the data in the first matrix are linear in a local range (it can also be understood as assuming that the movement range of a human feature in a local time period (number of frames) is linear). That is, as shown in formula (5), any data in the first matrix can be linearly represented by other adjacent data in the local range:

[0109]

[0110] Among them, T i represents the i-th item (also known as the i-th frame) data, Z(i) represents the k nearest neighboring data set of the i-th item data, T j Represents the jth data closest to it (j∈[1,k]∩j≠i), ω ij Indicates T j The corresponding weight coefficient. Among them, the weight coefficient needs to meet the normalization restriction, that is, the weight coefficient needs to meet

[0111] When reducing the data dimension of the first matrix, the projection of each data T in the low-dimensional space (i.e., the space after dimensionality reduction) and the projection of other data in its local range can be set to maintain the same linear relationship as before dimensionality reduction. That is, after the data dimensionality reduction, any dimensionality-reduced data can be linearly represented using almost the same weight coefficient as other dimensionality-reduced data.

[0112] Based on this, for the i-th data T in the first matrix i , you can first determine and T i The k nearest data (T i1 ,T i2 ,...T ik ); wherein, the value of k can be pre-set as needed, and the embodiment of the present application does not limit this; a method for determining the k nearest data may include selecting k data with the smallest Euclidean distance, or determining using a K-nearest neighbor (KNN, K-Nearest Neighbor) algorithm, and the embodiment of the present application does not limit this.

[0113] After determining the k nearest data, the local covariance matrix (also known as the second matrix) can be constructed by using the i-th data and the k nearest data in combination with formula (6):

[0114] Q i =(T i -T ij )(T i -T ij ) T (6)

[0115] Among them, Qi represents the local covariance matrix corresponding to the i-th data, T ij In this way, the local covariance matrix (i.e., the second matrix) can be used to obtain the motion features of the person to be evaluated, that is, to extract the body features of the person to be evaluated.

[0116] In one embodiment, the specific implementation of obtaining the motion characteristics of the person to be evaluated by using the second matrix may include:

[0117] Reconstructing a weight matrix using the second matrix;

[0118] The weight matrix is ​​used to determine the motion characteristics of the person to be evaluated from the human body characteristics.

[0119] Specifically, formula (5) can be used in combination with a gradient descent algorithm (specifically, a Lagrange multiplication optimization algorithm) to obtain formula (7), which is used to calculate the weight coefficient vector corresponding to the i-th data:

[0120]

[0121] Among them, W i =(ω i1 ,ω i2 ,...ω ik ) T Represents the weight coefficient vector corresponding to the i-th data, and each item in the weight coefficient vector represents the weight coefficient of one of the k nearest data; 1 k Represents a k-dimensional all-one vector. Here, it should be noted that in order to ensure the reversibility of the local covariance matrix, the local covariance matrix can be regularized using formula (8) after constructing it:

[0122] Q i =Q i +rI (8)

[0123] Where r represents the regularization parameter and I represents the k-dimensional identity matrix. The local covariance matrix after regularization is reversible and can meet the needs of calculating the weight coefficient vector in formula (7).

[0124] In this way, the weight coefficient vector corresponding to the i-th data can be obtained by using formula (6) and formula (7). Combining the weight coefficient vector corresponding to each data item can obtain the weight matrix (also called weight coefficient matrix) W = (W1, W2, ..., W m ), where m represents the total number of items in the first matrix.

[0125] The weight coefficient matrix can be further used to construct the matrix M through formula (9), and the first d+1 eigenvalues ​​of the matrix M and the corresponding eigenvectors {y1, y2, ..., y d+1}, where d represents the dimension of the data of the first matrix after dimensionality reduction, which can be two-dimensional (ie, d=2), and each item in the eigenvector is 1xm-dimensional data.

[0126] M=(IW)(IW) T (9)

[0127] Where I represents the k-dimensional identity matrix.

[0128] The eigenvectors from the 2nd to the d+1th item can be spanned to obtain the reduced-dimensional data matrix D = (y2, ..., y d+1 ). In this way, when each item in the first matrix is ​​three-dimensional data of a human feature, the above steps can be used to reduce the m three-dimensional data in the first matrix into m two-dimensional data, and use the m two-dimensional data to represent the motion trajectory of the human feature. Specifically, each row in the matrix D = (y2, y3) corresponds to a two-dimensional data, and the two-dimensional data can be represented by (X h ,Y h ), where h can be an integer between 1 and m.

[0129] In order to facilitate the analysis of the motion trajectory represented by the m two-dimensional data after dimensionality reduction, the m two-dimensional data can be fitted, and the fitting weight (or a series of two-dimensional data determined according to the fitting weight) can be used to represent the motion characteristics of the person to be evaluated.

[0130] Specifically, as shown in formula (10), the m two-dimensional data can be fitted in combination with Lagrange multiplication, and regularization can be used to avoid overfitting:

[0131]

[0132] Among them, W 2D represents the fitting weight of two-dimensional data, m represents the number of data, || || F represents the F norm, represents the regularization term, and λ represents the least squares regularization parameter.

[0133] As described above, by executing the above steps and using the weight matrix, the motion characteristics of the person to be evaluated can be determined from the human body characteristics. The determined motion characteristics can then be input into the first model to obtain a gait evaluation result of the person to be evaluated.

[0134] In actual application, the first model may specifically include a neural network model, and the neural network model may include: convolution neurons, deconvolution neurons, recurrent neural networks, deep neural networks, etc.

[0135] Exemplarily, the first model can be a long short-term memory (LSTM) network. After the motion features of the person to be evaluated are input into the LSTM network, the LSTM network uses the input motion features to match the motion features corresponding to different gaits (specifically, normal gait, scissors gait, rooster gait, duck gait, jumping gait, etc.), and determines the motion features of the person to be evaluated based on the matching results (also called matching correlation).

[0136] In actual application, since the gait of the person to be evaluated is mainly reflected by the movements of the legs and waist of the person to be evaluated, in order to improve the recognition effect of the first model, when the obtained motion features are input into the first model, only the motion features corresponding to the human body features of the legs and waist of the person to be evaluated (i.e., the left knee, right knee, left ankle, right ankle, left waist, right waist and other human posture key points) can be input into the first model, thereby removing the influence of non-important human features (which can also be understood as non-important factors) and improving the evaluation effect of the first model. At the same time, since the number of input motion features is reduced, computing resources can be saved and evaluation efficiency can be improved.

[0137] In actual application, before using the first model, the first model needs to be trained to ensure the accuracy of the gait assessment results output by the first model. That is, in one embodiment, the method may further include: training the first model.

[0138] The loss function of the first model is associated with normal gait, abnormal gait and the evaluation results output by the first model.

[0139] After obtaining image data of the person to be evaluated moving in the detection area, the motion information corresponding to the image data can be obtained by executing steps 102-103, and the evaluation result r (i.e., the evaluation result output by the first model) can be obtained using the first model. At the same time, experienced experts can judge the image data to obtain a manual evaluation result r' (which can also be understood as the true value of the gait evaluation result, which can specifically include normal gait or abnormal gait). In this way, r and r' can be used, combined with the loss function of the first model, to train the first model, thereby obtaining a first model with more accurate gait evaluation results.

[0140] Exemplarily, the loss function may specifically include formula (11):

[0141] Loss=rlogr'+(1-r)log(1-r') (11)

[0142] Here, r represents the evaluation result of the first model, r' represents the manual evaluation result, and Loss represents the loss value. This simple loss function calculation method allows for better and faster convergence during model training, reduces training overhead, and accurately distinguishes (or measures) subtle differences.

[0143] Here, in actual application, r can be data with a value range of [0,1]. When the value of r is "1", it indicates that the gait of the person to be evaluated is abnormal, and when the value of r is "0", it indicates that the gait of the person to be evaluated is normal. Of course, it can also be that when the value of r is "0", it indicates that the gait of the person to be evaluated is abnormal, and when the value of r is "1", it indicates that the gait of the person to be evaluated is normal.

[0144] r' can be Boolean data. When the value of r' is "1", it indicates that the gait of the person to be evaluated is abnormal; when the value of r' is "0", it indicates that the gait of the person to be evaluated is normal. Of course, it can also be that when the value of r' is "0", it indicates that the gait of the person to be evaluated is abnormal; when the value of r' is "1", it indicates that the gait of the person to be evaluated is normal.

[0145] When training the first model, the training objectives include: by adjusting the parameters of the first model, making the calculated value of Loss as close to zero as possible; based on this, a preset threshold can be set according to actual needs. When the calculated value of Loss is less than or equal to the preset threshold, it is determined that the training of the first model is completed, that is, it can be considered that the motion features are processed using the trained first model to obtain accurate gait evaluation results.

[0146] In practical applications, the gait assessment results can be used to determine the recovery status of the person being assessed. The closer the gait assessment results are to a normal gait, the higher the person's recovery level can be; conversely, the closer the gait assessment results are to an abnormal gait, the lower the person's recovery level can be. The recovery status can be determined based on the assessment results and a preset threshold value, which can be expressed as a percentage. The preset threshold value can be set as needed and is not limited in this embodiment of the application.

[0147] The gait assessment method provided by the embodiment of the present application obtains image data, wherein the image data includes data of the person to be assessed moving in the detection area; performs feature extraction on the image data to obtain the human body features of the person to be assessed; performs feature extraction on the obtained human body features based on the local covariance matrix to obtain the motion features of the person to be assessed; inputs the obtained motion features into a first model to obtain the gait assessment result of the person to be assessed. The solution provided by the embodiment of the present application obtains image data of the person to be assessed moving, extracts human body features from the image data, further extracts motion features from the human body features using the local covariance matrix, analyzes the motion features using the model, and obtains the gait assessment result of the person to be assessed. In this way, firstly, no human intervention is required, and gait assessment is fully automatic with high accuracy; secondly, the image data of the person to be assessed is convenient to collect, and the solution is easy to implement; thirdly, the amount of motion feature data obtained by extracting the local covariance is small, so the computing power and time cost of model analysis using motion features are low, and the assessment efficiency is high.

[0148] The present application is described in further detail below with reference to application examples.

[0149] The gait assessment system of the application example of this application includes: a data acquisition module, an image segmentation module, a feature extraction module, a noise processing module, a motion information feature acquisition module, and a matching output module. It has the advantages of high gait recognition accuracy, high assessment efficiency, and easy implementation.

[0150] like Figure 2 As shown, the process of gait assessment by the gait assessment system includes the following steps:

[0151] Step 201: capturing an image of a person to be assessed in a detection area;

[0152] In practical applications, the data acquisition module can be used to capture (or also be understood as acquiring) images (i.e., the aforementioned image data) of a subject to be evaluated (specifically, a child patient or sick child) within the detection area. The detection area can be set as a circular shooting area, the size of which can be set based on the age (or height, body shape, etc.) of the subject to be evaluated. The shape of the circular area can specifically be a traditional isosceles runway with a diameter range of 3 to 5 meters.

[0153] The data acquisition module may specifically include multiple sets of high-definition recording cameras arranged around a circular shooting area, and the angle division may be specifically evenly divided. In this way, when capturing an image of the person to be evaluated, the multiple sets of high-definition recording cameras capture video (i.e., image data) of the person to be evaluated from multiple angles.

[0154] Step 202: Segment the image using a difference method to obtain an image region of the person to be evaluated;

[0155] In practical applications, an image segmentation module can be used to perform region segmentation using a difference method to obtain the image region of the person to be evaluated. The difference method can specifically include an inter-frame difference method and a background difference method.

[0156] Step 203: determining shoulder and torso features of the person to be evaluated in the image region of the person to be evaluated;

[0157] In actual application, the feature extraction module can be used to locate the feature areas of the segmented image area of ​​the person to be evaluated. Specifically, it can include first locating the torso feature area of ​​the person to be evaluated in the image area of ​​the person to be evaluated, then locating the head feature area around the torso feature area, and then using the head feature area and the torso feature area, combined with the correlation between the various feature areas of the human body, to locate all the feature areas of the person to be evaluated (specifically including the head feature area, shoulder and torso feature areas, leg feature areas, hand feature areas, etc.).

[0158] Step 204: Acquire human body motion characteristics;

[0159] In practical applications, after locating the characteristic regions of the subject to be evaluated, a feature extraction module can be used to obtain human motion features (i.e., the aforementioned human features). Specifically, a human posture key point detection model based on FPN can be used, combining high-level and low-level features of the image, to determine the subject's left eye, right eye, left ear, right ear, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left waist, right waist, left knee, right knee, left ankle, and right ankle.

[0160] In practical applications, the corresponding human posture key points can be combined and judged based on the human feature area, which can more quickly obtain the human motion characteristics of the person to be evaluated.

[0161] Step 205: performing noise removal processing on the human body motion features;

[0162] In practical applications, after obtaining the human motion features of the person to be evaluated, the noise processing module can be used to perform normal distribution noise processing on the coordinate points corresponding to the human motion features, thereby improving the feature recognition ability during feature extraction and reducing the error of feature extraction.

[0163] Step 206: extracting motion information features;

[0164] In practical applications, the motion information feature extraction module can be used to extract the human motion features after noise removal to obtain motion information features (i.e., the above-mentioned motion features). Specifically, the original matrix can be constructed based on the human motion features, and at the same time, the local covariance matrix can be constructed based on the original matrix. In this way, the changes in the corresponding human feature information in the local covariance matrix can be used to characterize (also understood as mapping) the motion information features corresponding to the original matrix. Among them, the local covariance matrix can be used to reconstruct the weight matrix, and the weight matrix and the local covariance matrix can be used to determine the corresponding human feature information in the local covariance matrix.

[0165] Step 207: Input the motion information features into the neural network model to obtain a matching correlation; and determine the gait assessment result of the person to be assessed based on the correlation.

[0166] In actual application, the matching output module can be used to input the extracted motion information features into the neural network model. The neural network model performs correlation matching on the motion information features and the standard gait features (which can also be understood as reference features or reference features), outputs the matching correlation results, and determines the gait assessment results of the person to be assessed based on the correlation results. Then, the rehabilitation status of the person to be assessed can be determined (which can also be understood as judgment) based on the gait assessment results. Among them, the standard gait features can include preset physical feature thresholds, or motion information features obtained by image processing of images corresponding to standard gaits (including normal gait, scissors gait, cock gait, duck gait, and jumping gait, etc.).

[0167] The neural network model is a trained neural network model. The specific implementation of training the neural network model may include: obtaining a large number of images of children walking, extracting motion information features corresponding to the images, manually evaluating the motion information features by experienced doctors and experts, judging the rehabilitation status of the children corresponding to the images, and using the manually evaluated rehabilitation status and motion information features as training sets to train the neural network model.

[0168] In practical applications, only the motion information features corresponding to the waist and legs of the person being evaluated can be input into the neural network model, thereby removing human features that are not important for gait assessment (specifically, these may include head features). This can improve the recognition effect of the neural network model while saving computing resources.

[0169] In the application example of the present application, image data of the movement of the person to be evaluated is obtained, human motion features in the image data are extracted, and the motion information features in the human features are further extracted using the local covariance matrix. The motion information features are analyzed using a neural network model to obtain the gait evaluation results of the person to be evaluated. In this way, on the one hand, no human participation is required, and gait evaluation is realized automatically with high accuracy; on the other hand, the image data of the person to be evaluated is convenient to collect, and the solution is easy to implement; on the third hand, the amount of motion information feature data obtained by extracting the local covariance is small. In this way, the computing power and time cost of model analysis using motion features are low, and the evaluation efficiency is high.

[0170] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a gait assessment system, such as Figure 3 As shown, the system includes:

[0171] An acquisition device 301 is used to acquire image data, wherein the image data includes data of the person to be evaluated moving in the detection area;

[0172] The gait assessment device 302 is used to obtain the image data; perform feature extraction on the image data to obtain the human features of the person to be assessed; perform feature extraction on the human features based on the local covariance matrix to obtain the motion features of the person to be assessed; and input the motion features into the first model to obtain the gait assessment result of the person to be assessed.

[0173] In one embodiment, the gait assessment device 302 is specifically configured to:

[0174] constructing a first matrix using the body characteristics of the person to be evaluated;

[0175] constructing a second matrix using the first matrix, wherein the second matrix includes a local covariance matrix;

[0176] The second matrix is ​​used to obtain motion features of the person to be evaluated, where the motion features represent the motion range of the person to be evaluated.

[0177] In one embodiment, the gait assessment device 302 is specifically configured to:

[0178] Reconstructing a weight matrix using the second matrix;

[0179] The weight matrix is ​​used to determine the motion characteristics of the person to be evaluated from the human body characteristics.

[0180] In one embodiment, the gait assessment device 302 is specifically configured to:

[0181] Using an inter-frame difference method and the image data, the image region is segmented to obtain image region data containing the person to be evaluated;

[0182] The second model is used to extract features from the image region data containing the person to be evaluated to obtain the body features of the person to be evaluated. The second model is used to detect key points of human posture, and the second model includes an FPN network.

[0183] In one embodiment, the gait assessment device 302 is specifically configured to:

[0184] performing noise removal processing on the human body features;

[0185] Based on the local covariance matrix, feature extraction is performed on the human body features after noise removal.

[0186] In one embodiment, the collection device 301 includes at least two cameras, which are arranged around the detection area to form a ring-shaped collection point.

[0187] Here, it should be noted that the function of the acquisition device 301 is equivalent to the function of the data acquisition module in the application example; the function of the gait evaluation device 302 is equivalent to the function of the image segmentation module, feature extraction module, noise processing module, motion information feature acquisition module, and matching output module in the application example.

[0188] In order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a gait assessment device, which is set on an electronic device, such as Figure 4 As shown, the device includes:

[0189] An acquisition module 401 is used to acquire image data, wherein the image data includes data of the person to be evaluated moving in the detection area;

[0190] A human body feature extraction module 402 is used to extract features from the image data to obtain human body features of the person to be evaluated;

[0191] A motion feature extraction module 403 is used to extract the human body features to obtain the motion features of the person to be evaluated;

[0192] The output module 404 is configured to input the motion features into the first model to obtain a gait assessment result of the person to be assessed.

[0193] In one embodiment, the motion feature extraction module 403 is specifically configured to:

[0194] constructing a first matrix using the body characteristics of the person to be evaluated;

[0195] constructing a second matrix using the first matrix, wherein the second matrix includes a local covariance matrix;

[0196] The second matrix is ​​used to obtain motion features of the person to be evaluated, where the motion features represent the motion range of the person to be evaluated.

[0197] In one embodiment, the motion feature extraction module 403 is specifically configured to:

[0198] Reconstructing a weight matrix using the second matrix;

[0199] The weight matrix is ​​used to determine the motion characteristics of the person to be evaluated from the human body characteristics.

[0200] In one embodiment, the human feature extraction module 402 is specifically configured to:

[0201] Using a difference method and the image data, segmenting the image region to obtain image region data containing the person to be evaluated;

[0202] The second model is used to extract features from the image region data containing the person to be evaluated to obtain the body features of the person to be evaluated. The second model is used to detect key points of human posture, and the second model includes an FPN network.

[0203] In one embodiment, the human feature extraction module 402 is specifically configured to:

[0204] performing noise removal processing on the human body features;

[0205] Based on the local covariance matrix, feature extraction is performed on the human body features after noise removal.

[0206] In one embodiment, the acquisition module 401 is specifically configured to:

[0207] Acquiring the image data through at least two cameras;

[0208] The at least two cameras are arranged around the detection area.

[0209] Here, it should be noted that the function of the acquisition module 401 is equivalent to the function of the data acquisition module in the application example; the function of the human feature extraction module 402 is equivalent to the function of the image segmentation module, feature extraction module, and noise processing module in the application example; the function of the motion feature extraction module 403 is equivalent to the function of the motion information feature acquisition module in the application example; the function of the output module 404 is equivalent to the function of the matching output module in the application example.

[0210] In actual application, the acquisition module 401 can be implemented by the communication interface in the gait evaluation device; the human feature extraction module 402 and the motion feature extraction module 403 can be implemented by the processor in the gait evaluation device; and the output module 404 can be implemented by the processor in the gait evaluation device combined with the communication interface.

[0211] It should be noted that the gait assessment device provided in the above embodiment is merely illustrated by the division of the aforementioned program modules when performing gait assessment. In actual applications, the aforementioned processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the aforementioned processing. Furthermore, the gait assessment device provided in the above embodiment and the gait assessment method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0212] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides an electronic device, such as Figure 5 As shown, the electronic device 500 includes:

[0213] Communication interface 501, capable of interacting with other devices;

[0214] The processor 502 is connected to the communication interface 501 to interact with other devices and is used to execute the methods provided by one or more of the above technical solutions when running a computer program; the computer program is stored in the memory 503.

[0215] Specifically, the processor 502 is configured to:

[0216] Acquiring image data through the communication interface 501, wherein the image data includes data of the person to be assessed moving in the detection area;

[0217] Performing feature extraction on the image data to obtain the body features of the person to be evaluated;

[0218] Extracting the human body features based on the local covariance matrix to obtain the motion features of the person to be evaluated;

[0219] The motion characteristics are input into the first model to obtain a gait evaluation result of the person to be evaluated.

[0220] In one embodiment, the processor 502 is further configured to:

[0221] constructing a first matrix using the body characteristics of the person to be evaluated;

[0222] constructing a second matrix using the first matrix, wherein the second matrix includes a local covariance matrix;

[0223] The second matrix is ​​used to obtain motion features of the person to be evaluated, where the motion features represent the motion range of the person to be evaluated.

[0224] In one embodiment, the processor 502 is specifically configured to:

[0225] Reconstructing a weight matrix using the second matrix;

[0226] The weight matrix is ​​used to determine the motion characteristics of the person to be evaluated from the human body characteristics.

[0227] In one embodiment, the processor 502 is specifically configured to:

[0228] Using a difference method and the image data, segmenting the image region to obtain image region data containing the person to be evaluated;

[0229] The second model is used to extract features from the image region data containing the person to be evaluated to obtain the body features of the person to be evaluated. The second model is used to detect key points of human posture, and the second model includes an FPN network.

[0230] In one embodiment, the processor 502 is further configured to:

[0231] performing noise removal processing on the human body features;

[0232] Based on the local covariance matrix, feature extraction is performed on the human body features after noise removal.

[0233] In one embodiment, the processor 502 is specifically configured to:

[0234] The image data is acquired through at least two cameras and a communication interface 501 ; the at least two cameras are arranged around the detection area.

[0235] It should be noted that the specific processing process of the processor 502 and the communication interface 501 can be understood by referring to the above method.

[0236] Of course, in actual application, the various components in the electronic device 500 are coupled together through the bus system 504. It can be understood that the bus system 504 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 04 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 504 .

[0237] The memory 503 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 500. Examples of such data include: any computer program used to operate on the electronic device 500.

[0238] The methods disclosed in the above embodiments of the present application can be applied to the processor 502 or implemented by the processor 502. The processor 502 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above methods can be completed by hardware integrated logic circuits in the processor 502 or instructions in software form. The above processor 502 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 502 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 503. The processor 502 reads the information in the memory 503 and completes the steps of the above methods in combination with its hardware.

[0239] In an exemplary embodiment, the electronic device 500 can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0240] It can be understood that the memory (memory 503) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0241] In an exemplary embodiment, the present application also provides a storage medium, namely, a computer storage medium, specifically a computer-readable storage medium, including, for example, a memory 503 storing a computer program. The computer program can be executed by a processor 502 of an electronic device 500 to perform the steps of any of the aforementioned methods. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0242] In an exemplary embodiment, the present application further provides a computer program product, including a stored computer program, which can be executed by the processor 502 of the electronic device 500 to complete the steps of any of the aforementioned methods.

[0243] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0244] In addition, the technical solutions described in the embodiments of the present application can be arbitrarily combined without conflict.

[0245] The above description is merely a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application.

Claims

1. A gait assessment method, characterized in that: include: Acquiring image data, wherein the image data includes data of the person to be evaluated moving in the detection area; Performing feature extraction on the image data to obtain the body features of the person to be evaluated; Extracting the human body features based on the local covariance matrix to obtain the motion features of the person to be evaluated; The motion characteristics are input into the first model to obtain a gait evaluation result of the person to be evaluated.

2. The method according to claim 1, characterized in that The loss function of the first model is associated with normal gait, abnormal gait and evaluation results output by the first model.

3. The method according to claim 1, characterized in that The extracting the human body features based on the local covariance matrix includes: constructing a first matrix using the body characteristics of the person to be evaluated; constructing a second matrix using the first matrix, wherein the second matrix includes a local covariance matrix; The second matrix is ​​used to obtain motion features of the person to be evaluated, where the motion features represent the motion range of the person to be evaluated.

4. The method according to claim 3, characterized in that The step of obtaining the motion characteristics of the person to be evaluated by using the second matrix includes: Reconstructing a weight matrix using the second matrix; The weight matrix is ​​used to determine the motion characteristics of the person to be evaluated from the human body characteristics.

5. The method according to claim 1, wherein The extracting features from the image data to obtain the body features of the person to be evaluated includes: Using a difference method and the image data, segmenting the image region to obtain image region data containing the person to be evaluated; A second model is used to extract features from image region data containing the person to be evaluated to obtain body features of the person to be evaluated. The second model is used to detect key points of body postures, and the second model includes a feature pyramid network.

6. The method according to any one of claims 1 to 5, characterized in that The detection area includes a circular area.

7. The method according to claim 6, characterized in that The acquiring of image data comprises: Acquiring the image data through at least two cameras; The at least two cameras are arranged around the detection area.

8. A gait assessment system, characterized in that: include: an acquisition device for acquiring image data, wherein the image data includes data of the person to be evaluated moving in the detection area; A gait assessment device is used to obtain the image data; perform feature extraction on the image data to obtain the body features of the person to be assessed; Based on the local covariance matrix, feature extraction is performed on the human body features to obtain motion features of the person to be evaluated; and the motion features are input into a first model to obtain a gait evaluation result of the person to be evaluated.

9. The system according to claim 8, characterized in that The collection device includes at least two cameras, which are arranged around the detection area to form a ring-shaped collection point.

10. A gait assessment device, characterized in that: include: an acquisition module, configured to acquire image data, wherein the image data includes data of the person to be assessed moving in the detection area; A human body feature extraction module, configured to extract features from the image data to obtain the human body features of the person to be evaluated; A motion feature extraction module, configured to extract the human body features to obtain the motion features of the person to be evaluated; The output module is used to input the motion features into the first model to obtain the gait assessment result of the person to be assessed.

11. An electronic device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method according to any one of claims 1 to 7.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images