Gait anomaly detection method and device, medium and product
By using a combined model of gait reconstruction module, gait prediction module and adaptive scoring module in gait abnormality detection, the problems of insufficient generalization and fine-grainedness in the existing technology are solved, and more efficient gait abnormality detection is achieved. Detection, capable of accurate detection at sequence level, frame level and joint point level.
Patent Information
- Application Number
- CN202411789183.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-01-21
AI Technical Summary
The existing gait anomaly detection methods have shortcomings in generalization and fine-graining, and cannot accurately detect abnormal gaits not included in the data set, and it is difficult to perform accurate detection at the frame level and joint node level.
The gait abnormal detection model includes gait reconstruction modules, gait prediction modules, and adaptive scoring modules, and input through single/multi -eye sequence gesture data to achieve abnormal detection of gait sequence levels, frame levels, and joint nodes.
It improves the generalization and fine particle size of gait abnormal detection, and can more accurately detect a variety of abnormal gait, including sequence levels, frame levels, and joint node levels, and enhance the accuracy and coverage of the detection.
Smart Images
Figure CN119326405A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of gait detection, and in particular to a method, device, medium and product for detecting gait abnormality. Background Art
[0002] Gait has a wide range of applications, such as gait recognition and gait abnormality detection. Gait abnormality detection focuses on identifying abnormalities in gait sequences. At present, gait abnormality detection has been widely used in the medical field, because many movement disorder-type diseases show typical pathological gait patterns. For example, the typical characteristics of Parkinson's gait abnormality are gait posture disorder and freezing gait. Therefore, gait abnormality detection not only needs to locate whether the human body has gait abnormalities during walking, but also needs to detect in which frame and which joint the gait abnormality occurs. Based on the above operations, the time required for doctors to diagnose can be greatly reduced, and even the time and location of the abnormality can be directly located to provide doctors with diagnostic references. It is widely used and important. So far, vision-based gait abnormality detection has attracted much attention due to its low cost and more convenient data collection.
[0003] The following aspects need to be considered in the gait anomaly detection method: the granularity of gait anomaly detection, the generalization of gait anomaly detection, and the data modality of gait anomaly detection. The granularity of gait anomaly detection refers to the level of depth of gait anomaly detection. Currently, the main research on gait anomaly detection is limited to the sequence level. This means that these studies can only determine whether the gait sequence is abnormal or evaluate the severity of abnormal gait sequences. The generalization of gait detection refers to the types of anomalies that gait anomaly detection methods can detect. Most supervised learning methods can only detect anomalies that already exist in the data set. The data modality of gait anomaly detection refers to the type of data used, such as two-dimensional posture data, three-dimensional posture data, point clouds, images, acceleration signals collected by wearable devices, etc.
[0004] In terms of data modality, especially when focusing on vision-based methods, 3D posture data derived from videos, inertial measurement data, and other data types show superior scalability. Moreover, for the representation of gait sequences, 3D posture contains more spatiotemporal information compared to 2D posture and profile, and can better reflect some subtle anomalies. In addition, compared to images and point clouds, 3D posture occupies very little computing resources and storage space.
[0005] In the gait anomaly detection method based on three-dimensional posture, early researchers classified gait sequences as normal or abnormal by extracting physical parameters from the gait sequence. Since 2016, some neural network-based methods such as recurrent neural networks, long short-term memory neural networks, and attention mechanisms have emerged to complete this task. Among them, in the human motion prediction model based on the attention mechanism, the prediction model is used to extract the hidden gait information during human walking, and then based on the above hidden information, the severity of Parkinson's disease is rated by a multi-layer perceptron. The advantage of this method is that it can extract hidden information from the gait sequence for subsequent classification by the multi-layer perceptron. The disadvantage is that the final goal only stays at the sequence level, and can only judge the severity of the gait sequence of a Parkinson's patient, but cannot judge where the abnormality occurs, nor can it detect abnormalities of other diseases. It has low generalization and insufficient granularity.
[0006] In order to perform anomaly detection at the frame level and realize the binary classification task of abnormal frames, a multi-layer perceptron can be used to reconstruct the 3D posture and train only on normal data. During testing, the anomaly is identified by judging the reconstruction error. The advantage of this method is that it uses a self-supervised approach, which increases generalization and granularity. However, the boundary between normal and abnormal is very vague and difficult to define, resulting in low recognition accuracy. Moreover, the biggest drawback of this method is that it cannot identify abnormal gaits of a time series type, such as freezing gait, and can only identify gaits that are relatively abnormal in space.
[0007] Based on the above description, there are still the following technical problems when the gait abnormality detection algorithm is applied in clinical diagnosis:
[0008] 1. Most vision-based gait anomaly detection methods use supervised learning algorithms. Supervised learning for gait anomaly detection refers to providing anomaly types of gait sequence data, allowing the model to learn the characteristics of specific anomaly type data, and thus classify each gait anomaly type. Currently available datasets only contain a limited number of abnormal gait sequences, which results in weak generalization capabilities based on supervised learning methods, and means that these studies cannot accurately detect anomalies not included in the dataset. Among them, supervised learning requires providing input data and its corresponding label data, and then building a model. After training, the model accurately finds the optimal mapping relationship between the input data and the label data, thereby predicting or classifying new unlabeled data. The training data used in unsupervised learning only contains input samples, without corresponding labels or targets.
[0009] 2. In clinical applications, gait abnormality detection requires not only detecting whether the gait sequence is abnormal, but also detecting which frame is abnormal and which joint point in the frame is abnormal. Currently, there are few abnormal gait datasets accurately labeled at the frame level, and most known methods are limited to the sequence level. Moreover, few works can achieve frame-level anomaly detection, and currently there is no method to achieve joint-level anomaly detection, and the accuracy is often low, which leads to a lack of fine-grainedness. Summary of the invention
[0010] The purpose of this application is to provide a gait abnormality detection method, device, medium and product, which have better generalization of abnormality detection and can also improve the granularity of gait abnormality detection.
[0011] To achieve the above objectives, this application provides the following solutions:
[0012] In a first aspect, the present application provides a method for detecting gait abnormality, comprising:
[0013] Acquire data to be detected; the data to be detected includes a color image, a depth map, and gait data collected by an inertial measurement unit;
[0014] Acquire the single / multi-eye sequence posture data to be detected based on the data to be detected;
[0015] Constructing a gait abnormality detection model; the gait abnormality detection model comprises: a gait reconstruction module, a gait prediction module and an adaptive scoring module; the gait reconstruction module and the gait prediction module are both connected to the adaptive scoring module;
[0016] The single / multi-eye sequence posture data is input into the gait abnormality detection model to obtain a gait abnormality detection result; the gait abnormality detection result includes a gait sequence level abnormality detection result, a frame level abnormality detection result and a joint point level abnormality detection result.
[0017] Optionally, a gait abnormality detection model is constructed, including:
[0018] Acquire gait data, and generate a sample data set based on the gait data;
[0019] Constructing an initial gait reconstruction model, an initial gait prediction model and the adaptive scoring module;
[0020] The sample data set is used to train and test the initial gait reconstruction model and the initial gait prediction model respectively, until the difference between the output data of the trained initial gait reconstruction model and the labeled data in the sample data set and the difference between the output data of the trained initial gait prediction model and the labeled data in the sample data set are both less than a set value, and the trained initial gait prediction model and the trained initial gait prediction model are used to respectively construct the gait reconstruction module and the gait prediction module;
[0021] The gait reconstruction module and the gait prediction module are both connected to the adaptive scoring module to obtain the gait abnormality detection model.
[0022] Optionally, the single / multi-eye sequence posture data is input into the gait abnormality detection model to obtain a gait abnormality detection result, including:
[0023] Inputting the single / multi-eye sequence posture data into the gait reconstruction module to obtain a gait reconstruction module result;
[0024] Inputting the single / multi-objective sequence posture data into the gait prediction module to obtain a gait prediction module result;
[0025] The adaptive scoring module is used to obtain the gait abnormality detection result based on the gait reconstruction module result and the gait prediction module result.
[0026] Optionally, the initial gait reconstruction model includes a first fully connected layer, an encoder, a memory pool, a decoder and a second fully connected layer.
[0027] Optionally, the gait prediction module includes: an encoder and a decoder.
[0028] Optionally, inputting the single / multi-eye sequence posture data into the gait reconstruction module to obtain a gait reconstruction module result includes:
[0029] Adding noise and mask to the single / multi-objective sequence posture data to obtain processed sequence data;
[0030] Input each frame in the processed sequence data into the first fully connected layer, extract the association between the joint points, and obtain a feature map;
[0031] Inputting the feature map into the encoder to obtain an encoding vector;
[0032] Inputting the encoding vector into the decoder to obtain a gait reconstruction model result;
[0033] The error between the gait reconstruction model result and the input data is determined to obtain the gait reconstruction module result.
[0034] Optionally, the adaptive scoring module is used to obtain the abnormal gait detection result based on the gait reconstruction module result and the gait prediction module result, including:
[0035] Determining an average value of the gait reconstruction module results and an average value of the gait prediction module results; the gait prediction module results are obtained using the gait prediction module;
[0036] The result with a smaller average value between the results of the gait reconstruction module and the results of the gait prediction module is taken as the first module result, and the result with a larger average value between the results of the gait reconstruction module and the results of the gait prediction module is taken as the second module result;
[0037] Determine whether the first module result is less than a set threshold;
[0038] When the result of the first module is less than the set threshold, the result of the second module is used as the gait abnormality detection result;
[0039] When the result of the first module is greater than or equal to the set threshold, fusing the result of the gait reconstruction module and the result of the gait prediction module to obtain fusion information;
[0040] Determining whether the fusion information is less than an anomaly detection threshold;
[0041] When the fusion information is less than the abnormality detection threshold, there is no gait abnormality detection result;
[0042] When the fused information is greater than or equal to the abnormality detection threshold, the fused information is used as the gait abnormality detection result.
[0043] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-described gait abnormality detection methods.
[0044] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned gait abnormality detection methods.
[0045] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned gait abnormality detection methods.
[0046] According to the specific embodiments provided in this application, this application has the following technical effects:
[0047] The present application provides a gait abnormality detection method, device, medium and product, which obtains gait abnormality detection results by adopting a gait abnormality detection model including a gait reconstruction module, a gait prediction module and an adaptive scoring module, and has better generalization of abnormality detection. By taking single / multi-objective sequence posture data as input, gait abnormality detection results including gait sequence level abnormality detection results, frame level abnormality detection results and joint point level abnormality detection results can be obtained, which can improve the granularity of gait abnormality detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0049] Figure 1 A flowchart of a gait abnormality detection method provided in one embodiment of the present application;
[0050] Figure 2 A schematic diagram of an implementation framework of a gait abnormality detection method provided in an embodiment of the present application;
[0051] Figure 3 A schematic diagram of the structure of a gait reconstruction model provided in one embodiment of the present application;
[0052] Figure 4 A schematic diagram of the gait prediction model structure provided in one embodiment of the present application. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0054] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0055] In an exemplary embodiment, the present application provides a method for detecting gait abnormality, which can be executed by a computer device, specifically, by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to a server as an example for illustration. Movement disorder manifestation diseases often show the characteristics of gait abnormality, so it is hoped that the gait abnormality detection algorithm can be used to assist in the diagnosis or automated diagnosis of movement disorder manifestation diseases. Figure 1 As shown, the gait abnormality detection method provided by the present application includes:
[0056] Step 100: Acquire the data to be detected. The data to be detected includes a color image, a depth map, and gait data collected by an inertial measurement unit or the like.
[0057] Step 101: Acquire single / multi-eye sequence posture data to be detected based on the data to be detected.
[0058] Step 102: Construct a gait abnormality detection model. Figure 2 As shown, the gait abnormality detection model includes: a gait reconstruction module, a gait prediction module and an adaptive scoring module. The gait reconstruction module and the gait prediction module are both connected to the adaptive scoring module.
[0059] For example, from the perspective of gait abnormalities in movement disorder manifestation diseases, gait abnormalities can be divided into spatially significant abnormalities (such as scissor gait and staggering gait) and temporally significant abnormalities (such as freezing gait). Figure 2 The gait reconstruction module and gait prediction module shown in the figure finally fuse the information through the adaptive scoring module to make abnormal judgment. Figure 2 In the figure, the symbol C in the green circle represents the splicing operation, which is used to splice the T frames generated by each sliding window in time sequence to obtain the prediction result of the entire sequence. Figure 2 In the figure, the symbol in the green circle represents the calculated average joint point error.
[0060] Step 103: Input the single / multi-eye sequence posture data into the gait anomaly detection model to obtain the gait anomaly detection result. The gait anomaly detection result includes the gait sequence level anomaly detection result, the frame level anomaly detection result and the joint point level anomaly detection result.
[0061] In another exemplary embodiment of the present application, in order to adopt an unsupervised learning method, the gait anomaly detection model is allowed to learn the data characteristics of normal gait data, and the anomaly is judged by the normal, so that the method provided by the present application has better anomaly detection generalization. Based on this, in this embodiment, the implementation process of the above step 102 can be described as:
[0062] (1) Obtain gait data and generate a sample data set based on the gait data.
[0063] (2) Construct an initial gait reconstruction model, an initial gait prediction model and an adaptive scoring module.
[0064] (3) The sample data set is used to train and test the initial gait reconstruction model and the initial gait prediction model respectively until the difference between the output data of the trained initial gait reconstruction model and the labeled data in the sample data set and the difference between the output data of the trained initial gait prediction model and the labeled data in the sample data set are both less than the set value. Then, the trained initial gait prediction model and the trained initial gait prediction model are used to construct the gait reconstruction module and the gait prediction module respectively.
[0065] (4) Connect the gait reconstruction module and the gait prediction module with the adaptive scoring module to obtain a gait abnormality detection model.
[0066] During the training process of the above models, no labeled data is used. Only the position error of the input data is compared with the output data after learning the data characteristics of the normal gait data to obtain the trained gait reconstruction module and gait prediction module.
[0067] In another exemplary embodiment of the present application, the implementation process of the above step 103 may include:
[0068] (1) Input the single / multi-eye sequence posture data into the gait reconstruction module to obtain the gait reconstruction module results.
[0069] (2) Input the single / multi-eye sequence posture data into the gait prediction module to obtain the gait prediction module result.
[0070] (3) An adaptive scoring module is used to obtain gait abnormality detection results based on the results of the gait reconstruction module and the gait prediction module.
[0071] In another exemplary embodiment of the present application, in order to facilitate processing of a single frame, the gait reconstruction module provided in the present application adopts the same reconstruction model to process each frame in the sequence and reconstruct the posture of each frame. Figure 3 As shown, each initial gait reconstruction model includes a first fully connected layer, an encoder, a memory pool, a decoder and a second fully connected layer.
[0072] Based on this structure of the gait reconstruction module, an abnormal gait sequence of n frames is input. As an example, the training and data processing process of the gait reconstruction module is explained. In the formula, R represents a real number set, and S represents the number of joint points of the posture data.
[0073] The gait reconstruction model is trained with a large amount of normal gait posture data. When a posture data is input into the gait reconstruction model, noise and mask are added to improve the robustness and accuracy of the gait reconstruction model. Moreover, the mask operation method is to randomly mask a joint, which helps to extract the spatial relationship between the joints and the global information of the posture. For posture data composed of multiple joints, it is believed that in the gait process, not only there is a certain relationship between each joint, but also there is a certain relationship between the coordinates of each dimension of the joint. Therefore, the input data is passed through the fully connected layer to extract the relationship between the joint points, and a 1*M*M feature map is output. Then it is sent to the encoder of the convolutional neural network to output the encoded vector Z e In the training phase, the encoded vector Z e The initial gait reconstruction model is fed into the decoder and then passes through the fully connected layer to obtain the output reconstructed posture. The initial gait reconstruction model is trained by calculating the mean square error between the reconstructed posture and the input data posture.
[0074] When the training is finished, the memory pool is constructed. e The encoded vector Z e Stored in the memory pool. When the memory pool contains the encoded vector Z corresponding to all normal gait postures in the training set e When , the greedy algorithm is used to reduce the size of the memory pool. After the memory pool is built, when a posture is input during testing, the encoded vector Z is obtained e , the encoded vector Z e Calculate the Euclidean distance with each vector in the memory pool and select the vector e with the smallest Euclidean distance i As the encoding vector Z q , the encoding vector Z q The input is decoded into the decoder, and the output is the normal gait posture closest to the input. If the input is a normal posture, then the output is the normal gait posture closest to the input Will be the same as normal posture x i Very similar, but if the input is an abnormal posture, then the normal gait posture closest to the input Will be the same as normal posture x i Based on this, by calculating the average joint position error μ(x i ) is used to measure this difference. Among them, the average joint point error μ(x i ) is calculated as:
[0075]
[0076] In the formula, |||| represents the norm, represents the jth normal posture, represents the jth normal gait posture that is closest to the input.
[0077] Further, based on the training and data processing process of the gait reconstruction module given above, the implementation process of step (1) may include:
[0078] (1-1) Noise and mask are added to single / multi-eye sequence posture data to obtain processed sequence data.
[0079] (1-2) Each frame in the processed sequence data is input into the first fully connected layer, the relationship between the joint points is extracted, and the feature map is obtained.
[0080] (1-3) Input the feature map into the encoder to obtain the encoding vector.
[0081] (1-4) Input the encoded vector into the decoder to obtain the gait reconstruction model result.
[0082] (1-5) Determine the error between the gait reconstruction model result and the input data, and obtain the gait reconstruction module result.
[0083] In another exemplary embodiment of the present application, similar to the reconstruction module, the gait prediction module also performs data processing synchronously. The prediction module processes long sequences and sequentially sends the segmented sliding windows to the gait prediction model. Based on this, the gait prediction model of the gait prediction module used in the present application is as follows: Figure 4 As shown, it mainly includes: an encoder and a decoder.
[0084] Based on the structure of the gait prediction module given above, the input data is divided into β subsequences of length t+T, where β is the number of sliding windows and the step length of each sliding is T. It is only necessary to determine whether each T frame in the sliding window is abnormal to detect anomalies in the entire sequence. Each subsequence is input into the gait prediction model, and the next T frames are predicted through the first t frames. Finally, the posture is predicted. With input posture x i The abnormality is judged by comparing the input data with the input data. During training, the normal gait data is also used for training. The gait prediction model learns the body movement pattern of the normal gait, and thus infers the movement state of the next T frames based on the movement state of the previous t frames. During testing, when the input data is a normal gait, the predicted posture will be very similar to the input normal gait. When the input data is an abnormal gait, the predicted posture is still normal. At this time, there will be a large difference between the input abnormal gait and the input. Similarly, the average joint position error of the input and output postures (denoted as ρ(x i )) to measure this difference.
[0085] In another exemplary embodiment of the present application, after obtaining the average joint position error μ(xi ) and the average joint position error ρ(x i ), in order to further improve the accuracy of gait abnormality detection, it is necessary to fuse the abnormality detection results obtained by the gait reconstruction module and the gait prediction module to evaluate the results at the sequence level. Based on this, the implementation process of the above step (3) may include:
[0086] (3-1) Determine the average value of the results of the gait reconstruction module and the average value of the results of the gait prediction module.
[0087] (3-2) Compare the average value of the results of the gait reconstruction module and the average value of the results of the gait prediction module, and use the result with the smaller average value as the first module result V1, and use the result with the larger average value as the second module result V2.
[0088] (3-3) Determine whether the first module result V1 is less than a set threshold value. For example, this set threshold value is 0.6 times the abnormality detection threshold value θ.
[0089] (3-4) When the first module result V1 is less than the set threshold, it is considered that no abnormality is detected, and the second module result V2 is directly used as the gait abnormality detection result.
[0090] (3-5) When the result V1 of the first module is greater than or equal to the set threshold, the result of the gait reconstruction module and the result of the gait prediction module are fused to obtain fusion information. For example, each ρ(x i ) or μ(x i ) divided by the anomaly detection threshold θ, and then multiplied by ρ(x i ) or μ(x i ) to obtain fusion information.
[0091] (3-6) Determine whether the fused information is less than the anomaly detection threshold θ.
[0092] (3-7) When the fusion information is less than the anomaly detection threshold θ, it is non-abnormal, indicating that there is no gait abnormality detection result.
[0093] (3-8) When the fused information is greater than or equal to the abnormality detection threshold θ, it is judged as abnormal. At this time, the fused information is used as the gait abnormality detection result.
[0094] Based on the above description, based on the frame-level results, the sequence-level results can be evaluated. If a sequence contains at least N abnormal frames, it can be considered abnormal, and N can be adjusted. As for the joint point level, we can go deep into the detected abnormal frame, use the reconstructed or predicted normal posture results, compare it with the input posture, compare the Euclidean distance of each joint point, and select the joint point with the largest Euclidean distance as the most abnormal joint point. So far, based on frame-level anomaly detection, three levels of anomaly detection tasks can be completed.
[0095] Based on the above description, this application designs a self-supervised gait abnormality detection framework based on reconstruction and prediction (i.e., a gait reconstruction module and a gait prediction module). By learning the data characteristics of normal gait, highly generalized gait abnormality detection at the sequence level, frame level, and joint level can be achieved, and the granularity of gait abnormality detection is greatly improved.
[0096] The present application can realize temporal gait abnormality detection and spatial gait abnormality detection by designing a gait reconstruction module and a gait prediction module. By integrating spatial information and temporal information through adaptive scoring (i.e., designing an adaptive scoring module), different types of gait abnormalities can be effectively detected.
[0097] This application adopts an unsupervised learning method to allow the model to learn the data characteristics of normal gait data and judge the anomaly based on the normal, so that the method provided by this application has better anomaly detection generalization.
[0098] In summary, compared with the traditional wearable gait abnormality detection method, the method provided by this application is lower in cost and more convenient in data collection. Compared with the visual-based supervised learning gait abnormality detection method, the method provided by this application can improve the granularity of gait abnormality detection by learning the data characteristics of normal gait, and can detect abnormalities at the sequence level, frame level and joint level. Moreover, because it is a self-supervised learning method, the generalization is greatly enhanced, and more types of abnormalities can be detected.
[0099] Compared with the known frame-level gait abnormality detection method that can only detect one type of abnormality, the method provided in the present application divides gait abnormalities into spatial gait abnormalities and temporal gait abnormalities, which can achieve effective detection of these two types of abnormalities.
[0100] In addition, in the gait abnormality detection method provided in the present application, a supervised learning method can also be used to complete abnormality detection at the sequence level, frame level, and joint level.
[0101] In the process of using the gait reconstruction module to perform spatial gait abnormality detection in the present application, the convolutional neural network can be replaced by a deep learning network such as a multi-layer perceptron, a graph convolutional neural network, etc.
[0102] In the process of using the gait reconstruction module to perform spatial gait abnormality detection in the present application, the size of the memory pool can be reduced by using other methods such as downsampling to extract the main distribution of normal gait features.
[0103] In the adaptive scoring module designed in this application, the anomaly detection threshold can be adjusted according to the degree of anomaly of the data set to achieve the effect of accurately distinguishing between normal and abnormal.
[0104] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store gait abnormality detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a gait abnormality detection method is implemented.
[0105] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0106] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0109] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0110] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0111] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A method for detecting abnormal gait, characterized in that: The gait abnormality detection method comprises: Acquire data to be detected; the data to be detected includes a color image, a depth map, and gait data collected by an inertial measurement unit; Acquire the single / multi-eye sequence posture data to be detected based on the data to be detected; Constructing a gait abnormality detection model; the gait abnormality detection model comprises: a gait reconstruction module, a gait prediction module and an adaptive scoring module; the gait reconstruction module and the gait prediction module are both connected to the adaptive scoring module; The single / multi-eye sequence posture data is input into the gait abnormality detection model to obtain a gait abnormality detection result; the gait abnormality detection result includes a gait sequence level abnormality detection result, a frame level abnormality detection result and a joint point level abnormality detection result.
2. The method for detecting abnormal gait according to claim 1, characterized in that: Build a gait anomaly detection model, including: Acquire gait data, and generate a sample data set based on the gait data; Constructing an initial gait reconstruction model, an initial gait prediction model and the adaptive scoring module; The sample data set is used to train and test the initial gait reconstruction model and the initial gait prediction model respectively, until the difference between the output data of the trained initial gait reconstruction model and the labeled data in the sample data set and the difference between the output data of the trained initial gait prediction model and the labeled data in the sample data set are both less than a set value, and the trained initial gait prediction model and the trained initial gait prediction model are used to respectively construct the gait reconstruction module and the gait prediction module; The gait reconstruction module and the gait prediction module are both connected to the adaptive scoring module to obtain the gait abnormality detection model.
3. The method for detecting abnormal gait according to claim 2, characterized in that: Inputting the single / multi-eye sequence posture data into the gait abnormality detection model to obtain a gait abnormality detection result, including: Inputting the single / multi-eye sequence posture data into the gait reconstruction module to obtain a gait reconstruction module result; Inputting the single / multi-objective sequence posture data into the gait prediction module to obtain a gait prediction module result; The adaptive scoring module is used to obtain the gait abnormality detection result based on the gait reconstruction module result and the gait prediction module result.
4. The method for detecting abnormal gait according to claim 3, characterized in that: The initial gait reconstruction model includes a first fully connected layer, an encoder, a memory pool, a decoder and a second fully connected layer.
5. The method for detecting abnormal gait according to claim 3, characterized in that: The gait prediction module includes: an encoder and a decoder.
6. The method for detecting abnormal gait according to claim 4, characterized in that: Inputting the single / multi-eye sequence posture data into the gait reconstruction module to obtain a gait reconstruction module result, including: Adding noise and mask to the single / multi-eye sequence posture data to obtain processed sequence data; Input each frame in the processed sequence data into the first fully connected layer, extract the association between the joint points, and obtain a feature map; Inputting the feature map into the encoder to obtain an encoding vector; Inputting the encoding vector into the decoder to obtain a gait reconstruction model result; The error between the gait reconstruction model result and the input data is determined to obtain the gait reconstruction module result.
7. The method for detecting abnormal gait according to claim 6, characterized in that: The adaptive scoring module is used to obtain the abnormal gait detection result based on the gait reconstruction module result and the gait prediction module result, including: Determining an average value of the gait reconstruction module results and an average value of the gait prediction module results; the gait prediction module results are obtained using the gait prediction module; The result with a smaller average value between the results of the gait reconstruction module and the results of the gait prediction module is taken as the first module result, and the result with a larger average value between the results of the gait reconstruction module and the results of the gait prediction module is taken as the second module result; Determine whether the first module result is less than a set threshold; When the result of the first module is less than the set threshold, the result of the second module is used as the gait abnormality detection result; When the result of the first module is greater than or equal to the set threshold, fusing the result of the gait reconstruction module and the result of the gait prediction module to obtain fusion information; Determining whether the fusion information is less than an anomaly detection threshold; When the fusion information is less than the abnormality detection threshold, there is no gait abnormality detection result; When the fused information is greater than or equal to the abnormality detection threshold, the fused information is used as the gait abnormality detection result.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the gait abnormality detection method according to any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the gait abnormality detection method according to any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the gait abnormality detection method according to any one of claims 1 to 7 is implemented.