Method for training gait recognition model, gait recognition method and related device

Abstract

Embodiments of the present application relate to the technical field of gait recognition, and disclose a method for training a gait recognition model, a gait recognition method and related devices. A plurality of gait signal data are acquired, and each gait signal data includes triaxial acceleration signals and triaxial angular velocity signals collected by a motion sensor on the gait of both feet within a preset time. The gait signal data are divided into a plurality of gait short sequences in the time dimension. The gait short sequences are encoded into image data to obtain multi-channel gait images. The pre-set neural network is iteratively trained according to a plurality of gait images to obtain a gait recognition model. In this embodiment, the gait signal data with time sequence are converted into image data, which can not only retain the complete information of the signal, but also retain the dependence of the signal on time; on the other hand, it is conducive to fully utilizing the recognition advantage of the neural network on the image data. Thus, it is conducive to improving the accuracy of the model.

Description

Technical Field

[0001] This application relates to the field of gait recognition technology, and in particular to a method for training a gait recognition model, a gait recognition method, and related apparatus. Background Technology

[0002] Parkinson's disease is a chronic neurological disorder that primarily affects the elderly. In the early stages, gait abnormalities mainly manifest as changes in stride length and walking rhythm. Common abnormalities include shuffling gait and frozen gait. Shuffling gait is one of the most typical and unique gait disturbances in Parkinson's patients, characterized by rapid, small steps taken to maintain balance between the feet. After initiating the movement, the body leans forward and is difficult to stop, as if attempting to retrace. As the disease progresses, it gradually develops into frozen gait. Frozen gait often occurs in the middle and late stages of the disease, mainly characterized by transient motor blockage, difficulty lifting the foot, and difficulty taking the first step. Shuffling gait often coexists with frozen gait, but it can also exist independently. Shuffling gait may precede frozen gait, initially manifesting as rapid steps and shortened stride length, which then becomes completely frozen.

[0003] Parkinson's disease often has insidious onset in its early stages, making it difficult to detect. Many patients are diagnosed only when the disease has progressed to the middle stage, making early detection crucial. However, current methods primarily rely on sensors placed on the soles of the feet to detect abnormal gait, which have relatively low accuracy. Summary of the Invention

[0004] In view of this, some embodiments of this application provide a method for training a gait recognition model, a gait recognition method and related apparatus, and the trained gait recognition model can accurately identify the category to which the gait belongs.

[0005] In a first aspect, embodiments of this application provide a method for training a gait recognition model, including:

[0006] Multiple gait signal data are acquired. Each gait signal data includes triaxial acceleration signal waves and triaxial angular velocity signal waves obtained by the motion sensor collecting the gait of both feet within a preset time.

[0007] Gait signal data is divided into multiple short gait sequences in the time dimension;

[0008] Gait short sequences are encoded into image data to obtain multi-channel gait images;

[0009] Based on several gait images, a pre-set neural network is iteratively trained to obtain a gait recognition model.

[0010] In some embodiments, the aforementioned gait signal data is segmented into multiple short gait sequences along the time dimension, including:

[0011] Obtain the trough of a signal wave in gait signal data;

[0012] For gait signal data, a preset length of signal data is extracted from each trough to obtain multiple short gait sequences.

[0013] In some embodiments, the aforementioned acquisition of the trough of a signal wave in gait signal data includes:

[0014] Invert the signal wave to obtain the reverse signal wave;

[0015] For reverse signal waves, a peak-finding algorithm is used to search for peaks and obtain troughs of the signal wave.

[0016] In some embodiments, the aforementioned peak-finding algorithm is used to search for peaks and obtain troughs of the signal wave for the reverse signal wave;

[0017] Obtain the index of the signal value greater than the preset value in the reverse signal wave to obtain the first index set;

[0018] The reverse signal wave is processed by first-order differential processing to obtain differential data;

[0019] After inserting 0 at the end of the difference data, obtain the index of the data less than 0 to get the second index set;

[0020] After inserting 0 at the beginning of the difference data, obtain the index of the data greater than 0 to get the third index set;

[0021] The peak index set is determined based on the intersection of the first index set, the second index set, and the third index set;

[0022] Based on the peak index set, obtain the valley index set in the signal wave.

[0023] In some embodiments, prior to the aforementioned step of segmenting the gait signal data into multiple short gait sequences in the time dimension, the method further includes:

[0024] The gait signal data is filtered.

[0025] In some embodiments, the aforementioned encoding of short gait sequences into image data to obtain multi-channel gait images includes:

[0026] Normalize each signal band in the short gait sequence;

[0027] Map the normalized signal bands to polar coordinates;

[0028] Substitute the data mapped to polar coordinates into the Gram angle field to generate the corresponding single-channel image;

[0029] A multi-channel gait image is obtained by combining multiple single-channel images corresponding to short gait sequences.

[0030] In some embodiments, the aforementioned gait images include a left foot acceleration image, a left foot angular velocity image, a right foot acceleration image, and a left foot angular velocity image;

[0031] The aforementioned method of combining multiple single-channel images corresponding to short gait sequences to obtain multi-channel gait images includes:

[0032] The single-channel images of the triaxial acceleration corresponding to the left foot are stitched together to obtain the acceleration image of the left foot.

[0033] The single-channel images of the triaxial angular velocities corresponding to the left foot are stitched together to obtain the left corner angular velocity image;

[0034] The single-channel images of the triaxial acceleration corresponding to the right foot are stitched together to obtain the acceleration image of the right foot.

[0035] The single-channel images of the triaxial angular velocities corresponding to the right foot are stitched together to obtain the angular velocity image of the right foot.

[0036] Secondly, this application provides a method for identifying gait, including:

[0037] Acquire test gait signal data;

[0038] The test gait signal data is divided into multiple short test gait sequences in the time dimension;

[0039] The short test gait sequence is encoded into test image data to obtain multi-channel test gait images;

[0040] The test gait image is input into the gait recognition model, which outputs the category to which the gait belongs. The gait recognition model is trained using the method in the first aspect.

[0041] Thirdly, this application provides an electronic device, including:

[0042] At least one processor; and

[0043] A memory communicatively connected to at least one processor; wherein the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to perform the method of the first aspect.

[0044] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions for causing a computer device to perform the method of the first aspect.

[0045] The beneficial effects of this application's embodiments are as follows: Unlike existing technologies, the method for training a gait recognition model provided in this application first acquires multiple gait signal data, including triaxial acceleration signals and triaxial angular velocity signals obtained by motion sensors from both feet within a preset time period. Then, the gait signal data is segmented into multiple short gait sequences along the time dimension; these short gait sequences are encoded into image data to obtain multi-channel gait images. Finally, based on several gait images, a pre-set neural network is iteratively trained to obtain a gait recognition model.

[0046] In this embodiment, a large amount of gait signal data is segmented into several short gait sequences and then encoded into multi-channel gait images. On the one hand, converting temporally sequential gait signal data into image data preserves both the complete information of the signal and its temporal dependence; on the other hand, it facilitates full utilization of the neural network's advantage in recognizing image data. Furthermore, multi-channel gait images effectively preserve signal data across multiple dimensions, which is beneficial for the neural network to learn to consider signal data from multiple dimensions, thus improving the model's accuracy. Therefore, the gait recognition model trained using a large number of multi-channel gait images can accurately identify the category to which the gait belongs. Attached Figure Description

[0047] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0048] Figure 1 This is a schematic diagram of the gait recognition system in some embodiments of this application;

[0049] Figure 2 This is a schematic diagram of the structure of an electronic device in some embodiments of this application;

[0050] Figure 3 This is a schematic diagram illustrating the process of training a gait recognition model in some embodiments of this application;

[0051] Figure 4 This is a schematic diagram illustrating the interception of signal bands in some embodiments of this application;

[0052] Figure 5 This is a schematic diagram of the signal waves before and after filtering in some embodiments of this application;

[0053] Figure 6 This is a schematic diagram illustrating the process of image encoding of signal bands in some embodiments of this application;

[0054] Figure 7 This is a flowchart illustrating the gait recognition method in some embodiments of this application. Detailed Implementation

[0055] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0058] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0059] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0060] To facilitate understanding of the methods provided in the embodiments of this application, the terms used in the embodiments of this application will first be introduced:

[0061] (1) Neural Network

[0062] A neural network can be composed of neural units, specifically understood as a neural network with input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. Neural networks with many hidden layers are called deep neural networks (DNNs). The work of each layer in a neural network can be described by the mathematical expression y = a(W·x + b). From a physical perspective, the work of each layer in a neural network can be understood as transforming the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space: 1. Dimensionality increase / decrease; 2. Magnification / scaling; 3. Rotation; 4. Translation; 5. "Bending". Operations 2 and 3 are performed by "W·x", operation 4 by "+b", and operation 5 by "a()". The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the set of all individuals within that class. W is the weight matrix of each layer in the neural network, where each value represents the weight of a neuron in that layer. This matrix W determines the spatial transformation from the input space to the output space, as described above; that is, the W of each layer of the neural network controls how the space is transformed. The purpose of training the neural network is to ultimately obtain the weight matrices of all layers of the trained neural network. Therefore, the training process of a neural network is essentially learning how to control spatial transformation, more specifically, learning the weight matrix.

[0063] It should be noted that in the embodiments of this application, the models used for machine learning tasks are essentially neural networks. Common components in neural networks include convolutional layers, pooling layers, and normalization layers. By assembling these common components in neural networks, a model is designed. When the model parameters (weight matrices of each layer) are determined so that the model error meets a preset condition or the number of model parameters is adjusted to reach a preset threshold, the model converges.

[0064] Before introducing the embodiments of this application, a brief introduction to gait recognition methods known to the inventors of this application will be given to facilitate the understanding of the embodiments of this application.

[0065] In some solutions, a six-axis sensor is used on the foot. Based on the collected three-axis acceleration data and three-axis angular velocity data, the three-axis acceleration signal is converted into a resultant acceleration (acc), and the three-axis Euler angle signal is converted into a resultant Euler angle signal (euler). Wavelet transform is used to extract detail components and approximate components. Statistical analysis is used to obtain training gait parameters, train a random forest model, and monitor whether frozen gait occurs.

[0066] This approach primarily relies on data from six-axis sensors placed on both feet to calculate the resultant acceleration and Euler angles of both feet; however, it neglects information in the time dimension. Furthermore, using features obtained through wavelet transform analysis in statistical analysis as the final gait parameters makes the model's performance overly dependent on the comprehensiveness of the extracted features and the choice of wavelet basis. Consequently, the gait recognition results are unsatisfactory, failing to accurately determine the gait category.

[0067] In some approaches, pressure sensors are used to collect plantar pressure data during the subject's gait, and abnormal gait is detected based on a long short-term memory neural network.

[0068] In this scheme,

[0069] This method identifies human gait phase and lower limb movement patterns by analyzing changes in plantar pressure distribution over time. However, this method can only obtain pressure changes when the foot contacts the ground, and therefore cannot obtain gait information for a complete gait cycle. This results in unsatisfactory gait recognition results and makes it impossible to accurately determine the gait category.

[0070] To address the aforementioned problems, this application provides a method, apparatus, and technique for training a gait recognition model. First, multiple gait signal data are acquired, including triaxial acceleration and triaxial angular velocity signals obtained by motion sensors collecting data on the gait of both feet within a preset time period. Then, the gait signal data is segmented into multiple short gait sequences along the time dimension; these short gait sequences are encoded into image data to obtain multi-channel gait images. Finally, a pre-set neural network is iteratively trained based on several gait images to obtain a gait recognition model.

[0071] In this embodiment, a large amount of gait signal data is segmented into several short gait sequences and then encoded into multi-channel gait images. On the one hand, converting temporally sequential gait signal data into image data preserves both the complete information of the signal and its temporal dependence; on the other hand, it facilitates full utilization of the neural network's advantage in recognizing image data. Furthermore, multi-channel gait images effectively preserve signal data across multiple dimensions, which is beneficial for the neural network to learn to consider signal data from multiple dimensions, thus improving the model's accuracy. Therefore, the gait recognition model trained using a large number of multi-channel gait images can accurately identify the category to which the gait belongs.

[0072] The following describes exemplary applications of the electronic devices provided in this application for training gait recognition models or for gait recognition. The electronic devices provided in this application may be servers, such as servers deployed in the cloud. Some embodiments of this application may provide electronic devices that are various types of terminals, such as laptops, desktop computers, or mobile devices.

[0073] As an example, see Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of the gait recognition system provided in this embodiment. Terminal 10 connects to server 20 via a network, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0074] Terminal 10 can be used to acquire training data and build neural networks. For example, those skilled in the art can download prepared training data to the terminal and build the network structure of the neural network. The training data includes several time-series data reflecting foot movements. It is understood that terminal 10 can also be used to acquire test samples. For example, the motion sensor sends the collected test data to terminal 10, thereby allowing terminal 10 to acquire the test data. In some embodiments, terminal 10 can be integrated with the motion sensor.

[0075] In some embodiments, terminal 10 locally executes the method for training a gait recognition model provided in this application to train a designed neural network using training data, determine the final model parameters, and then configure the neural network with the final model parameters to obtain a gait recognition model. In some embodiments, terminal 10 may also send training data and a constructed neural network stored on the terminal by those skilled in the art to server 20 via a network. Server 20 receives the training data and the neural network, trains the neural network using the training data, determines the final model parameters, and then sends the final model parameters to terminal 10. Terminal 10 saves the final model parameters, and then configures the neural network with the final model parameters to obtain a gait recognition model.

[0076] The structure of the electronic device in the embodiments of this application is described below. Figure 2 This is a schematic diagram of the structure of an electronic device 500 in an embodiment of this application. The electronic device 500 includes at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together through a bus system 540. It is understood that the bus system 540 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0077] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0078] User interface 530 includes one or more output devices 531 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 530 also includes one or more input devices 532, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0079] Memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory. Memory 550 may optionally include one or more storage devices physically located away from processor 510.

[0080] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0081] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks.

[0082] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0083] Display module 553 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 531 (e.g., a display screen, a speaker, etc.) associated with user interface 530;

[0084] The input processing module 554 is used to detect and translate one or more user inputs or interactions from one or more input devices 532.

[0085] As can be understood from the above, the method for training a gait recognition model provided in this application embodiment can be implemented by various types of electronic devices with processing capabilities, such as by the processor of an electronic device or by other devices with computing capabilities. Other devices with computing capabilities may be smart terminals or servers that are communicatively connected to the electronic device.

[0086] The following describes the method for training a gait recognition model provided in this application, using exemplary applications and implementations of the electronic device provided in the embodiments of this application. Please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the method for training a gait recognition model provided in an embodiment of this application. It is understood that the execution entity of this training method can be one or more processors of an electronic device.

[0087] Please refer to it again. Figure 3 The method S100 may specifically include the following steps:

[0088] S10: Acquire multiple gait signal data, including triaxial acceleration signal waves and triaxial angular velocity signal waves obtained by the motion sensor collecting the gait of both feet within a preset time.

[0089] In some embodiments, the motion sensor may be a six-axis sensor, which includes a three-axis accelerometer and a three-axis gyroscope, capable of acquiring acceleration and angular velocity signals of a moving object along the x-axis, y-axis, and z-axis, respectively. In some embodiments, one six-axis sensor is positioned on the left foot of the human body, and one six-axis sensor is positioned on the right foot of the human body, thereby acquiring acceleration and angular velocity signals of the left foot along the x-axis, y-axis, and z-axis, and acceleration and angular velocity signals of the right foot along the x-axis, y-axis, and z-axis, respectively.

[0090] Understandably, two six-axis sensors are installed in the smart shoe. When collecting multiple gait signal data, several testers walk in the smart shoe for a preset time. After collecting the gait signal data within the preset time, the smart shoe sends it to the electronic device. Understandably, in some embodiments, the gait categories of these testers are not entirely the same; some have Parkinson's gait, and some have normal gait, i.e., non-Parkinson's gait. In some embodiments, some testers have normal gait, some have flustered gait, and some have frozen gait, i.e., further distinguishing between the early and late stages of Parkinson's gait. Thus, the electronic device can acquire these multiple gait signal data. Each gait signal data corresponds to a gait category.

[0091] Gait signal data is obtained by collecting triaxial acceleration and triaxial angular velocity signals of both feet from motion sensors within a preset time. Thus, a gait signal data has 12 dimensions of signal waves, including: left foot x-axis acceleration signal wave, left foot y-axis acceleration signal wave, left foot z-axis acceleration signal wave, left foot x-axis angular velocity signal wave, left foot y-axis angular velocity signal wave, left foot z-axis angular velocity signal wave, and right foot x-axis acceleration signal wave, right foot y-axis acceleration signal wave, right foot z-axis acceleration signal wave, right foot x-axis angular velocity signal wave, right foot y-axis angular velocity signal wave, and right foot z-axis angular velocity signal wave.

[0092] In some embodiments, the preset time is an empirical value set by those skilled in the art based on actual needs; in some embodiments, the preset time can be 200 seconds, etc. No limitations are imposed on the preset time.

[0093] S20: Divide the gait signal data into multiple short gait sequences in the time dimension.

[0094] In some embodiments, each gait signal data point in a plurality of gait signal data points is segmented into multiple gait short sequences of equal length. For example, it may be segmented into gait short sequences of 200 data points each. Another example is segmenting each gait signal data point into multiple gait short sequences of 5 seconds each. It is understood that the gait signal data includes 12 signal waves. Each signal wave in the gait signal data is divided into multiple signal bands along the time dimension. The 12 signal bands corresponding to each time constitute a gait short sequence. In this embodiment, by setting the length of the gait short sequence, it is made possible that the gait short sequence can reflect the gait characteristics of taking one step.

[0095] It is understandable that, based on the fact that a gait signal data includes 12 signal waves, a short gait sequence includes 12 signal bands, including: the signal band corresponding to the left foot x-axis acceleration signal, the signal band corresponding to the left foot y-axis acceleration signal, the signal band corresponding to the left foot z-axis acceleration signal, the signal band corresponding to the left foot x-axis angular velocity signal, the signal band corresponding to the left foot y-axis angular velocity signal, the signal band corresponding to the left foot z-axis angular velocity signal, and the signal band corresponding to the right foot x-axis acceleration signal, the signal band corresponding to the right foot y-axis acceleration signal, the signal band corresponding to the right foot z-axis acceleration signal, the signal band corresponding to the right foot x-axis angular velocity signal, the signal band corresponding to the right foot y-axis angular velocity signal, and the signal band corresponding to the right foot z-axis angular velocity signal.

[0096] In some embodiments, the aforementioned step S20 specifically includes:

[0097] S21: Obtain the trough of a signal wave in the gait signal data.

[0098] S22: For gait signal data, extract signal data of a preset length starting from each trough to obtain multiple short gait sequences.

[0099] Based on the periodicity of the movement of the feet during human walking, the acceleration signal waves and angular velocity signal waves on the x, y, and z axes all possess the same periodicity. In this embodiment, the troughs of a signal wave are first obtained. This signal wave can be the left foot's z-axis acceleration signal wave. That is, the indices of all the troughs in the left foot's z-axis acceleration signal wave are obtained. It can be understood that the troughs in the left foot's z-axis acceleration signal wave reflect the foot placement point.

[0100] like Figure 4 As shown, a signal band is obtained by extracting signal data of a preset length from each trough of the left foot z-axis acceleration signal wave. Specifically, a signal band is obtained by extracting the signal data between the starting point and the ending point, using the trough index as the starting point and the index obtained by adding the preset length to the trough index as the ending point. In some embodiments, the preset length can be 200 data points.

[0101] Understandably, for other signal waves in the gait signal data, the signal between the start and end points is extracted to obtain the corresponding signal bands. The 12 signal bands corresponding to the index constitute a short gait sequence. After traversing all the troughs in the left foot z-axis acceleration signal wave, multiple short gait sequences can be extracted.

[0102] In this embodiment, the gait short sequence is extracted from the landing point, so that the gait short sequence includes complete gait cycle data, which is beneficial for training an accurate gait recognition model.

[0103] In some embodiments, the aforementioned step S21 specifically includes:

[0104] S211: Perform an inversion operation on the signal wave to obtain the reverse signal wave.

[0105] S212: For reverse signal waves, a peak-finding algorithm is used to search for peaks and obtain troughs of the signal waves.

[0106] The inversion operation refers to taking the opposite value of the signal value in the signal wave. For example, taking the opposite value of the left foot z-axis acceleration signal wave yields its corresponding reverse signal wave.

[0107] It is understandable that the peaks in the reverse signal wave are the troughs in the original signal wave. Therefore, by using a peak-finding algorithm to search for the peaks of the reverse signal wave, the troughs of the original signal wave can be obtained.

[0108] In some embodiments, the aforementioned step S211 specifically includes:

[0109] (1) Obtain the index of the signal value greater than the preset value in the reverse signal wave to obtain the first index set.

[0110] (2) Perform first-order differential processing on the reverse signal wave to obtain differential data.

[0111] (3) After inserting 0 at the end of the difference data, obtain the index of the data less than 0 to get the second index set.

[0112] (4) After inserting 0 at the beginning of the differential data, obtain the index of the data greater than 0 to get the third index set.

[0113] (5) Determine the peak index set based on the intersection of the first index set, the second index set, and the third index set.

[0114] (6) Obtain the set of indices of the valleys in the signal wave based on the set of peak indices.

[0115] Signal values ​​with amplitudes greater than a certain level are filtered out using a preset value; that is, the indices of signal values ​​in the reverse signal wave that are greater than the preset value are obtained, resulting in a first index set. In some embodiments, the preset value can be A*(max(y)-min(y))+min(y), where A is an empirical value, such as 0.6, y is the reverse signal wave, max(y) is the maximum value in the reverse signal wave, and min(y) is the minimum value in the reverse signal wave. The indices of signal values ​​greater than A*(max(y)-min(y))+min(y) constitute the first index set.

[0116] The reverse signal wave y undergoes first-order differential processing to obtain differential data y. *Specifically, first-order differential processing refers to subtracting the previous signal value from the subsequent signal value in the time dimension of the reverse signal wave.

[0117] It is understandable that the difference data y * The differential data y is one value less than the reverse signal wave y. To ensure that both have the same length, the differential data y... * Inserting 0s at the end of the data creates a new data B. Data B and the difference data y are then compared. * The lengths are the same. Obtain the indices of the data in data B that are less than 0, and obtain the second set of indices.

[0118] In some embodiments, in the differential data y * Insert 0s at the beginning of the data to form a new data C, so that the index alignment of data C can be aligned with the difference data y. * Index alignment. Obtain the indices of data greater than 0 in data C to obtain the third index set.

[0119] It is understandable that in data B, the first-order difference corresponding to the peak index is less than 0; while in data C, the first-order difference corresponding to the peak index is greater than 0. Therefore, the peak index set can be determined based on the intersection of the first, second, and third index sets. The intersection of these three sets is then used as the peak index set.

[0120] It is understandable that the set of peak indices is the same as the set of trough indices in the original signal wave. Therefore, the set of trough indices in the signal wave can be obtained from the set of peak indices. For example, the set of peak indices can be used as the set of trough indices in the signal wave.

[0121] In this embodiment, by using the above-described filtering method, the index set of valleys in the signal wave can be accurately obtained, thereby enabling the accurate acquisition of valleys in the signal wave.

[0122] In some embodiments, the preceding step S20 may include: filtering the gait signal data.

[0123] Filtering each signal wave in the gait signal data can effectively eliminate high-frequency noise. Please refer to [link / reference]. Figure 5 A fourth-order Basswater filter was used to filter the original gray signal wave to remove interference noise, resulting in a processed blue signal wave. It can be seen that the processed signal wave is smoother and the waveform is clearer.

[0124] Therefore, using the filtered left-foot z-axis acceleration signal wave to search for the valley is beneficial for accurately obtaining the valley.

[0125] Furthermore, segmenting the gait signal data using filtering and obtaining multiple short gait sequences with reduced interference noise is beneficial for improving the accuracy of subsequent gait images and accelerating the convergence of subsequent gait recognition models.

[0126] S30: Encode the short gait sequence into image data to obtain a multi-channel gait image.

[0127] Understandably, a short gait sequence comprises 12 signal bands. Each signal band can be encoded into image data, resulting in 12 single-channel images. Based on these 12 single-channel images, a multi-channel gait image is obtained. The gait image has more than 1 and less than or equal to 12 channels.

[0128] In some embodiments, the aforementioned step S30 specifically includes:

[0129] S31: Normalize each signal band in the gait short sequence.

[0130] S32: Map the normalized signal bands to polar coordinates.

[0131] S33: Substitute the data mapped to polar coordinates into the Gram angle field to generate the corresponding single-channel image.

[0132] S34: Combine multiple single-channel images corresponding to short gait sequences to obtain multi-channel gait images.

[0133] For any one of the 12 signal bands in the gait short sequence, the signal value is first normalized using the following formula to scale the signal value to the [0,1] interval.

[0134]

[0135] Where X is any signal value in the signal band before normalization, and X′ is the signal value after normalization. nean X is the average value of all signal values ​​in the signal band. max X is the signal value with the largest value in the signal band. min It is the signal value with the smallest numerical value in the signal band.

[0136] Then, the normalized signal bands are mapped to polar coordinates. In some embodiments, the following formula is used to map the normalized signal bands to polar coordinates:

[0137]

[0138]

[0139] Where, x iIt is the i-th signal value, t i It is the signal value x i The timestamp, where N is the length of the signal band, t i ∈N, i It is x i The transformed angle, r i It is t i The converted radius.

[0140] It is understandable that the data mapped to polar coordinates... i and r i It can reflect the relationship between signal value and time in the original signal band, that is, reflect the time domain characteristics of the signal value.

[0141] Then, the data mapped to polar coordinates is substituted into the Gram angle field to generate the corresponding single-channel image. The Gram angle field is an encoding method that converts one-dimensional time-series data into an image. Specifically, the following formula is used to generate the corresponding single-channel image;

[0142]

[0143] Here, GASF is an n*n matrix, where the value in the i-th row and j-th column is the pixel value in the i-th row and j-th column of a single-channel image.

[0144] It is understandable that when all 12 signal bands in a short gait sequence are encoded into images using the above method, 12 single-channel images are obtained.

[0145] Finally, multiple single-channel images corresponding to the short gait sequence are combined to obtain a multi-channel gait image. For example, 12 single-channel images are stitched together to obtain a 12-channel gait image.

[0146] In this implementation, the one-dimensional signal band is converted into two-dimensional image data using Gram angle field, which amplifies the temporal information of the signal band and preserves the time dependence of the signal values, thus reflecting more hidden information. In other words, multi-channel gait images preserve the time dependence of signal values, thereby reflecting more hidden information and facilitating the training of an accurate gait recognition model.

[0147] In some embodiments, the gait images include a left foot acceleration image, a left foot angular velocity image, a right foot acceleration image, and a left foot angular velocity image.

[0148] The aforementioned step S34 specifically includes:

[0149] S341: Perform channel stitching on the single-channel images of the triaxial acceleration corresponding to the left foot to obtain the acceleration image of the left foot;

[0150] S342: Perform channel stitching on the single-channel image of the three-axis angular velocity corresponding to the left foot to obtain the left corner angular velocity image;

[0151] S343: Perform channel stitching on the single-channel image of the triaxial acceleration corresponding to the right foot to obtain the acceleration image of the right foot;

[0152] S344: The single-channel images of the three-axis angular velocities corresponding to the right foot are stitched together to obtain the angular velocity image of the right foot.

[0153] Please see Figure 6 The acceleration of the left foot corresponds to three signal bands: the x-axis, y-axis, and z-axis. After encoding these signal bands into single-channel images, three single-channel images corresponding to the left foot's acceleration can be obtained, such as an x-axis single-channel image, a y-axis single-channel image, and a z-axis single-channel image. These three single-channel images reflecting the left foot's acceleration are then stitched together to obtain the left foot acceleration image. The left foot acceleration image is a three-channel color image.

[0154] The angular velocity of the left foot corresponds to three signal bands: the x-axis, y-axis, and z-axis. After encoding these signal bands into single-channel images, three single-channel images corresponding to the left foot's angular velocity can be obtained, such as an x-axis single-channel image, a y-axis single-channel image, and a z-axis single-channel image. These three single-channel images reflecting the left foot's angular velocity are then stitched together to obtain the left foot angular velocity image.

[0155] Similarly, based on the acceleration of the right foot, there are signal bands corresponding to the x-axis, y-axis, and z-axis. After encoding the signal bands into single-channel images, three single-channel images corresponding to the acceleration of the right foot can be obtained, such as a single-channel image of the x-axis, a single-channel image of the y-axis, and a single-channel image of the z-axis. By stitching these three single-channel images reflecting the acceleration of the right foot together, the right foot acceleration image is obtained.

[0156] The angular velocity of the right foot corresponds to three signal bands: the x-axis, y-axis, and z-axis. After encoding these signal bands into single-channel images, three single-channel images corresponding to the right foot's angular velocity can be obtained, such as an x-axis single-channel image, a y-axis single-channel image, and a z-axis single-channel image. These three single-channel images reflecting the right foot's angular velocity are then stitched together to obtain the right foot angular velocity image.

[0157] In this embodiment, the x-axis, y-axis, and z-axis data corresponding to acceleration are converted into a three-channel color image, and the x-axis, y-axis, and z-axis data corresponding to angular velocity are also converted into a three-channel color image. The gait image, which includes the left foot acceleration image, left foot angular velocity image, right foot acceleration image, and left foot angular velocity image, is used as the input to the neural network. Compared with multiple single-channel images as the input to the neural network, the input dimension of the model is reduced, which helps to compress the size of the trained model, effectively saves computing time and energy consumption, and saves hardware costs.

[0158] S40: Based on several gait images, iteratively train a pre-set neural network to obtain a gait recognition model.

[0159] Here, several gait images are used as training data for the neural network. The pre-defined neural network is trained by continuously adjusting its parameters. Under the constraint of the loss function, the predicted gait category output by the neural network will become increasingly closer to the true gait category. When the loss calculated by the loss function fluctuates within a certain range or reaches a certain value, the neural network converges. The parameters at convergence are used as model parameters to obtain the gait recognition model.

[0160] In this embodiment, a large amount of gait signal data is segmented into several short gait sequences and then encoded into multi-channel gait images. On the one hand, converting temporally sequential gait signal data into image data preserves both the complete information of the signal and its temporal dependence; on the other hand, it facilitates full utilization of the neural network's advantage in recognizing image data. Furthermore, multi-channel gait images effectively preserve signal data across multiple dimensions, which is beneficial for the neural network to learn to consider signal data from multiple dimensions, thus improving the model's accuracy. Therefore, the gait recognition model trained using a large number of multi-channel gait images can accurately identify the category to which the gait belongs.

[0161] After training a gait recognition model using the method provided in this application, the gait recognition model can be applied to identify Parkinson's gait. The gait recognition method provided in this application can be implemented by various types of electronic devices with computing capabilities, such as smart terminals, servers, or monitoring devices with motion sensors.

[0162] The gait recognition method provided in this application is described below with reference to exemplary applications and implementations of the terminal provided in the embodiments of this application. See also Figure 7 , Figure 7 This is a flowchart illustrating the gait recognition method provided in an embodiment of this application. Method S200 includes the following steps:

[0163] S201: Acquire test gait signal data.

[0164] It is understandable that the test gait signal data consists of triaxial acceleration and triaxial angular velocity signal waves collected by the motion sensor from both feet within a preset time period in a real-world application scenario. The motion sensor then sends the test gait signal data to the terminal, which then obtains the test gait signal data.

[0165] S202: Divide the test gait signal data into multiple short test gait sequences in the time dimension.

[0166] Referring to the method for dividing gait signal data in step S20, the test gait signal data is divided into multiple short test gait sequences. The specific division method is detailed in the description of step S20 and will not be repeated here.

[0167] S203: Encode the short test gait sequence into test image data to obtain multi-channel test gait images.

[0168] Referring to the encoding method in step S30, each short sequence of test gait is encoded into test image data to obtain multi-channel test gait images. The specific encoding method is detailed in the description of step S30 and will not be repeated here.

[0169] S204: Input the test gait image into the gait recognition model and output the category to which the gait belongs. The gait recognition model is trained using the method for training a gait recognition model in any of the above embodiments.

[0170] It is understood that the terminal has a built-in gait recognition application, and the gait recognition model is encapsulated in this application. The gait recognition model is called to perform gait recognition on the aforementioned test gait images. After a series of calculations, the gait category corresponding to each test gait image is output. This gait recognition model is trained using the method described in any of the above embodiments, and has the same structure and function as the gait recognition model in the above embodiments, which will not be elaborated further here.

[0171] In this embodiment, based on the test gait images reflecting the motion characteristics of each step, the gait recognition model can identify the gait category of each step taken by the user. In some embodiments, a recognition result of 1 indicates a Parkinsonian gait, and a recognition result of 0 indicates a non-Parkinsonian gait. If the proportion of 1 in the recognition result is greater than the proportion of 0, then the output gait belongs to the Parkinsonian gait. In some embodiments, the proportion of 1 is used as the similarity of the Parkinsonian gait; for example, if the proportion of 1 is 57%, then the output Parkinsonian gait similarity is 57%. By recognizing each step of the user and combining the recognition results of all steps during a walking process, the final gait category is determined, which can accurately detect the Parkinsonian gait and help remind users to detect Parkinson's syndrome in a timely manner.

[0172] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for training a gait recognition model, characterized in that, include: Acquire multiple gait signal data, wherein the gait signal data includes triaxial acceleration signal waves and triaxial angular velocity signal waves obtained by motion sensors collecting the gait of both feet within a preset time. The gait signal data is divided into multiple short gait sequences in the time dimension; The gait short sequence is encoded into image data to obtain a multi-channel gait image; Based on several gait images, a pre-set neural network is iteratively trained to obtain the gait recognition model. The step of encoding the short gait sequence into image data to obtain a multi-channel gait image includes: Normalize each signal band in the gait short sequence; Map the normalized signal bands to polar coordinates; Substitute the data mapped to polar coordinates into the Gram angle field to generate the corresponding single-channel image; The multi-channel gait image is obtained by combining multiple single-channel images corresponding to the short gait sequence.

2. The method according to claim 1, characterized in that, The step of dividing the gait signal data into multiple short gait sequences in the time dimension includes: Obtain the trough of a signal wave in the gait signal data; For the gait signal data, a preset length of signal data is extracted from each of the troughs to obtain the multiple short gait sequences.

3. The method according to claim 2, characterized in that, The step of obtaining the trough of a signal wave in the gait signal data includes: Invert the signal wave to obtain the reverse signal wave; For the reverse signal wave, a peak-finding algorithm is used to search for the peaks and obtain the troughs of the signal wave.

4. The method according to claim 3, characterized in that, The peak-finding algorithm is used to search for the peaks of the reverse signal wave and to obtain the troughs of the signal wave. Obtain the index of the signal value greater than a preset value in the reverse signal wave to obtain the first index set; The reverse signal wave is subjected to first-order differential processing to obtain differential data; After inserting 0 at the end of the differential data, the index of the data less than 0 is obtained to get the second index set; After inserting 0 at the beginning of the differential data, the index of the data greater than 0 is obtained to get the third index set; The peak index set is determined based on the intersection of the first index set, the second index set, and the third index set; Based on the set of peak indices, obtain the set of trough indices for the signal wave.

5. The method according to claim 1, characterized in that, Prior to the step of segmenting the gait signal data into multiple short gait sequences in the time dimension, the method further includes: The gait signal data is filtered.

6. The method according to claim 1, characterized in that, The gait images include left foot acceleration image, left foot angular velocity image, right foot acceleration image, and left foot angular velocity image; The step of combining multiple single-channel images corresponding to the short gait sequence to obtain the multi-channel gait image includes: The single-channel images of the triaxial acceleration corresponding to the left foot are stitched together to obtain the acceleration image of the left foot. The single-channel image of the triaxial angular velocity corresponding to the left foot is stitched together to obtain the angular velocity image of the left foot. The right foot acceleration image is obtained by stitching together the single-channel images of the triaxial acceleration corresponding to the right foot. The single-channel images of the three-axis angular velocities corresponding to the right foot are stitched together to obtain the right foot angular velocity image.

7. A method for recognizing gait, characterized in that, include: Acquire test gait signal data; The test gait signal data is divided into multiple short test gait sequences in the time dimension; The short test gait sequence is encoded into test image data to obtain multi-channel test gait images; The test gait image is input into the gait recognition model, and the gait category is output. The gait recognition model is trained using the method described in any one of claims 1-6.

8. An electronic device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer device to perform the method as described in any one of claims 1-7.

Images