Modularization-based lower limb multi-joint intention recognition method and system

Abstract

The invention discloses a lower limb multi-joint intention recognition method and system based on modularization. The invention relates to lower limb rehabilitation equipment which comprises a plurality of modules connected with signal lines through power supply lines. The device collects module signal data of a user to form multi-channel data, divides a plurality of periods based on a motion period, and generates matrix-based analog image data. And performing consciousness model training by using the simulation image data and the operation result of the historical time period. Furthermore, second image data is generated through data clearing and zero setting, and secondary training of the model is carried out. Meanwhile, a GAN-based generation model is constructed, the feature vectors are integrated for training, and simulated feature vector data are generated and used for expanding training data. And finally, collecting user data in real time through the intention model and automatically setting an equipment operation mode. By means of the method and device, accurate recognition of the lower limb consciousness of the user is achieved, the requirement for system computing power is lowered, and meanwhile the applicability of lower limb rehabilitation equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of limb equipment and data analysis, and more specifically, to a modular-based lower limb multi-joint intention recognition method and system. Background Art

[0002] Neurological diseases such as stroke are seriously affecting people's health. With the improvement of people's living standards, the incidence of stroke has further increased and is showing a trend of becoming younger. Therefore, the prevention and treatment of stroke and the rehabilitation of motor function after stroke are particularly important, especially the motor rehabilitation of the lower limbs, which is the key to whether people can move freely. At the same time, trauma, disability and other problems can also lead to motor dysfunction, which further increases people's demand for rehabilitation training. Traditional methods use manual techniques to achieve limb rehabilitation of patients, but traditional techniques are inefficient and have poor effects. Therefore, rehabilitation robots have gradually emerged, which are as effective as manual ones, and are more efficient and comfortable.

[0003] However, at the current stage of walking rehabilitation, modular robots have the characteristic of free reorganization, which means that the algorithm has multiple inputs and it is difficult for the algorithm to adapt to different situations. Most of the current algorithms use preset algorithms with fixed modes, which cannot effectively utilize data and occupy a huge capacity. When there are many modules, it is difficult to cover all module combinations and achieve better use effects. In addition, it is impossible to identify the user's lower limb movement intentions and it is difficult to match and analyze the best training plan. Summary of the invention

[0004] The present invention overcomes the defects of the prior art and proposes a modular-based lower limb multi-joint intention recognition method and system.

[0005] The first aspect of the present invention provides a modular lower limb multi-joint intention recognition method, comprising: One power supply module and six limb modules are set for the lower limb rehabilitation equipment, and a power supply line and a signal line are set for each module to connect the equipment; During an exercise period, the signal data of each module of the target user when using the lower limb rehabilitation device is collected to form N-channel module data, and M cycles are divided based on the exercise period. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated; Construct a CNN intention recognition model, collect simulated image data of multiple motion periods in a historical time period to form first image data, and obtain corresponding operation result data, import the first image data and the operation result data into the intention recognition model for intention recognition training, the intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer, the first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, the intermediate feature data is flattened to form a one-dimensional feature vector, the one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result; Randomly select a plurality of selected simulated image data from the first image data, clear and zero the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generate second image data, and import the second image data and the operation result data into the intent recognition model for secondary model training; Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training; Through the intention recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

[0006] In this solution, the lower limb rehabilitation equipment is configured with 1 power supply module and 6 limb modules, and each module is provided with multiple sensors to obtain module data.

[0007] In this solution, in the generation of N×M dimensional analog image data based on the module data of M cycles and N channels, for each channel, the corresponding M cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

[0008] In this solution, the CNN intention recognition model is constructed. In a historical time period, simulated image data of multiple motion periods are collected to form first image data, and corresponding operation result data are obtained. The first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through a double-layer neural network and an output layer to obtain a two-layer output result, specifically: In a historical time period, simulated image data of a plurality of motion time periods are collected to form first image data, and a current simulated image is selected from the first image data; Import the current simulated image into the intention recognition model, extract features through the convolution layer, the size of the convolution layer is 12×12, and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimension reduction to prevent overfitting. The size of the pooling layer is 3×3, and the intermediate feature data is obtained. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, the two-layer output results are obtained through the output layer. The first layer output results represent the movement mode, and the second layer output results represent the movement gait. Compare the output results of the two layers with the running result data and optimize the intent recognition model through the preset loss function; All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

[0009] In this solution, in the double-layer neural network, the first layer of the network is set with 20 neural units, and the second layer of the network is set with 9 neural units.

[0010] In this solution, a plurality of selected simulated image data are randomly selected from the first image data, and data is cleared and zeroed according to a preset percentage so that the amount of zeroed data reaches a preset percentage, and second image data is generated, and the second image data and the operation result data are imported into the intent recognition model for secondary model training, specifically: randomly selecting a plurality of selected simulated image data from the first image data; Setting a plurality of preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data accounts for a certain preset percentage in the selected analog image data; For each preset percentage, a data clearing operation is performed on a plurality of selected simulation image data respectively; After the data clearing operation, the second image data is obtained.

[0011] In this solution, the GAN-based generative model is constructed, corresponding one-dimensional feature vectors are extracted based on the first image data and the second image data, and the feature vectors are integrated to obtain a one-dimensional feature vector set, the one-dimensional feature vector set is imported into the generative model, the generative model is trained, and simulated feature vector data is generated based on the trained generative model, and the simulated feature vector data is imported into the intent recognition model for training, specifically: Build a GAN-based generative model, which includes a generator and a discriminator; Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set; Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation; Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error back propagation algorithm until the generator and discriminator reach Nash equilibrium; Generate a preset amount of simulated feature vector data through the trained generative model.

[0012] In this solution, the intention recognition model is used to collect user module data in real time and import it into the recognition model for intention recognition, and the device operation mode is automatically set based on the recognition result, specifically: Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated by multi-joint awareness to generate the user's real-time limb evaluation status; According to the user's real-time limb assessment status, the corresponding operation mode is set for the equipment module and a rehabilitation plan is generated; The equipment module includes 1 power supply module and 6 limb modules.

[0013] The second aspect of the present invention further provides a modular lower limb multi-joint intention recognition system, the system comprising: a memory, a processor, the memory comprising a modular lower limb multi-joint intention recognition program, the modular lower limb multi-joint intention recognition program being executed by the processor to implement the following steps: One power supply module and six limb modules are set for the lower limb rehabilitation equipment, and a power supply line and a signal line are set for each module to connect the equipment; During an exercise period, the signal data of each module of the target user when using the lower limb rehabilitation device is collected to form N-channel module data, and M cycles are divided based on the exercise period. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated; Construct a CNN intention recognition model, collect simulated image data of multiple motion periods in a historical time period to form first image data, and obtain corresponding operation result data, import the first image data and the operation result data into the intention recognition model for intention recognition training, the intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer, the first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, the intermediate feature data is flattened to form a one-dimensional feature vector, the one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result; Randomly select a plurality of selected simulated image data from the first image data, clear and zero the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generate second image data, and import the second image data and the operation result data into the intent recognition model for secondary model training; Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training; Through the intention recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

[0014] The third aspect of the present invention also provides a computer-readable storage medium, which includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by a processor, the steps of the modular lower limb multi-joint intention recognition method as described in any one of the above are implemented.

[0015] The present invention discloses a modular lower limb multi-joint intention recognition method and system. The present invention relates to lower limb rehabilitation equipment, including multiple modules, and the modules are connected to signal lines through power supply lines. The equipment collects module signal data of the user to form multi-channel data, and divides multiple cycles based on the exercise period to generate matrix-based simulated image data. The simulated image data and operation results of the historical period are used to train the consciousness model. Furthermore, the second image data is generated by clearing and zeroing the data for secondary model training. At the same time, a generative model based on GAN is constructed, and feature vectors are integrated for training to generate simulated feature vector data, which is used to expand the training data. Finally, user data is collected in real time through the intention model and the equipment operation mode is automatically set. Through the present invention, accurate recognition of the user's lower limb consciousness is achieved, the requirements for system computing power are reduced, and at the same time, the applicability of lower limb rehabilitation equipment is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a modular lower limb multi-joint intention recognition method of the present invention is shown; Figure 2 Shows a schematic diagram of the connection of each module of the present invention; Figure 3 The schematic diagram of analog image data conversion of the present invention is shown; Figure 4 A schematic diagram of the intention recognition model of the present invention is shown; Figure 5 A block diagram of a modular lower limb multi-joint intention recognition system of the present invention is shown. DETAILED DESCRIPTION

[0017] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0019] Figure 1 A flow chart of a modular lower limb multi-joint intention recognition method of the present invention is shown.

[0020] like Figure 1 As shown, the first aspect of the present invention provides a modular lower limb multi-joint intention recognition method, comprising: S102, setting one power supply module and six limb modules for the lower limb rehabilitation equipment, and setting a power supply line and a signal line for each module to connect the equipment; S104, collecting signal data of each module of the target user when using the lower limb rehabilitation device during an exercise period to form module data of N channels, dividing M cycles based on the exercise period, and generating N×M dimensional simulated image data based on the M cycles and the module data of the N channels; S106, constructing a CNN intention recognition model, collecting simulated image data of multiple motion periods in a historical time period to form first image data, and obtaining corresponding operation result data, importing the first image data and the operation result data into the intention recognition model for intention recognition training, the intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer, the first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, the intermediate feature data is flattened to form a one-dimensional feature vector, and the one-dimensional feature vector is subjected to the double-layer neural network and the output layer to obtain a two-layer output result; S108, randomly selecting a plurality of selected simulated image data from the first image data, clearing and zeroing the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, and generating second image data, and importing the second image data and the operation result data into the intent recognition model for secondary model training; S110, constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training; S112, through the intention recognition model, collect user module data in real time and import the recognition model to perform intention recognition, and automatically set the device operation mode based on the recognition result.

[0021] It is worth mentioning that the present invention proposes a multi-channel consciousness recognition algorithm model based on convolutional neural network, which can realize the simultaneous input of multiple channel data, realize the detection of the current state and gait of the human body, and can also adapt to the lack of channel data and still maintain recognition without switching modes.

[0022] The present invention combines the module data (time series data) of a stage of multi-channel data into a square matrix, which has the characteristics of an image, and recognizes and processes the data through a convolutional neural network, thereby achieving an accurate recognition effect. Considering the timeliness of data processing and the fact that the sequence data itself has continuous characteristics and does not change much, and the correlation between the front and back is strong, only one convolution layer and one pooling layer are used, and then the neural network is used to obtain the final required result data, thereby greatly reducing the amount of calculation, enabling high-speed computing to be achieved on embedded systems, and reducing the requirements for system computing power.

[0023] The algorithm model will collect sensor data with complete data (including full module collection), randomly remove or not remove data from certain channels (all removed data are set to zero as input) to implement model training. The result obtained in this way can realize user intention recognition and completion of lower limb training process when all sensor data are complete or partial data exists, achieving the effect of no need to switch modes, thereby improving the applicability of lower limb rehabilitation equipment.

[0024] According to an embodiment of the present invention, the lower limb rehabilitation equipment is set with one power supply module and six limb modules, and each module is provided with multiple sensors to obtain module data.

[0025] It should be noted that the lower limb rehabilitation equipment is set with 1 power supply module and 6 limb modules. Based on the needs of lower limb rehabilitation, the number of various modules can be appropriately increased.

[0026] Figure 2 Shows a schematic diagram of the connection of each module of the present invention; like Figure 2As shown, the left side is a schematic diagram of the position of each module in the lower limb rehabilitation equipment, including 7 modules 101-107, and the right side is a schematic diagram of the signal circuit. Module 101 is a power supply module, which is responsible for providing power to each module, collecting the sensor data and status data of each module, detecting the current state, calculating the output of each joint, and sending the control command to each module through the signal bus. 102-107 are 6 limb modules, which are responsible for detecting the sensor data of each module, sending it to the control center through the bus, receiving the status command of the control center, and each module determines the output according to the state and current gait. The right side is a schematic diagram of the signal circuit, which includes a power supply bus, using bus communication such as CAN communication or 485 communication, and the power supply is also bus-powered. It can realize the random combination of any number of nodes of the device. All modules have two input and output sockets, and the power supply and control center have two outputs. The input of all modules can be plugged into the output of another module or the control center, and connected to the control center, which can realize the combination of any modules. The intention recognition model is used to realize the data acquisition and state recognition of all modules and some modules, and control commands are applied to different joints according to different states.

[0027] According to an embodiment of the present invention, in the generation of N×M dimensional analog image data based on the module data of M cycles and N channels, for each channel, the corresponding M cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

[0028] It should be noted that the N×M dimensional simulated image data includes N×M data points.

[0029] Figure 3 The schematic diagram of analog image data conversion of the present invention is shown; like Figure 3As shown in the figure, the left side of the figure shows the specific joint information of the module data corresponding to channels 1-23. Here, both N and M are set to 23, generating a 23×23 data image, that is, simulated image data. The simulated image data can be complete data acquisition or partial data acquisition. The complete data is the neat state of 23-channel data. If only some modules are used, only some channels will have data, and other channels will have no data. Channels without data can be directly set to zero. Based on analysis requirements, the values ​​of N and M can be changed. The data of channels 1-23 correspond to waist pitch angle, waist roll angle, waist sagittal axis angular velocity, left thigh pitch angle, left thigh roll angle, left thigh sagittal axis angular velocity, right thigh pitch angle, right thigh roll angle, right thigh sagittal axis angular velocity, left calf pitch angle, left calf roll angle, left calf sagittal axis angular velocity, right thigh pitch angle, right thigh roll angle, right thigh sagittal axis angular velocity, left hip joint angle, left hip joint angular velocity, right hip joint angle, right hip joint angular velocity, left knee joint angle, left knee joint angular velocity, right knee joint angle, right knee joint angular velocity.

[0030] According to an embodiment of the present invention, the CNN intention recognition model is constructed, and within a historical time period, simulated image data of multiple motion periods are collected to form first image data, and corresponding operation result data are obtained, and the first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, and the intermediate feature data is flattened to form a one-dimensional feature vector, and the one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result, specifically: In a historical time period, simulated image data of a plurality of motion time periods are collected to form first image data, and a current simulated image is selected from the first image data; Import the current simulated image into the intention recognition model, extract features through the convolution layer, the size of the convolution layer is 12×12, and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimension reduction to prevent overfitting. The size of the pooling layer is 3×3, and the intermediate feature data is obtained. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, the two-layer output results are obtained through the output layer. The first layer output results represent the movement mode, and the second layer output results represent the movement gait. Compare the output results of the two layers with the running result data and optimize the intent recognition model through the preset loss function; All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

[0031] Figure 4 A schematic diagram of the intention recognition model of the present invention is shown; like Figure 4 As shown in the figure, the final output result is a two-layer output. The first layer is the output result R1, which indicates the current motion mode, and the result range is 1 to 6: 1 indicates normal walking, 2 indicates standing and resting, 3 indicates sitting, 4 indicates squatting, 5 indicates going up stairs, and 6 indicates going down stairs. The second layer is the output result R2, which indicates the current motion gait, and the result range is: 1 to 8 indicates 8 different gaits. The motion gait here is based on the traditional 8-point cutting method, corresponding to 8 states.

[0032] According to an embodiment of the present invention, in the double-layer neural network, the first layer of the network is provided with 20 neural units, and the second layer of the network is provided with 9 neural units.

[0033] According to an embodiment of the present invention, a plurality of selected simulated image data are randomly selected from the first image data, data is cleared and zeroed according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, and second image data is generated, and the second image data and the operation result data are imported into the intent recognition model for secondary model training, specifically: randomly selecting a plurality of selected simulated image data from the first image data; Setting a plurality of preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data accounts for a certain preset percentage in the selected analog image data; For each preset percentage, a data clearing operation is performed on a plurality of selected simulation image data respectively; After the data clearing operation, the second image data is obtained.

[0034] It should be noted that, in the process of making the amount of zeroed data reach a preset percentage and generating the second image data, specifically, some simulated image data are selected, the data is zeroed, and the percentage of the amount of data to be cleared and zeroed needs to be set in multiple ways, limited by a preset percentage, to simulate the situation in which some sensors may lack data in actual data collection. The second image data is obtained by clearing data based on the first image data, and the first image data can be backed up based on data security.

[0035] According to an embodiment of the present invention, the construction of a GAN-based generative model extracts corresponding one-dimensional feature vectors based on the first image data and the second image data, and integrates the feature vectors to obtain a one-dimensional feature vector set, imports the one-dimensional feature vector set into the generative model, trains the generative model, and generates simulated feature vector data based on the trained generative model, and imports the simulated feature vector data into the intent recognition model for training, specifically: Build a GAN-based generative model, which includes a generator and a discriminator; Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set; Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation; Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error back propagation algorithm until the generator and discriminator reach Nash equilibrium; Generate a preset amount of simulated feature vector data through the trained generative model.

[0036] It should be noted that, based on user rehabilitation training, due to factors such as the small number of user groups and low motion feature extraction capabilities, it is difficult for existing technologies to obtain more feature training data, which further affects the optimization and iteration capabilities of the consciousness recognition model, and it is difficult to generate accurate lower limb equipment training programs for different users. Based on this, the present invention intercepts a one-dimensional feature vector in the CNN intention recognition model, intercepts and integrates the feature data sets for both the first image data and the second image data, and performs feature learning and simulated data generation through the GAN model; in addition, the flattened one-dimensional data has a high degree of feature information, and the data volume is low, and the main feature accounts for a high proportion. Using this feature for simulation can generate simulated data with low redundancy, and using the simulated feature vector data for training can greatly improve the training efficiency and shorten the training and model parameter tuning time.

[0037] According to an embodiment of the present invention, the user module data is collected in real time through the intention recognition model and imported into the recognition model for intention recognition, and the device operation mode is automatically set based on the recognition result, specifically: Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated by multi-joint awareness to generate the user's real-time limb evaluation status; According to the user's real-time limb assessment status, the corresponding operation mode is set for the equipment module and a rehabilitation plan is generated; The equipment module includes 1 power supply module and 6 limb modules.

[0038] According to an embodiment of the present invention, the generating of simulated feature vector data based on the trained generative model further includes: Collect module data of N channels of the target user before and after rehabilitation training in real time, and generate simulated image data before training and simulated image data after training; Based on the intention recognition model, feature extraction is performed on the pre-training simulated image data and the post-training simulated image data, and the corresponding one-dimensional feature vectors are marked respectively to obtain a first feature vector and a second feature vector; A low-dimensional mapping space is constructed by an autoencoder, a low-dimensional mapping is performed on the first eigenvector and the second eigenvector to form low-dimensional identification data, and the low-dimensional identification data is reconstructed based on a multi-layer perceptron to form a reconstructed first eigenvector and a reconstructed second eigenvector respectively; Divide the simulated feature vector into a plurality of feature data groups, and compare the feature difference of each feature data group with the reconstructed first feature vector and the reconstructed second feature vector, respectively, wherein the feature difference comparison is based on cosine distance calculation to obtain a first feature distance and a second feature distance; Setting a first preset distance value and a second preset distance value, if the first feature distance is less than the first preset distance value and the second feature distance is less than the second preset distance value, marking the feature data group as a target user training data group; All target user training data groups are screened out, and the data in the groups are integrated into target user training feature vector data.

[0039] It should be noted that, in the multiple feature data groups, the amount of data in each feature data group is consistent. In the set distance value, the first preset distance value needs to be less than the second preset distance value, so that the characteristic components of the data screened out by the simulated feature vector data fit the trained user simulated image data. The present invention screens the module data before and after the training of the target user, performs feature dimension reduction and extracts the main features through the autoencoder, and screens out the training data of the target user from the simulated feature vector data by setting the difference value, so as to realize the targeted training of individual users by modular lower limb equipment and improve the model's body consciousness recognition ability for individual users.

[0040] Figure 5 A block diagram of a modular lower limb multi-joint intention recognition system of the present invention is shown.

[0041] The second aspect of the present invention further provides a modular lower limb multi-joint intention recognition system 5, the system comprising: a memory 51, a processor 52, the memory 51 comprising a modular lower limb multi-joint intention recognition program, the modular lower limb multi-joint intention recognition program is executed by the processor 52 to implement the following steps: One power supply module and six limb modules are set for the lower limb rehabilitation equipment, and a power supply line and a signal line are set for each module to connect the equipment; During an exercise period, the signal data of each module of the target user when using the lower limb rehabilitation device is collected to form N-channel module data, and M cycles are divided based on the exercise period. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated; Construct a CNN intention recognition model, collect simulated image data of multiple motion periods in a historical time period to form first image data, and obtain corresponding operation result data, import the first image data and the operation result data into the intention recognition model for intention recognition training, the intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer, the first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, the intermediate feature data is flattened to form a one-dimensional feature vector, the one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result; Randomly select a plurality of selected simulated image data from the first image data, clear and zero the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generate second image data, and import the second image data and the operation result data into the intent recognition model for secondary model training; Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training; Through the intention recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

[0042] It is worth mentioning that the present invention proposes a multi-channel consciousness recognition algorithm model based on convolutional neural network, which can realize the simultaneous input of multiple channel data, realize the detection of the current state and gait of the human body, and can also adapt to the lack of channel data and still maintain recognition without switching modes.

[0043] The present invention combines the module data (time series data) of a stage of multi-channel data into a square matrix, which has the characteristics of an image, and recognizes and processes the data through a convolutional neural network, thereby achieving an accurate recognition effect. Considering the timeliness of data processing and the fact that the sequence data itself has continuous characteristics and does not change much, and the correlation between the front and back is strong, only one convolution layer and one pooling layer are used, and then the neural network is used to obtain the final required result data, thereby greatly reducing the amount of calculation, enabling high-speed computing to be achieved on embedded systems, and reducing the requirements for system computing power.

[0044] The algorithm model will collect sensor data with complete data (including full module collection), randomly remove or not remove data from certain channels (all removed data are set to zero as input) to implement model training. The result obtained in this way can realize user intention recognition and completion of lower limb training process when all sensor data are complete or partial data exists, achieving the effect of no need to switch modes, thereby improving the applicability of lower limb rehabilitation equipment.

[0045] According to an embodiment of the present invention, the lower limb rehabilitation equipment is set with one power supply module and six limb modules, and each module is provided with multiple sensors to obtain module data.

[0046] It should be noted that the lower limb rehabilitation equipment is set with 1 power supply module and 6 limb modules. Based on the needs of lower limb rehabilitation, the number of various modules can be appropriately increased.

[0047] According to an embodiment of the present invention, in the generation of N×M dimensional analog image data based on the module data of M cycles and N channels, for each channel, the corresponding M cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

[0048] It should be noted that the N×M dimensional simulated image data includes N×M data points.

[0049] According to an embodiment of the present invention, the CNN intention recognition model is constructed, and within a historical time period, simulated image data of multiple motion periods are collected to form first image data, and corresponding operation result data are obtained, and the first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, and the intermediate feature data is flattened to form a one-dimensional feature vector, and the one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result, specifically: In a historical time period, simulated image data of a plurality of motion time periods are collected to form first image data, and a current simulated image is selected from the first image data; Import the current simulated image into the intention recognition model, extract features through the convolution layer, the size of the convolution layer is 12×12, and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimension reduction to prevent overfitting. The size of the pooling layer is 3×3, and the intermediate feature data is obtained. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, the two-layer output results are obtained through the output layer. The first layer output results represent the movement mode, and the second layer output results represent the movement gait. Compare the output results of the two layers with the running result data and optimize the intent recognition model through the preset loss function; All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

[0050] According to an embodiment of the present invention, in the double-layer neural network, the first layer of the network is provided with 20 neural units, and the second layer of the network is provided with 9 neural units.

[0051] According to an embodiment of the present invention, a plurality of selected simulated image data are randomly selected from the first image data, data is cleared and zeroed according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, and second image data is generated, and the second image data and the operation result data are imported into the intent recognition model for secondary model training, specifically: randomly selecting a plurality of selected simulated image data from the first image data; Setting a plurality of preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data accounts for a certain preset percentage in the selected analog image data; For each preset percentage, a data clearing operation is performed on a plurality of selected simulation image data respectively; After the data clearing operation, the second image data is obtained.

[0052] It should be noted that, in the process of making the amount of zeroed data reach a preset percentage and generating the second image data, specifically, some simulated image data are selected, the data is zeroed, and the percentage of the amount of data to be cleared and zeroed needs to be set in multiple ways, limited by a preset percentage, to simulate the situation in which some sensors may lack data in actual data collection. The second image data is obtained by clearing data based on the first image data, and the first image data can be backed up based on data security.

[0053] According to an embodiment of the present invention, the construction of a GAN-based generative model extracts corresponding one-dimensional feature vectors based on the first image data and the second image data, and integrates the feature vectors to obtain a one-dimensional feature vector set, imports the one-dimensional feature vector set into the generative model, trains the generative model, and generates simulated feature vector data based on the trained generative model, and imports the simulated feature vector data into the intent recognition model for training, specifically: Build a GAN-based generative model, which includes a generator and a discriminator; Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set; Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation; Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error back propagation algorithm until the generator and discriminator reach Nash equilibrium; Generate a preset amount of simulated feature vector data through the trained generative model.

[0054] It should be noted that, based on user rehabilitation training, due to factors such as the small number of user groups and low motion feature extraction capabilities, it is difficult for existing technologies to obtain more feature training data, which further affects the optimization and iteration capabilities of the consciousness recognition model, and it is difficult to generate accurate lower limb equipment training programs for different users. Based on this, the present invention intercepts a one-dimensional feature vector in the CNN intention recognition model, intercepts and integrates the feature data sets for both the first image data and the second image data, and performs feature learning and simulated data generation through the GAN model; in addition, the flattened one-dimensional data has a high degree of feature information, and the data volume is low, and the main feature accounts for a high proportion. Using this feature for simulation can generate simulated data with low redundancy, and using the simulated feature vector data for training can greatly improve the training efficiency and shorten the training and model parameter tuning time.

[0055] According to an embodiment of the present invention, the user module data is collected in real time through the intention recognition model and imported into the recognition model for intention recognition, and the device operation mode is automatically set based on the recognition result, specifically: Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated by multi-joint awareness to generate the user's real-time limb evaluation status; According to the user's real-time limb assessment status, the corresponding operation mode is set for the equipment module and a rehabilitation plan is generated; The equipment module includes 1 power supply module and 6 limb modules.

[0056] The third aspect of the present invention also provides a computer-readable storage medium, which includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by a processor, the steps of the modular lower limb multi-joint intention recognition method as described in any one of the above are implemented.

[0057] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0058] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0059] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0060] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0061] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0062] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A modular lower limb multi-joint intention recognition method, characterized in that: include: One power supply module and six limb modules are set for the lower limb rehabilitation equipment, and a power supply line and a signal line are set for each module to connect the equipment; During an exercise period, the signal data of each module of the target user when using the lower limb rehabilitation device is collected to form N-channel module data, and M cycles are divided based on the exercise period. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated; Construct a CNN intention recognition model, collect simulated image data of multiple motion periods in a historical time period to form first image data, and obtain corresponding operation result data, import the first image data and the operation result data into the intention recognition model for intention recognition training, the intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer, the first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, the intermediate feature data is flattened to form a one-dimensional feature vector, the one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result; Randomly select a plurality of selected simulated image data from the first image data, clear and zero the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generate second image data, and import the second image data and the operation result data into the intent recognition model for secondary model training; Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training; Through the intention recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

2. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The lower limb rehabilitation equipment is configured with one power supply module and six limb modules, and each module is provided with multiple sensors to obtain module data.

3. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: In the generation of N×M dimensional analog image data based on M cycles and N channels of module data, for each channel, the corresponding M cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

4. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The CNN intention recognition model is constructed, within a historical time period, simulated image data of multiple motion periods are collected to form first image data, and corresponding operation result data are obtained, and the first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, and the intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result, specifically: In a historical time period, simulated image data of a plurality of motion time periods are collected to form first image data, and a current simulated image is selected from the first image data; Import the current simulated image into the intention recognition model, extract features through the convolution layer, the size of the convolution layer is 12×12, and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimension reduction to prevent overfitting. The size of the pooling layer is 3×3, and the intermediate feature data is obtained. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, the two-layer output results are obtained through the output layer. The first layer output results represent the movement mode, and the second layer output results represent the movement gait. Compare the output results of the two layers with the running result data and optimize the intent recognition model through the preset loss function; All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

5. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: In the double-layer neural network, the first layer of the network is provided with 20 neural units, and the second layer of the network is provided with 9 neural units.

6. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The method randomly selects a plurality of selected simulated image data from the first image data, performs data clearing and zeroing on the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generates second image data, and imports the second image data and the operation result data into the intention recognition model for secondary model training, specifically: randomly selecting a plurality of selected simulated image data from the first image data; Setting a plurality of preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data accounts for a certain preset percentage in the selected analog image data; For each preset percentage, a data clearing operation is performed on a plurality of selected simulation image data respectively; After the data clearing operation, the second image data is obtained.

7. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The GAN-based generative model is constructed, corresponding one-dimensional feature vectors are extracted based on the first image data and the second image data, and the feature vectors are integrated to obtain a one-dimensional feature vector set, the one-dimensional feature vector set is imported into the generative model, the generative model is trained, and simulated feature vector data is generated based on the trained generative model, and the simulated feature vector data is imported into the intent recognition model for training, specifically: Build a GAN-based generative model, which includes a generator and a discriminator; Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set; Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation; Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error back propagation algorithm until the generator and discriminator reach Nash equilibrium; Generate a preset amount of simulated feature vector data through the trained generative model.

8. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The intention recognition model is used to collect user module data in real time and import it into the recognition model for intention recognition, and the device operation mode is automatically set based on the recognition result, specifically: Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated by multi-joint awareness to generate the user's real-time limb evaluation status; According to the user's real-time limb assessment status, the corresponding operation mode is set for the equipment module and a rehabilitation plan is generated; The equipment module includes 1 power supply module and 6 limb modules.

9. A modular lower limb multi-joint intention recognition system, characterized in that: The system includes: a memory and a processor, wherein the memory includes a modular lower limb multi-joint intention recognition program, and when the modular lower limb multi-joint intention recognition program is executed by the processor, the following steps are implemented: One power supply module and six limb modules are set for the lower limb rehabilitation equipment, and a power supply line and a signal line are set for each module to connect the equipment; During an exercise period, the signal data of each module of the target user when using the lower limb rehabilitation device is collected to form N-channel module data, and M cycles are divided based on the exercise period. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated; Construct a CNN intention recognition model, collect simulated image data of multiple motion periods in a historical time period to form first image data, and obtain corresponding operation result data, import the first image data and the operation result data into the intention recognition model for intention recognition training, the intention recognition model includes a convolution layer, a pooling layer, a double-layer neural network and an output layer, the first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data, the intermediate feature data is flattened to form a one-dimensional feature vector, the one-dimensional feature vector is passed through the double-layer neural network and the output layer to obtain a two-layer output result; Randomly select a plurality of selected simulated image data from the first image data, clear and zero the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generate second image data, and import the second image data and the operation result data into the intent recognition model for secondary model training; Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training; Through the intention recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by a processor, the steps of the modular lower limb multi-joint intention recognition method as described in any one of claims 1 to 8 are implemented.

Images