A modular lower limb multi-joint intention recognition method and system

Abstract

The present invention discloses a modular lower limb multi-joint intention recognition method and system. The present invention relates to lower limb rehabilitation equipment, which includes multiple modules and the modules are connected to signal lines through power supply lines. The equipment collects the user's module signal data to form multi-channel data, and divides it into multiple cycles based on the exercise period to generate matrix-based simulated image data. The simulated image data and operation results of the historical period are used to train the consciousness model. Furthermore, the second image data is generated by clearing and zeroing the data to perform secondary training of the model. At the same time, a generative model based on GAN is constructed, and the feature vectors are integrated for training to generate simulated feature vector data, which is used to expand the training data. Finally, the user data is collected in real time through the intention model and the device operation mode is automatically set. Through the present invention, accurate recognition of the user's lower limb consciousness is achieved, the requirements for system computing power are reduced, and at the same time, the applicability of lower limb rehabilitation equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of limb equipment and data analysis, and more specifically, to a modular lower limb multi-joint intention recognition method and system. Background Art

[0002] Neurological diseases such as stroke are seriously affecting people's health. With the improvement of people's living standards, the incidence of stroke has further increased and is showing a trend of younger age. Therefore, the prevention and treatment of stroke and the rehabilitation of motor function after stroke are particularly important, especially the motor rehabilitation of the lower limbs, which is related to whether people can move freely and is more prominent. At the same time, trauma, disability and other problems can also lead to motor dysfunction, which further increases people's demand for rehabilitation training. Traditional methods use manual techniques to achieve limb rehabilitation of patients, but traditional techniques are inefficient and have poor effects. Therefore, rehabilitation robots have gradually emerged, which have effects that are not inferior to manual ones, and are more efficient and comfortable.

[0003] However, at the current stage of walking rehabilitation, modular robots have the characteristic of free reorganization, which means that the algorithm input will have multiple inputs, and the algorithm is difficult to adapt to different situations. Most of the current algorithms use preset algorithms with fixed modes, which cannot effectively utilize data and occupy a huge capacity. When there are many modules, it is difficult to cover all module combinations and achieve better use effects. In addition, it is impossible to identify the user's lower limb movement intentions and it is difficult to match and analyze the best training plan. Summary of the Invention

[0004] The present invention overcomes the defects of the prior art and proposes a modular lower limb multi-joint intention recognition method and system.

[0005] A first aspect of the present invention provides a modular lower limb multi-joint intention recognition method, comprising:

[0006] The lower limb rehabilitation equipment is equipped with one power supply module and six limb modules, and each module is equipped with a power supply line and a signal line to connect the equipment.

[0007] During an exercise period, the signal data of each module of the target user using the lower limb rehabilitation device is collected to form N-channel module data. The exercise period is divided into M cycles. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated.

[0008] Construct a CNN intent recognition model. Within a historical time period, collect simulated image data from multiple motion periods to form first image data, and obtain corresponding operation result data. Import the first image data and operation result data into the intent recognition model for intent recognition training. The intent recognition model includes a convolutional layer, a pooling layer, a two-layer neural network, and an output layer. The first image data is subjected to feature extraction through the convolutional layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result.

[0009] Randomly selecting a plurality of selected simulated image data from the first image data, clearing and zeroing the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generating second image data, and importing the second image data and the operation result data into the intent recognition model for secondary model training;

[0010] Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training;

[0011] Through the intent recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

[0012] In this solution, the lower limb rehabilitation equipment is configured with one power supply module and six limb modules, and each module is provided with multiple sensors to obtain module data.

[0013] In this solution, in the generation of N×M dimensional analog image data based on the module data of M cycles and N channels, for each channel, the corresponding M cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

[0014] In this solution, the CNN intention recognition model is constructed. Within a historical time period, simulated image data of multiple motion periods are collected to form first image data, and corresponding operation result data are obtained. The first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a two-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result, specifically:

[0015] In a historical time period, simulated image data of a plurality of motion periods are collected to form first image data, and a current simulated image is selected from the first image data;

[0016] Import the current simulated image into the intent recognition model and perform feature extraction through the convolution layer. The convolution layer size is 12×12 and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimensionality reduction to prevent overfitting. The pooling layer size is 3×3. The intermediate feature data obtained is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, two layers of output results are obtained through the output layer. The first layer output result represents the movement pattern, and the second layer output result represents the movement gait.

[0017] Compare the output of the two layers with the running result data and optimize the intent recognition model using the preset loss function;

[0018] All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

[0019] In this solution, in the double-layer neural network, the first layer of the network is set with 20 neural units, and the second layer of the network is set with 9 neural units.

[0020] In this solution, a plurality of selected simulated image data are randomly selected from the first image data, and data is cleared and zeroed according to a preset percentage so that the amount of zeroed data reaches a preset percentage, and second image data is generated. The second image data and the operation result data are imported into the intent recognition model for secondary model training, specifically:

[0021] randomly selecting a plurality of selected simulated image data from the first image data;

[0022] Setting multiple preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data in the selected analog image data accounts for a certain preset percentage;

[0023] For each preset percentage, performing a data cleaning operation on a plurality of selected simulation image data respectively;

[0024] After the data clearing operation, the second image data is obtained.

[0025] In this solution, the GAN-based generative model is constructed, corresponding one-dimensional feature vectors are extracted based on the first image data and the second image data, and the feature vectors are integrated to obtain a one-dimensional feature vector set. The one-dimensional feature vector set is imported into the generative model, the generative model is trained, and simulated feature vector data is generated based on the trained generative model. The simulated feature vector data is imported into the intent recognition model for training, specifically:

[0026] Build a GAN-based generative model, which includes a generator and a discriminator;

[0027] Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set;

[0028] Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation;

[0029] Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error backpropagation algorithm until the generator and discriminator reach Nash equilibrium;

[0030] Generate simulated feature vector data of a preset amount through the trained generative model.

[0031] In this solution, the intention recognition model is used to collect user module data in real time and import it into the recognition model for intention recognition. The device operation mode is automatically set based on the recognition results. Specifically:

[0032] Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated for multi-joint awareness to generate the user's real-time limb assessment status;

[0033] According to the user's real-time limb assessment status, the corresponding operating mode is set for the equipment module and a rehabilitation plan is generated;

[0034] The equipment module includes 1 power supply module and 6 limb modules.

[0035] A second aspect of the present invention further provides a modular lower limb multi-joint intention recognition system, the system comprising: a memory and a processor, the memory comprising a modular lower limb multi-joint intention recognition program, the modular lower limb multi-joint intention recognition program being executed by the processor to implement the following steps:

[0036] The lower limb rehabilitation equipment is equipped with one power supply module and six limb modules, and each module is equipped with a power supply line and a signal line to connect the equipment.

[0037] During an exercise period, the signal data of each module of the target user using the lower limb rehabilitation device is collected to form N-channel module data. The exercise period is divided into M cycles. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated.

[0038] Construct a CNN intent recognition model. Within a historical time period, collect simulated image data from multiple motion periods to form first image data, and obtain corresponding operation result data. Import the first image data and operation result data into the intent recognition model for intent recognition training. The intent recognition model includes a convolutional layer, a pooling layer, a two-layer neural network, and an output layer. The first image data is subjected to feature extraction through the convolutional layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result.

[0039] Randomly selecting a plurality of selected simulated image data from the first image data, clearing and zeroing the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generating second image data, and importing the second image data and the operation result data into the intent recognition model for secondary model training;

[0040] Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training;

[0041] Through the intent recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

[0042] The third aspect of the present invention also provides a computer-readable storage medium, which includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by a processor, it implements the steps of the modular lower limb multi-joint intention recognition method as described in any one of the above items.

[0043] The present invention discloses a modular lower limb multi-joint intention recognition method and system. The present invention relates to lower limb rehabilitation equipment, which includes multiple modules and the modules are connected to signal lines through power supply lines. The equipment collects the user's module signal data to form multi-channel data, and divides it into multiple cycles based on the exercise period to generate matrix-based simulated image data. The simulated image data and operation results of the historical period are used to train the consciousness model. Furthermore, the second image data is generated by clearing and zeroing the data to perform secondary training of the model. At the same time, a generative model based on GAN is constructed, and the feature vectors are integrated for training to generate simulated feature vector data, which is used to expand the training data. Finally, the user data is collected in real time through the intention model and the device operation mode is automatically set. Through the present invention, accurate recognition of the user's lower limb consciousness is achieved, the requirements for system computing power are reduced, and at the same time, the applicability of lower limb rehabilitation equipment is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a modular lower limb multi-joint intention recognition method according to the present invention is shown;

[0045] Figure 2 Shows a schematic diagram of the connection of various modules of the present invention;

[0046] Figure 3 shows a schematic diagram of analog image data conversion according to the present invention;

[0047] Figure 4 Shown is a schematic diagram of the intention recognition model of the present invention;

[0048] Figure 5 A block diagram of a modular lower limb multi-joint intention recognition system of the present invention is shown. DETAILED DESCRIPTION

[0049] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0051] Figure 1 A flowchart of a modular lower limb multi-joint intention recognition method of the present invention is shown.

[0052] like Figure 1As shown, the first aspect of the present invention provides a modular lower limb multi-joint intention recognition method, comprising:

[0053] S102, setting one power supply module and six limb modules for the lower limb rehabilitation device, and setting a power supply line and a signal line for each module to connect the devices;

[0054] S104, during an exercise period, collecting signal data from various modules of the target user while using the lower limb rehabilitation device to form N-channel module data, dividing the exercise period into M cycles, and generating N×M-dimensional simulated image data based on the M cycles and the N-channel module data;

[0055] S106: Construct a CNN intent recognition model. Within a historical time period, collect simulated image data of multiple motion periods to form first image data, and obtain corresponding operation result data. The first image data and the operation result data are imported into the intent recognition model for intent recognition training. The intent recognition model includes a convolutional layer, a pooling layer, a two-layer neural network, and an output layer. The first image data is subjected to feature extraction through the convolutional layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result.

[0056] S108, randomly selecting a plurality of selected simulated image data from the first image data, performing data clearing and zeroing on the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, and generating second image data, and importing the second image data and the operation result data into the intent recognition model for secondary model training;

[0057] S110: constructing a generative model based on a GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model. Importing the simulated feature vector data into the intent recognition model for training;

[0058] S112, through the intention recognition model, collect user module data in real time and import it into the recognition model for intention recognition, and automatically set the device operation mode based on the recognition result.

[0059] It is worth mentioning that the present invention proposes a multi-channel consciousness recognition algorithm model based on convolutional neural networks, which can realize the simultaneous input of multiple channel data, realize the detection of the current state and gait of the human body, and can also adapt to the lack of channel data and still maintain recognition without switching modes.

[0060] This method combines the modular data (time series data) from a single phase of multi-channel data into a square matrix with image-like properties. This matrix is ​​then processed through a convolutional neural network to achieve accurate recognition. Considering the timeliness of data processing and the inherent continuity and minimal variation of sequence data, with strong correlations between the preceding and following, only one convolutional layer and one pooling layer are used. The neural network is then used to obtain the final desired result data, significantly reducing the amount of computation required, enabling high-speed computing on embedded systems and lowering the requirements for system computing power.

[0061] The algorithm model will collect sensor data with complete data (including full module collection), randomly remove or not remove data from certain channels (all removed data are set to zero as input) to implement model training. The result obtained in this way can realize user intention recognition and completion of lower limb training process when all sensor data are complete or partial data exists, achieving the effect of no need to switch modes, and improving the applicability of lower limb rehabilitation equipment.

[0062] According to an embodiment of the present invention, the lower limb rehabilitation equipment is configured with one power supply module and six limb modules, and each module is provided with multiple sensors to obtain module data.

[0063] It should be noted that the lower limb rehabilitation equipment is set with 1 power supply module and 6 limb modules. Based on the needs of lower limb rehabilitation, the number of various modules can be appropriately increased.

[0064] Figure 2 Shows a schematic diagram of the connection of various modules of the present invention;

[0065] like Figure 2The left side shows the schematic diagram of the modules in the lower limb rehabilitation device, including seven modules (101-107). The right side shows the signal circuit diagram. Module 101 is the power supply module, responsible for providing power to each module. It also collects sensor data and status data from each module, detects the current state, calculates the output of each joint, and sends control instructions to each module via a signal bus. Modules 102-107 are the six limb modules, responsible for detecting sensor data from each module and sending it to the control center via the bus. Each module receives status instructions from the control center, and each module independently determines its output based on its status and current gait. The signal circuit diagram on the right side includes the power supply bus, which uses bus-based communication such as CAN or 485 communication. The power supply is also bus-based, allowing for random combination of any number of nodes in the device. All modules have two input and output sockets, and the power supply and control center have two outputs. The inputs of all modules can be connected to the outputs of another module or the control center, allowing for arbitrary module combinations. An intention recognition model is used to collect data and identify the status of all or some modules, applying control instructions to different joints based on their status.

[0066] According to an embodiment of the present invention, in the generation of N×M dimensional analog image data based on M-cycle and N-channel module data, for each channel, the corresponding M-cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

[0067] It should be noted that the N×M dimensional simulated image data includes N×M data points.

[0068] Figure 3 shows a schematic diagram of analog image data conversion according to the present invention;

[0069] like Figure 3As shown in the figure, the left side shows the specific joint information for the module data corresponding to channels 1-23. Here, both N and M are set to 23, generating a 23×23 data image, i.e., simulated image data. Simulated image data can be acquired in full or in part. Complete data refers to the neat state of all 23 channels. If only some modules are used, only some channels will have data, while others will not. Channels without data can be simply set to zero. The values ​​of N and M can be changed based on analysis needs. The data for channels 1-23 correspond to waist pitch angle, waist roll angle, waist sagittal axis angular velocity, left thigh pitch angle, left thigh roll angle, left thigh sagittal axis angular velocity, right thigh pitch angle, right thigh roll angle, right thigh sagittal axis angular velocity, left calf pitch angle, left calf roll angle, left calf sagittal axis angular velocity, right thigh pitch angle, right thigh roll angle, right thigh sagittal axis angular velocity, left hip joint angle, left hip joint angular velocity, right hip joint angle, right hip joint angular velocity, left knee joint angle, left knee joint angular velocity, right knee joint angle, right knee joint angular velocity.

[0070] According to an embodiment of the present invention, the CNN intention recognition model is constructed by collecting simulated image data of multiple motion periods within a historical time period to form first image data, and obtaining corresponding operation result data. The first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a two-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result, specifically:

[0071] In a historical time period, simulated image data of a plurality of motion periods are collected to form first image data, and a current simulated image is selected from the first image data;

[0072] Import the current simulated image into the intent recognition model and perform feature extraction through the convolution layer. The convolution layer size is 12×12 and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimensionality reduction to prevent overfitting. The pooling layer size is 3×3. The intermediate feature data obtained is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, two layers of output results are obtained through the output layer. The first layer output result represents the movement pattern, and the second layer output result represents the movement gait.

[0073] Compare the output of the two layers with the running result data and optimize the intent recognition model using the preset loss function;

[0074] All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

[0075] Figure 4 Shown is a schematic diagram of the intention recognition model of the present invention;

[0076] like Figure 4 As shown, the final output is a two-layer output. The first layer is output R1, which represents the current motion mode. The result ranges from 1 to 6: 1 represents normal walking, 2 represents standing and resting, 3 represents sitting, 4 represents squatting, 5 represents climbing stairs, and 6 represents descending stairs. The second layer is output R2, which represents the current motion gait. The result ranges from 1 to 8, representing eight different gaits. The motion gait here is based on the traditional 8-point division method, corresponding to eight states.

[0077] According to an embodiment of the present invention, in the double-layer neural network, the first layer of the network is set with 20 neural units, and the second layer of the network is set with 9 neural units.

[0078] According to an embodiment of the present invention, the method randomly selects a plurality of selected simulated image data from the first image data, performs data clearing and zeroing on the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generates second image data, and imports the second image data and the operation result data into the intent recognition model for secondary model training, specifically as follows:

[0079] randomly selecting a plurality of selected simulated image data from the first image data;

[0080] Setting multiple preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data in the selected analog image data accounts for a certain preset percentage;

[0081] For each preset percentage, performing a data cleaning operation on a plurality of selected simulation image data respectively;

[0082] After the data clearing operation, the second image data is obtained.

[0083] It should be noted that in generating the second image data by achieving a preset percentage of zeroed data, specifically selecting some simulated image data and performing a zeroing operation on the data, the percentage of data to be cleared of zeros may be set to various values, each limited by a preset percentage, to simulate the situation in which some sensors may lack data during actual data collection. The second image data is obtained by clearing data from the first image data, and for data security, a backup of the first image data may be provided.

[0084] According to an embodiment of the present invention, the GAN-based generative model is constructed, and corresponding one-dimensional feature vectors are extracted based on the first image data and the second image data, and the feature vectors are integrated to obtain a one-dimensional feature vector set, and the one-dimensional feature vector set is imported into the generative model. The generative model is trained, and simulated feature vector data is generated based on the trained generative model. The simulated feature vector data is imported into the intent recognition model for training, specifically:

[0085] Build a GAN-based generative model, which includes a generator and a discriminator;

[0086] Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set;

[0087] Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation;

[0088] Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error backpropagation algorithm until the generator and discriminator reach Nash equilibrium;

[0089] Generate simulated feature vector data of a preset amount through the trained generative model.

[0090] It should be noted that, in user rehabilitation training, due to factors such as the small number of user groups and low motion feature extraction capabilities, it is difficult for existing technologies to obtain more feature training data, which further affects the optimization and iteration capabilities of the consciousness recognition model, and it is difficult to generate precise lower limb equipment training plans for different users. Based on this, the present invention intercepts a one-dimensional feature vector in the CNN intention recognition model, intercepts and integrates the feature data sets for both the first image data and the second image data, and performs feature learning and simulated data generation through the GAN model; in addition, the flattened one-dimensional data has a high degree of feature information, and the data volume is low, and the main features account for a high proportion. Using this feature for simulation can generate simulated data with low redundancy, and using the simulated feature vector data for training can greatly improve training efficiency and shorten the training and model parameter tuning time.

[0091] According to an embodiment of the present invention, the user module data is collected in real time through the intention recognition model and imported into the recognition model for intention recognition, and the device operation mode is automatically set based on the recognition result, specifically:

[0092] Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated for multi-joint awareness to generate the user's real-time limb assessment status;

[0093] According to the user's real-time limb assessment status, the corresponding operating mode is set for the equipment module and a rehabilitation plan is generated;

[0094] The equipment module includes 1 power supply module and 6 limb modules.

[0095] According to an embodiment of the present invention, generating simulated feature vector data based on the trained generative model further includes:

[0096] Collect N-channel module data of the target user before and after rehabilitation training in real time, and generate pre-training simulated image data and post-training simulated image data;

[0097] Based on the intention recognition model, feature extraction is performed on the pre-training simulated image data and the post-training simulated image data, and the corresponding one-dimensional feature vectors are marked to obtain the first feature vector and the second feature vector;

[0098] A low-dimensional mapping space is constructed by an autoencoder, and the first eigenvector and the second eigenvector are low-dimensionally mapped to form low-dimensional identification data. The low-dimensional identification data is reconstructed based on a multi-layer perceptron to form a reconstructed first eigenvector and a reconstructed second eigenvector respectively.

[0099] Divide the simulated feature vector into multiple feature data groups, perform feature difference comparison between each feature data group and the reconstructed first feature vector and the reconstructed second feature vector, and calculate the feature difference based on the cosine distance to obtain the first feature distance and the second feature distance;

[0100] Setting a first preset distance value and a second preset distance value, if the first feature distance is less than the first preset distance value and the second feature distance is less than the second preset distance value, marking the feature data group as a target user training data group;

[0101] Filter out all target user training data groups and integrate the data in the groups into target user training feature vector data.

[0102] It should be noted that, among the multiple feature data groups, the amount of data in each feature data group is consistent. Among the set distance values, the first preset distance value needs to be smaller than the second preset distance value, so that the characteristic components of the data filtered out by the simulated feature vector data fit the trained user simulated image data. The present invention filters the module data before and after the training of the target user, performs feature dimensionality reduction and extracts the main features through the autoencoder, and filters the training data of the target user from the simulated feature vector data by setting the difference value, thereby realizing the targeted training of individual users by modular lower limb equipment and improving the model's ability to recognize limb awareness for individual users.

[0103] Figure 5 A block diagram of a modular lower limb multi-joint intention recognition system of the present invention is shown.

[0104] A second aspect of the present invention further provides a modular lower limb multi-joint intention recognition system 5, which includes: a memory 51 and a processor 52. The memory 51 includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by the processor 52, the following steps are implemented:

[0105] The lower limb rehabilitation equipment is equipped with one power supply module and six limb modules, and each module is equipped with a power supply line and a signal line to connect the equipment.

[0106] During an exercise period, the signal data of each module of the target user using the lower limb rehabilitation device is collected to form N-channel module data. The exercise period is divided into M cycles. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated.

[0107] Construct a CNN intent recognition model. Within a historical time period, collect simulated image data from multiple motion periods to form first image data, and obtain corresponding operation result data. Import the first image data and operation result data into the intent recognition model for intent recognition training. The intent recognition model includes a convolutional layer, a pooling layer, a two-layer neural network, and an output layer. The first image data is subjected to feature extraction through the convolutional layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result.

[0108] Randomly selecting a plurality of selected simulated image data from the first image data, clearing and zeroing the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generating second image data, and importing the second image data and the operation result data into the intent recognition model for secondary model training;

[0109] Constructing a generative model based on GAN, extracting corresponding one-dimensional feature vectors based on the first image data and the second image data, integrating the feature vectors to obtain a one-dimensional feature vector set, importing the one-dimensional feature vector set into the generative model, training the generative model, and generating simulated feature vector data based on the trained generative model, and importing the simulated feature vector data into the intent recognition model for training;

[0110] Through the intent recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

[0111] It is worth mentioning that the present invention proposes a multi-channel consciousness recognition algorithm model based on convolutional neural networks, which can realize the simultaneous input of multiple channel data, realize the detection of the current state and gait of the human body, and can also adapt to the lack of channel data and still maintain recognition without switching modes.

[0112] This method combines the modular data (time series data) from a single phase of multi-channel data into a square matrix with image-like properties. This matrix is ​​then processed through a convolutional neural network to achieve accurate recognition. Considering the timeliness of data processing and the inherent continuity and minimal variation of sequence data, with strong correlations between the preceding and following, only one convolutional layer and one pooling layer are used. The neural network is then used to obtain the final desired result data, significantly reducing the amount of computation required, enabling high-speed computing on embedded systems and lowering the requirements for system computing power.

[0113] The algorithm model will collect sensor data with complete data (including full module collection), randomly remove or not remove data from certain channels (all removed data are set to zero as input) to implement model training. The result obtained in this way can realize user intention recognition and completion of lower limb training process when all sensor data are complete or partial data exists, achieving the effect of no need to switch modes, and improving the applicability of lower limb rehabilitation equipment.

[0114] According to an embodiment of the present invention, the lower limb rehabilitation equipment is configured with one power supply module and six limb modules, and each module is provided with multiple sensors to obtain module data.

[0115] It should be noted that the lower limb rehabilitation equipment is set with 1 power supply module and 6 limb modules. Based on the needs of lower limb rehabilitation, the number of various modules can be appropriately increased.

[0116] According to an embodiment of the present invention, in the generation of N×M dimensional analog image data based on M-cycle and N-channel module data, for each channel, the corresponding M-cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

[0117] It should be noted that the N×M dimensional simulated image data includes N×M data points.

[0118] According to an embodiment of the present invention, the CNN intention recognition model is constructed by collecting simulated image data of multiple motion periods within a historical time period to form first image data, and obtaining corresponding operation result data. The first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a two-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result, specifically:

[0119] In a historical time period, simulated image data of a plurality of motion periods are collected to form first image data, and a current simulated image is selected from the first image data;

[0120] Import the current simulated image into the intent recognition model and perform feature extraction through the convolution layer. The convolution layer size is 12×12 and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimensionality reduction to prevent overfitting. The pooling layer size is 3×3. The intermediate feature data obtained is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, two layers of output results are obtained through the output layer. The first layer output result represents the movement pattern, and the second layer output result represents the movement gait.

[0121] Compare the output of the two layers with the running result data and optimize the intent recognition model using the preset loss function;

[0122] All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

[0123] According to an embodiment of the present invention, in the double-layer neural network, the first layer of the network is set with 20 neural units, and the second layer of the network is set with 9 neural units.

[0124] According to an embodiment of the present invention, the method randomly selects a plurality of selected simulated image data from the first image data, performs data clearing and zeroing on the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generates second image data, and imports the second image data and the operation result data into the intent recognition model for secondary model training, specifically as follows:

[0125] randomly selecting a plurality of selected simulated image data from the first image data;

[0126] Setting multiple preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data in the selected analog image data accounts for a certain preset percentage;

[0127] For each preset percentage, performing a data cleaning operation on a plurality of selected simulation image data respectively;

[0128] After the data clearing operation, the second image data is obtained.

[0129] It should be noted that in generating the second image data by achieving a preset percentage of zeroed data, specifically selecting some simulated image data and performing a zeroing operation on the data, the percentage of data to be cleared of zeros may be set to various values, each limited by a preset percentage, to simulate the situation in which some sensors may lack data during actual data collection. The second image data is obtained by clearing data from the first image data, and for data security, a backup of the first image data may be provided.

[0130] According to an embodiment of the present invention, the GAN-based generative model is constructed, and corresponding one-dimensional feature vectors are extracted based on the first image data and the second image data, and the feature vectors are integrated to obtain a one-dimensional feature vector set, and the one-dimensional feature vector set is imported into the generative model. The generative model is trained, and simulated feature vector data is generated based on the trained generative model. The simulated feature vector data is imported into the intent recognition model for training, specifically:

[0131] Build a GAN-based generative model, which includes a generator and a discriminator;

[0132] Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set;

[0133] Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation;

[0134] Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error backpropagation algorithm until the generator and discriminator reach Nash equilibrium;

[0135] Generate simulated feature vector data of a preset amount through the trained generative model.

[0136] It should be noted that, in user rehabilitation training, due to factors such as the small number of user groups and low motion feature extraction capabilities, it is difficult for existing technologies to obtain more feature training data, which further affects the optimization and iteration capabilities of the consciousness recognition model, and it is difficult to generate precise lower limb equipment training plans for different users. Based on this, the present invention intercepts a one-dimensional feature vector in the CNN intention recognition model, intercepts and integrates the feature data sets for both the first image data and the second image data, and performs feature learning and simulated data generation through the GAN model; in addition, the flattened one-dimensional data has a high degree of feature information, and the data volume is low, and the main features account for a high proportion. Using this feature for simulation can generate simulated data with low redundancy, and using the simulated feature vector data for training can greatly improve training efficiency and shorten the training and model parameter tuning time.

[0137] According to an embodiment of the present invention, the user module data is collected in real time through the intention recognition model and imported into the recognition model for intention recognition, and the device operation mode is automatically set based on the recognition result, specifically:

[0138] Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated for multi-joint awareness to generate the user's real-time limb assessment status;

[0139] According to the user's real-time limb assessment status, the corresponding operating mode is set for the equipment module and a rehabilitation plan is generated;

[0140] The equipment module includes 1 power supply module and 6 limb modules.

[0141] The third aspect of the present invention also provides a computer-readable storage medium, which includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by a processor, it implements the steps of the modular lower limb multi-joint intention recognition method as described in any one of the above items.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0143] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0144] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0145] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0146] Alternatively, if the integrated units described above are implemented as software modules and sold or used as standalone products, they can also be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as removable storage devices, ROM, RAM, magnetic disks, or optical disks.

[0147] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A modular lower limb multi-joint intention recognition method, characterized in that: include: The lower limb rehabilitation equipment is equipped with one power supply module and six limb modules, and each module is equipped with a power supply line and a signal line to connect the equipment. During an exercise period, the signal data of each module of the target user using the lower limb rehabilitation device is collected to form N-channel module data. The exercise period is divided into M cycles. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated. Construct a CNN intent recognition model. Within a historical time period, collect simulated image data from multiple motion periods to form first image data, and obtain corresponding operation result data. Import the first image data and operation result data into the intent recognition model for intent recognition training. The intent recognition model includes a convolutional layer, a pooling layer, a two-layer neural network, and an output layer. The first image data is subjected to feature extraction through the convolutional layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result. Randomly selecting a plurality of selected simulated image data from the first image data, clearing and zeroing the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generating second image data, and importing the second image data and the operation result data into the intent recognition model for secondary model training; Construct a GAN-based generative model, extract corresponding one-dimensional feature vectors based on the first image data and the second image data, integrate the feature vectors to obtain a one-dimensional feature vector set, import the one-dimensional feature vector set into the generative model, train the generative model, and generate simulated feature vector data based on the trained generative model. Import the simulated feature vector data into the intent recognition model for training, specifically including: Build a GAN-based generative model, which includes a generator and a discriminator; Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set; Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation; Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error backpropagation algorithm until the generator and discriminator reach Nash equilibrium; Generate a preset amount of simulated feature vector data through the trained generative model; Through the intent recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

2. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The lower limb rehabilitation equipment is configured with one power supply module and six limb modules, and each module is provided with multiple sensors to obtain module data.

3. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: In the generation of N×M dimensional analog image data based on the module data of M cycles and N channels, for each channel, the corresponding M cycle module data are serialized, and a matrix-based image is constructed in a sequence form to generate N×M dimensional analog image data.

4. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The CNN intention recognition model is constructed by collecting simulated image data of multiple motion periods within a historical time period to form first image data, and obtaining corresponding operation result data. The first image data and the operation result data are imported into the intention recognition model for intention recognition training. The intention recognition model includes a convolution layer, a pooling layer, a two-layer neural network and an output layer. The first image data is subjected to feature extraction through the convolution layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result, specifically: In a historical time period, simulated image data of a plurality of motion periods are collected to form first image data, and a current simulated image is selected from the first image data; Import the current simulated image into the intent recognition model and perform feature extraction through the convolution layer. The convolution layer size is 12×12 and 6 channels are set. After the convolution layer, the data is input into the pooling layer for feature data dimensionality reduction to prevent overfitting. The pooling layer size is 3×3. The intermediate feature data obtained is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is integrated through a two-layer neural network. Finally, two layers of output results are obtained through the output layer. The first layer output result represents the movement pattern, and the second layer output result represents the movement gait. Compare the output of the two layers with the running result data and optimize the intent recognition model using the preset loss function; All simulated image data of the first image data are sequentially imported into the intention recognition model for training and model optimization.

5. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: In the double-layer neural network, the first layer of the network is set with 20 neural units, and the second layer of the network is set with 9 neural units.

6. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The method randomly selects a plurality of selected simulated image data from the first image data, performs data clearing and zeroing on the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generates second image data, and imports the second image data and the operation result data into the intent recognition model for secondary model training, specifically: randomly selecting a plurality of selected simulated image data from the first image data; Setting multiple preset percentages, performing random data clearing and zeroing on the selected analog image data, so that the amount of zeroed data in the selected analog image data accounts for a certain preset percentage; For each preset percentage, performing a data cleaning operation on a plurality of selected simulation image data respectively; After the data clearing operation, the second image data is obtained.

7. A modular lower limb multi-joint intention recognition method according to claim 1, characterized in that: The intention recognition model is used to collect user module data in real time and import it into the recognition model for intention recognition, and the device operation mode is automatically set based on the recognition result, specifically: Through the recognition results, the user's current running mode and movement gait are analyzed, and the running mode and movement gait are evaluated for multi-joint awareness to generate the user's real-time limb assessment status; According to the user's real-time limb assessment status, the corresponding operating mode is set for the equipment module and a rehabilitation plan is generated; The equipment module includes 1 power supply module and 6 limb modules.

8. A modular lower limb multi-joint intention recognition system, characterized in that: The system includes: a memory and a processor. The memory includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by the processor, the following steps are implemented: The lower limb rehabilitation equipment is equipped with one power supply module and six limb modules, and each module is equipped with a power supply line and a signal line to connect the equipment. During an exercise period, the signal data of each module of the target user using the lower limb rehabilitation device is collected to form N-channel module data. The exercise period is divided into M cycles. Based on the M cycles and the N-channel module data, N×M-dimensional simulated image data is generated. Construct a CNN intent recognition model. Within a historical time period, collect simulated image data from multiple motion periods to form first image data, and obtain corresponding operation result data. Import the first image data and operation result data into the intent recognition model for intent recognition training. The intent recognition model includes a convolutional layer, a pooling layer, a two-layer neural network, and an output layer. The first image data is subjected to feature extraction through the convolutional layer and the pooling layer to obtain intermediate feature data. The intermediate feature data is flattened to form a one-dimensional feature vector. The one-dimensional feature vector is passed through the two-layer neural network and the output layer to obtain a two-layer output result. Randomly selecting a plurality of selected simulated image data from the first image data, clearing and zeroing the selected simulated image data according to a preset percentage, so that the amount of zeroed data reaches a preset percentage, generating second image data, and importing the second image data and the operation result data into the intent recognition model for secondary model training; Construct a GAN-based generative model, extract corresponding one-dimensional feature vectors based on the first image data and the second image data, integrate the feature vectors to obtain a one-dimensional feature vector set, import the one-dimensional feature vector set into the generative model, train the generative model, and generate simulated feature vector data based on the trained generative model. Import the simulated feature vector data into the intent recognition model for training, specifically including: Build a GAN-based generative model, which includes a generator and a discriminator; Based on the intention recognition model, feature extraction is performed on the first image data and the second image data, and the obtained one-dimensional feature vectors are integrated to form a one-dimensional feature vector set; Perform data standardization preprocessing on the one-dimensional feature vector set, import it into the generator to generate simulated feature data, and import the generated simulated data into the discriminator for probability evaluation; Perform cyclic adversarial generative training on the generator and discriminator, and update the parameters of the generator and discriminator based on the error backpropagation algorithm until the generator and discriminator reach Nash equilibrium; Generate a preset amount of simulated feature vector data through the trained generative model; Through the intent recognition model, user module data is collected in real time and imported into the recognition model for intent recognition, and the device operation mode is automatically set based on the recognition results.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a modular lower limb multi-joint intention recognition program. When the modular lower limb multi-joint intention recognition program is executed by the processor, the steps of the modular lower limb multi-joint intention recognition method as described in any one of claims 1 to 7 are implemented.

Images