A method and system for predicting motion trajectory of an upper limb rehabilitation robot

Through multi-layer time dependence and sequence feature extraction method, combined with LSTM and Transformer modules, the problems of low accuracy in prediction of rehabilitation movement trajectory of the robotic arm upper limbs and delay in force feedback are solved, and high-precision and stable real-time prediction are achieved.

CN118664607BActive Publication Date: 2024-11-05JIANGXI QIUSHI INST OF ADVANCED STUDIES
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411154871.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-05
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

In the prior art, when predicting the upper limb rehabilitation trajectory of the robot arm, the end-drive method results in low prediction accuracy, and the force feedback delay of mathematical algorithms cannot provide real-time power control.

Method used

Multi-layer time-dependent relationship extraction and multi-layer sequence feature extraction methods are adopted to obtain and preprocess the user's data, and perform three-dimensional feature extraction, LSTM and Transformer module processing. Combined with the cross attention mechanism, predictive output is generated to improve prediction accuracy and stability.

Benefits of technology

Effectively capture long-term dependencies in the time series, enhance the performance of time series feature data in the prediction process, improve prediction accuracy and stability, and realize real-time prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118664607B_ABST
    Figure CN118664607B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for predicting the motion trajectory of an upper limb rehabilitation robot. The method includes obtaining user usage data, preprocessing the usage data to obtain processed data, normalizing the processed data to obtain normalized data; performing three-dimensional feature extraction on the normalized data to obtain extracted data; performing multi-layer time dependency extraction on the extracted data to obtain a number of output vectors, performing multi-layer sequence feature extraction on the extracted data to obtain a number of output features; performing corresponding splicing processing on the number of output vectors and the number of output features to obtain a splicing feature set, and aggregating the splicing feature set to obtain a predicted output. The present invention can effectively capture long-term dependencies in time series and enhance the performance of time series feature data in the prediction process, thereby further improving prediction accuracy and stability and realizing real-time prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of motion prediction, and in particular relates to a motion trajectory prediction method and system for an upper limb rehabilitation robot. Background Art

[0002] For robot motion, most trajectory predictions are driven by the end point, and only the trajectory of the end point is predicted. However, the robotic arm exoskeleton is driven by multiple motors, and each joint is closely related to the movement changes of other joints. Therefore, only predicting the trajectory of the end point will affect the prediction accuracy.

[0003] At the same time, for rehabilitation exercises in the power-assisted mode, the robotic arm will provide external force compensation for the patient's upper limb movements. However, the original mathematical algorithm's compensation method cannot provide a predicted trajectory for implementation and make corresponding power-assisted control because of the short delay in force feedback. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method and system for predicting the motion trajectory of an upper limb rehabilitation robot, which are used to solve the technical problems in the prior art.

[0005] On the one hand, the present invention provides the following technical solution: a method for predicting the motion trajectory of an upper limb rehabilitation robot, wherein the upper limb rehabilitation robot comprises a base and a plurality of joint links arranged on the base, wherein the plurality of joint links are connected end to end in sequence, and relevant nodes are arranged between two adjacent joint links and between the base and the joint links, and the method comprises:

[0006] Acquiring usage data of a user, preprocessing the usage data to obtain processed data, and normalizing the processed data to obtain normalized data;

[0007] Performing three-dimensional feature extraction on the normalized data to obtain extracted data;

[0008] Performing multi-layer time dependency extraction on the extracted data to obtain a number of output vectors, and performing multi-layer sequence feature extraction on the extracted data to obtain a number of output features;

[0009] The output vectors and the output features are concatenated to obtain a concatenated feature set, and the concatenated feature sets of multiple layers are aggregated to obtain a prediction output.

[0010] Compared with the prior art, the beneficial effects of the present application are as follows: the present invention first obtains the user's usage data, pre-processes the usage data to obtain processed data, and normalizes the processed data to obtain normalized data; then performs three-dimensional feature extraction on the normalized data to obtain extracted data; then performs multi-layer time dependency extraction on the extracted data to obtain a number of output vectors, and performs multi-layer sequence feature extraction on the extracted data to obtain a number of output features; finally, the output vectors and the output features are correspondingly spliced ​​to obtain a spliced ​​feature set, and the multi-layer spliced ​​feature sets are aggregated to obtain a predicted output. The present invention can effectively capture the long-term dependencies in the time series and enhance the performance of the time series feature data in the prediction process by performing multi-layer time dependency extraction and multi-layer sequence feature extraction on the data, thereby further improving the prediction accuracy and stability and realizing real-time prediction.

[0011] Preferably, the usage data includes user identity information, start and end time of upper limb rehabilitation robot use, average movement speed, end point deviation, path ratio, user grip strength, user elbow strength, user shoulder strength, angles of each joint point, speeds of each joint point, accelerations of each joint point, and positions of each joint connecting rod.

[0012] Preferably, in the step of obtaining the user's usage data and preprocessing the usage data to obtain processed data, the preprocessing process includes sequentially performing missing value processing, outlier processing, data deduplication, and data smoothing and denoising on the usage data to obtain processed data.

[0013] Preferably, the step of performing three-dimensional feature extraction on the normalized data to obtain extracted data includes:

[0014] A first preset formula is used to perform three-dimensional feature extraction on the normalized data to obtain extracted data, wherein the first preset formula is:

[0015] ;

[0016] In the formula, To normalize the data, To extract data, is the activation function, represents the pooling layer, Represents a multi-layer 3D convolutional network.

[0017] Preferably, the step of performing multi-layer time dependency extraction on the extracted data to obtain a plurality of output vectors comprises:

[0018] Output the extracted data to the multi-layer LSTM and determine the activation value of the forget gate And the input gate activation value :

[0019] ;

[0020] ;

[0021] In the formula, represents the sigmoid function, , Represent the first and second learnable weight matrices respectively, , represent the first and second bias terms respectively, express Extract data at the moment, express Output vector at time instant;

[0022] Determine the current candidate cell state and update the cell state to obtain Cell state at a moment :

[0023] ;

[0024] In the formula, express The cell state at a moment in time, represents the hyperbolic tangent function, , Respectively represent the third learnable weight matrix and bias term;

[0025] based on Cell state at a moment Sure Output vector at time :

[0026] ;

[0027] In the formula, , They represent the fourth learnable weight matrix and bias term respectively.

[0028] Preferably, the step of performing multi-layer sequence feature extraction on the extracted data to obtain a plurality of output features comprises:

[0029] Position encoding is performed on the extracted data, and the obtained position encoding is added to the extracted data to obtain encoded data, wherein the obtained position encoding is:

[0030] ;

[0031] ;

[0032] In the formula, , are the first position code and the second position code respectively. is the encoding position, Represents the feature dimension of the extracted data, Represents dimension index;

[0033] Use the self-attention layer to calculate the key vector of the encoded data , value vector With query vector , and based on the key vector With the query vector Calculating attention scores :

[0034] ;

[0035] In the formula, represents the dimension of the key vector;

[0036] Based on the attention score with the value vector Calculate the self-attention layer output , based on the output of the self-attention layer And apply residual connection and layer normalization to get the output features of each sub-layer :

[0037] ;

[0038]

[0039] In the formula, Indicates The output features of the sub-layers are Indicates The value vector of the sublayer , represents the normalization function, To extract data.

[0040] Preferably, the step of aggregating the multiple layers of the concatenated feature sets to obtain a prediction output includes:

[0041] Each feature vector in the concatenated feature set is input into three convolutional layers of size 1×1 for feature mapping to obtain the first feature map , the second feature map With the third feature map ;

[0042] The first feature map With the second feature map Perform affine operation to obtain the attention map :

[0043] ;

[0044] For the third feature map Each position in the spatial dimension of Generate a feature set ,gather By the third feature map Center and position The eigenvectors in the same row or column;

[0045] Based on feature set With attention map Output location Aggregation results at :

[0046] ;

[0047] In the formula, Indicates the number of channels, Representing Attention Map Middle Channel Location The scalar value at Indicates channel The feature set at Indicates that the stitching feature is concentrated at position The eigenvector at , Represent the first and second learnable parameters respectively;

[0048] After merging the aggregation results of each position of a single layer, the merging results of multiple layers are aggregated to obtain the prediction output.

[0049] In a second aspect, the present invention provides the following technical solutions: a motion trajectory prediction system for an upper limb rehabilitation robot, the upper limb rehabilitation robot comprising a base and a plurality of joint links arranged on the base, the plurality of joint links being connected end to end in sequence, and relevant nodes being arranged between two adjacent joint links and between the base and the joint links, the system comprising:

[0050] A processing module, used for obtaining the usage data of the user, preprocessing the usage data to obtain processed data, and normalizing the processed data to obtain normalized data;

[0051] An extraction module, used for performing three-dimensional feature extraction on the normalized data to obtain extracted data;

[0052] A feature module, used for performing multi-layer time dependency extraction on the extracted data to obtain a plurality of output vectors, and performing multi-layer sequence feature extraction on the extracted data to obtain a plurality of output features;

[0053] The aggregation module is used to perform corresponding splicing processing on the plurality of output vectors and the plurality of output features to obtain a splicing feature set, and to perform aggregation processing on the splicing feature sets of multiple layers to obtain a prediction output.

[0054] In a third aspect, the present invention provides the following technical solution: a computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned upper limb rehabilitation robot motion trajectory prediction method when executing the computer program.

[0055] In a fourth aspect, the present invention provides the following technical solution: a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned method for predicting the motion trajectory of an upper limb rehabilitation robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A flow chart of a method for predicting motion trajectory of an upper limb rehabilitation robot provided in Embodiment 1 of the present invention;

[0058] Figure 2 A structural diagram of an upper limb rehabilitation robot provided in Embodiment 1 of the present invention;

[0059] Figure 3 A structural diagram of a prediction output model provided in Embodiment 1 of the present invention;

[0060] Figure 4 This is a flowchart of the execution of the cross-fusion module provided in the first embodiment of the present invention;

[0061] Figure 5 A structural block diagram of a motion trajectory prediction system for an upper limb rehabilitation robot provided in Embodiment 2 of the present invention;

[0062] Figure 6 A schematic diagram of the hardware structure of a computer provided in another embodiment of the present invention.

[0063] The embodiments of the present invention will be further described below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0064] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0065] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0066] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0067] In the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0068] Embodiment 1

[0069] In the first embodiment of the present invention, Figure 1 , Figure 2 As shown, a method for predicting the motion trajectory of an upper limb rehabilitation robot is provided, wherein the upper limb rehabilitation robot comprises a base and a plurality of joint connecting rods arranged on the base, wherein the plurality of joint connecting rods are sequentially connected end to end, and relevant nodes are arranged between two adjacent joint connecting rods and between the base and the joint connecting rods;

[0070] Specific as Figure 2 As stated, Figure 2 The right side in the middle is the base position, and the base is connected to the first joint connecting rod through the first joint motor, the first joint connecting rod is connected to the second joint connecting rod through the second joint motor, the second joint connecting rod is connected to the third joint connecting rod through the third joint motor, the third joint connecting rod is connected to the fourth joint connecting rod through the fourth joint motor, and the fourth joint connecting rod is connected to the fifth joint connecting rod through the fifth joint motor. For the joint point, the position between the two joint connecting rods, that is, the position of the joint motor is the joint point. At the same time, for the fifth joint connecting rod, the end away from the fifth joint motor is also the joint point. Therefore, for the shoulder four-axis rehabilitation robot, it has six joint points, five joint motors and five joint connecting rods, and according to the connection relationship, the joint point at the base is set as the first joint point, the end of the fifth joint connecting rod away from the fifth joint motor is the sixth joint point, and the remaining joint points are arranged in sequence according to the connection relationship.

[0071] The method specifically comprises:

[0072] S1. Acquire usage data of a user, pre-process the usage data to obtain processed data, and normalize the processed data to obtain normalized data;

[0073] Specifically, the usage data includes user identity information, the start and end time of the upper limb rehabilitation robot use, the average movement speed, the end point deviation, the path ratio, the user's grip strength, the user's elbow strength, the user's shoulder strength, the angles of each joint, the speed of each joint, the acceleration of each joint, and the position of each joint connecting rod, while the user identity information includes the user's ID, gender, age, etc.

[0074] Furthermore, in step S1, the preprocessing process includes sequentially performing missing value processing, outlier processing, data deduplication, and data smoothing and denoising on the usage data to obtain processed data;

[0075] The missing value processing process is mainly used to check whether there are missing values ​​in the data, delete the records with missing time data or user ID; fill the records with zero for missing other attributes; outlier processing involves using a variety of methods to identify and process anomalies in the data, and detects outliers in the data through statistical standard deviation methods and visualization tools such as box plots and scatter plots. When processing time data, the abnormal time points are recorded and deleted; for arm angle data, correction processing will be performed to adjust the values ​​that exceed the predetermined range to the boundary value; for outliers of other attributes, replacement processing is performed to replace the abnormal values ​​with the mean to ensure the accuracy and consistency of the data; and if there are duplicate records in the data, deduplication is required to ensure the uniqueness of each record; and data smoothing and denoising are performed to eliminate some noise data in the data, and the data normalization processing uses the maximum and minimum normalization method to unify the format of the data.

[0076] Since the above preprocessing process is a commonly used data processing method in the prior art, it will not be described again.

[0077] S2. performing three-dimensional feature extraction on the normalized data to obtain extracted data;

[0078] Specifically, step S2 can be implemented through a three-dimensional feature extraction module. In order to extract more comprehensive spatiotemporal features from complex historical data, the present invention designs a three-dimensional feature extraction module. When the input data is passed into the model, feature extraction is first performed through a multi-layer three-dimensional convolutional network and a pooling layer, and then the extracted three-dimensional features are subjected to residual processing to obtain the extracted data.

[0079] Wherein, the step S2 is specifically as follows:

[0080] A first preset formula is used to perform three-dimensional feature extraction on the normalized data to obtain extracted data, wherein the first preset formula is:

[0081] ;

[0082] In the formula, To normalize the data, To extract data, is the activation function, represents the pooling layer, Represents a multi-layer 3D convolutional network.

[0083] S3, performing multi-layer time dependency extraction on the extracted data to obtain a plurality of output vectors, and performing multi-layer sequence feature extraction on the extracted data to obtain a plurality of output features;

[0084] Specifically, step S3 specifically includes: S31, performing multi-layer time dependency extraction on the extracted data to obtain a number of output vectors; S32, performing multi-layer sequence feature extraction on the extracted data to obtain a number of output features.

[0085] As for step S31, it inputs the extracted data into LSTM, and uses LSTM to more effectively capture and remember the long-term dependencies in the sequence, thereby outputting a number of output vectors.

[0086] Specifically, step S31 includes:

[0087] S311, output the extracted data to the multi-layer LSTM and determine the forget gate activation value And the input gate activation value :

[0088] ;

[0089] ;

[0090] In the formula, represents the sigmoid function, , Represent the first and second learnable weight matrices respectively, , represent the first and second bias terms respectively, express Extract data at the moment, express The output vector at time instant.

[0091] S312, determine the current candidate cell state and update the cell state to obtain Cell state at a moment :

[0092] ;

[0093] In the formula, express The cell state at a moment in time, represents the hyperbolic tangent function, , They represent the third learnable weight matrix and bias term respectively.

[0094] S313, based on Cell state at a moment Sure Output vector at time :

[0095] ;

[0096] In the formula, , They represent the fourth learnable weight matrix and bias term respectively.

[0097] Specifically, the learnable matrix and bias term in the above formula can be updated through the model training process.

[0098] Among them, for step S32, it can use the Transformer module to further implement multi-layer sequence feature extraction on the extracted data, and then output a number of output features.

[0099] Step S32 specifically includes:

[0100] S321, position coding the extracted data, and adding the obtained position coding to the extracted data to obtain coded data, wherein the obtained position coding is:

[0101] ;

[0102] ;

[0103] In the formula, , are the first position code and the second position code respectively. is the encoding position, Represents the feature dimension of the extracted data, Represents a dimension index.

[0104] S322, using the self-attention layer to calculate the key vector of the encoded data , value vector With query vector , and based on the key vector With the query vector Calculating attention scores :

[0105] ;

[0106] In the formula, Indicates the dimension of the key vector.

[0107] S323, based on the attention score with the value vector Calculate the self-attention layer output , based on the output of the self-attention layer And apply residual connection and layer normalization to get the output features of each sub-layer :

[0108] ;

[0109]

[0110] In the formula, Indicates The output features of the sub-layers are Indicates The value vector of the sublayer , represents the normalization function, To extract data.

[0111] S4, performing corresponding splicing processing on the plurality of output vectors and the plurality of output features to obtain a splicing feature set, and performing aggregation processing on the multi-layer splicing feature sets to obtain a prediction output;

[0112] Specifically, for the above LSTM and Transformer, they have a multi-layer structure, so there are also several output vectors and output features. Therefore, the output vectors and output features of each layer are concatenated to obtain a concatenated feature set. .

[0113] Wherein, the step S4 comprises:

[0114] S41, input each feature vector in the concatenated feature set into three convolutional layers of size 1×1 for feature mapping, so as to obtain first feature maps respectively. , the second feature map With the third feature map ;

[0115] Specifically, in this application, a cross-attention mechanism is used to achieve aggregation between features.

[0116] S42, mapping the first feature With the second feature map Perform affine operation to obtain the attention map :

[0117] .

[0118] S43: For the third feature map Each position in the spatial dimension of Generate a feature set ,gather By the third feature map Center and position The feature vectors in the same row or column.

[0119] S44, based on feature set With attention map Output location Aggregation results at :

[0120] ;

[0121] In the formula, Indicates the number of channels, Representing Attention Map Middle Channel Location The scalar value at Indicates channel The feature set at Indicates that the stitching feature is concentrated at position The eigenvector at , Represent the first and second learnable parameters respectively;

[0122] Specifically, by adding contextual information to multi-layer outputs, the pixel-by-pixel representation is enhanced, a broad contextual view is provided, and multi-step spatiotemporal feature information is selectively aggregated based on the spatial attention map, thereby improving the prediction accuracy.

[0123] S45, after merging the aggregation results of each position of a single layer, aggregate the merging results of multiple layers to obtain a prediction output.

[0124] Specifically, for the above steps, the three-dimensional feature extraction module, the multi-layer prediction module composed of multiple Transformers and LSTMs, and the cross-fusion module in the above steps constitute the prediction output model in the present application, the three-dimensional feature extraction module executes step S2, the multi-layer prediction module executes step S3, and the cross-fusion module executes step S4, and the model structure diagram and the execution flow chart of the cross-fusion model are respectively as follows: Figure 3 , Figure 4 As shown in the figure, after determining the prediction output model, historical training sets from multiple users can be collected and the model can be trained with the historical training sets. During the training process, the weights are updated by the Adam optimizer, which effectively improves the convergence speed and generalization ability of the model. At the same time, evaluation indicators such as RMSE (root mean square error) are used to measure the performance of the model in prediction accuracy, ensuring that the model achieves the expected performance level during the training phase and providing a reliable foundation for subsequent practical applications.

[0125] The upper limb rehabilitation robot motion trajectory prediction method provided in Example 1 of the present invention first obtains the user's usage data, pre-processes the usage data to obtain processed data, and normalizes the processed data to obtain normalized data; then performs three-dimensional feature extraction on the normalized data to obtain extracted data; then performs multi-layer time dependency extraction on the extracted data to obtain a number of output vectors, and performs multi-layer sequence feature extraction on the extracted data to obtain a number of output features; finally, the output vectors and the output features are correspondingly spliced ​​to obtain a spliced ​​feature set, and the multi-layer spliced ​​feature sets are aggregated to obtain a predicted output. The present invention can effectively capture the long-term dependencies in the time series and enhance the performance of the time series feature data in the prediction process by performing multi-layer time dependency extraction and multi-layer sequence feature extraction on the data, thereby further improving the prediction accuracy and stability and realizing real-time prediction.

[0126] Embodiment 2

[0127] like Figure 5 As shown, in the second embodiment of the present invention, a motion trajectory prediction system for an upper limb rehabilitation robot is provided, wherein the upper limb rehabilitation robot comprises a base and a plurality of joint connecting rods arranged on the base, wherein the plurality of joint connecting rods are sequentially connected end to end, and relevant nodes are arranged between two adjacent joint connecting rods and between the base and the joint connecting rods, and the system comprises:

[0128] Processing module 1, used for obtaining the usage data of the user, preprocessing the usage data to obtain processed data, and normalizing the processed data to obtain normalized data;

[0129] Extraction module 2, used for performing three-dimensional feature extraction on the normalized data to obtain extracted data;

[0130] Feature module 3, used for performing multi-layer time dependency extraction on the extracted data to obtain a plurality of output vectors, and performing multi-layer sequence feature extraction on the extracted data to obtain a plurality of output features;

[0131] Aggregation module 4 is used to perform corresponding splicing processing on the plurality of output vectors and the plurality of output features to obtain a splicing feature set, and to perform aggregation processing on the splicing feature sets of multiple layers to obtain a prediction output.

[0132] Wherein, the extraction module 2 is specifically used for:

[0133] A first preset formula is used to perform three-dimensional feature extraction on the normalized data to obtain extracted data, wherein the first preset formula is:

[0134] ;

[0135] In the formula, To normalize the data, To extract data, is the activation function, represents the pooling layer, Represents a multi-layer 3D convolutional network.

[0136] The feature module 3 includes:

[0137] The activation value determination submodule is used to output the extracted data to the multi-layer LSTM and determine the forget gate activation value And the input gate activation value :

[0138] ;

[0139] ;

[0140] In the formula, represents the sigmoid function, , Represent the first and second learnable weight matrices respectively, , denote the first and second bias terms respectively, express Extract data at the moment, express Output vector at time instant;

[0141] The update submodule is used to determine the current candidate cell state and update the cell state to obtain Cell state at a moment :

[0142] ;

[0143] In the formula, express The cell state at a moment in time, represents the hyperbolic tangent function, , Respectively represent the third learnable weight matrix and bias term;

[0144] Vector output submodule for Cell state at a moment Sure Output vector at time :

[0145] ;

[0146] In the formula, , They represent the fourth learnable weight matrix and bias term respectively.

[0147] The feature module 3 also includes:

[0148] The encoding submodule is used to perform position encoding on the extracted data and add the obtained position encoding to the extracted data to obtain encoded data, wherein the obtained position encoding is:

[0149] ;

[0150] ;

[0151] In the formula, , are the first position code and the second position code respectively. is the encoding position, Represents the feature dimension of the extracted data, Represents dimension index;

[0152] The score submodule is used to calculate the key vector of the encoded data using a self-attention layer , value vector With query vector , and based on the key vector With the query vector Calculating attention scores :

[0153] ;

[0154] In the formula, represents the dimension of the key vector;

[0155] The feature output submodule is used to output the attention score based on the with the value vector Calculate the self-attention layer output , based on the output of the self-attention layer And apply residual connection and layer normalization to get the output features of each sub-layer :

[0156] ;

[0157]

[0158] In the formula, Indicates The output features of the sub-layers are Indicates The value vector of the sublayer , represents the normalization function, To extract data.

[0159] The aggregation module 4 includes:

[0160] The convolution submodule is used to input each feature vector in the concatenated feature set into three convolution layers of size 1×1 for feature mapping, so as to obtain the first feature map respectively. , the second feature map With the third feature map ;

[0161] Affine submodule, used to map the first feature With the second feature map Perform affine operation to obtain the attention map :

[0162] ;

[0163] A collection submodule for mapping the third feature Each position in the spatial dimension of Generate a feature set ,gather By the third feature map Center and position The eigenvectors in the same row or column;

[0164] Aggregation submodule for feature collection With attention map Output location Aggregation results at :

[0165] ;

[0166] In the formula, Indicates the number of channels, Representing Attention Map Middle Channel Location The scalar value at Indicates channel The feature set at Indicates that the stitching feature is concentrated at position The eigenvector at , Represent the first and second learnable parameters respectively;

[0167] Merge After merging the aggregation results of each position of a single layer, the merging results of multiple layers are aggregated to obtain the prediction output.

[0168] In some other embodiments of the present invention, the embodiments of the present invention provide the following technical solutions: a computer, comprising a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101; when the processor 101 executes the computer program, the upper limb rehabilitation robot motion trajectory prediction method as described above is implemented.

[0169] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0170] Among them, the memory 102 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 102 may be inside or outside the data processing device. In a specific embodiment, the memory 102 is a non-volatile memory. In a specific embodiment, the memory 102 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0171] The memory 102 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101 .

[0172] The processor 101 implements the above-mentioned upper limb rehabilitation robot motion trajectory prediction method by reading and executing the computer program instructions stored in the memory 102.

[0173] In some embodiments, the computer may further include a communication interface 103 and a bus 100. Figure 6 As shown, the processor 101, the memory 102, and the communication interface 103 are connected via a bus 100 and communicate with each other.

[0174] The communication interface 103 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication interface 103 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0175] The bus 100 includes hardware, software or both, and couples the components of the computer device to each other. The bus 100 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 100 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0176] The computer can execute the upper limb rehabilitation robot motion trajectory prediction method of the present application based on the acquired upper limb rehabilitation robot motion trajectory prediction system, thereby realizing the inverse kinematics solution of the shoulder four-axis rehabilitation robot.

[0177] In some further embodiments of the present invention, in combination with the above-mentioned upper limb rehabilitation robot motion trajectory prediction method, the embodiments of the present invention provide the following technical solutions, a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned upper limb rehabilitation robot motion trajectory prediction method is implemented.

[0178] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0179] More specific examples of readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0180] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0181] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0182] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for predicting the motion trajectory of an upper limb rehabilitation robot, the upper limb rehabilitation robot comprising a base and a plurality of joint links arranged on the base, the plurality of joint links being connected end to end in sequence, and relevant nodes being arranged between two adjacent joint links and between the base and the joint links, characterized in that: The method comprises: Acquiring usage data of a user, preprocessing the usage data to obtain processed data, and normalizing the processed data to obtain normalized data; Performing three-dimensional feature extraction on the normalized data to obtain extracted data; Performing multi-layer time dependency extraction on the extracted data to obtain a number of output vectors, and performing multi-layer sequence feature extraction on the extracted data to obtain a number of output features; Performing corresponding splicing processing on a plurality of the output vectors and a plurality of the output features to obtain a splicing feature set, and performing aggregation processing on the splicing feature sets of multiple layers to obtain a prediction output; The step of performing three-dimensional feature extraction on the normalized data to obtain extracted data comprises: A first preset formula is used to perform three-dimensional feature extraction on the normalized data to obtain extracted data, wherein the first preset formula is: ; In the formula, To normalize the data, To extract data, is the activation function, represents the pooling layer, Represents a multi-layer 3D convolutional network; The step of performing multi-layer time dependency extraction on the extracted data to obtain a plurality of output vectors comprises: Output the extracted data to the multi-layer LSTM and determine the activation value of the forget gate And the input gate activation value : ; ; In the formula, represents the sigmoid function, , Represent the first and second learnable weight matrices respectively, , represent the first and second bias terms respectively, express Extract data at the moment, express Output vector at time instant; Determine the current candidate cell state and update the cell state to obtain Cell state at a moment : ; In the formula, express The cell state at a moment in time, represents the hyperbolic tangent function, , Respectively represent the third learnable weight matrix and bias term; based on Cell state at a moment Sure Output vector at time : ; In the formula, , Respectively represent the fourth learnable weight matrix and bias term; The step of performing multi-layer sequence feature extraction on the extracted data to obtain a plurality of output features comprises: Position encoding is performed on the extracted data, and the obtained position encoding is added to the extracted data to obtain encoded data, wherein the obtained position encoding is: ; ; In the formula, , are the first position code and the second position code respectively. is the encoding position, Represents the feature dimension of the extracted data, Represents dimension index; Use the self-attention layer to calculate the key vector of the encoded data , value vector With query vector , and based on the key vector With the query vector Calculating attention scores : ; In the formula, represents the dimension of the key vector; Based on the attention score with the value vector Calculate the self-attention layer output , based on the output of the self-attention layer And apply residual connection and layer normalization to get the output features of each sub-layer : ; In the formula, Indicates The output features of the sub-layers are Indicates The value vector of the sublayer , represents the normalization function, To extract data; The step of aggregating the multiple layers of the concatenated feature sets to obtain a prediction output includes: Each feature vector in the concatenated feature set is input into three convolutional layers of size 1×1 for feature mapping to obtain the first feature map , the second feature map With the third feature map ; The first feature map With the second feature map Perform affine operation to obtain the attention map : ; For the third feature map Each position in the spatial dimension of Generate a feature set ,gather By the third feature map Center and position The eigenvectors in the same row or column; Based on feature set With attention map Output location Aggregation results at : ; In the formula, Indicates the number of channels, Representing Attention Map Middle Channel Location The scalar value at Indicates channel The feature set at Indicates that the stitching feature is concentrated at position The eigenvector at , Represent the first and second learnable parameters respectively; After merging the aggregation results of each position of a single layer, the merging results of multiple layers are aggregated to obtain the prediction output.

2. The method for predicting the motion trajectory of an upper limb rehabilitation robot according to claim 1, characterized in that: The usage data includes user identity information, start and end time of upper limb rehabilitation robot use, average movement speed, end point deviation, path ratio, user grip strength, user elbow strength, user shoulder strength, angles of each joint point, speeds of each joint point, accelerations of each joint point, and positions of each joint connecting rod.

3. The method for predicting the motion trajectory of an upper limb rehabilitation robot according to claim 1, characterized in that: In the step of obtaining the user's usage data and preprocessing the usage data to obtain processed data, the preprocessing process includes sequentially performing missing value processing, outlier processing, data deduplication, and data smoothing and denoising on the usage data to obtain processed data.

4. An upper limb rehabilitation robot motion trajectory prediction system, the system adopts the upper limb rehabilitation robot motion trajectory prediction method as claimed in claim 1, the upper limb rehabilitation robot comprises a base and a plurality of joint links arranged on the base, the plurality of joint links are connected end to end in sequence, and relevant nodes are arranged between two adjacent joint links and between the base and the joint links, characterized in that: The system comprises: A processing module, used for obtaining the usage data of the user, preprocessing the usage data to obtain processed data, and normalizing the processed data to obtain normalized data; An extraction module, used for performing three-dimensional feature extraction on the normalized data to obtain extracted data; A feature module, used for performing multi-layer time dependency extraction on the extracted data to obtain a plurality of output vectors, and performing multi-layer sequence feature extraction on the extracted data to obtain a plurality of output features; The aggregation module is used to perform corresponding splicing processing on the plurality of output vectors and the plurality of output features to obtain a splicing feature set, and to perform aggregation processing on the splicing feature sets of multiple layers to obtain a prediction output.

5. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for predicting the motion trajectory of an upper limb rehabilitation robot according to any one of claims 1 to 3 is implemented.

6. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for predicting the motion trajectory of an upper limb rehabilitation robot according to any one of claims 1 to 3 is implemented.