Muscle deformation signal and exoskeleton joint angle mapping system and method

Abstract

The application provides a muscle deformation signal and exoskeleton joint angle mapping system and method, which utilizes the multi-head self-attention algorithm of the transformer to extract data correlation features by performing three linear transformations on the vector of each position in the input sequence respectively. By extracting five muscle deformation variables around the knee pad of the knee joint, the angle of the human knee joint is mapped to achieve accurate prediction of the knee joint angle. The application can quickly collect and analyze data, and can be used offline. The muscle deformation data around the knee pad of the knee joint is generated by a specific method and mapped to the angle of the human knee joint to ensure quality and consistency. The original signal is coded, dimensioned, and fused with a multi-layer perceptron to realize knee joint function evaluation and accurately infer the angle of the knee joint from the muscle deformation data. The muscle deformation data is serialized to facilitate model processing and analysis. The accuracy of the algorithm is verified by collecting data from a flexible sensor to determine the exoskeleton motor parameters.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation and intelligent assistive devices, specifically to a transformer-based mapping system and method for human knee joint muscle deformation signals and exoskeleton joint angles. Background Technology

[0002] In the fields of medical rehabilitation and intelligent assistive devices, accurately acquiring human muscle deformation signals and mapping them precisely to exoskeleton joint angles is of great significance. Previous technologies were slow in data acquisition, making it difficult to obtain muscle deformation information in real time and quickly, resulting in insufficient data timeliness and usability. Furthermore, the lack of efficient algorithms and models in data analysis and processing makes it difficult to accurately extract key features from muscle deformation data and accurately infer knee joint angles.

[0003] Meanwhile, the lack of targeted and standardized data generation methods leads to inconsistent data quality and difficulty in ensuring consistency, affecting the accuracy and reliability of the model. Furthermore, the organization of muscle deformation data is not reasonable, hindering model processing and analysis. In determining the exoskeleton motor parameters, there is also a lack of effective data support and accurate verification methods, making it difficult to fully guarantee the accuracy of the algorithm.

[0004] In summary, existing technologies have many shortcomings in mapping muscle deformation signals to exoskeleton joint angles, and cannot fully realize the role of exoskeleton devices in assisting movement and rehabilitation. There is an urgent need for an innovative system and method to solve these problems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention aims to establish a human-machine collaborative Transformer model, and to generate data on the deformation of muscles around the knee joint and map it to the angle of the human knee joint.

[0006] A mapping system between muscle deformation signals and exoskeleton joint angles provided by the present invention includes: a sensor module, a Transformer model, an exoskeleton device, a control module, a communication module, a motor drive module, and a harmonic reducer;

[0007] Sensor module, used to collect muscle deformation signals;

[0008] The Transformer model is used to process the received muscle deformation signals using a multi-head attention mechanism, and then combined with a multilayer perceptron to train and analyze the processed features and knee joint angle.

[0009] Exoskeleton devices used to assist in the movement of human joints;

[0010] After the sensor module collects the muscle deformation signal, it sends it to the control module through the communication module, and then inputs it into the Transformer model to obtain the output result. The control module then sends the output result to the motor drive module through the communication module, and the motor drive module controls the harmonic reducer to drive the exoskeleton device to move.

[0011] Preferably, the sensor module includes five distributed strain sensors disposed on the knee brace body. The sensors include a stretchable base layer and a stretchable conductor unit. The stretchable base layer is laid on the knee brace body, and the stretchable conductor unit is used to generate changes in electrical properties in response to the deformation of the stretchable base layer.

[0012] Preferably, the data processing flow of the Transformer model is as follows: First, the data is encoded and the encoded and upgraded data is transmitted to the encoder. In the encoder, the data features are calculated through a multi-head self-attention mechanism. After the feature calculation is completed, the knee joint angle information is combined with the multilayer perceptron neural network for training and analysis.

[0013] The algorithm process includes data dimensionality enhancement, multi-head self-attention computation, layer normalization and residual connection, data dimensionality reduction, data fusion and multilayer perceptron;

[0014] The encoder consists of multiple stacked encoder layers. Each encoder layer includes a multi-head self-attention mechanism, layer normalization, and residual connection operations. Each attention head of the multi-head self-attention mechanism calculates different attention weights, thereby capturing the correlation between different parts of the input data.

[0015] Preferably, the exoskeleton device includes an ultra-flat joint actuator composed of four DC brushless external rotor disc motors. The ultra-flat joint actuator is equipped with a Hall sensor and a harmonic reducer of matching size. The ultra-flat joint actuator is powered by 24V DC voltage.

[0016] The ultra-flat joint actuators correspond one-to-one with the positions of each major joint. In the lower limb, two ultra-flat joint actuators are placed at each of the left and right hip joints, and one ultra-flat joint actuator is placed at each of the left and right knee joints.

[0017] Preferably, the host computer of the control module is a PC, the slave computer is an STM32F103VET6 microcontroller, and the communication module uses both CAN Open communication and serial communication. The control flow of the control module is as follows: after the sensor module collects the muscle deformation signal, it sends the data to the PC through the serial port, inputs the data into the Transformer model, calculates the angle, and then the PC sends the data to the STM32F103VET6 microcontroller through the serial port. The microcontroller then sends the data to the motor drive module through CAN Open communication, thereby enabling the motor drive module to control the harmonic reducer for transmission.

[0018] This invention provides a method for mapping muscle deformation signals to exoskeleton joint angles. The method utilizes the aforementioned mapping system for muscle deformation signals and exoskeleton joint angles and includes the following steps:

[0019] S1, extract feature data from the input muscle deformation signal to obtain key feature vectors;

[0020] S2, using a multi-head attention layer to process the key feature vectors, obtains the feature representation of the key focus area;

[0021] S3, perform layer standardization on the feature representation to obtain standardized feature data;

[0022] S4, using residual connection to add the feature representation and feature data to obtain the added feature data;

[0023] S5, perform dimensionality reduction on the summed feature data to obtain the dimensionality-reduced data;

[0024] S6, perform feature enhancement on the dimensionality-reduced features to obtain the enhanced data;

[0025] S7. Based on the increased data, data mapping is performed to obtain the mapping results between muscle deformation signals and exoskeleton joint angles.

[0026] Preferably, in step S1, the input muscle deformation signal is {a1, a2, a3, a4, a5}, where a1 to a5 are data contents and are integer data;

[0027] Transpose the data {a1, a2, a3, a4, a5} to get (a1, a2, a3, a4, a5). T A matrix with 5 rows and 1 column is multiplied by a trainable coefficient matrix with 1 row and n columns to obtain matrix A, as shown in the following formula:

[0028]

[0029] In matrix A, each row of elements abstractly represents its original data through a coefficient matrix.

[0030] Preferably, in step S2, the self-attention mechanism is implemented using three matrices (query matrix Q, key matrix K, and value matrix V). The input is a matrix containing a sequence, typically represented as (X) (with a shape of ((n, d)), where (n) is the sequence length and (d) is the feature dimension). The input is transformed through three different linear transformations to obtain the query matrix (Q), key matrix (K), and value matrix (V). The expression formula for the attention mechanism is as follows:

[0031]

[0032] Where Q (Query), K (Key), and V (Value) are input vectors, the formula represents performing a linear transformation on K and V, then calculating the dot product of Q and K, obtaining the attention weights after softmax normalization, and then multiplying them with V to obtain the final output;

[0033] Multi-head self-attention is an extension of self-attention, linearly transforming the input vectors Q (Query), K (Key), and V (Value) into multiple heads, each with a reduced dimensionality, typically d / h, where d is the dimension of the input vector and h is the number of heads. The process of multi-head self-attention includes the following steps:

[0034] Step 1: Each head independently calculates the attention mechanism;

[0035] Step 2: Concatenate the outputs of all the heads, perform a linear transformation, and obtain the final output;

[0036] The formula is expressed as:

[0037] MultiHead(Q,K,V)=Concat(head1,head2,...,head n W o (3)

[0038] The calculation process for each head is as follows:

[0039] head i =Attention(Q) i K i V i )#(4)

[0040] Multi-head self-attention is performed on the upgraded data, and the upgraded matrix A is dot-multiplied by three different linear transformation matrices W. Q W K W V Thus, we obtain three matrices Q, K, and V, as shown in the following formula:

[0041]

[0042] Among them, the three matrices Q, K, and V are 5-row m-column matrices, and the value of m should be an integer multiple of the number of heads;

[0043] Then, the three matrices Q, K, and V are split. The number of matrices resulting from the split is the corresponding head number. The matrix is ​​split into h heads, and the number of columns in each matrix should be... The corresponding formula is as follows:

[0044]

[0045] As shown above, the three matrices Q, K, and V are split into h-headed matrices, resulting in Q1, ..., Q. h ,K1,…K h V1,…V h Each of these matrices is subjected to self-attention weights to obtain matrix Z. According to the self-attention calculation formula, firstly, Q1,…Q h ,K1,…K h Multiplying them separately, the formula is as follows:

[0046]

[0047] Performing a softmax transformation on the resulting matrix, we get:

[0048]

[0049] Here, we perform softmax normalization on the row elements of the matrix so that the sum of the elements in each row is 1.

[0050]

[0051]

[0052] Next, let's discuss the dot product V of the matrix after the softmax transformation:

[0053]

[0054] in,

[0055] In multi-head attention calculations, the Q of each head... i K i V i Self-attention calculation is required to obtain matrix Z. i Finally, the matrix Z of each head will be obtained. i The matrices are concatenated to obtain matrix Z, which is used to calculate the multi-head self-attention function. The formula is as follows:

[0056]

[0057] Preferably, in step S3, the layer normalization formula is as follows:

[0058]

[0059] Where E(x) is the mean of x, σ(x) is the standard deviation of x, ε is a very small number to prevent the denominator from being 0; γ and β are learnable parameters.

[0060] After multi-head attention calculation is completed, we need to perform layer normalization on each row of the matrix, as shown in the following formula:

[0061]

[0062] in, for:

[0063]

[0064] According to the layer normalization formula, γ is 1, β is 0, and ε is (1e-6), which is 10 to the power of -6. 1e-6 is scientific notation, which is 1 multiplied by 10 to the power of -6.

[0065] Preferably, in step S4, the normalized layer is... The matrix is ​​added to the initially upgraded matrix A. Let A be a 1x1m matrix and A be a 1x1n matrix. Let n = m. The matrix output from the multi-head attention layer does not need to be multiplied by W. o The residual join formula after dimensional transformation of the matrix is ​​shown below:

[0066]

[0067] In step S5, make The formula for reducing the data to one row and five columns is shown below:

[0068]

[0069] Then matrix C is the matrix after dimensionality reduction;

[0070] Step S6: After dimensionality reduction of the encoder output data, the reduced features are added to the original data, i.e., feature enhancement, as shown in the following formula:

[0071] (a1,a2,a3,a4,a5)+(c1,c2,c3,c4,c5)=(α1,α2,α3,α4,α5) (21)

[0072] Step S7: Combine the encoder output with the knee angle and send it to the multilayer sensor for training and analysis to achieve knee joint function assessment.

[0073] The activation function to choose is the leaky_relu function:

[0074]

[0075] If the input x is greater than 0, the output is x; if the input x is less than or equal to 0, the output is α multiplied by the input.

[0076] Let the knee angle be y, and combine it with the data output from the encoder to form a neural network, as shown in the following formula:

[0077]

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] 1. This invention establishes a human-machine collaborative Transformer model, which can quickly collect and parse data, and can be used offline.

[0080] 2. This invention employs a specific training data generation method to generate muscle deformation data around the knee brace and map it to the angle of the human knee joint, ensuring the quality and consistency of the data.

[0081] 3. This invention encodes and upscales the original signal, uses a multi-head attention algorithm to extract features from the encoded muscle change signal, and then fuses the dimensionality-reduced signal with the original data. It also combines a multilayer perceptron (MLP) to train and analyze the extracted features and knee joint angle, thereby achieving accurate knee joint function assessment and enabling it to accurately infer the knee joint angle from muscle deformation data.

[0082] 4. This invention transforms muscle deformation data from raw time-domain signals into serialized data, enabling the model to better process and analyze the data.

[0083] 5. This invention can determine the precise parameters of the exoskeleton motor by acquiring data from flexible sensors, thereby verifying the accuracy of the algorithm. Attached Figure Description

[0084] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0085] Figure 1 This is a flowchart illustrating the overall process of the human-machine collaboration model based on the transformer in this embodiment of the invention.

[0086] Figure 2 This is a flowchart of the multi-head attention algorithm in an embodiment of the present invention;

[0087] Figure 3 This is a flexible sensor for measuring human muscle deformation in an embodiment of the present invention;

[0088] Figure 4 This is a structural diagram of the exoskeleton device in an embodiment of the present invention;

[0089] Figure 5 This is a control flowchart in an embodiment of the present invention;

[0090] Figure 6 This is a physical diagram of the mapping system between muscle deformation signals and exoskeleton joint angles in an embodiment of the present invention.

[0091] Figure 7 This is the overall driving flowchart in an embodiment of the present invention. Detailed Implementation

[0092] This invention introduces a Transformer multi-head attention mechanism, which weights the importance of different parts of a feature, improving the model's perceptual and feature representation capabilities, thereby enabling coordinated movement of human muscles and exoskeleton joint angles. The multi-head self-attention mechanism can improve the detection accuracy of object detection networks such as Faster R-CNN, reducing false negatives and false positives, thus establishing a human-machine collaborative Transformer model. By extracting five muscle deformation variables around the knee brace and correlating them with the angle of the human knee joint, accurate prediction of the knee joint angle is achieved. First, we collected large-scale data on muscle deformation around the knee brace and knee joint angles, and performed rigorous data preprocessing to ensure data quality and consistency. Then, feature extraction was performed on the muscle change signals around the knee joint, and a multilayer perceptron (MLP) was used to train and analyze the extracted features and knee joint angles, enabling the inference of the knee joint angle from the muscle deformation data. Finally, through experimental testing, the muscle deformation data can be mapped to the exoskeleton joint angle, completing the data acquisition and prediction for the experimental testing section.

[0093] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0094] Example 1:

[0095] Figure 1This is a flowchart illustrating the overall process of the human-machine collaboration model based on the transformer in this embodiment of the invention.

[0096] like Figure 1 As shown, this embodiment of the invention converts the muscle motion signals around the knee into angles. Features of the muscle change signals around the knee joint are extracted and enhanced using a multi-head attention mechanism in the Transformer encoder module. Then, a multilayer perceptron (MLP) is used to train and analyze the enhanced features and knee joint angles, thereby achieving accurate knee joint function assessment.

[0097] This invention provides a mapping system between muscle deformation signals and exoskeleton joint angles, including a sensor module, a Transformer model, an exoskeleton device, a control module, a communication module, a motor drive module, and a harmonic reducer.

[0098] The sensor module includes five distributed strain sensors installed on the knee brace body. The sensors include a stretchable base layer and a stretchable conductor unit. The stretchable base layer is laid on the knee brace body, and the stretchable conductor unit is used to generate changes in electrical properties in response to the deformation of the stretchable base layer.

[0099] Figure 3 This is a flexible sensor for measuring human muscle deformation in an embodiment of the present invention.

[0100] like Figure 3 As shown, the sensor in this embodiment is a flexible sensor for wearable knee braces. By setting distributed strain sensors on the knee brace body, the stretchable conductor unit can be deformed when the knee brace deforms, thereby generating changes in electrical properties. The deformation of the knee brace can be mapped through the changes in electrical properties, which has high accuracy and is not easily affected by external interference.

[0101] The wearable knee brace sensors collect deformation data from five muscles around the knee joint, which also serve as the input to this system. When the knee begins to bend, it stretches or compresses the five sensors on the knee brace. These sensors detect this deformation and return an analog signal.

[0102] In summary, the distributed strain sensors on the knee brace can acquire data in real time by monitoring the deformation variables of five muscle regions around the knee joint. These sensors work as follows: when the knee begins to bend or undergoes other forms of movement, the five sensors on the knee brace are subjected to varying degrees of stretching or compression. These sensors convert this deformation into electrical signals and return analog data. This data can be used to analyze knee joint movement patterns.

[0103] The data processing flow of the Transformer model is as follows: First, the data is encoded and then transmitted to the encoder. In the encoder, the data features are calculated through a multi-head self-attention mechanism. After the feature calculation is completed, the knee joint angle information is combined with the multilayer perceptron neural network for training and analysis.

[0104] Figure 2 This is a flowchart of the multi-head attention algorithm in an embodiment of the present invention.

[0105] like Figure 2 As shown, the algorithm flow is as follows: data dimensionality enhancement (data encoding) → multi-head self-attention calculation → layer normalization and residual connection → data dimensionality reduction → data fusion → multilayer perceptron.

[0106] The encoder of the Transformer model consists of multiple stacked encoder layers. Each encoder layer includes a multi-head self-attention mechanism (MSM), layer normalization (LN), and residual connection (RC) operations. In the multi-head self-attention mechanism, each attention head calculates different attention weights, thereby capturing the correlation between different parts of the input data.

[0107] Exoskeletons are devices used to assist in the movement of human joints. The weight of lower limb exoskeletons needs to be borne by the human body to varying degrees, so the structural design needs to be small in size and lightweight. At the same time, the design of the exoskeleton should minimize the interference or collision of the mechanical structure with the human lower limbs and the surrounding environment.

[0108] Figure 4 This is a control flowchart of the exoskeleton device in an embodiment of the present invention.

[0109] like Figure 4 As shown, based on the number of joints involved in the movement of the lower body legs, the exoskeleton device in this embodiment includes an ultra-flat joint actuator composed of four DC brushless external rotor disc motors. Each ultra-flat joint actuator is equipped with a Hall sensor and a harmonic reducer of matching size. Under a rated output speed of 60 rpm, the ultra-flat joint actuator outputs a torque of 23 N·m, with a maximum instantaneous output torque of 80 N·m. It can also maintain continuous operation for 5 seconds under overload conditions, providing sufficient power-off time for unexpected situations. The joint actuator is powered by 24V DC and can utilize common switching power supplies and lithium batteries. The weight of a single joint actuator is 940g.

[0110] Furthermore, the ultra-flat joint actuators correspond one-to-one with the positions of each major joint. In the lower limb, two ultra-flat joint actuators are placed at each of the left and right hip joints, and one ultra-flat joint actuator is placed at each of the left and right knee joints. Therefore, a single lower limb exoskeleton has 4 degrees of freedom. The degrees of freedom include rotation of the femur, flexion and extension of the knee.

[0111] The motor drive module is a key component in controlling the operation of the ultra-flat joint actuator (motor), which is one of the core components of the exoskeleton device.

[0112] The host computer for the control module is a PC, and the slave computer is an STM32F103VET6 microcontroller.

[0113] The communication module uses both CAN Open communication and serial communication.

[0114] Figure 5 This is a control flowchart in an embodiment of the present invention.

[0115] like Figure 5 As shown, the control flow of this embodiment of the invention is as follows:

[0116] After the sensor module acquires muscle deformation signals, it sends the data to the PC via serial port. The data is then input into the Transformer model, and the angle is calculated. The PC then sends the data to the STM32F103VET6 microcontroller via serial port. The microcontroller then sends the data to the motor drive module via CAN Open communication, thereby enabling the motor drive module to control the harmonic reducer for transmission. In this embodiment, the knee rotation angle is aligned with the hip joint rotation angle; whatever angle the motor rotates at the knee, the motor at the hip joint rotates accordingly.

[0117] Example 2:

[0118] This invention applies the muscle deformation signal-exoskeleton joint angle mapping system described in Embodiment 1 above. It utilizes a multi-head self-attention algorithm using transformers, a technique widely used in natural language processing and machine learning. This algorithm performs a cubic linear transformation on the vector at each position in the input sequence to generate a query matrix Q, a key matrix K, and a value matrix V, and then performs multi-head attention calculations to extract relevant data features. The method involves extracting five muscle deformation variables around the knee brace and mapping them to the angle of the human knee joint, thereby achieving accurate prediction of the knee joint angle.

[0119] This invention provides a method for mapping muscle deformation signals to exoskeleton joint angles, comprising the following steps:

[0120] S1 extracts feature data from the input muscle deformation signal to obtain key feature vectors.

[0121] The device for collecting data on changes in the muscles around the knee is a wearable smart knee brace. This knee brace is equipped with distributed strain sensors, which cause the stretchable conductor units to deform during knee deformation, resulting in changes in electrical properties. These changes in electrical properties are then used to map the knee brace's deformation. The knee brace integrates a data processing unit capable of collecting and processing sensor data in real time and transmitting the data to external devices via a wireless module. The data returned by the knee brace is in the format {a1, a2, a3, a4, a5}, where a1 to a5 represent the data content, which is integer data.

[0122] To process this one-dimensional data, we need to reshape it into a two-dimensional space. Therefore, we need to encode the data to increase its dimensionality and then map it to the two-dimensional space using a trainable linear projection. First, we transpose the data "{a1,a2,a3,a4,a5}" to become (a1,a2,a3,a4,a5). T A 5x1 matrix. Then, multiply this matrix by a 1xn matrix of trainable coefficients, as shown in the following formula:

[0123]

[0124] From the above formula, we can see that in matrix A, each row of elements can be abstractly represented by a coefficient matrix, which represents its original data. For example, the data in the first row of matrix A (a... 11 …a 1n ) is linearly expressed through a coefficient matrix w, a1, (a 21 …a 2n ) is linearly expressed through a coefficient matrix w, (a 31 …a 3n ) is linearly expressed through a coefficient matrix w, (a 41 …a 4n ) is linearly expressed through a coefficient matrix w, (a 51 …a 5n a5 is expressed linearly through a coefficient matrix w.

[0125] This is essentially equivalent to encoding the raw data. The purpose of encoding the raw data is to map one-dimensional data to a multi-dimensional space so that it can be processed by more complex models and capture more features and information.

[0126] By transforming the original one-dimensional vector into a two-dimensional matrix, we enhance the expressive power of the data. A linear transformation (dot product operation) allows each element of the original data to be mapped to a new space through the weight matrix. In this encoding mechanism, w (the weight matrix) is a trainable parameter, meaning that during training, the value of w is continuously adjusted through backpropagation so that the model can better fit the data. This allows the encoding process to dynamically adapt to the data. Through training, the parameters in the weight matrix are tuned to their optimal state, enabling the encoded representation to be better used for downstream tasks. Trainable encoding parameters give the model greater flexibility and representational power when handling different datasets, thereby improving overall performance.

[0127] By upscaling a one-dimensional vector into a multi-dimensional matrix, the feature representation capability is enhanced, allowing each original data point to be expressed more richly through the coefficient matrix. This upscaling operation makes the data suitable for the input requirements of sequence-to-sequence models like the Transformer, as these models typically deal with multi-dimensional feature matrices rather than simple one-dimensional vectors. This transformation ensures that the data can be processed more effectively by the model.

[0128] Through linear mapping, each piece of raw data can be mapped to a multidimensional space via a coefficient matrix, thereby capturing more information and features and improving the model's performance.

[0129] In summary, the above operations are equivalent to encoding the original data, mapping one-dimensional vector data to a high-dimensional matrix through transpose and linear transformation operations. This encoding method not only enhances the expressive power of the data but also helps adapt to the input requirements of the model. Furthermore, through trainable parameters, the encoding process dynamically adapts to the data, thereby improving the overall performance of the model.

[0130] S2 utilizes a multi-head attention layer to process the key feature vectors, obtaining the feature representation of the key focus area.

[0131] Self-attention is one of the core mechanisms of the Transformer model, enabling the model to dynamically focus on different parts of the input sequence. The self-attention mechanism allows each position in the sequence to pay attention to all other positions in the sequence through a weighted sum. This mechanism is implemented using three matrices (query matrix Q, key matrix K, and value matrix V).

[0132] The input is a matrix containing a sequence, typically represented as (X) (with a shape of ((n,d)), where (n) is the sequence length and (d) is the feature dimension). The input is then subjected to three different linear transformations to obtain the query matrix (Q), the key matrix (K), and the value matrix (V). The attention mechanism can then be represented by a simple formula:

[0133]

[0134] Where Q (Query), K (Key), and V (Value) are the input vectors. This formula means that we perform a linear transformation on K and V, then calculate the dot product of Q and K, normalize it using softmax to obtain the attention weights, and then multiply them by V to obtain the final output.

[0135] Multi-head self-attention is an extension of attention mechanisms. It divides the input into multiple "heads" and allows the model to compute attention independently in different subspaces, thus capturing more layers of features.

[0136] Specifically, the multi-head self-attention mechanism includes the following steps:

[0137] The input vectors Q (Query), K (Key), and V (Value) are linearly transformed into multiple heads, with the dimension of each head decreasing, typically by d / h, where d is the dimension of the input vector and h is the number of heads.

[0138] Step 1: Calculate the attention mechanism independently for each head.

[0139] Step 2: Concatenate the outputs of all the heads and perform a linear transformation to obtain the final output.

[0140] The formula is expressed as:

[0141] MultiHead(Q,K,V)=Concat(head1,head2,...,head n W o (3)

[0142] The calculation process for each head is as follows:

[0143] head i =Attention(Q) i K i V i )#(4)

[0144] Multi-head self-attention is performed on the upgraded data, and the upgraded matrix A is dot-multiplied by three different linear transformation matrices W. Q W K W V Thus, we obtain three matrices Q, K, and V, as shown in the following formula:

[0145]

[0146] Among them, the three matrices Q, K, and V are 5-row m-column matrices, and the value of m should be an integer multiple of the number of heads;

[0147] Then, the three matrices Q, K, and V are split. The number of matrices resulting from the split is the corresponding head number. The matrix is ​​split into h heads, and the number of columns in each matrix should be... The corresponding formula is as follows:

[0148]

[0149] As shown above, the three matrices Q, K, and V are split into h-heads, resulting in Q1, ..., Q1. h K1, ... K h V1, ...V h Each of these matrices is subjected to self-attention weights to obtain matrix Z. According to the self-attention calculation formula, first, Q1, ... Q... h K1, ... K h Multiplying them separately, the formula is as follows:

[0150]

[0151] Performing a softmax transformation on the resulting matrix, we get:

[0152]

[0153] Here, we perform softmax normalization on the row elements of the matrix so that the sum of the elements in each row is 1.

[0154]

[0155] Multiplying the matrix after the softmax transformation by V, we get:

[0156]

[0157] in,

[0158] In multi-head attention calculations, the Q of each head... i K i V i Self-attention calculation is required to obtain matrix Z. i Finally, the matrix Z of each head will be obtained. i The matrices are concatenated to obtain matrix Z, which is used to calculate the multi-head self-attention function. The formula is as follows:

[0159]

[0160] S3 performs layer standardization on the feature representation to obtain standardized feature data.

[0161] Layer normalization normalizes the feature dimensions of each input sample, ensuring that each feature dimension has the same distribution across different samples. This differs from batch normalization (BatchNorm), which normalizes each feature within a single batch.

[0162] In neural network training, normalization refers to transforming the input data to make its distribution more consistent with a certain standard or uniform distribution, thereby helping to improve the training effect of the model.

[0163] Common normalization methods include batch normalization and layer normalization. Batch normalization normalizes the data at each batch level, while layer normalization normalizes the data at each layer level. Here, we use layer normalization. It can help solve problems such as gradient explosion in deep neural networks, thereby accelerating the training process and improving the model's convergence speed.

[0164] The layer normalization formula is shown below:

[0165]

[0166] Where E(x) is the mean of x, σ(x) is the standard deviation of x, ε is a very small number to prevent the denominator from being 0; γ and β are learnable parameters.

[0167] After multi-head attention calculation is completed, we need to perform layer normalization on each row of the matrix, as shown in the following formula:

[0168]

[0169] in, for:

[0170]

[0171] According to the layer normalization formula, γ is 1, β is 0, and ε is (1e-6), which is 10 to the power of -6. 1e-6 is scientific notation, which is 1 multiplied by 10 to the power of -6.

[0172] S4 uses residual connection to add the feature representation and feature data to obtain the added feature data.

[0173] Residual connections are a special type of connection that allows network layers to directly add input data to the output data, forming a "skip connection" structure. This structure makes it easier for the network to learn identity mappings during deep learning, thus avoiding the vanishing gradient problem during deep network training.

[0174] The data calculated by the multi-head self-attention part is layer normalized to prevent gradient explosion, and a residual connection is performed to prevent gradient vanishing.

[0175] We need to normalize the layers. We add the matrix to the initially upgraded matrix A, but at this point we find... Let A be a 1x1m matrix and A be a 1x1n matrix. Therefore, in this invention, we let n = m, so that the matrix output from the multi-head attention layer does not need to be multiplied by W. o The matrix undergoes dimensionality transformation. The residual join formula is shown below:

[0176]

[0177] S5. Perform dimensionality reduction on the summed feature data to obtain the dimensionality-reduced data.

[0178] In the initial data processing, all operations are based on dimensionality increase. After the encoder completes its data processing, dimensionality reduction is required to restore the data to its original dimensions. Specifically, a linear layer is applied to transform the n-column, five-row data in the sample into a single row with five columns. This process involves applying a linear transformation to the high-dimensional data, thereby compressing and transforming the data dimensions to ensure that the data returns to the expected form for subsequent processing and analysis.

[0179] We need to make The formula for reducing the data to one row and five columns is shown below:

[0180]

[0181] Then matrix C is the matrix after dimensionality reduction;

[0182] S6 performs feature enhancement on the dimensionality-reduced features to obtain the enhanced data.

[0183] It is worth noting that although the data has undergone dimensionality reduction, it is still not suitable for direct use at this stage because the encoder's job is only to extract feature representations of the data, not the original data itself. Therefore, a crucial step is to merge the features extracted by the encoder with the original data before dimensionality reduction. This process involves combining the feature representations with the actual data to ensure that the final data retains important features while also containing the original information.

[0184] During the data processing by an encoder, a set of high-level features is typically extracted. These features better represent the deeper information of the data. However, these features are not the original data itself, but rather an abstraction and condensation of the original data. The encoder's feature extraction capability can help identify potential patterns and relationships in the data, but it loses some of the original, specific information.

[0185] To compensate for this deficiency, after dimensionality reduction of the encoder output data, we add these dimensionality-reduced features to the original data. This process is called feature enhancement. The specific formula is as follows:

[0186] (a1,a2,a3,a4,a5)+(c1,c2,c3,c4,c5)=(α1,α2,α3,α4,α5) (21)

[0187] This operation ensures that each feature in each sample of the original data includes not only its own data but also data from other features within the same sample. Therefore, when performing linear regression, a feature needs to consider not only its own influence but also the influence of other features. This can improve the training performance of the neural network, making the model easier to optimize and generalize.

[0188] Feature enhancement allows us to simultaneously leverage the specific information in the original data and the high-level features extracted by the encoder, thereby improving model performance. This approach is widely used in many machine learning and deep learning tasks because it can improve model accuracy and robustness by utilizing deep features extracted by the encoder while preserving important information from the original data.

[0189] S7. Based on the increased data, data mapping is performed to obtain the mapping results between muscle deformation signals and exoskeleton joint angles.

[0190] The above process describes only one decoder layer. However, in actual operation, multiple encoder layers are used to form the encoder. In this step, the encoder output is combined with the knee angle and fed into a multilayer perceptron (MLP) for training and analysis, thereby achieving accurate knee joint function assessment.

[0191] The activation function to choose is the leaky_relu function:

[0192]

[0193] If the input x is greater than 0, the output is x; if the input x is less than or equal to 0, the output is α multiplied by the input.

[0194] Compared to the ReLU activation function, this means it solves the dead ReLU problem because the gradient value is no longer restricted to 0. Additionally, this function also avoids the vanishing gradient problem.

[0195] We define the knee angle as y, and combine it with the data output from the encoder to form a neural network, as shown in the following formula:

[0196]

[0197] Example 3:

[0198] To ensure efficient and accurate collection of deformation data of the muscles around the knee, this embodiment provides an innovative acquisition method. First, data from each frame transmitted back by the knee brace is collected, specifically in the format "{a,b,c,d,e}", where "a,b,c,d,e" represent the deformation data (integer data) of the five muscles around the knee. This data changes with the knee angle. For example, the deformation data might be "{3140,2850,1910,2320,2560}".

[0199] To train the model, we need 750 data samples ranging from knee fully extended (0 degrees) to bent at 90 degrees. However, considering the differences in sensor feedback each time the smart knee brace is worn, collecting data from each angle is not only time-consuming and laborious, but the data may also be inconsistent under different wearing conditions. Therefore, we only collect deformation data of the knee at two key angles: 0 degrees and 90 degrees.

[0200] By uniformly dividing the deformation data at 0 degrees and 90 degrees into 750 parts, and also dividing the corresponding angles into 750 parts, we can efficiently generate training samples. This method not only simplifies the data collection process but also enables online data collection and training after each wear, ensuring the model's accuracy and adaptability under different conditions. Fifty samples are extracted as the validation set.

[0201] The entire model was built in a PyTorch environment. After acquiring the knee brace data via serial port, it was parsed into training samples and sent to the model for training. Here, we need to use regression mode instead of classification. If we use classification, the angle output will only be limited to one of 750 angles, resulting in very poor accuracy. Therefore, a regression model is required. The loss function is chosen as mean absolute error. The optimizer is chosen as gradient descent (SGD) algorithm. Because the training is conducted online, the training time cannot be too long; therefore, the number of training iterations is chosen to be 1000.

[0202] The main adjustable parameters are: the dimension of the data during dimensionality upgrade, the number of heads in the multi-head self-attention system, the learning rate step size, the momentum factor, and the number of encoder layers.

[0203] Dimensionality Upscaling: To fully utilize the dimensionality-reduced feature data, we need to upscale it to an appropriate dimension for input into the various layers of the model. The choice of dimension for upscaling affects model performance, therefore multiple experiments and adjustments are necessary.

[0204] Number of attention heads in multi-head self-attention: In a model, multi-head self-attention captures the correlations between different parts of the data. Adjusting the number of attention heads can affect the depth and accuracy of the model's understanding of the data.

[0205] Learning rate step size: The learning rate step size determines the adjustment range of model parameters with each update. An appropriate learning rate step size can accelerate the model's convergence process and avoid overfitting or underfitting.

[0206] The number of encoder layers: The choice of the number of encoder layers directly affects the depth and complexity of the model. More layers can increase the model's expressive power, but may also lead to increased training time and a higher risk of overfitting.

[0207] Momentum Factor: The momentum factor is used in the gradient descent algorithm to accelerate convergence and avoid local minima. We try different momentum factors to find the best value for the current dataset.

[0208] Evaluation Metrics: A fundamental requirement during neural network training is that the loss function of the training set must converge and stabilize. When the training set loss is stable, the validation set loss function is typically slightly larger than the training set loss function, indicating good generalization ability. With the same validation set loss, a smaller training set loss is better, indicating a higher fit of the model to the training data. With the same training set loss, we want the validation set loss to be as small as possible, but it should be slightly larger than the training set loss, indicating a certain level of generalization ability.

[0209] To select suitable neural network training parameters, this invention designs a specific experimental scheme. Using the controlled variable method, when selecting one parameter, the other four parameters remain constant. Therefore, at least five sets of experiments are required to select all parameters. Each set of experiments focuses on the selection of a specific parameter. In each experiment, the selection of each parameter requires 10 trials, and the average of the 10 loss values ​​is taken as the loss value corresponding to that parameter selection.

[0210] Experiment 1: Learning Rate Selection: Among the five parameters above, the learning rate step size and momentum factor have the greatest impact on the training results, directly affecting whether the training converges. Therefore, based on experience, we started by gradually adjusting the learning rate to 0.00001 and the momentum factor to 0.5. We first conducted an experiment to select the learning rate. The remaining parameter settings are shown in Table 1, and the maximum number of training iterations was 1000. The training results are shown in Table 1.

[0211] Table 1 Learning Rate Settings

[0212]

[0213] As shown in the table above, the training results converged only when the learning rate was set to 0.0000001. Experiments revealed that either the step size was set too large, failing to find the minimum value and causing the loss function to fluctuate excessively and not converge, or it was set too small, resulting in the loss function still decreasing and not converging even when the maximum number of training iterations was reached. Therefore, we chose a learning rate of 0.0000001.

[0214] Experiment 2: Momentum Factor Selection: A larger momentum factor results in faster convergence, but if too large, it may cause the loss function to decrease too quickly, skipping the minimum and leading to large fluctuations and non-convergence. A smaller momentum factor results in slower convergence, but a more stable update process, though it may get stuck in local minima. Therefore, a balance needs to be found between these two factors to ensure that the model can stably converge to the global optimum. The training results are shown in Table 2:

[0215] Table 2 Momentum Factor Settings

[0216]

[0217] As shown in the table above, a momentum factor of 0.85 is most suitable. If it is too small, the loss function decreases too slowly and lacks sufficient momentum to escape local minima, thus affecting the model's ability to find the global optimum. If it is too large, the update magnitude becomes too aggressive, causing the loss function to decrease too quickly and skip the global minimum. This results in large fluctuations in the loss function during training, making it difficult to converge to a stable minimum.

[0218] Experiment 3: Selection of Data Dimensions: The data dimension refers to the dimension of the original data after dimensionality increase, or it can be understood as the dimension after data encoding. Too large or too small a dimension will negatively impact model performance. The training results are shown in Table 3.

[0219] Table 3 Data Dimension Settings

[0220]

[0221] As shown in the table above, based on the experimental samples of this invention, if the dimension of the data is too small, it will lead to an explosion of the gradient of the loss function; if it is slightly smaller, the loss function will be larger. If it is too large, the loss value will not only fail to converge, but overfitting will also occur. Therefore, a data dimension of 10 is most suitable.

[0222] Selection of the number of self-attention heads in Experiment 4:

[0223] In the Transformer model, the number of self-attention heads directly affects the model's representational power, computational efficiency, and ability to capture contextual information. Appropriately increasing the number of heads can improve model performance, but a balance needs to be struck between computational resources, memory requirements, and the risk of overfitting. The choice of head number needs to be optimized based on the specific task, data volume, and hardware resources. Too few heads result in poor model performance, while too many heads can lead to overfitting. The training results are shown in Table 4:

[0224] Table 4 Data Dimension Settings

[0225]

[0226] As shown in the table above, 5 self-attention heads are the most suitable. Too many heads will lead to overfitting, while too few heads will result in a slightly larger loss function.

[0227] Experiment 5: Selection of the number of encoder layers:

[0228] In the Transformer model, the number of encoder layers has a significant impact on the model's performance and behavior. The main effects of increasing the number of encoder layers are analyzed in detail below: Increasing the number of encoder layers increases the model's depth, enabling it to learn more complex and abstract representations and capture deeper features and patterns. Deeper encoder layers can progressively improve the level of feature abstraction through layer-by-layer feature transformation and combination, giving the model a stronger understanding of complex input data. Increasing the number of encoder layers also increases the risk of overfitting, especially when training data is insufficient or of low quality. In such cases, the model may overfit the training data, leading to a decrease in generalization ability. The training results are shown in Table 5:

[0229] Table 5 Encoder Layer Settings

[0230]

[0231] As shown in the table above, a encoder layer count of 6 is most suitable. Too many layers can lead to overfitting, consume more hardware resources, and increase training time. Too few layers will result in a slightly larger loss function.

[0232] Based on the above five sets of experiments, the selection of parameters such as learning rate, momentum factor, number of self-attention heads, encoder layer, and data dimension in the architecture of this invention is shown in Table 6:

[0233] Table 6 Final Parameter Selection

[0234]

[0235] At this point, the entire system setup and internal parameter selection are complete. Next, this system can be used to collect knee brace data, convert it into knee joint / motor control angles in real time, and send it to the exoskeleton control unit.

[0236] Figure 6 This is a physical diagram of the mapping system between muscle deformation signals and exoskeleton joint angles in an embodiment of the present invention. Figure 7 This is the overall driving flowchart in an embodiment of the present invention.

[0237] like Figure 6 , 7As shown, this invention provides a mapping system between muscle deformation signals and exoskeleton joint angles. It employs separate sensing devices to drive the exoskeleton motors, achieving precise sensing of lower limb joint angles. The powered exoskeleton can move in sync with the human body within the constructed collaborative motion framework, without any time delay. A method based on the Transformer model is proposed. This invention collects signals from the muscles around the knee joint, processes and analyzes the signals using the Transformer algorithm, and effectively transforms the extracted muscle signals into the knee joint angle signals required for exoskeleton control. This allows for real-time adjustment of the exoskeleton's motion mode, thereby achieving precise human-machine collaborative control.

[0238] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, enabling the system and its various devices, modules, and units to function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

[0239] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0240] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A mapping system between muscle deformation signals and exoskeleton joint angles, characterized in that, include: Sensor module, Transformer model, exoskeleton device, control module, communication module, motor drive module, and harmonic reducer; The sensor module is used to collect muscle deformation signals; The Transformer model is used to perform feature processing on the received muscle deformation signals using a multi-head attention mechanism, and then combined with a multilayer perceptron to train and analyze the processed features and knee joint angle. The exoskeleton device is used to assist in the movement of human joints; After the sensor module collects the muscle deformation signal, it sends it to the control module through the communication module, and then inputs it into the Transformer model to obtain the output result. The control module then sends the output result to the motor drive module through the communication module, and the motor drive module controls the harmonic reducer to drive the exoskeleton device to move.

2. The muscle deformation signal to exoskeleton joint angle mapping system of claim 1, wherein, The sensor module includes five distributed strain sensors disposed on the knee brace body. Each sensor includes a stretchable base layer and a stretchable conductor unit. The stretchable base layer is laid on the knee brace body, and the stretchable conductor unit is used to generate changes in electrical properties in response to the deformation of the stretchable base layer.

3. The muscle deformation signal to exoskeleton joint angle mapping system of claim 1, wherein, The data processing flow of the Transformer model is as follows: First, the data is encoded and then transmitted to the encoder. In the encoder, the data features are calculated through a multi-head self-attention mechanism. After the feature calculation is completed, the knee joint angle information is combined with the multilayer perceptron neural network for training and analysis. The algorithm process includes data dimensionality enhancement, multi-head self-attention computation, layer normalization and residual connection, data dimensionality reduction, data fusion and multilayer perceptron; The encoder comprises multiple stacked encoder layers. Each encoder layer includes a multi-head self-attention mechanism, layer normalization, and residual connection operations. Each attention head of the multi-head self-attention mechanism calculates different attention weights, thereby capturing the correlation between different parts of the input data.

4. The muscle deformation signal to exoskeleton joint angle mapping system of claim 1, wherein, The exoskeleton device includes an ultra-flat joint actuator composed of four DC brushless external rotor disc motors. The ultra-flat joint actuator is equipped with a Hall sensor and a harmonic reducer of matching size. The ultra-flat joint actuator is powered by 24V DC voltage. The ultra-flat joint actuators correspond one-to-one with the positions of each major joint. In the lower limbs, two ultra-flat joint actuators are arranged at each of the left and right hip joints, and one ultra-flat joint actuator is arranged at each of the left and right knee joints.

5. The muscle deformation signal to exoskeleton joint angle mapping system of claim 1, wherein, The host computer of the control module adopts a PC, the lower computer adopts a single-chip microcomputer STM32F103VET6, and the communication module adopts two modes of CANOpen communication and serial communication; the control flow of the control module is as follows: after the sensor module collects the muscle deformation signal, the data is sent to the PC through the serial port, the data is input into the Transformer model, the angle is calculated, the PC sends the data to the single-chip microcomputer STM32F103VET6 through the serial port, and the single-chip microcomputer sends the data to the motor driving module through CAN Open communication, so that the motor driving module controls the harmonic reducer to drive.

6. A method of mapping a muscle deformation signal to an exoskeleton joint angle, using the mapping system of any one of claims 1-5, characterized in that, Comprise the following steps: S1, the input muscle deformation signal is extracted for feature data, and a key feature vector is obtained; S2, the key feature vector is processed by using a multi-head attention layer to obtain a feature representation of a part of interest; S3, the feature representation is subjected to layer normalization to obtain normalized feature data; S4, the feature representation and the feature data are added by using residual connection to obtain added feature data; S5, the added feature data is subjected to data dimension reduction to obtain reduced data; S6, the reduced feature is subjected to feature enhancement to obtain increased data; S7, based on the increased data, data mapping is performed to obtain a mapping result of the muscle deformation signal and the exoskeleton joint angle.

7. The method of mapping muscle deformation signals to exoskeleton joint angles of claim 6, wherein, In step S1, the input muscle deformation signal is { }, wherein is the data content, which is an integer type data; The data { } is transposed to become a matrix of five rows and one column, and the matrix is point-multiplied by a trainable coefficient matrix of one row and n columns to obtain the matrix A, as shown in the following formula: In matrix A, each row element abstractly represents its original data through a coefficient matrix.

8. The method of mapping muscle deformation signals to exoskeleton joint angles of claim 6, wherein, In step S2, the self-attention mechanism is realized through three matrices; the input is a matrix containing a sequence, denoted as X, the input is subjected to three different linear transformations to obtain a query matrix, a key matrix and a value matrix, and the representation formula of the attention mechanism is: Wherein, Q, K, V are input vectors, the formula represents that K and V are linearly transformed, then the dot product of Q and K is calculated, the attention weight is obtained after softmax normalization, and then V is multiplied to obtain the final output; The multi-head self-attention mechanism is an extension of the self-attention mechanism, which linearly transforms the input vectors Q, K, V into multiple heads, and the dimension of each head is reduced, usually d / h, wherein d is the dimension of the input vector, and h is the number of heads, the process of the multi-head self-attention mechanism comprises the following steps: Step 1, each head independently calculates the attention mechanism; Step 2, the outputs of all heads are spliced, and the final output is obtained after linear transformation; The formula is as follows: Wherein, the calculation process of each head is as follows: The multi-head self-attention calculation is performed on the data after dimension increasing, and the matrix A after dimension increasing is respectively multiplied by three different linear transformation matrices , thereby obtaining three matrices, and the formula is as follows: wherein The three matrices are 5-row m-column matrices, and the value of m should be an integer multiple of the number of heads. Then the three matrices are split, the number of split matrices is the number of corresponding heads, the matrix is split into h heads, the number of columns of each matrix should be , and the corresponding formula is as follows: Splitting the three matrices as shown above after splitting into h heads, respectively get , , , respectively, the respective self-attention weight calculation matrix Z is obtained, according to the self-attention calculation formula, first , are multiplied respectively, the formula is as follows: The obtained matrix is subjected to softmax transformation, and there is: Wherein, we perform softmax normalization on the row elements of the matrix, so that the sum of each row element is 1, and there is Then, the matrix after the softmax transformation is multiplied by V, and there is: wherein, In multi-head attention calculations, each head's... , , Self-attention calculation is required to obtain the matrix. Finally, the matrix obtained for each head will be... The matrices are concatenated to obtain matrix Z, which is used to calculate the multi-head self-attention function. The formula is as follows: 。 9. The method of mapping muscle deformation signals to exoskeleton joint angles of claim 6, wherein, In step S3, the layer normalization formula is as follows: where E(x) is the mean of x, is the standard deviation of x, is a very small number, the purpose is to prevent the denominator to 0; and are learnable parameters; After the multi-head attention calculation is completed, we need to perform layer normalization on each row of the matrix, and the formula is as follows: wherein is: According to the layer normalization formula, Take 1, Take 0, Take That is 10 to the power of -6, (1e-6) is scientific notation, 1 times 10 to the power of -6.

10. The method of mapping muscle deformation signals to exoskeleton joint angles of claim 6, wherein, In step S4, the normalized matrix in step S3 is added to the A matrix after the dimension is increased at the beginning, The matrix after the dimension is increased at the beginning is added to the normalized matrix in step S3, is a matrix with 1 row and m columns, A is a matrix with 1 row and n columns, n=m, and the matrix after the multi-head attention layer is output and does not need to be multiplied by a dimension transformation matrix, and the residual connection formula is as follows: In step S5, the data is reduced to one row of five columns, as shown in the following equation: Dimensional reduction to one row of five columns, as shown in the following equation: Therefore, the C matrix is the reduced matrix; Step S6, after dimensionality reduction processing of the data output by the encoder, the dimensionally reduced features are added to the original data, i.e., feature enhancement, and the formula is as follows: Step S7, the result output by the encoder is combined with the knee angle and delivered to the multi-layer perceptron for training and analysis to realize the knee function evaluation; The activation function is selected as the leaky_relu function: If the input x is greater than 0, the output is x; If the input x is less than or equal to 0, the output is α times the input; Let the knee angle be y, and the data after the encoder output form a neural network, and the formula is as follows: 。

Images