Upper arm muscle force estimation method based on double-branch self-attention mechanism network
Through the dual-branch self-attention mechanism network and the time-frequency domain feature fusion module, the problem of insufficient detection of shallow muscle information in traditional methods is solved, and the accuracy of deep information of upper arm muscles is achieved. Estimation and high-precision muscle strength detection.
Patent Information
- Application Number
- CN202411233739.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-04
AI Technical Summary
Traditional muscle strength estimation methods can only detect shallow muscle information, ignore deep muscle information during muscle contraction, and fail to make full use of the frequency domain characteristics of electrical impedance imaging signals.
Using a dual-branch self-attention mechanism network, data is collected through electrical impedance imaging sensors and six-dimensional force sensors, Gaussian-Newtonian image reconstruction and Fourier transform, and combined with the time-frequency domain feature fusion module, the electrical impedance image and frequency domain signal are extracted. , muscle strength estimation is performed.
It realizes accurate detection of deep information of upper arm muscles, improves the accuracy and accuracy of muscle strength estimation, has the advantages of lightweight and damage-free, and is more accurate than traditional methods.
Smart Images

Figure CN119055216A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of human biological signal processing and muscle strength estimation, and in particular to an upper arm muscle strength estimation method based on a dual-branch self-attention mechanism network. Background Art
[0002] Muscle force estimation can reflect the body's movement intention, muscle fatigue level, and muscle health status, and has an important research basis for biomechanics, human-computer interaction, sports, and rehabilitation medicine. From the perspective of data acquisition methods, it can be divided into kinematics / dynamics-based and bioinformatics-based methods. In recent years, due to the rapid development of deep learning technology and its hardware cost advantages, bioinformatics-based methods have become a research hotspot.
[0003] The muscle force estimation method based on traditional biological information uses human biological signals such as electromyographic signals and muscle sound signals. In the early stage, attempts were made to extract artificially designed manual features from them for further classification and recognition. In recent years, the recognition effect of methods based on deep learning has been greatly improved compared with the early methods based on manual features. Deep neural networks, represented by convolutional neural networks, can effectively extract spatiotemporal features from data. However, these signals can only detect shallow muscle information, and crosstalk between muscles will have a negative impact on the estimation results.
[0004] As a new type of bio-information sensor, electrical impedance imaging records changes in muscle cross-sectional information based on its own characteristics of containing two-dimensional image information. At present, research based on electrical impedance imaging has covered many fields such as clinical medicine, human-computer interaction, and sensors. In the field of human-computer interaction, the sensitivity of electrical impedance imaging to impedance changes can effectively detect changes in human muscle tissue during skeletal muscle contraction, which is conducive to identification. Therefore, it is necessary to design a novel upper arm muscle force estimation method based on the characteristics of electrical impedance imaging signals. Summary of the invention
[0005] In order to solve the problem that traditional estimation methods can only detect superficial muscle information, ignore deep muscle information during muscle contraction, and ignore the frequency domain signal characteristics of electrical impedance imaging itself, the purpose of the present invention is to provide a lightweight, high estimation accuracy, non-destructive upper arm muscle strength estimation method based on a dual-branch self-attention mechanism network.
[0006] To achieve the above object, the present invention adopts the following technical solution: a method for estimating upper arm muscle strength based on a dual-branch self-attention mechanism network, comprising:
[0007] (1) In one measurement cycle, the original electrical impedance imaging voltage signal generated by the contraction of the upper arm muscles of the human body is measured by the electrical impedance imaging sensor and normalized to obtain the normalized original electrical impedance imaging voltage sequence. Among them, M 1is the total number of normalized raw electrical impedance imaging voltage sequences recorded in a measurement cycle, i is the index subscript of the normalized raw electrical impedance imaging voltage sequence at the current moment, i is an integer and 1≤i≤M 1 At the same time, the six-dimensional force sensor is used to collect the forces Fx, Fy, Fz and the moments Mz, My, Mz in three directions generated by the contraction of the upper arm muscles of the human body. Fx, Fy, Fz and Mz, My, Mz together constitute the real force sequence. The real force sequence is normalized to obtain the normalized real force sequence Among them, M 2 is the total number of true forces recorded in a measurement cycle, j is the index subscript of the true force at the current moment, j is an integer and 1≤j≤M 2 ;
[0008] (2) The normalized original electrical impedance imaging voltage sequence Perform one-step Gauss-Newton image reconstruction and Fourier transform to obtain a two-dimensional electrical impedance image. Voltage sequence frequency domain characterization Among them, M 3 is the total number of two-dimensional electrical impedance images recorded in a measurement cycle, v is the index subscript of the two-dimensional electrical impedance image at the current moment, v is an integer and 1≤v≤M 3 ;M 4 is the total number of frequency domain representations of the voltage sequence recorded in a measurement cycle, u is the index subscript of the frequency domain representation of the voltage sequence at the current moment, u is an integer and 1≤u≤M 4 ;
[0009] (3) Frequency domain characterization of voltage sequence 2D electrical impedance imaging and the normalized true force series The dataset is composed and divided into batches to obtain an electrical impedance image tensor V of size batchsize×1×64×64 IMG , the voltage frequency domain tensor V of size batchsize×1×28 fft and the real force tensor V of size batchsize×1×6 F , batchsize indicates the amount of data contained in each batch;
[0010] (4) constructing a dual-branch self-attention mechanism network, wherein the dual-branch self-attention mechanism network consists of a first branch network, a second branch network, and a time-frequency domain feature fusion module; converting the electrical impedance image tensor V IMG , voltage frequency domain tensor V fft Input the dual-branch self-attention mechanism network, and the dual-branch self-attention mechanism network outputs the muscle strength estimation result;
[0011] (5) The muscle force estimation result is compared with the actual force sequence collected by the six-dimensional force sensor, and the mean square error is used to reflect the degree of difference between the two.
[0012] The step (1) specifically comprises the following steps:
[0013] (1a) Repeat the measurement of the impedance imaging sensor multiple times, and set each measurement from the beginning to the end as a measurement cycle; in each measurement cycle, the excitation current is injected along the voltage on each adjacent electrode in turn, and the response voltage V on each adjacent electrode is obtained. l , N is the number of electrodes and N=8, that is, The 28 response voltage values obtained by the 8 electrodes in the adjacent excitation measurement mode at the current moment are set as one frame; after continuous measurement, the original voltage sequence within a measurement cycle is obtained vol(i raw )={V 1 ,V 2 ,...,V 28} is the response voltage value of the current frame; the original voltage sequence collected in each frame is normalized to the maximum and minimum values, and all voltages are linearly shrunk to [0,1]:
[0014]
[0015] Among them, vol is the response voltage value at the current moment, vol max is the maximum response voltage value in the current measurement cycle, vol min is the minimum response voltage value in the current measurement cycle; the normalized original electrical impedance imaging voltage sequence is obtained Set the voltage sequence vol measured in the first frame of each measurement norm (1) is the initial value;
[0016] (1b) In each measurement cycle of the six-dimensional force sensor, the real force sequence collected at the current moment is recorded as force(j)={F x (j),F y (j),F z (j),M x (j),M y (j),M z (j)}, the collected real force magnitude is normalized to the maximum and minimum to obtain the normalized real force sequence
[0017] The step (2) specifically comprises the following steps:
[0018] (2a) Reconstruct the 2D electrical impedance image using the one-step Gauss-Newton reconstruction algorithm:
[0019] σ=σ 0 -[[J(σ 0 )] T J(σ 0 )+αdiag(J T J)] -1 [J(σ 0 )] T [V(σ 0 )-U]
[0020] Where σ is the conductivity distribution vector, σ 0 is the initial conductivity distribution, α is the regularization factor, J T J is the regularization matrix, V(σ 0 ) is the conductivity distribution σ 0 The calculated boundary voltage value under 0 ) is the Jacobian matrix, U is the boundary voltage measurement value;
[0021] After reconstruction, the two-dimensional electrical impedance image is obtained, which is recorded as Among them, img(v) is the two-dimensional electrical impedance image of the current frame, which is used to reflect the conductivity distribution information of the cross section of the upper arm muscle of the human body at the current moment;
[0022] (2b) The normalized original electrical impedance imaging voltage sequence The frequency domain information contained in is decoupled to obtain the frequency domain representation of the voltage sequence
[0023]
[0024] Among them, M 1 =M 3 =M 4 =M, M is the total number of samples in a measurement cycle; fft(u) is the amplitude spectrum of the frequency domain signal of the original voltage sequence of the current frame, which is used to reflect the frequency domain information of the cross section of the upper arm muscle of the human body at the current moment.
[0025] In step (4), the first branch network includes an image serialization module and a first multi-head self-attention mechanism module, and the second branch network includes a first fully connected layer and a second multi-head self-attention mechanism module;
[0026] The time-frequency domain feature fusion module includes a first global average pooling layer, a second global average pooling layer, a first RELU activation layer, a second RELU activation layer, a first Sigmoid activation layer, a second Sigmoid activation layer, and second, third, fourth, fifth, sixth, seventh, eighth, and ninth fully connected layers;
[0027] First, the electrical impedance image tensor VIMG Send to the first branch network for processing: the image serialization module converts V IMG The image is divided into patchsize×patchsize small images, where patchsize=16, indicating that the image is evenly divided into 16 rows and 16 columns; the segmented image is first linearly transformed, and then position coding is added to each part of the linearly transformed image; finally, the image data of each part after adding position coding is spliced together, and the whole data obtained by splicing is As a sequence, it is sent to the first multi-head self-attention mechanism module for operation; the first multi-head self-attention mechanism module adopts the scaled dot product attention mechanism, and the formula is as follows:
[0028]
[0029] Among them, Q, K and V represent query, key and value respectively, Attention represents the output tensor, Indicates the number of dimensions of matrix Q;
[0030] Will Through linear transformation by multiplying the trainable weight matrix, three initial vector representations are obtained. The formula is as follows:
[0031] Q=W q V IMG
[0032] K=W k V IMG
[0033] V=W v V IMG
[0034] Among them, W q , W k and W v Both represent trainable weight matrices;
[0035] Substitute the obtained Q, K and V into formula (1) to calculate the output tensor Attention; IMG After multiple independent learning, multiple outputs are concatenated and fed into a fully connected layer of the first multi-head self-attention mechanism module to obtain a feature tensor of size batchsize×patchsize×dim. The first dimension batchsize represents the amount of data contained in each batch, the second dimension patchsize=16, and the third dimension dim=1024. Dim represents the length of each image vector after image serialization. The mean is calculated on the second dimension to finally obtain a time domain feature tensor U with a size of 8×1024. T ;
[0036] The voltage frequency domain tensor V fft Send it to the second branch network for processing: First, V with size batchsize×1×28 fft Input the first fully connected layer, V fft Expand in the third dimension to obtain a feature tensor of size batchsize×1×1024, which is then sent to the second multi-head self-attention mechanism module to obtain a frequency domain feature tensor U of size 8×1024 F ;
[0037] The obtained time domain feature tensor U T and the frequency domain feature tensor U F Reshaped into 8×1024×1 and sent to the time-frequency domain feature fusion module: First, the global average pooling operation is performed through the first global average pooling layer and the second global average pooling layer to convert the time domain feature tensor U T and the frequency domain feature tensor U F Compress to obtain the compressed time domain feature tensor and frequency domain feature tensor Then and They are sent to the second fully connected layer and the fourth fully connected layer respectively, and then they are respectively sent to the first ReLU activation layer and the second ReLU activation layer to reduce the channel dimension, and then they are respectively sent to the third fully connected layer and the fifth fully connected layer, and they are respectively sent to the first Sigmoid activation layer and the second Sigmoid activation layer to reduce the channel dimension. and Each element in the vector is compressed to between 0 and 1 to obtain the weight of each channel. and Next, the weight and the time domain feature tensor U T After multiplication, it is sent to the sixth fully connected layer to obtain the time domain channel attention feature tensor Weight and the frequency domain feature tensor U F After multiplication, it is sent to the seventh fully connected layer to obtain the frequency domain channel attention feature tensor At the same time, U T , U F The downsampled time domain feature tensor is obtained by sending them to the eighth and ninth fully connected layers respectively. And the downsampled frequency domain feature tensor Then, the frequency domain channel attention feature tensor And the downsampled time domain feature tensor After matrix multiplication, the time domain feature tensor after downsampling is Add together to get the first fused tensor U Fusion1 ; At the same time, the time domain channel attention feature tensor And the downsampled frequency domain feature tensor After matrix multiplication, the frequency domain feature tensor after downsampling is obtained. Add them together to get the second fused tensor U Fusion2 ; The first fused tensor U Fusion1 , the second fused tensor U Fusion2 Add together to get an output of size 8×6 The specific formula is as follows:
[0038]
[0039] In the formula, GAP represents global average pooling, W 1 , W 2 , W 3 , W 4 Both represent fully connected layers, δ and σ represent the ReLU activation function and the Sigmoid activation function respectively, and ⊙ is the matrix element multiplication.
[0040] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention takes the original electrical impedance signal of the upper arm muscle as the research object, thereby effectively filling the deep muscle information missing from other biological signals such as electromyographic signals and muscle sound signals; second, the present invention extracts the frequency domain signal of the original electrical impedance imaging voltage signal and the two-dimensional electrical impedance image reconstructed by the one-step Newton-Gaussian method from the original electrical impedance signal, and calculates more complex features by extracting more global information, thereby improving the accuracy of detection; third, the present invention designs a time-frequency domain feature fusion module, which fuses the time domain features and frequency domain features of the original electrical impedance imaging voltage signal, and can obtain more time-frequency domain features at the same time; fourth, compared with the typical neural network algorithm, the present invention can more accurately realize the estimation of the size of the upper arm muscle without damage, and the present invention has the advantages of lightweight, high estimation accuracy, and good mobility. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 is a flow chart of the method of the present invention;
[0042] Figure 2 A schematic diagram of the excitation-measurement sequence of the electrical impedance sensor in the present invention;
[0043] Figure 3 It is a schematic diagram of the muscle strength estimation experimental process in the present invention;
[0044] Figure 4 This is a flow chart of preprocessing the original electrical impedance imaging voltage signal in the present invention;
[0045] Figure 5 A schematic diagram of the structure of the first branch network and the second branch network in the present invention;
[0046] Figure 6 It is a structural diagram of the time-frequency domain feature fusion module in the present invention. DETAILED DESCRIPTION
[0047] like Figure 1 As shown, a method for estimating upper arm muscle strength based on a dual-branch self-attention mechanism network includes:
[0048] (1) In one measurement cycle, the original electrical impedance imaging voltage signal generated by the contraction of the upper arm muscles of the human body is measured by the electrical impedance imaging sensor and normalized to obtain the normalized original electrical impedance imaging voltage sequence. Among them, M 1 is the total number of normalized raw electrical impedance imaging voltage sequences recorded in a measurement cycle, i is the index subscript of the normalized raw electrical impedance imaging voltage sequence at the current moment, i is an integer and 1≤i≤M 1 At the same time, the six-dimensional force sensor is used to collect the forces Fx, Fy, Fz and the moments Mz, My, Mz in three directions generated by the contraction of the upper arm muscles of the human body. Fx, Fy, Fz and Mz, My, Mz together constitute the real force sequence. The real force sequence is normalized to obtain the normalized real force sequence Among them, M 2 is the total number of true forces recorded in a measurement cycle, j is the index subscript of the true force at the current moment, j is an integer and 1≤j≤M 2 ;
[0049] (2) The normalized original electrical impedance imaging voltage sequence Perform one-step Gauss-Newton image reconstruction and Fourier transform to obtain a two-dimensional electrical impedance image. Voltage sequence frequency domain characterization Among them, M 3 is the total number of two-dimensional electrical impedance images recorded in a measurement cycle, v is the index subscript of the two-dimensional electrical impedance image at the current moment, v is an integer and 1≤v≤M 3 ;M 4 is the total number of frequency domain representations of the voltage sequence recorded in a measurement cycle, u is the index subscript of the frequency domain representation of the voltage sequence at the current moment, u is an integer and 1≤u≤M 4 ;
[0050] (3) Frequency domain characterization of voltage sequence 2D electrical impedance imaging and the normalized true force series The dataset is composed and divided into batches to obtain an electrical impedance image tensor V of size batchsize×1×64×64 IMG, the voltage frequency domain tensor V of size batchsize×1×28 fft and the real force tensor V of size batchsize×1×6 F , batchsize indicates the amount of data contained in each batch;
[0051] (4) constructing a dual-branch self-attention mechanism network, wherein the dual-branch self-attention mechanism network consists of a first branch network, a second branch network, and a time-frequency domain feature fusion module; converting the electrical impedance image tensor V IMG , voltage frequency domain tensor V fft Input the dual-branch self-attention mechanism network, and the dual-branch self-attention mechanism network outputs the muscle strength estimation result;
[0052] (5) The muscle force estimation result is compared with the actual force sequence collected by the six-dimensional force sensor, and the mean square error is used to reflect the degree of difference between the two.
[0053] The step (1) specifically comprises the following steps:
[0054] (1a) Figure 2 As shown, the measurement of the electrical impedance imaging sensor is repeated multiple times, and each measurement from the beginning to the end is set as a measurement cycle; in each round of measurement cycle, the excitation current is injected along the voltage on each adjacent electrode in turn, and the response voltage V on each adjacent electrode is obtained. l , N is the number of electrodes and N=8, that is, The 28 response voltage values obtained by the 8 electrodes in the adjacent excitation measurement mode at the current moment are set as one frame; after continuous measurement, the original voltage sequence within a measurement cycle is obtained vol(i raw )={V 1 ,V 2 ,...,V 28} is the response voltage value of the current frame; the original voltage sequence collected in each frame is normalized to the maximum and minimum values, and all voltages are linearly shrunk to [0,1]:
[0055]
[0056] Among them, vol is the response voltage value at the current moment, vol max is the maximum response voltage value in the current measurement cycle, vol min is the minimum response voltage value in the current measurement cycle; the normalized original electrical impedance imaging voltage sequence is obtained Set the voltage sequence vol measured in the first frame of each measurement norm (1) is the initial value;
[0057] (1b) Figure 3 As shown, in each measurement cycle of the six-dimensional force sensor, the real force sequence collected at the current moment is recorded as force(j)={F x (j),F y (j),F z (j),M x (j),M y (j),M z (j)}, the collected real force magnitude is normalized to the maximum and minimum to obtain the normalized real force sequence
[0058]
[0059] like Figure 4 As shown, the first 100 groups of original electrical impedance imaging voltage signals are preprocessed, and the step (2) specifically includes the following steps:
[0060] (2a) Reconstruct the 2D electrical impedance image using the one-step Gauss-Newton reconstruction algorithm:
[0061] σ=σ 0 -[[J(σ 0 )] T J(σ 0 )+αdiag(J T J)] -1 [J(σ 0 )] T [V(σ 0 )-U]
[0062] Where σ is the conductivity distribution vector, σ 0 is the initial conductivity distribution, α is the regularization factor, J T J is the regularization matrix, V(σ 0 ) is the conductivity distribution σ 0 The calculated boundary voltage value under 0 ) is the Jacobian matrix, U is the boundary voltage measurement value;
[0063] After reconstruction, the two-dimensional electrical impedance image is obtained, which is recorded as Among them, img(v) is the two-dimensional electrical impedance image of the current frame, which is used to reflect the conductivity distribution information of the cross section of the upper arm muscle of the human body at the current moment;
[0064] (2b) The normalized original electrical impedance imaging voltage sequence The frequency domain information contained in is decoupled to obtain the frequency domain representation of the voltage sequence
[0065]
[0066] Among them, M1 =M 3 =M 4 =M, M is the total number of samples in a measurement cycle; fft(u) is the amplitude spectrum of the frequency domain signal of the original voltage sequence of the current frame, which is used to reflect the frequency domain information of the cross section of the upper arm muscle of the human body at the current moment.
[0067] In step (4), if Figure 5 As shown, the first branch network includes an image serialization module and a first multi-head self-attention mechanism module, and the second branch network includes a first fully connected layer and a second multi-head self-attention mechanism module;
[0068] like Figure 6 As shown, the time-frequency domain feature fusion module includes a first global average pooling layer, a second global average pooling layer, a first RELU activation layer, a second RELU activation layer, a first Sigmoid activation layer, a second Sigmoid activation layer, and second, third, fourth, fifth, sixth, seventh, eighth, and ninth fully connected layers;
[0069] First, the electrical impedance image tensor V IMG Send to the first branch network for processing: the image serialization module converts V IMG The image is divided into patchsize×patchsize small images, where patchsize=16, indicating that the image is evenly divided into 16 rows and 16 columns; the segmented image is first linearly transformed, and then position coding is added to each part of the linearly transformed image; finally, the image data of each part after adding position coding is spliced together, and the whole data obtained by splicing is As a sequence, it is sent to the first multi-head self-attention mechanism module for operation; the first multi-head self-attention mechanism module adopts the scaled dot product attention mechanism, and the formula is as follows:
[0070]
[0071] Among them, Q, K and V represent query, key and value respectively, Attention represents the output tensor, Indicates the number of dimensions of matrix Q;
[0072] Will Through linear transformation by multiplying the trainable weight matrix, three initial vector representations are obtained. The formula is as follows:
[0073] Q=W q V IMG
[0074] K=W k V IMG
[0075] V=W v VIMG
[0076] Among them, W q , W k and W v Both represent trainable weight matrices;
[0077] Substitute the obtained Q, K and V into formula (1) to calculate the output tensor Attention; IMG After multiple independent learning, multiple outputs are concatenated and fed into a fully connected layer of the first multi-head self-attention mechanism module to obtain a feature tensor of size batchsize×patchsize×dim. The first dimension batchsize represents the amount of data contained in each batch, the second dimension patchsize=16, and the third dimension dim=1024. Dim represents the length of each image vector after image serialization. The mean is calculated on the second dimension to finally obtain a time domain feature tensor U with a size of 8×1024. T ;
[0078] The voltage frequency domain tensor V fft Send it to the second branch network for processing: First, V with size batchsize×1×28 fft Input the first fully connected layer, V fft Expand in the third dimension to obtain a feature tensor of size batchsize×1×1024, which is then sent to the second multi-head self-attention mechanism module to obtain a frequency domain feature tensor U of size 8×1024 F ;
[0079] The obtained time domain feature tensor U T and the frequency domain feature tensor U F Reshaped into 8×1024×1 and sent to the time-frequency domain feature fusion module: First, the global average pooling operation is performed through the first global average pooling layer and the second global average pooling layer to convert the time domain feature tensor U T and the frequency domain feature tensor U F Compress to obtain the compressed time domain feature tensor and frequency domain feature tensor Then and They are sent to the second fully connected layer and the fourth fully connected layer respectively, and then they are respectively sent to the first ReLU activation layer and the second ReLU activation layer to reduce the channel dimension, and then they are respectively sent to the third fully connected layer and the fifth fully connected layer, and they are respectively sent to the first Sigmoid activation layer and the second Sigmoid activation layer to reduce the channel dimension. and Each element in the vector is compressed to between 0 and 1 to obtain the weight of each channel. and Next, the weight and the time domain feature tensor U T After multiplication, it is sent to the sixth fully connected layer to obtain the time domain channel attention feature tensor Weight and the frequency domain feature tensor U F After multiplication, it is sent to the seventh fully connected layer to obtain the frequency domain channel attention feature tensor At the same time, U T , U F The downsampled time domain feature tensor is obtained by sending them to the eighth and ninth fully connected layers respectively. And the downsampled frequency domain feature tensor Then, the frequency domain channel attention feature tensor And the downsampled time domain feature tensor After matrix multiplication, the time domain feature tensor after downsampling is Add together to get the first fused tensor U Fusion1 ; At the same time, the time domain channel attention feature tensor And the downsampled frequency domain feature tensor After matrix multiplication, the frequency domain feature tensor after downsampling is obtained. Add them together to get the second fused tensor U Fusion2 ; The first fused tensor U Fusion1 , the second fused tensor U Fusion2 Add together to get an output of size 8×6 The specific formula is as follows:
[0080]
[0081] In the formula, GAP represents global average pooling, W 1 , W 2 , W 3 , W 4 Both represent fully connected layers, δ and σ represent ReLU activation function and Sigmoid activation function respectively, and ⊙ is matrix element multiplication
[0082] As shown in Table 1, Table 1 compares the recognition effects of the present invention and multiple upper arm muscle strength estimation methods based on traditional machine learning methods on the test set:
[0083] Table 1
[0084]
[0085] In Table 1, LSTM, VGG16, ResNet18, DensNet121, ShuffleNet_v2_x0_5, MobileNet_v2, EfficientNet_b0, and ConvNeXt are upper arm muscle strength estimation methods based on traditional machine learning methods. By comparing the experimental results in the table, it can be found that the present invention can more accurately realize the estimation of the size of the upper arm muscle strength without damage compared with the upper arm muscle strength estimation method based on the traditional machine learning method. At the same time, the present invention has the advantages of lightweight, high estimation accuracy, good mobility, etc.; the present invention effectively fills the deep muscle information missing from other biological signals such as electromyographic signals and muscle sound signals by taking the original electrical impedance imaging voltage signal as the research object; the present invention calculates more complex features by extracting more global information from the original electrical impedance signal, thereby improving the accuracy of detection; at the same time, the present invention designs a time-frequency domain feature fusion module to fuse the time domain features and frequency domain features of the original electrical impedance imaging voltage signal, so as to obtain more time-frequency domain features at the same time.
Claims
1. A method for estimating upper arm muscle strength based on a dual-branch self-attention mechanism network, characterized in that: The method comprises the following steps in order: (1) In one measurement cycle, the original electrical impedance imaging voltage signal generated by the contraction of the upper arm muscles of the human body is measured by the electrical impedance imaging sensor and normalized to obtain the normalized original electrical impedance imaging voltage sequence. Wherein, M1 is the total number of normalized original electrical impedance imaging voltage sequences recorded in a measurement cycle, i is the index subscript of the normalized original electrical impedance imaging voltage sequence at the current moment, i is an integer and 1≤i≤M1; at the same time, the forces Fx, Fy, Fz and moments Mz, My, Mz in three directions generated by the contraction of the upper arm muscles of the human body are collected by the six-dimensional force sensor, and Fx, Fy, Fz and Mz, My, Mz together constitute the real force sequence, and the real force sequence is normalized to obtain the normalized real force sequence Where M2 is the total number of true forces recorded in a measurement cycle, j is the index subscript of the true force at the current moment, j is an integer and 1≤j≤M2; (2) The normalized original electrical impedance imaging voltage sequence Perform one-step Gauss-Newton image reconstruction and Fourier transform to obtain a two-dimensional electrical impedance image. Voltage sequence frequency domain characterization Wherein, M3 is the total number of two-dimensional electrical impedance images recorded in one measurement cycle, v is the index subscript of the two-dimensional electrical impedance image at the current moment, v is an integer and 1≤v≤M3; M4 is the total number of frequency domain representations of the voltage sequence recorded in one measurement cycle, u is the index subscript of the frequency domain representation of the voltage sequence at the current moment, u is an integer and 1≤u≤M4; (3) Frequency domain characterization of voltage sequence 2D electrical impedance imaging and the normalized true force series The dataset is composed and divided into batches to obtain an electrical impedance image tensor V of size batchsize×1×64×64 IMG , the voltage frequency domain tensor V of size batchsize×1×28 fft and the real force tensor V of size batchsize×1×6 F , batchsize indicates the amount of data contained in each batch; (4) constructing a dual-branch self-attention mechanism network, wherein the dual-branch self-attention mechanism network consists of a first branch network, a second branch network, and a time-frequency domain feature fusion module; converting the electrical impedance image tensor V IMG , voltage frequency domain tensor V fft Input the dual-branch self-attention mechanism network, and the dual-branch self-attention mechanism network outputs the muscle strength estimation result; (5) The muscle force estimation result is compared with the actual force sequence collected by the six-dimensional force sensor, and the mean square error is used to reflect the degree of difference between the two.
2. The upper arm muscle strength estimation method based on the dual-branch self-attention mechanism network according to claim 1 is characterized in that: The step (1) specifically comprises the following steps: (1a) Repeat the measurement of the impedance imaging sensor multiple times, and set each measurement from the beginning to the end as a measurement cycle; in each measurement cycle, the excitation current is injected along the voltage on each adjacent electrode in turn, and the response voltage V on each adjacent electrode is obtained. l , N is the number of electrodes and N=8, that is, The 28 response voltage values obtained by the 8 electrodes in the adjacent excitation measurement mode at the current moment are set as one frame; after continuous measurement, the original voltage sequence within a measurement cycle is obtained vol(i raw )={V1,V2,...,V 28 } is the response voltage value of the current frame; the original voltage sequence collected in each frame is normalized to the maximum and minimum values, and all voltages are linearly shrunk to [0,1]: Among them, vol is the response voltage value at the current moment, vol max is the maximum response voltage value in the current measurement cycle, vol min is the minimum response voltage value in the current measurement cycle; the normalized original electrical impedance imaging voltage sequence is obtained Set the voltage sequence vol measured in the first frame of each measurement norm (1) is the initial value; (1b) In each measurement cycle of the six-dimensional force sensor, the real force sequence collected at the current moment is recorded as force(j)={F x (j),F y (j),F z (j),M x (j),M y (j),M z (j)}, the collected real force magnitude is normalized to the maximum and minimum to obtain the normalized real force sequence 3. The upper arm muscle strength estimation method based on the dual-branch self-attention mechanism network according to claim 1 is characterized in that: The step (2) specifically comprises the following steps: (2a) Reconstruct the 2D electrical impedance image using the one-step Gauss-Newton reconstruction algorithm: σ=σ0-[[J(σ0)] T J(σ0)+αdiag(J T J)] -1 [J(σ0)] T [V(σ0)-U] Where σ is the conductivity distribution vector, σ0 is the initial conductivity distribution, α is the regularization factor, and J T J is the regularization matrix, V(σ0) is the calculated value of the boundary voltage under the conductivity distribution of σ0, J(σ0) is the Jacobian matrix, and U is the measured value of the boundary voltage; After reconstruction, the two-dimensional electrical impedance image is obtained, which is recorded as Among them, img(v) is the two-dimensional electrical impedance image of the current frame, which is used to reflect the conductivity distribution information of the cross section of the upper arm muscle of the human body at the current moment; (2b) The normalized original electrical impedance imaging voltage sequence The frequency domain information contained in is decoupled to obtain the frequency domain representation of the voltage sequence Among them, M1=M3=M4=M, M is the total number of samples in a measurement cycle; fft(u) is the amplitude spectrum of the frequency domain signal of the original voltage sequence of the current frame, which is used to reflect the frequency domain information of the cross-section of the upper arm muscle of the human body at the current moment.
4. The upper arm muscle strength estimation method based on a dual-branch self-attention mechanism network according to claim 1, characterized in that: In step (4), the first branch network includes an image serialization module and a first multi-head self-attention mechanism module, and the second branch network includes a first fully connected layer and a second multi-head self-attention mechanism module; The time-frequency domain feature fusion module includes a first global average pooling layer, a second global average pooling layer, a first RELU activation layer, a second RELU activation layer, a first Sigmoid activation layer, a second Sigmoid activation layer, and second, third, fourth, fifth, sixth, seventh, eighth, and ninth fully connected layers; First, the electrical impedance image tensor V IMG Send to the first branch network for processing: the image serialization module converts V IMG The image is divided into patchsize×patchsize small images, where patchsize=16, indicating that the image is evenly divided into 16 rows and 16 columns; the segmented image is first linearly transformed, and then position coding is added to each part of the linearly transformed image; finally, the image data of each part after adding position coding is spliced together, and the whole data obtained by splicing is As a sequence, it is sent to the first multi-head self-attention mechanism module for operation; the first multi-head self-attention mechanism module adopts the scaled dot product attention mechanism, and the formula is as follows: Among them, Q, K and V represent query, key and value respectively, Attention represents the output tensor, Indicates the number of dimensions of matrix Q; Will Through linear transformation by multiplying the trainable weight matrix, three initial vector representations are obtained. The formula is as follows: Q=W q V IMG K=W k V IMG V=W v V IMG Among them, W q , W k and W v Both represent trainable weight matrices; Substitute the obtained Q, K and V into formula (1) to calculate the output tensor Attention; IMG After multiple independent learning, multiple outputs are concatenated and fed into a fully connected layer of the first multi-head self-attention mechanism module to obtain a feature tensor of size batchsize×patchsize×dim. The first dimension batchsize represents the amount of data contained in each batch, the second dimension patchsize=16, and the third dimension dim=1024. Dim represents the length of each image vector after image serialization. The mean is calculated on the second dimension to finally obtain a time domain feature tensor U with a size of 8×1024. T ; The voltage frequency domain tensor V fft Send it to the second branch network for processing: First, V with size batchsize×1×28 fft Input the first fully connected layer, V fft Expand in the third dimension to obtain a feature tensor of size batchsize×1×1024, which is then sent to the second multi-head self-attention mechanism module to obtain a frequency domain feature tensor U of size 8×1024 F ; The obtained time domain feature tensor U T and the frequency domain feature tensor U F Reshaped into 8×1024×1 and sent to the time-frequency domain feature fusion module: First, the global average pooling operation is performed through the first global average pooling layer and the second global average pooling layer to convert the time domain feature tensor U T and the frequency domain feature tensor U F Compress to obtain the compressed time domain feature tensor and frequency domain feature tensor Then and They are sent to the second fully connected layer and the fourth fully connected layer respectively, and then they are respectively sent to the first ReLU activation layer and the second ReLU activation layer to reduce the channel dimension, and then they are respectively sent to the third fully connected layer and the fifth fully connected layer, and they are respectively sent to the first Sigmoid activation layer and the second Sigmoid activation layer to reduce the channel dimension. and Each element in the vector is compressed to between 0 and 1 to obtain the weight of each channel. and Next, the weight and the time domain feature tensor U T After multiplication, it is sent to the sixth fully connected layer to obtain the time domain channel attention feature tensor Weight and the frequency domain feature tensor U F After multiplication, it is sent to the seventh fully connected layer to obtain the frequency domain channel attention feature tensor At the same time, U T , U F The downsampled time domain feature tensor is obtained by sending them to the eighth and ninth fully connected layers respectively. And the downsampled frequency domain feature tensor Then, the frequency domain channel attention feature tensor And the downsampled time domain feature tensor After matrix multiplication, the time domain feature tensor after downsampling is Add together to get the first fused tensor U Fusion1 ; At the same time, the time domain channel attention feature tensor And the downsampled frequency domain feature tensor After matrix multiplication, the frequency domain feature tensor after downsampling is obtained. Add them together to get the second fused tensor U Fusion2 ; The first fused tensor U Fusion1 , the second fused tensor U Fusion2 Add together to get an output of size 8×6 The specific formula is as follows: Where GAP represents global average pooling, W1, W2, W3, and W4 all represent fully connected layers, δ and σ represent ReLU activation function and Sigmoid activation function respectively, and ⊙ represents matrix element multiplication.
Citation Information
Patent Citations
System and method for monitoring muscle fatigue
CN110464351A
Random vector functional-link network technique-based human mechanomyography (MMG) signal prediction method
CN111110268A
Electromyography gesture recognition method of feature pyramid network fused with double-attention mechanism
CN113627401A
Deep learning electrical impedance imaging system based on FPGA and imaging evaluation method
CN114544708A
Binocular image super-resolution reconstruction method based on multistage intensified attention mechanism
CN116797461A
Cited By
Muscle force estimation method based on combination of long and short term memory and attention mechanism
CN119924838A
Electrical impedance imaging method, medium and equipment for dynamic muscle imaging
CN120477740A
An electrical impedance imaging method, medium and apparatus for muscle dynamic imaging
CN120477740B
Lower limb motion intention prediction method and device and exoskeleton robot
CN120724097A
