Finger continuous estimation method based on multi-scale cavity convolution and Transform model

Abstract

The invention provides a finger continuous estimation method based on multi-scale cavity convolution and a Transform model, and belongs to the technical field of deep learning method.The finger continuous estimation method comprises the following steps that surface electromyogram signals and joint angles in Ninapro DB2 serve as original data, and a sliding window is used for constructing a data set of the surface electromyogram signals and the corresponding joint angles; local features of the surface electromyogram signals are extracted based on multi-scale cavity convolution, and global features of the surface electromyogram signals are extracted based on a Transform architecture; the extracted local features and global features are fused through a fusion module, a full connection layer and a smooth layer are connected behind the fusion module, and model building is achieved; training the built model to obtain an estimation model; and inputting the surface electromyogram signals into the estimation model, and outputting the joint angle of each joint. According to the method disclosed by the invention, relatively good accuracy, generalization and stability are achieved whether in a single subject or in a cross-subject manner.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning methods, and in particular, to a finger continuous estimation method based on multi-scale dilated convolution and Transformer model. Background Art

[0002] Currently, deep learning models are used to achieve continuous estimation of finger movements based on surface electromyography signals. For continuous estimation of fingers of specific subjects, convolutional neural networks (CNN) or recurrent neural networks (RNN) are used, or a stack of CNN and RNN or attention. For cross-individual subjects, BERT focuses on using its global features to achieve continuous estimation prediction.

[0003] Currently, in human-computer interaction systems, continuous estimation methods play a vital role in achieving natural and intuitive interactions. In previous work, LSTM relies on sequential calculations of time steps and cannot be calculated in parallel, which is time-consuming. TCN relies on convolution operations to capture long-term dependencies, which can lead to instability and fluctuations in prediction results. BERT, as a bidirectional structure, can capture contextual information of surface electromyography signals. BERT has more advantages in capturing long-term dependencies. In continuous estimation, local features are equally important, and BERT's global attention mechanism may lead to insufficient attention to local details.

[0004] References:

[0005] [1] C.Lin,

[0006] [2]W.Guo et al., "Long exposure convolutional memory network for accurate estimation of finger kinematics from surface electromyographicsignals," J.Neural Eng., vol.18, no.2, 2021, Art.no.026027.

[0007] [3] Lin, Chuang. "sEMG-Based Continuous Estimation of Finger Kinematicsvia Large-Scale Temporal Convolutional Network." Applied Sciences 11 (2021).

[0008] [4]A CNN-Attention Network for Continuous Estimation of FingerKinematics from Surface Electromyograms, IEEE ROBOTICS AND AUTOMATION LETTERS, 2022 Summary of the invention

[0009] According to the technical problems mentioned in the above background technology, a finger continuous estimation method based on multi-scale hole convolution and Transformer model is provided.

[0010] The technical means adopted by the present invention are as follows:

[0011] A finger continuous estimation method based on multi-scale dilated convolution and Transformer model includes the following steps:

[0012] Step 1: Based on the surface electromyographic signals and joint angles in Ninapro DB2 as raw data, a sliding window is used to construct a dataset of surface electromyographic signals and corresponding joint angles;

[0013] Step 2: extracting local features of the surface electromyographic signal based on multi-scale dilated convolution, and extracting global features of the surface electromyographic signal based on the Transformer architecture;

[0014] Step 3: The extracted local features and the global features are fused through a fusion module, and the fusion module is followed by a fully connected layer and a smoothing layer to realize model construction; the constructed model is trained to obtain an estimated model;

[0015] Step 4: Input the surface electromyography signal into the estimation model and output the joint angle of each joint.

[0016] Furthermore, the use of a sliding window to construct a data set of surface electromyographic signals and corresponding joint angles comprises the following steps:

[0017] Step 11: Process the original surface electromyographic signal and joint angle by using the sliding window method, setting the sliding window length to 100ms and the step length to 0.5ms;

[0018] Step 12: Calculate the root mean square RMS for the surface electromyography signal, perform μ-law normalization on the root mean square RMS, and use the normalized root mean square RMS as the input data of the model;

[0019] Step 13: For the joint angle, use the last joint angle in the sliding window as the target data.

[0020] Furthermore, the formula for the root mean square RMS and μ-law normalization is:

[0021]

[0022] Among them, N represents the window size, x t represents the electromyographic data sampled at time t; μ represents the scaling parameter of the signal, μ=2^20.

[0023] Furthermore, the step 2 comprises the following steps:

[0024] Step 21: Use multi-scale dilated convolution to extract local features of surface electromyographic signals, and use a fixed convolution kernel and dilated convolutions with dilated rates of 1, 2, and 4 to extract electromyographic signal features in parallel;

[0025] Step 22: Use the encoding layer in the Transformer architecture to extract the global features of the surface electromyography signal, where the encoding layer includes: a multi-head attention layer and a feedforward neural network.

[0026] Furthermore, the step 21 includes the following steps:

[0027] Step 211: In the first branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 1 is used. After batch normalization and activation function, the convolution layer is input into the second convolution operation, and a convolution kernel of 5 and a dilation rate of 1 are used. The activation function is used to obtain the first branch feature data;

[0028] Step 212: In the second branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 2 is used. After batch normalization and activation function, the convolution layer is input into the second convolution operation. A convolution kernel of 5 and a dilation rate of 2 are used. The activation function is used to obtain the feature data of the second branch.

[0029] Step 213: In the third branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 4 is used. After batch normalization and activation function, it is input into the second convolution operation, and a convolution layer with a convolution kernel of 5 and a dilation rate of 4 is used. The activation function is used to obtain the feature data of the third branch;

[0030] Step 214: splicing the first branch characteristic data, the second branch characteristic data and the third branch characteristic data in the channel dimension;

[0031] Step 215: A convolution layer with a convolution kernel of 1 is connected after the channel attention mechanism to perform dimensionality reduction and reduce the number of channels. Batch normalization and activation function are performed after the convolution layer, and finally the local features extracted by the multi-scale dilated convolution module are output.

[0032] Furthermore, the attention mechanism ECA is:

[0033] Use global average pooling GAP to process the splicing features, and then use the convolution layer followed by the Sigmoid activation function to multiply the result with the original feature map to give different weights to each channel. The size of the convolution kernel in the convolution layer adjusts the size of the convolution kernel k according to the channel. The convolution kernel is calculated as follows:

[0034]

[0035] Here, γ=2, b=1 are set; C represents the number of channels.

[0036] Further, the step 22 comprises the following steps:

[0037] Step 221: using a linear layer as an embedding layer for the surface electromyography signal, and adding time embedding to the linear layer output to capture time information; the time embedding is used to train a one-dimensional vector;

[0038] Step 222: The data X of the embedding time is input into the multi-head attention layer through the layer normalization layer; the output X of the multi-head attention layer is M Add to X to form an add layer, i.e. residual connection, and get the output X ′ M ;

[0039] Step 223: For output X ′ MUse layer normalization and input to a feedforward neural network; the feedforward neural network includes: two linear layers; after the first linear layer, use GELU as the activation function to convert the output X of the feedforward neural network F With X ′ M The addition forms the add layer, i.e., the residual connection, and finally outputs the extracted global features.

[0040] Furthermore, the step 3 comprises the following steps:

[0041] Step 31: construct a fusion module;

[0042] Step 32: Use a fully connected layer to map the obtained fusion features into joint angles. In order to make the output results more consistent with the actual movement, a smoothing layer is added after the fully connected layer.

[0043] Step 33: Obtain an estimation model using the Pearson correlation coefficient PCC and the normalized root mean square error NRMSE; the Pearson correlation coefficient is:

[0044]

[0045]

[0046] Where N is the number of samples; θ est represents the estimated joint angle value; represents the estimated joint angle mean; θ real Indicates the actual joint angle value, represents the actual joint angle mean; θ max Indicates the maximum value of the actual joint angle; θ min Indicates the minimum value of the actual joint angle.

[0047] Furthermore, the smoothing layer uses a weighted average smoothing method using a sliding window size, which is calculated as follows:

[0048] w i =α*(1-α) i ;

[0049]

[0050] Among them, w i represents the weight of the i-th position, α represents the weighting coefficient, α = 0.3; out t represents the result after smoothing at time t; N represents the window size, N=3; for the data output by the fully connected layer, the output values ​​of the initial two sampling points remain unchanged, and smoothing is performed from the third sampling point according to the sliding weighted average.

[0051] Furthermore, constructing the fusion module comprises the following steps:

[0052] Step 311: performing channel splicing on the extracted local features and global features;

[0053] Step 312: Build a convolution block with a convolution kernel of 1. Each convolution block consists of two convolution layers with a convolution kernel of 1. After the first convolution layer, relu is used as the activation function to add the data input to the first convolution layer and the data output by the second convolution layer to form an add layer, i.e., a residual connection.

[0054] Step 313: Build three convolution blocks in the fusion module, add the concatenated features obtained in the first step to the output results of the three convolution blocks to form an add layer, i.e., residual connection, and output the fusion features.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] In order to make full use of the local features and global features in the surface electromyography signal, the present invention designs a method of extracting the local features of the surface electromyography signal using multi-scale dilated convolution, uses the Transformer architecture to capture the global features of the surface electromyography signal, and designs a fusion module to fuse the local features and the global features, which can fully utilize the advantages of the local features and the global features. The method of the present invention has achieved good accuracy, generalization and stability both in a single subject and across subjects. It is proved that this method has good application prospects in continuous motion estimation based on surface electromyography. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0060] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0061] like Figure 1 As shown, the present invention provides a finger continuous estimation method based on multi-scale dilated convolution and Transformer model, comprising the following steps:

[0062] Step 1: Based on the surface electromyographic signals and joint angles in Ninapro DB2 as raw data, a data set of surface electromyographic signals and corresponding joint angles is constructed using a sliding window; constructing a data set of surface electromyographic signals and corresponding joint angles using a sliding window includes the following steps:

[0063] Step 11: Process the original surface electromyographic signal and joint angle by using the sliding window method, setting the sliding window length to 100ms and the step length to 0.5ms;

[0064] Step 12: Calculate the root mean square (RMS) of the surface electromyographic signal and perform μ-law normalization as the input data of the model; the formulas for the root mean square (RMS) and μ-law normalization are:

[0065] Step 12: Calculate the root mean square RMS for the surface electromyography signal, perform μ-law normalization on the root mean square RMS, and use the normalized root mean square RMS as the input data of the model;

[0066]

[0067] Among them, N represents the window size, x t Indicates the electromyographic data sampled at time t; μ indicates the scaling parameter of the signal, μ = 2^20. In the present invention, the sampling rate of the surface electromyographic signal is set to 2000 Hz; N = 2000*(100ms) = 200

[0068] Step 13: For the joint angle, use the last joint angle in the sliding window as the target data.

[0069] Step 2: Extract local features of surface electromyographic signals based on multi-scale dilated convolution, and extract global features of surface electromyographic signals based on Transformer architecture; Step 2 includes the following steps:

[0070] Step 21: Use multi-scale dilated convolution to extract local features of surface electromyographic signals, and use a fixed convolution kernel and dilated convolutions with dilated rates of 1, 2, and 4 to extract electromyographic signal features in parallel; Step 21 includes the following steps:

[0071] Step 211: In the first branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 1 is used. After batch normalization and activation function, the convolution layer is input into the second convolution operation, and a convolution kernel of 5 and a dilation rate of 1 are used. The activation function is used to obtain the first branch feature data;

[0072] Step 212: In the second branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 2 is used. After batch normalization and activation function, the convolution layer is input into the second convolution operation. A convolution kernel of 5 and a dilation rate of 2 are used. The activation function is used to obtain the feature data of the second branch.

[0073] Step 213: In the third branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 4 is used. After batch normalization and activation function, it is input into the second convolution operation, and a convolution layer with a convolution kernel of 5 and a dilation rate of 4 is used. The activation function is used to obtain the feature data of the third branch;

[0074] Step 214: splicing the first branch characteristic data, the second branch characteristic data and the third branch characteristic data in the channel dimension;

[0075] Step 215: After the channel attention mechanism, a convolution layer with a convolution kernel of 1 is connected to reduce the dimension and reduce the number of channels. Batch normalization and activation function are performed after the convolution layer, and finally the local features extracted by the multi-scale hole convolution module are output. The attention mechanism ECA is:

[0076] Use global average pooling GAP to process the splicing features, then use the convolution layer followed by the Sigmoid activation function, multiply the result with the original feature map, and give different weights to each channel. The size of the convolution kernel in the convolution layer adjusts the size of the convolution kernel k according to the channel. The convolution kernel is calculated as follows:

[0077]

[0078] Here, γ=2, b=1 are set; C represents the number of channels.

[0079] Step 22: Use the encoding layer in the Transformer architecture to extract the global features of the surface electromyography signal. The encoding layer includes: a multi-head attention layer and a feedforward neural network. The following steps are included:

[0080] Step 221: using a linear layer as an embedding layer for the surface electromyography signal, and adding time embedding to the output of the linear layer to capture time information; the time embedding is used to train the one-dimensional vector;

[0081] Step 222: The embedded time data X is input into the multi-head attention layer through the layer normalization layer; the output X M Add to X to form an add layer, i.e. residual connection, and get the output X ′ M ;

[0082] Step 223: For output X ′ M Use layer normalization and input to the feedforward neural network; the feedforward neural network includes: two linear layers; after the first linear layer, GELU is used as the activation function to convert the output X of the feedforward neural network F With X ′ M The addition forms the add layer, i.e., the residual connection, and finally outputs the extracted global features.

[0083] Step 3: The extracted local features and global features are fused through the fusion module, and the fusion module is followed by a fully connected layer and a smoothing layer to realize the construction of the model; the constructed model is trained to obtain an estimated model; Step 3 includes the following steps:

[0084] Step 31: constructing a fusion module; constructing a fusion module includes the following steps:

[0085] Step 311: performing channel splicing on the extracted local features and global features;

[0086] Step 312: Build a convolution block with a convolution kernel of 1. Each convolution block consists of two convolution layers with a convolution kernel of 1. After the first convolution layer, relu is used as the activation function to add the data input to the first convolution layer and the data output by the second convolution layer to form an add layer, i.e., a residual connection.

[0087] Step 313: Build three convolution blocks in the fusion module, add the concatenated features obtained in the first step to the output results of the three convolution blocks to form an add layer, i.e., residual connection, and output the fusion features.

[0088] Step 32: The obtained fusion features are mapped to joint angles using a fully connected layer. In order to make the output result more consistent with the actual movement, a smoothing layer is added after the fully connected layer; the smoothing layer uses a weighted average smoothing method using a sliding window size, and the calculation is as follows:

[0089] w i =α*(1-α) i ;

[0090]

[0091] Among them, w i represents the weight of the i-th position, α represents the weighting coefficient, α = 0.3; out t represents the result after smoothing at time t; N represents the window size, N=3; for the data output by the fully connected layer, the output values ​​of the initial two sampling points remain unchanged, and smoothing is performed from the third sampling point according to the sliding weighted average.

[0092] Step 33: Use the Pearson correlation coefficient PCC and normalized root mean square error NRMSE to obtain the estimated model; the Pearson correlation coefficient is:

[0093]

[0094] Where N is the number of samples; θ est represents the estimated joint angle value; represents the estimated joint angle mean; θ real Indicates the actual joint angle value, represents the actual joint angle mean; θ max Indicates the maximum value of the actual joint angle; θ min Indicates the minimum value of the actual joint angle.

[0095] Step 4: Input the surface electromyography signal into the estimation model and output the joint angles of each joint.

[0096] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A finger continuous estimation method based on multi-scale dilated convolution and Transformer model, characterized in that: The following steps are involved: Step 1: Based on the surface electromyographic signals and joint angles in Ninapro DB2 as raw data, a sliding window is used to construct a dataset of surface electromyographic signals and corresponding joint angles; Step 2: extracting local features of the surface electromyographic signal based on multi-scale dilated convolution, and extracting global features of the surface electromyographic signal based on the Transformer architecture; Step 3: The extracted local features and the global features are fused through a fusion module, and the fusion module is followed by a fully connected layer and a smoothing layer to realize model construction; the constructed model is trained to obtain an estimation model; Step 4: Input the surface electromyography signal into the estimation model and output the joint angle of each joint.

2. According to claim 1, a finger continuous estimation method based on multi-scale dilated convolution and Transformer model is characterized in that: The method of using a sliding window to construct a data set of surface electromyographic signals and corresponding joint angles includes the following steps: Step 11: Process the original surface electromyographic signal and joint angle by using the sliding window method, setting the sliding window length to 100ms and the step length to 0.5ms; Step 12: Calculate the root mean square RMS for the surface electromyography signal, perform μ-law normalization on the root mean square RMS, and use the normalized root mean square RMS as the input data of the model; Step 13: For the joint angle, use the last joint angle in the sliding window as the target data.

3. According to claim 2, a finger continuous estimation method based on multi-scale dilated convolution and Transformer model is characterized in that: The formulas for the root mean square RMS and μ-law normalization are: Among them, N represents the window size, x t represents the electromyographic data sampled at time t; μ represents the scaling parameter of the signal, μ=2^20.

4. According to claim 1, the finger continuous estimation method based on multi-scale dilated convolution and Transformer model is characterized in that: The step 2 comprises the following steps: Step 21: Use multi-scale dilated convolution to extract local features of surface electromyographic signals, and use a fixed convolution kernel and dilated convolutions with dilated rates of 1, 2, and 4 to extract electromyographic signal features in parallel; Step 22: Use the encoding layer in the Transformer architecture to extract the global features of the surface electromyography signal, where the encoding layer includes: a multi-head attention layer and a feedforward neural network.

5. According to claim 4, a finger continuous estimation method based on multi-scale dilated convolution and Transformer model is characterized in that: The step 21 comprises the following steps: Step 211: In the first branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 1 is used. After batch normalization and activation function, the convolution layer is input into the second convolution operation, and a convolution kernel of 5 and a dilation rate of 1 are used. The activation function is used to obtain the first branch feature data; Step 212: In the second branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 2 is used. After batch normalization and activation function, the convolution layer is input into the second convolution operation. A convolution kernel of 5 and a dilation rate of 2 are used. The activation function is used to obtain the feature data of the second branch. Step 213: In the third branch, a convolution layer with a convolution kernel size of 1 and a dilation rate of 4 is used. After batch normalization and activation function, it is input into the second convolution operation, and a convolution layer with a convolution kernel of 5 and a dilation rate of 4 is used. The activation function is used to obtain the feature data of the third branch; Step 214: splicing the first branch characteristic data, the second branch characteristic data and the third branch characteristic data in the channel dimension; Step 215: A convolution layer with a convolution kernel of 1 is connected after the channel attention mechanism to perform dimensionality reduction and reduce the number of channels. Batch normalization and activation function are performed after the convolution layer, and finally the local features extracted by the multi-scale dilated convolution module are output.

6. According to claim 5, a finger continuous estimation method based on multi-scale dilated convolution and Transformer model is characterized in that: The attention mechanism ECA is: Use global average pooling GAP to process the splicing features, and then use the convolution layer followed by the Sigmoid activation function to multiply the result with the original feature map to give different weights to each channel. The size of the convolution kernel in the convolution layer adjusts the size of the convolution kernel k according to the channel. The convolution kernel is calculated as follows: Here, γ=2, b=1 are set; C represents the number of channels.

7. According to claim 4, a finger continuous estimation method based on multi-scale dilated convolution and Transformer model is characterized in that: The step 22 comprises the following steps: Step 221: using a linear layer as an embedding layer for the surface electromyography signal, and adding time embedding to the linear layer output to capture time information; the time embedding is used to train a one-dimensional vector; Step 222: The data X of the embedding time is input into the multi-head attention layer through the layer normalization layer; the output X of the multi-head attention layer is M Add to X to form an add layer, i.e. residual connection, and get the output X′ M ; Step 223: For output X′ M Use layer normalization and input to a feedforward neural network; the feedforward neural network includes: two linear layers; after the first linear layer, use GELU as the activation function to convert the output X of the feedforward neural network F With X′ M The addition forms the add layer, i.e., the residual connection, and finally outputs the extracted global features.

8. The finger continuous estimation method based on multi-scale dilated convolution and Transformer model according to claim 1, characterized in that: The step 3 comprises the following steps: Step 31: construct a fusion module; Step 32: Use a fully connected layer to map the obtained fusion features into joint angles. In order to make the output results more consistent with the actual movement, a smoothing layer is added after the fully connected layer. Step 33: Obtain an estimation model using the Pearson correlation coefficient PCC and the normalized root mean square error NRMSE; the Pearson correlation coefficient is: Where N is the number of samples; θ est represents the estimated joint angle value; represents the estimated joint angle mean; θ real Indicates the actual joint angle value, represents the actual joint angle mean; θ max Indicates the maximum value of the actual joint angle; θ min Indicates the minimum value of the actual joint angle.

9. The finger continuous estimation method based on multi-scale dilated convolution and Transformer model according to claim 1, characterized in that: The smoothing layer uses a weighted average smoothing method with a sliding window size, which is calculated as follows: w i ＝α*(1-α) i ； Among them, w i represents the weight of the i-th position, α represents the weighting coefficient, α = 0.3; out t represents the result after smoothing at time t; N represents the window size, N=3; for the data output by the fully connected layer, the output values ​​of the initial two sampling points remain unchanged, and smoothing is performed from the third sampling point according to the sliding weighted average.

10. The finger continuous estimation method based on multi-scale dilated convolution and Transformer model according to claim 1, characterized in that: Constructing the fusion module includes the following steps: Step 311: performing channel splicing on the extracted local features and global features; Step 312: Build a convolution block with a convolution kernel of 1. Each convolution block consists of two convolution layers with a convolution kernel of 1. After the first convolution layer, relu is used as the activation function to add the data input to the first convolution layer and the data output by the second convolution layer to form an add layer, i.e., a residual connection. Step 313: Build three convolution blocks in the fusion module, add the concatenated features obtained in the first step to the output results of the three convolution blocks to form an add layer, i.e., residual connection, and output the fusion features.

Images