Escalator behavior detection method based on multi-branch human body feature fusion network

Abstract

The invention discloses an escalator behavior detection method based on a multi-branch human body feature fusion network, and relates to the technical field of image processing, and the method comprises the steps: processing a matrix through a multi-branch network comprising a joint flow, a skeleton flow, a velocity flow and a skeleton angle flow, and obtaining a joint feature matrix, a skeleton feature matrix, a velocity feature matrix and an angle feature matrix respectively; the features are fused through a self-attention mechanism; capturing dynamic changes of features in time by using multi-scale time sequence convolution, and generating a time sequence feature matrix; interacting with a time sequence feature matrix through attention weighted edge convolution, and converting a result into a high-dimensional feature vector; and inputting to a full connection layer, mapping to a category space to predict probability distribution of escalator behaviors, and selecting a category with the highest probability as a final behavior prediction result. According to the method, the multi-branch human body feature fusion network is constructed, and the self-attention mechanism and the multi-scale time sequence convolution are combined, so that detection and recognition of pedestrian behaviors on the escalator are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an escalator behavior detection method based on a multi-branch human feature fusion network. Background Art

[0002] Escalators are the most common means of transporting pedestrians in public places. They are used by people of all ages, from the elderly to babies in strollers. Therefore, the safety monitoring of escalators has always been a public safety issue that needs to be solved urgently. While escalator equipment improves the efficiency of people transportation, it also brings safety hazards. In actual applications, due to improper use by passengers, such as running fast, walking in the opposite direction, children playing, etc., as well as untimely emergency rescue, escalators are prone to cause accidents such as passengers falling, colliding, and squeezing. Common causes of escalator accidents are mostly caused by pedestrians climbing escalators, falling, or walking in the opposite direction.

[0003] However, in the narrow space of the escalator, there is a high degree of occlusion in front, behind, left and right, and many human behaviors do not change much, resulting in the consistent spatial structure of the action and lack of significant geometric differences. In addition, the temporal and spatial features are unevenly distributed, with some key frame features of the action in time being too significant, while the features of other frames are weak or not obvious, resulting in the model being unable to effectively capture the complete action dynamics.

[0004] In the existing technology, many methods based on graph convolutional networks can identify human behavior, but there is still a high degree of misjudgment for similar spatial actions. The dynamic changes of human actions in the time dimension are not fully captured, resulting in the model relying mainly on static spatial features for classification. Different joints or bone connections have different importance in the spatial dimension. Some joints or bones contribute more to action recognition, while other parts may be redundant, affecting the efficiency and effect of feature extraction. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes an escalator behavior detection method based on a multi-branch human feature fusion network, which accurately identifies the unsafe actions of passengers on the escalator and solves the problems of height occlusion, similarity of human spatial movements, and uneven distribution of temporal and spatial features from multiple dimensions of human features.

[0006] The specific plan is as follows:

[0007] The escalator behavior detection method based on a multi-branch human feature fusion network includes:

[0008] S1, obtain the action video of the pedestrian on the escalator; extract the coordinates of the human joint points of each frame in the action video through the human posture estimator; merge the coordinates of the human joint points of each frame into a coordinate sequence in time order, and convert the coordinate sequence into an original feature matrix;

[0009] S2, construct a multi-branch human feature fusion network; the multi-branch human feature fusion network includes joint flow, bone flow, velocity flow, bone angle flow, attention mechanism fusion module and multiple graph convolution sub-blocks; the bone flow calculates the vector length and vector direction of the bone edge based on the original feature matrix to obtain the bone feature matrix; the velocity flow calculates the velocity vectors of the human joint points and bone edges based on the original feature matrix to obtain the velocity feature matrix; the bone angle flow calculates the limb angle and limb angular velocity based on the original feature matrix to obtain the bone angle feature matrix; each graph convolution sub-block includes graph convolution, multi-scale temporal convolution and attention-weighted edge convolution;

[0010] S3, captures the temporal dynamic changes of the original feature matrix, the bone feature matrix, the velocity feature matrix, and the bone angle feature matrix through multi-scale temporal convolution to obtain the temporal feature matrix;

[0011] S4, weights the temporal feature matrix through attention-weighted edge convolution, and inputs the attention-weighted feature matrix into the global pooling layer to convert it into a high-dimensional feature vector;

[0012] S5, weighted fusion of high-dimensional feature vectors is performed through the self-attention mechanism fusion module, and the fused high-dimensional feature vectors are mapped to the category space through the fully connected layer to calculate the probability distribution of escalator behaviors in each category, and the category with the highest probability is selected as the prediction result of the escalator behavior.

[0013] Furthermore, in S2, the vector length and vector direction of the bone edge are calculated as follows:

[0014] ;

[0015] ;

[0016] ;

[0017] in, Represents the original feature matrix in the tth frame The coordinates of the joint points; Represents the original feature matrix in the tth frame The parent node of each joint point in the skeleton topology diagram; The vector length representing the bone edge; Indicates that the bone amplitude is the norm of the vector length, The vector direction representing the bone edge.

[0018] Furthermore, in S2, the velocity vectors of the human joints and bone edges are calculated as follows:

[0019] ;

[0020] ;

[0021] ;

[0022] in, Indicates the joint point number; , Indicates joint points In time The coordinates of time, Indicates the total number of frames of the input video; Indicates the time difference between adjacent frames; Represents the bone segment at time The coordinates of the center point at the time; N represents the total number of center points of joint points and bone edges; Represents the velocity vector of human joints and bone edges.

[0023] Furthermore, in S2, the calculation formulas for the limb angle and limb angular velocity are as follows:

[0024] ;

[0025] ;

[0026] in, Indicates the first The limb angles of the bones, Indicates the first Angular velocity of limbs between frames for each skeleton edge; Represents the component of the i-th bone edge in the positive direction of the z-axis in the n-th frame; Represents the component of the i-th bone edge in the n-th frame in the positive direction of the x-axis; It represents the component of the ith bone edge in the positive direction of the y-axis in the nth frame; l represents the bone edge; t represents the time.

[0027] Furthermore, in S4, the time series feature matrix is ​​weighted by attention-weighted edge convolution, and the calculation formula is as follows:

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] in, () represents a multi-layer perceptron; represents the time series feature matrix; Represents the feature matrix after attention weighting; Indicates the calculation of key points based on multi-layer perceptron MLP and key points Features; Represents the weight coefficient of the dynamic attention mechanism; represents edge convolution; Representation Node The set of neighbors of ; Representation Node and neighbor nodes The characteristic difference represents the adjacency matrix; Indicates the target node With neighbor nodes The attention score of the neighbor node For the target node the importance of and Represents the output feature tensor of the feature matrix X through graph convolution The key points and key points; Represents element-wise multiplication; represents the activation function; Indicates bias.

[0033] Furthermore, in S5, the high-dimensional feature vector is weightedly fused by the self-attention mechanism fusion module, and the specific formula is as follows:

[0034] ;

[0035] Q, K, and V represent the query matrix, key matrix, and value matrix obtained by linear transformation from the input feature stream; Represents the similarity weight distribution between different flows; Represents the weighted fusion feature.

[0036] The present invention adopts the above technical solution and has the following beneficial effects:

[0037] (1) The present invention effectively solves the problem of front, back, left, and right height occlusion caused by the narrow and long space of the escalator and the problem of misjudgment caused by similar actions in the same spatial structure through an escalator behavior detection method based on a multi-branch human feature fusion network. Multi-stream fusion combines the characteristic advantages of joints, bone segments, speed, and bone angles to fully capture the static structure and dynamic changes of the action. The introduction of velocity flow allows the motion characteristics of joints and bone segments to be expressed and the time domain characteristics can be fully utilized. The introduction of bone angle flow focuses on the geometric relationship between bones, which complements the feature flow of joint flow and bone flow based on position or distance, and can effectively enhance the ability to distinguish complex actions.

[0038] (2) The present invention constructs a multi-scale temporal convolution module (MSTCN) and implements different dilation rates to construct convolutions of various scales to capture information of different time spans. The multi-scale temporal convolution perceives the logic of the behavior by combining the information of the previous and next adjacent frames and restores the action trajectory of certain fragment frames during occlusion.

[0039] (3) The present invention constructs an attention-weighted edge convolution module (AEC) and introduces a dynamic attention mechanism based on edge convolution to enhance the overall network's capture of important spatial information and ignore secondary information, thereby screening out important features and improving the accuracy of unsafe action recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of an escalator behavior detection method based on a multi-branch human feature fusion network according to an embodiment of the present invention;

[0041] Figure 2 This is a working architecture diagram of a system applied to an escalator according to an embodiment of the present invention;

[0042] Figure 3 It is a structural diagram of a graph convolution sub-block in a multi-branch human feature fusion network according to an embodiment of the present invention;

[0043] Figure 4 This is a diagram of a multi-scale temporal convolution structure of an embodiment of the present invention;

[0044] Figure 5 This is a diagram of the attention-weighted edge convolution structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0046] like Figure 1 As shown, the escalator behavior detection method based on a multi-branch human feature fusion network of the present invention includes:

[0047] S1, obtain the action video of pedestrians on the escalator; extract the coordinates of the human joint points of each frame in the action video through the human posture estimator; merge the coordinates of the human joint points of each frame into a coordinate sequence in time order, and convert the coordinate sequence into the original feature matrix.

[0048] Specifically, in this embodiment, the input to the network is the key point coordinates of each person in each frame of the video , these coordinates are used to construct joint flow, bone flow, velocity flow and bone angle flow to solve the problem of misjudgment of similar spatial actions, and multi-scale temporal convolution modules and attention-weighted edge convolution modules are used to improve the graph convolution network to solve the problem of uneven distribution of spatiotemporal features; specifically, the configuration obtains escalator monitoring video (30FPS), segments the video into short videos of 10s every 300 frames, and extracts the coordinates of the key points of each person in each frame of the video The text sequences are formed in time order to be input into the network for training.

[0049] Specifically, Figure 2 As shown, in this embodiment, a working architecture diagram of a system applied to escalators is used, video is collected by a camera, and after preprocessing, features are extracted using a network model including joint flow, bone flow, velocity flow and bone angle flow, and then the features are integrated through a self-attention mechanism fusion module to finally determine and alarm unsafe behavior.

[0050] S2, construct a multi-branch human feature fusion network; the multi-branch human feature fusion network includes joint flow, bone flow, velocity flow, bone angle flow, attention mechanism fusion module and multiple graph convolution sub-blocks; the bone flow calculates the vector length and vector direction of the bone edge based on the original feature matrix to obtain the bone feature matrix; the velocity flow calculates the velocity vectors of the human joints and bone edges based on the original feature matrix to obtain the velocity feature matrix; the bone angle flow calculates the limb angles and limb angular velocities based on the original feature matrix to obtain the bone angle feature matrix; each graph convolution sub-block includes graph convolution, multi-scale temporal convolution and attention-weighted edge convolution.

[0051] Specifically, the calculation formulas for the vector length and vector direction of the bone edge are as follows:

[0052] ;

[0053] ;

[0054] ;

[0055] in, Represents the original feature matrix in the tth frame The coordinates of the joint points; Represents the original feature matrix in the tth frame The parent node of each joint point in the skeleton topology diagram; The vector length representing the bone edge; Indicates that the bone amplitude is the norm of the vector length, The vector direction representing the bone edge.

[0056] Specifically, the velocity vectors of human joints and bone edges are calculated as follows:

[0057] ;

[0058] ;

[0059] ;

[0060] in, Indicates the joint point number; , Indicates joint points In time The coordinates of time, Indicates the total number of frames of the input video; Indicates the time difference between adjacent frames; Represents the bone segment at time The coordinates of the center point at the time; N represents the total number of center points of joint points and bone edges; Represents the velocity vector of human joints and bone edges.

[0061] Specifically, the calculation formulas for limb angle and limb angular velocity are as follows:

[0062] ;

[0063] ;

[0064] in, Indicates the first The limb angles of the bones, Indicates the first Angular velocity of limbs between frames for each skeleton edge; Represents the component of the i-th bone edge in the positive direction of the z-axis in the n-th frame; Represents the component of the i-th bone edge in the positive direction of the x-axis in the n-th frame; It represents the component of the ith bone edge in the positive direction of the y-axis in the nth frame; l represents the bone edge; t represents the time.

[0065] Specifically, Figure 3As shown in FIG. 1 , the joint stream, bone stream, velocity stream, and angle stream in the embodiment of the present invention are shown in FIG. 1 . Each branch stream uses Figure 3 The improved graph convolution block shown in the figure is used to recognize human behavior. In the figure, the 1×1 convolution layer represents feature mapping, linearly combines the input feature matrix, and generates a new feature representation; the GCN graph convolution representation is used to aggregate the features of adjacent nodes and adjacent bone edges;

[0066] The BN layer represents batch normalization, which normalizes each small batch of data to stabilize the training process, accelerate convergence and regularize the effect. The ReLU layer represents the use of the ReLU activation function, which enables the neural network to learn and represent complex nonlinear relationships through simple nonlinear transformations. The GAP layer represents the conversion of the 4-dimensional feature matrix into a high-dimensional feature matrix; the fully connected layer represents the mapping of the high-dimensional feature matrix to the category space Softmax layer; the Softmax layer represents the conversion of the category score into a probability distribution, and the output layer selects the category with the highest probability as the prediction result of human behavior based on the probability distribution output by Softmax.

[0067] S3, through multi-scale temporal convolution, captures the dynamic changes of the original feature matrix, bone feature matrix, velocity feature matrix and bone angle feature matrix in time to obtain the temporal feature matrix.

[0068] Specifically, in this embodiment, the feature extraction part of the multi-scale temporal convolution module includes six convolution branches: a max pooling branch and five one-dimensional convolution branches with expansion rates ranging from 1 to 5 and a convolution kernel of 3x1; first, a dynamic channel allocation module is constructed to dynamically allocate channels to each branch, and the number of remaining channels is calculated to ensure that the channel allocation of each branch in the network is reasonable. The number of intermediate channels is obtained by dividing the number of output channels by the total number of branches. The first branch is allocated to the remaining number of channels, and the remaining 5 branches are allocated to the intermediate number of channels. The features are input into the dynamic channel allocation module, divided into six groups from the channel dimension, and input into each convolution branch respectively.

[0069] In order to reduce the increase in computational complexity caused by the six branches, a residual bottleneck structure is added before the max pooling branch and the five one-dimensional convolution branches to reduce the number of channels. The basic number of channels is 64, which is reduced to 32 through the bottleneck structure.

[0070] The input skeleton data feature map is ,according to To predict the results to get the feature map , as shown below:

[0071]

[0072] is the value of the output sequence at time step t, Indicates The weight of the convolution kernel, k represents the length of the convolution kernel, which determines the receptive field of the convolution kernel. The feature maps output by the five branches are And the maximum pooling branch is spliced ​​in the channel dimension to form a high-dimensional feature matrix At the same time, the concatenated channels are mapped to the final number of output channels, and finally the residual network resnet50 is used to perform high-level feature fusion on the concatenated features. .

[0073]

[0074]

[0075] Where L represents the number of residual blocks, Represents the weights and biases in the residual network.

[0076] Specifically, a multi-scale temporal convolution module (MSTCN) is embedded in each graph convolution block in the multi-branch human feature fusion network. Different convolution kernel sizes can extract features at different scales, and different convolution kernel sizes and expansion rates are used to process features at different time scales. The feature extraction part of the multi-scale temporal convolution consists of 6 convolution branches, 1 max pooling branch and 5 one-dimensional convolution branches with expansion rates ranging from 1 to 5 and convolution kernels of 3x1, including the following parts:

[0077] Dilation rate 1: standard 3×1 convolution with a receptive field of 3, focusing on local details.

[0078] Dilation rate 2: 3×1 convolution with an interval of 1 pixel and a receptive field of 5 to capture features in a slightly larger range.

[0079] Dilation rate 3: 3×1 convolution with a spacing of 2 pixels and a receptive field of 7.

[0080] Dilation rate 4: 3×1 convolution with a spacing of 3 pixels and a receptive field of 9.

[0081] Dilation rate 5: 3×1 convolution with an interval of 4 pixels and a receptive field of 11 to capture global information in a longer range.

[0082] like Figure 4The figure shows the multi-scale time series convolution structure diagram of the present embodiment, in which the bottleneck structure indicates that a 1×1 convolution kernel is used to reduce the number of channels; the maximum pooling layer is used to reduce the spatial size of the data and reduce the computational complexity; the activation layer uses the ReLU activation function, which enables the neural network to learn and represent complex nonlinear relationships through simple nonlinear transformations; the dilated convolution is composed of 5 branches, each consisting of a 3×1 convolution kernel with a dilation rate of 1 to 5; the maximum pooling branch consists of a bottleneck mechanism, a maximum pooling layer, and an activation layer; the convolution branches with different expansion rates are composed of a bottleneck structure, a dilated convolution, and an activation layer; the input feature matrix: the channel allocation module re-allocates channels according to the six convolution branches to obtain six feature matrices; the output feature matrix: obtained after the convolution layer Feature matrix; use the residual network resnet50 to perform high-level feature fusion on the concatenated features Output.

[0083] S4, the time series feature matrix is ​​weighted by attention-weighted edge convolution, and the attention-weighted feature matrix is ​​input into the global pooling layer to be converted into a high-dimensional feature vector.

[0084] Specifically, the time series feature matrix is ​​weighted by attention-weighted edge convolution, and the calculation formula is as follows:

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] in, () represents a multi-layer perceptron; represents the time series feature matrix; Represents the feature matrix after attention weighting; Indicates the calculation of key points based on multi-layer perceptron MLP and key points Features; Represents the weight coefficient of the dynamic attention mechanism; represents edge convolution; Representation Node The set of neighbors of ; Representation Node and neighbor nodes The characteristic difference represents the adjacency matrix; Indicates the target node With neighbor nodes The attention score of the neighbor node For the target node Importance and Represents the output feature tensor of the feature matrix X through graph convolution The key points and key points; Represents element-wise multiplication; represents the activation function; Indicates bias.

[0090] In this embodiment, it is assumed that the MLP consists of L layers, and each layer performs an affine transformation and a nonlinear activation function. Then the definition of the MLP is:

[0091] ;

[0092] represents the weight matrix of the Lth layer, represents the bias vector of the Lth layer, is a nonlinear activation function, x is the input vector, x= .

[0093] Specifically, an attention-weighted edge convolution module (AEC) is embedded in the graph convolution block of the multi-branch human feature fusion network. By default, the contribution of edge convolution to neighboring nodes is equal, and the introduction of dynamic attention mechanism can give weights to different neighboring nodes, thereby focusing on more important edge information, such as Figure 5 As shown in the figure, the role of the adjacency matrix A is to define the graph structure and calculate the feature difference with the input feature matrix; MLP convolution represents the multi-layer perceptron used to enhance the feature expression capability; Edge convolution (EdgeConv): calculates the feature difference between the adjacency matrix and the input feature matrix; the batch normalization layer normalizes each small batch of data to stabilize the training process, accelerate convergence and regularize the effect; the role of the maximum pooling layer: reduce the spatial size of the data and reduce the computational complexity; dynamic attention: dynamically adjust the weights of the nodes in the feature matrix.

[0094] S5, weighted fusion of high-dimensional feature vectors is performed through the self-attention mechanism fusion module, and the fused high-dimensional feature vectors are mapped to the category space through the fully connected layer to calculate the probability distribution of escalator behaviors in each category, and the category with the highest probability is selected as the prediction result of the escalator behavior.

[0095] Specifically, the high-dimensional feature vectors are weightedly fused through the self-attention mechanism fusion module. The specific formula is as follows:

[0096] ;

[0097] Q, K, and V represent the query matrix, key matrix, and value matrix obtained by linear transformation from the input feature stream; Represents the similarity weight distribution between different flows; Represents the weighted fusion feature.

[0098] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. An escalator behavior detection method based on a multi-branch human feature fusion network, characterized in that: include: S1, obtain the action video of pedestrians on the escalator; The human body joint point coordinates of each frame in the action video are extracted through the human body posture estimator; the human body joint point coordinates of each frame are merged into a coordinate sequence in time order, and the coordinate sequence is converted into the original feature matrix; S2, construct a multi-branch human feature fusion network; the multi-branch human feature fusion network includes joint flow, bone flow, velocity flow, bone angle flow, attention mechanism fusion module and multiple graph convolution sub-blocks; the bone flow calculates the vector length and vector direction of the bone edge based on the original feature matrix to obtain the bone feature matrix; the velocity flow calculates the velocity vectors of the human joint points and bone edges based on the original feature matrix to obtain the velocity feature matrix; the bone angle flow calculates the limb angle and limb angular velocity based on the original feature matrix to obtain the bone angle feature matrix; each graph convolution sub-block includes graph convolution, multi-scale temporal convolution and attention-weighted edge convolution; S3, captures the temporal dynamic changes of the original feature matrix, the bone feature matrix, the velocity feature matrix, and the bone angle feature matrix through multi-scale temporal convolution to obtain the temporal feature matrix; S4, weights the temporal feature matrix through attention-weighted edge convolution, and inputs the attention-weighted feature matrix into the global pooling layer to convert it into a high-dimensional feature vector; S5, weighted fusion of high-dimensional feature vectors is performed through the self-attention mechanism fusion module, and the fused high-dimensional feature vectors are mapped to the category space through the fully connected layer to calculate the probability distribution of escalator behaviors in each category, and the category with the highest probability is selected as the prediction result of the escalator behavior.

2. The escalator behavior detection method based on a multi-branch human feature fusion network according to claim 1 is characterized in that: In S2, the vector length and vector direction of the bone edge are calculated as follows: ； ； ； in, Represents the original feature matrix in the tth frame The coordinates of the joint points; Represents the original feature matrix in the tth frame The parent node of each joint point in the skeleton topology diagram; The vector length representing the bone edge; Indicates that the bone amplitude is the norm of the vector length, The vector direction representing the bone edge.

3. The escalator behavior detection method based on a multi-branch human feature fusion network according to claim 1 is characterized in that: In S2, the velocity vectors of human joints and bone edges are calculated as follows: ； ； ； in, Indicates the joint point number; , Indicates joint points In time The coordinates of time, Indicates the total number of frames of the input video; Indicates the time difference between adjacent frames; Represents the bone segment at time The coordinates of the center point at the time; N represents the total number of center points of joint points and bone edges; Represents the velocity vector of human joints and bone edges.

4. The escalator behavior detection method based on a multi-branch human feature fusion network according to claim 1 is characterized in that: In S2, the calculation formulas for limb angle and limb angular velocity are as follows: ； ； in, Indicates the first The limb angles of the bones, Indicates the first Angular velocity of limbs between frames for each skeleton edge; Represents the component of the i-th bone edge in the positive direction of the z-axis in the n-th frame; Represents the component of the i-th bone edge in the n-th frame in the positive direction of the x-axis; It represents the component of the ith bone edge in the positive direction of the y-axis in the nth frame; l represents the bone edge; t represents the time.

5. The escalator behavior detection method based on a multi-branch human feature fusion network according to claim 4 is characterized in that: In S4, the time series feature matrix is ​​weighted by attention-weighted edge convolution, and the calculation formula is as follows: ； ； ； ； in, () represents a multi-layer perceptron; represents the time series feature matrix; Represents the feature matrix after attention weighting; Indicates the calculation of key points based on multi-layer perceptron MLP and key points Features; Represents the weight coefficient of the dynamic attention mechanism; represents edge convolution; Representation Node The set of neighbors of ; Representation Node and neighbor nodes The characteristic difference represents the adjacency matrix; Indicates the target node With neighbor nodes The attention score of the neighbor node For the target node the importance of and Represents the output feature tensor of the feature matrix X through graph convolution The key points and key points; Represents element-wise multiplication; represents the activation function; Indicates bias.

6. The escalator behavior detection method based on a multi-branch human feature fusion network according to claim 1 is characterized in that: In S5, the high-dimensional feature vector is weightedly fused through the self-attention mechanism fusion module. The specific formula is as follows: ； Q, K, and V represent the query matrix, key matrix, and value matrix obtained by linear transformation from the input feature stream; Represents the similarity weight distribution between different flows; Represents the weighted fusion feature.

Images