A human motion direction determination method and system based on a fusion network

Abstract

The embodiment of the specification provides a human motion direction determination method and system based on a fusion network, wherein the method comprises the following steps: collecting a human echo signal through a radar, performing fast Fourier transform (FFT) on the human echo signal to generate a range-Doppler map, filtering out stationary clutter, and then performing short-time Fourier transform (STFT) to generate a data set containing a micro-Doppler signature (MDS) image; dividing the data set into a training set and a test set, using a fusion network of a convolutional neural module and a time sequence module to extract Doppler features from the training set for model training, using the test set for model testing, and obtaining an optimal model; and judging the human motion direction based on the trained optimal model and the collected human echo signal.

Description

Technical Field

[0001] This document relates to the field of human behavior recognition technology, and in particular to a method and system for determining the direction of human movement based on a fusion network. Background Technology

[0002] Human Motion Direction Determination (HMDD) is a crucial component of Human Action Recognition (HAR) and has broad application prospects in multiple fields. HMDD uses radar to accurately measure human activity. From the raw data collected by radar, time-frequency analysis techniques can be used to extract Micro-Doppler Features (MDS) in the joint time-frequency domain to describe the micro-motions of moving objects. Initial HAR research focused on keeping human targets within radar detection range and moving towards or away from the radar. However, in practical applications, the direction of human target movement is often unpredictable. When a human target deviates from the radial direction of the radar antenna, the radial velocity and radial distance of each limb on the radar also change with the change in the azimuth angle of the human movement. Therefore, the preprocessed MDS will also differ, which makes HMDD based on MDS images possible.

[0003] HMDD based on MDS faces two main challenges. First, since the micro-Doppler features of human motion are related to the radial velocity of the human body, the micro-Doppler features in two radially symmetrical directions are almost identical, which can affect the accurate recognition of HMDDs. Second, in the same direction, because the human body is a non-rigid object, the MDS it generates exhibits diversity, making recognition in the same direction potentially more challenging. HMDD based on MDS images is currently a research hotspot, with increasing research focusing on applying deep learning (DL) techniques to the field. In related HMDD research, researchers have found that not only are the spatial features of MDS important, but the temporal dependencies of MDS are also noteworthy. Therefore, different temporal networks (RNN, LSTM, Bi-LSTM, etc.) are gradually being applied to HMDD. However, the capabilities of using a single temporal network for HMDD are limited. Summary of the Invention

[0004] This specification provides one or more embodiments of a method for determining the direction of human motion based on a fusion network, including:

[0005] S1. Collect human body echo signals by radar, perform Fast Fourier Transform (FFT) on the human body echo signals to generate a range Doppler map, filter out stationary clutter, and then perform Short Time Fourier Transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images.

[0006] S2. Divide the dataset into a training set and a test set, and use a fusion network of convolutional neural modules and temporal modules to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model;

[0007] S3. Based on the trained optimal model, determine the direction of human movement according to the collected human echo signals.

[0008] Furthermore, step S1 specifically includes:

[0009] After transmitting a signal using radar, the echo signal reflected back from the human body and the surrounding environment is collected and mixed with the transmitted signal to generate an intermediate frequency signal.

[0010] The acquired multi-frame echo signals are incoherently accumulated, and the accumulated signal is subjected to a Fast Fourier Transform (FFT) to obtain a range Doppler map, where the signal can be represented as S(r, v, t). Then, a high-pass filter is applied to S(r, v, t) to remove stationary clutter, generating a two-dimensional matrix.

[0011] The generated two-dimensional matrix is ​​subjected to a short-time Fourier transform (STFT) to obtain a dataset of micro-Doppler images containing micro-Doppler features (MDS).

[0012] The micro-Doppler features on the micro-Doppler image represent the different velocities of various parts of the body over time, thereby determining the current state of human movement.

[0013] Furthermore, the dataset is divided into a training set and a test set in an 8:2 ratio.

[0014] Furthermore, the specific method for extracting Doppler features from the training set using a fusion network of convolutional neural modules and temporal modules for model training is as follows:

[0015] The training dataset is trained using Conv-GRU, a variant of LSTM, and the loss value is calculated during the training process.

[0016] To minimize the loss value, the gradient of each neuron in the backpropagation algorithm is calculated, the parameters in the fusion network are updated, and the gradient descent algorithm is used to update the parameters.

[0017] By iterating the above training repeatedly, the recognition accuracy of the fusion network of convolutional neural modules and temporal modules is improved, and a well-trained model is obtained.

[0018] Furthermore, the convolutional neural module includes a feature extraction part and a feature enhancement part. The feature extraction part consists of two MCblock modules, which include a depthwise separable convolution module, an SE attention module, and a residual module, used to capture multi-scale local micro-Doppler features on the MDS. The feature enhancement part includes MCblock and an improved ViT module with a self-attention mechanism, used to further process and optimize the global features in the MDS feature map through positional embedding.

[0019] Furthermore, the temporal module consists of two Conv-GRU layers. The Conv-GRU is used to construct spatial features of the input feature map within the unit through convolution operations, and introduces a gating mechanism to extract temporally correlated features, which includes four computational parts: update gate r t Reset door z t Activation status and the hidden state h t .

[0020] Furthermore, the specific method for using the test set to test the model and obtain the optimal model is as follows:

[0021] The trained model was tested using a test set, and the performance of the model fusion network of convolutional neural modules and temporal modules was evaluated based on performance evaluation metrics.

[0022] The performance evaluation metrics include: model accuracy, precision, recall, specificity, and F1 score.

[0023] This specification provides one or more embodiments of a human motion direction determination system based on a fusion network, including:

[0024] Data processing module: used to collect human body echo signals through radar, perform Fast Fourier Transform (FFT) on the human body echo signals to generate range Doppler maps, filter out stationary clutter, and then perform Short Time Fourier Transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images.

[0025] Model training module: used to divide the dataset into training set and test set, and use a network that combines convolutional neural module and temporal module to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model;

[0026] Direction determination module: Used to determine the direction of human movement based on the collected human echo signals, using a well-trained optimal model.

[0027] This specification provides one or more embodiments of an electronic device, including:

[0028] Processor; and,

[0029] A memory is configured to store computer-executable instructions, which, when executed, cause the processor to implement the steps of the above-described method for determining the direction of human movement based on a fusion network.

[0030] This specification provides one or more embodiments of a storage medium for storing computer-executable instructions, which, when executed, implement the steps of the above-described method for determining the direction of human movement based on a fusion network.

[0031] Using the embodiments of the present invention, the present invention combines micro-Doppler features and a fusion network of convolutional neural modules and temporal modules. On the HMDD task, it achieves the effects of fast operation speed, low operation complexity, small number of parameters, fast convergence speed and high recognition accuracy, which can meet the requirements of real-time and accurate determination of human movement direction.

[0032] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart illustrating a method for determining the direction of human motion based on a fusion network, provided for one or more embodiments of this specification;

[0035] Figure 2 This is a flowchart illustrating the overall process of training a fusion network model in one or more embodiments of this specification.

[0036] Figure 3 A schematic diagram illustrating the composition of a human motion direction determination system based on a fusion network, provided for one or more embodiments of this specification;

[0037] Figure 4 This is a schematic diagram of the structure of an electronic device provided for one or more embodiments of this specification. Detailed Implementation

[0038] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0039] Method Implementation Examples

[0040] According to embodiments of the present invention, a method for determining the direction of human motion based on a fusion network is provided. Figure 1 A flowchart illustrating a method for determining human motion direction based on a fusion network, provided for one or more embodiments of this specification, is shown below. Figure 1 As shown, the human motion direction determination method based on a fusion network according to an embodiment of the present invention specifically includes:

[0041] S1. Collect human body echo signals through radar, perform Fast Fourier Transform (FFT) on the human body echo signals to generate a range Doppler image, filter out stationary clutter, and then perform Short Time Fourier Transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images.

[0042] After transmitting a signal using radar, the echo signals reflected back from the human body and the surrounding environment are collected and mixed with the transmitted signal to generate an intermediate frequency signal S(n, l, t).

[0043] The acquired multi-frame echo signals are incoherently accumulated to obtain the accumulated signal. A single Fast Fourier Transform (FFT) is applied to the accumulated signal to obtain a range-Doppler image, where a single signal is represented as S(r, v, t). To avoid stationary clutter obscuring the active signal, a high-pass filter is used to filter out stationary clutter from S(r, v, t), generating a two-dimensional matrix.

[0044] For two-dimensional matrix Perform a short-time Fourier transform (STFT) to obtain the final time-frequency map, which is a dataset of micro-Doppler images containing micro-Doppler feature data (MDS).

[0045] The expression for STFT is as follows:

[0046]

[0047] Among them W n It is a window function, where t is the time index and w is the frequency index;

[0048] The micro-Doppler features on the micro-Doppler feature image represent the different velocities of various parts of the body over time, thereby determining the current state of human movement.

[0049] S2. Divide the dataset into a training set and a test set, and use a fusion network of convolutional neural modules and temporal modules to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model.

[0050] To train the optimal network model and test its performance, the dataset needs to be divided into training and testing datasets. In the experimental example, the obtained micro-Doppler dataset is divided in an 8:2 ratio, with 80% of the data used as the training dataset to train the model and the remaining 20% ​​used as the testing dataset to evaluate the model's real-world performance. The overall flowchart for training the fusion network is shown below. Figure 2 As shown:

[0051] The convolutional neural module includes a feature extraction part and a feature enhancement part. The feature extraction part consists of two MCblock modules, which include a depthwise separable convolution module, an SE attention module, and a residual module, used to capture multi-scale local micro-Doppler features on the MDS.

[0052] Depthwise separable convolution decomposes standard convolution into depthwise convolution and pointwise convolution. Depthwise convolution extracts features within each channel, while pointwise convolution integrates information across channels, increasing the semantic information of the extracted features. Furthermore, it reduces computational cost and network size while extracting spatial features. The computational cost comparison between depthwise separable convolution and standard convolution can be expressed by the following formula:

[0053]

[0054] Where N is the number of output channels, D K It refers to the spatial dimension of the convolution kernel. For example, when the kernel size is 3x3 and N is 128, depthwise separable convolution requires only 0.02 times the computation of standard convolution.

[0055] The SE attention module can weigh the spatial feature weights extracted by depthwise separable convolutions. By explicitly modeling channel dependencies, it implements inter-channel attention within the structure, thereby enhancing feature sensitivity along the channel dimension. This allows the convolutional neural module to adaptively perceive important weights, improving the network's perceptual ability. The specific calculation process is shown below:

[0056]

[0057] w sq=σ(W2δ(W1(x) sq )));

[0058] x se =W sq x dsc ;

[0059] Where x dsc Given the deep separable input features, W1 and W2 are the learned weight matrices, where δ represents the ReLU activation function and σ represents the Sigmoid activation function. Through this calculation process, the SE attention module can precisely adjust the weights of each channel, allowing the network to focus more on learning key features.

[0060] The residual module further optimizes the structure by mixing the original features with deep features, effectively mitigating the vanishing or exploding gradient problems that may occur in the network. This optimization strategy helps ensure that feature information propagates more stably and efficiently in the network, further improving the model's performance and generalization ability.

[0061] The feature enhancement part includes MCblock and an improved ViT module with a self-attention mechanism, which is used to further process and optimize the global features in the MDS feature map through position embedding, thereby improving the generalization ability of the model.

[0062] Unfolding and collapsing operations, along with the multi-head attention mechanism, are key components in improving ViT. The unfolding and collapsing operations split a feature map of size H×W into P units, which are then fed into the multi-head attention mechanism for parallel computation. Finally, the output feature map is folded back into a feature map of size H×W. This operation can improve network computation speed by splitting and concatenating feature maps without affecting recognition performance. The multi-head attention mechanism can be viewed as a combination of multiple self-attention modules. The external input is linearly mapped to three variables of the same dimension through a learning matrix: the query vector Q, the key vector K, and the value vector V. Attention weights are then calculated using these variables. The calculation process of the self-attention mechanism is as follows:

[0063]

[0064] Where d k For the overall dimension, the normalized similarity between Q and K is used as the weight of V in this calculation. Furthermore, a shortcut mechanism is employed in the self-attention mechanism to effectively prevent degradation caused by excessive network depth. Finally, the outputs from different attention mechanisms are combined using Concat. Using multi-head attention allows the network to learn rich feature information from different feature subspaces, and its global receptive field enhances the global dependencies of features, strengthening the learning of image semantics.

[0065] When the human body is active, the movements are continuous and uninterrupted. In micro-Doppler images, micro-Doppler features also change over time; however, micro-Doppler images exist in a discrete temporal form. To make the relationships between continuous movements in micro-Doppler images more coherent, a temporal module is added to the network, using a variant of LSTM called Conv-GRU to handle the dynamic relationships of micro-Doppler features, capture long-range dependencies in the image, and enhance the network's understanding of the spatiotemporal relationships of human movements in micro-Doppler images. This allows the model to more comprehensively and coherently perceive and analyze the dynamic changes of human activity.

[0066] The temporal module consists of two Conv-GRU layers. The Conv-GRU is used to construct spatial features of the input feature map within a unit through convolutional operations and introduces a gating mechanism to extract temporally correlated features. This includes four computational parts: updating the gate r... t Reset door z t Activation status and the hidden state h t The calculations for each part are as follows:

[0067] r t =σ g (W r *x t +U r *h t-1 +b r );

[0068] z t =σ g (W z *x t +U z *h t-1 +b z );

[0069]

[0070]

[0071] Among them W (·) Let b be the weight matrix. (·) For bias, σ g , σ h The activation function is as follows:

[0072]

[0073]

[0074] The classification layer at the end of the network consists of a softmax layer and a fully connected layer. The softmax layer transforms the raw numerical values ​​of the network output into a probability distribution, ensuring that the sum of the probabilities for each category is 1. The fully connected layer linearly combines the features from the previous layer to form the final output. The fully connected layer can learn the complex relationships between different features, mapping abstract high-level features to the final classification result. The classification layer summarizes the features of the entire network, generating probability distributions for different categories, thereby achieving the classification of the input samples.

[0075] The training dataset is trained using a fusion network. The AdamW optimization algorithm is used to adjust the model parameters. During training, the loss value is calculated. Based on the loss value, the gradient of each neuron in each layer is calculated using the backpropagation algorithm, and the parameters are updated using the gradient descent algorithm, so that the loss value gradually decreases. This training process is iterated repeatedly to minimize the prediction error and loss function, thereby improving the recognition accuracy of the fusion network of convolutional neural modules and temporal modules.

[0076] The trained model is tested using the test set to verify its performance and obtain the optimal model. The specific method is as follows:

[0077] The performance of the network integrating convolutional and temporal neural modules was evaluated based on performance metrics by validating it on a 20% test dataset separated from the main dataset. These metrics included model accuracy, precision, recall, specificity, and the F1 score. This testing phase comprehensively analyzed model performance using these metrics, providing a strong reference for further network model optimization. The testing phase also aimed to confirm the model's robustness in real-world scenarios and fully validate its feasibility in practical applications.

[0078] S3. Based on the trained optimal model, determine the direction of human movement according to the collected human echo signals.

[0079] The beneficial effects of this invention are as follows:

[0080] This invention combines micro-Doppler features with a fusion network of convolutional neural modules and temporal modules. On the HMDD task, it achieves high computational speed, low computational complexity, small number of parameters, fast convergence speed, and high recognition accuracy, which can meet the requirements of real-time and accurate determination of human motion direction.

[0081] System Implementation Examples

[0082] According to embodiments of the present invention, a human motion direction determination system based on a fusion network is provided. Figure 3 A schematic diagram illustrating the composition of a human motion direction determination system based on a fusion network, provided for one or more embodiments of this specification, is shown below. Figure 3As shown, the human motion direction determination system based on a fusion network according to an embodiment of the present invention specifically includes:

[0083] Data processing module 30: used to collect human body echo signals through radar, perform fast Fourier transform (FFT) on the human body echo signals to generate range Doppler images, filter out stationary clutter, and then perform short-time Fourier transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images.

[0084] Model training module 32: used to divide the dataset into training set and test set, and use a network that combines convolutional neural module and temporal module to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model;

[0085] Direction determination module 34: Used to determine the direction of human movement based on the collected human echo signals, using the best trained model.

[0086] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operation of each module can be understood by referring to the description of the method embodiments, and will not be repeated here.

[0087] Device Example 1

[0088] This invention provides an electronic device, such as... Figure 4 As shown, it includes: a memory 40, a processor 42, and a computer program stored in the memory 40 and executable on the processor 42. When the computer program is executed by the processor 42, it performs the following method steps:

[0089] S1. Collect human body echo signals by radar, perform Fast Fourier Transform (FFT) on the human body echo signals to generate a range Doppler map, filter out stationary clutter, and then perform Short Time Fourier Transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images.

[0090] S2. Divide the dataset into a training set and a test set, and use a fusion network of convolutional neural modules and temporal modules to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model;

[0091] S3. Based on the trained optimal model, determine the direction of human movement according to the collected human echo signals.

[0092] Device Example 2

[0093] This invention provides a computer-readable storage medium storing an information transmission implementation program. When executed by a processor 42, the program performs the following method steps:

[0094] S1. Collect human body echo signals by radar, perform Fast Fourier Transform (FFT) on the human body echo signals to generate a range Doppler map, filter out stationary clutter, and then perform Short Time Fourier Transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images.

[0095] S2. Divide the dataset into a training set and a test set, and use a fusion network of convolutional neural modules and temporal modules to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model;

[0096] S3. Based on the trained optimal model, determine the direction of human movement according to the collected human echo signals.

[0097] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining the direction of human motion based on a fusion network, characterized in that, include: S1. Collect human body echo signals by radar, perform Fast Fourier Transform (FFT) on the human body echo signals to generate a range Doppler map, filter out stationary clutter, and then perform Short Time Fourier Transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images. S2. Divide the dataset into a training set and a test set, and use a fusion network of convolutional neural modules and temporal modules to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model; The specific method for extracting Doppler features from the training set using a fusion network of convolutional neural modules and temporal modules for model training is as follows: The training dataset is trained using Conv-GRU, a variant of LSTM, and the loss value is calculated during the training process. To minimize the loss value, the gradient of each neuron in the backpropagation algorithm is calculated, the parameters in the fusion network are updated, and the gradient descent algorithm is used to update the parameters. By iterating the above training repeatedly, the recognition accuracy of the network that fuses convolutional neural modules and temporal modules is improved, and a well-trained model is obtained. The convolutional neural module includes a feature extraction part and a feature enhancement part. The feature extraction part consists of two MCblock modules, which include a depthwise separable convolution module, an SE attention module, and a residual module, used to capture multi-scale local micro-Doppler features on the MDS. The feature enhancement part includes MCblock and an improved ViT module with a self-attention mechanism, which is used to further process and optimize the global features in the MDS feature map through position embedding. The temporal module consists of two Conv-GRU layers. The Conv-GRU is used to construct spatial features of the input feature map within a unit through convolution operations, and introduces a gating mechanism to extract temporally correlated features, which includes four computational parts: update gate. Reset the door Activation status and hidden state ; S3. Based on the trained optimal model, determine the direction of human movement according to the collected human echo signals.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: After transmitting a signal using radar, the echo signal reflected back from the human body and the surrounding environment is collected and mixed with the transmitted signal to generate an intermediate frequency signal. The acquired multi-frame echo signals are incoherently accumulated, and the accumulated signals are subjected to Fast Fourier Transform (FFT) to obtain the range Doppler map, where the signal can be represented as... Then on Use a high-pass filter to filter out stationary clutter and generate a two-dimensional matrix. ; The generated two-dimensional matrix is ​​subjected to a short-time Fourier transform (STFT) to obtain a dataset of micro-Doppler images containing micro-Doppler features (MDS). The micro-Doppler features on the micro-Doppler image represent the different velocities of various parts of the body over time, thereby determining the current state of human movement.

3. The method according to claim 1, characterized in that, The dataset is divided into a training set and a test set in an 8:2 ratio.

4. The method according to claim 1, characterized in that, The specific method for using the test set to test the model and obtain the best model is as follows: The trained model was tested using a test set, and the performance of the model fusion network of convolutional neural modules and temporal modules was evaluated based on performance evaluation metrics. The performance evaluation metrics include: model accuracy, precision, recall, specificity, and F1 score.

5. A human motion direction determination system based on a fusion network, characterized in that, include: Data processing module: used to collect human body echo signals through radar, perform Fast Fourier Transform (FFT) on the human body echo signals to generate range Doppler maps, filter out stationary clutter, and then perform Short Time Fourier Transform (STFT) to generate a dataset containing micro-Doppler feature (MDS) images. Model training module: used to divide the dataset into training set and test set, and use a network that combines convolutional neural module and temporal module to extract Doppler features from the training set for model training, and use the test set to test the model to obtain the best model; The model training module is specifically used for: The training dataset is trained using Conv-GRU, a variant of LSTM, and the loss value is calculated during the training process. To minimize the loss value, the gradient of each neuron in the backpropagation algorithm is calculated, the parameters in the fusion network are updated, and the gradient descent algorithm is used to update the parameters. By iterating the above training repeatedly, the recognition accuracy of the network that fuses convolutional neural modules and temporal modules is improved, and a well-trained model is obtained. The convolutional neural module includes a feature extraction part and a feature enhancement part. The feature extraction part consists of two MCblock modules, which include a depthwise separable convolution module, an SE attention module, and a residual module, used to capture multi-scale local micro-Doppler features on the MDS. The feature enhancement part includes MCblock and an improved ViT module with a self-attention mechanism, which is used to further process and optimize the global features in the MDS feature map through position embedding. The temporal module consists of two Conv-GRU layers. The Conv-GRU is used to construct spatial features of the input feature map within a unit through convolution operations, and introduces a gating mechanism to extract temporally correlated features, which includes four computational parts: update gate. Reset the door Activation status and hidden state ; Direction determination module: Used to determine the direction of human movement based on the collected human echo signals, using a well-trained optimal model.

6. An electronic device, characterized in that, include: processor; as well as, A memory configured to store computer-executable instructions, which, when executed, cause the processor to implement the steps of the human motion direction determination method based on a fusion network as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, Used to store computer-executable instructions, which, when executed, implement the steps of the human motion direction determination method based on a fusion network as described in any one of claims 1 to 4.

Images