A method and system for identifying paddle motion trajectory based on temporal memory enhancement network

Abstract

The present invention discloses a method and system for identifying paddle motion trajectory based on a temporal memory enhancement network, the method comprising: obtaining paddle motion trajectory sequence data and a Grammar two-dimensional image of the paddle motion trajectory; introducing a motion trajectory memory feature extraction network and a Grammar image feature extraction network to construct a temporal memory enhancement network model; based on the temporal memory enhancement network model, performing paddle motion trajectory feature recognition processing on the paddle motion trajectory sequence data and the Grammar two-dimensional image of the paddle motion trajectory to obtain a paddle motion trajectory recognition result. By using the present invention, more detailed spatiotemporal feature information of the paddle motion trajectory can be extracted, the standard of the athlete's paddle motion can be improved, and the athlete's training efficiency can be improved. As a method and system for identifying paddle motion trajectory based on a temporal memory enhancement network, the present invention can be widely used in the field of sports motion recognition technology.

Description

Technical Field

[0001] The present invention relates to the technical field of sports action recognition, and in particular to a method and system for recognizing paddle action trajectories based on a temporal memory enhancement network. Background Art

[0002] In existing water sports, speed racing competitions are mostly conducted through observation by coaches, who then provide corresponding guidance. This is limited to naked-eye observation and guidance through experience, but without objective data, it is impossible to conduct effective breakthrough training. The relevant equipment for collecting the motion data of athletes is not lightweight, intelligent, and accurate, and cannot provide intuitive feedback on the training status of athletes. The collected motion data can be regarded as a motion trajectory. For this type of motion trajectory, the relevant methods for identification and modeling through deep learning often only perform feature learning on a single motion trajectory. Due to insufficient feature information, the accuracy of motion standard recognition is reduced. In addition, most existing methods only focus on learning the spatial features or temporal features of the motion trajectory, and lack effective methods for learning the spatiotemporal features of the paddling motion trajectory. Summary of the invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a paddle motion trajectory recognition method and system based on a temporal memory enhanced network, which can extract more detailed spatiotemporal feature information of the paddle motion trajectory, improve the standardization of the athlete's paddle motion and thus improve the athlete's training efficiency.

[0004] The first technical solution adopted by the present invention is: a method for identifying paddle motion trajectories based on a temporal memory enhancement network, comprising the following steps:

[0005] Obtaining paddle motion trajectory sequence data and paddle motion trajectory Gram two-dimensional image;

[0006] The motion trajectory memory feature extraction network and the Gram image feature extraction network are introduced to construct a temporal memory enhancement network model;

[0007] Based on the temporal memory enhanced network model, the paddle motion trajectory sequence data and the paddle motion trajectory Gram 2D ​​image are processed for paddle motion trajectory feature recognition to obtain the paddle motion trajectory recognition result.

[0008] Furthermore, the step of obtaining the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image specifically includes:

[0009] The change data of the propeller force is obtained through the strain gauge and the swing angle data of the propeller is obtained through the ten-axis gyroscope sensor;

[0010] Based on the LoRa wireless signal transceiver module, combined with the terminal data coordinate axis, the time change parameter is introduced to draw the propeller force change-time curve diagram and the propeller swing angle change-time curve diagram;

[0011] Integrate the paddle force change-time curve graph and the paddle swing angle change-time curve graph to construct the paddle motion trajectory sequence data;

[0012] The paddle motion trajectory sequence data is processed into a two-dimensional image and a Gram two-dimensional image of the paddle motion trajectory is obtained.

[0013] Furthermore, the temporal memory enhancement network model specifically includes a motion trajectory memory feature extraction network, a Grammar image feature extraction network, a spatiotemporal association fusion network and a feature joint attention network, wherein the output end of the motion trajectory memory feature extraction network is connected to the first input end of the spatiotemporal association fusion network, the output end of the Grammar image feature extraction network is connected to the second input end of the spatiotemporal association fusion network, and the output end of the spatiotemporal association fusion network is connected to the input end of the feature joint attention network.

[0014] Furthermore, the step of performing paddle motion trajectory feature recognition processing on the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image based on the temporal memory enhanced network model to obtain the paddle motion trajectory recognition result specifically includes:

[0015] Input the paddle motion trajectory sequence data and the paddle motion trajectory Gram 2D ​​image into the temporal memory augmented network model;

[0016] Based on the motion trajectory memory feature extraction network of the temporal memory enhanced network model, temporal feature extraction processing is performed on the paddle motion trajectory sequence data to obtain the paddle motion trajectory temporal feature data;

[0017] Based on the Gram image feature extraction network of the temporal memory enhanced network model, feature extraction processing is performed on the Gram two-dimensional image of the paddle motion trajectory to obtain the Gram two-dimensional feature data of the paddle motion trajectory;

[0018] Based on the spatiotemporal correlation fusion network of the temporal memory enhanced network model, the temporal feature data of the paddle motion trajectory and the Gram 2D ​​feature data of the paddle motion trajectory are fused to obtain a multi-scale paddle motion trajectory feature fusion map.

[0019] Based on the feature joint attention network of the temporal memory enhanced network model, feature recognition processing is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the paddle motion trajectory recognition result.

[0020] Furthermore, the step of performing temporal feature extraction processing on the paddle motion trajectory sequence data by the motion trajectory memory feature extraction network based on the temporal memory enhanced network model to obtain the temporal feature data of the paddle motion trajectory specifically includes:

[0021] Inputting the paddle motion trajectory sequence data into the motion trajectory memory feature extraction network of the temporal memory enhancement network model, the motion trajectory memory feature extraction network comprising a first convolutional layer, a second convolutional layer, a third convolutional layer, a first long short-term memory network, a second long short-term memory network and a third long short-term memory network;

[0022] Based on the first convolutional layer of the motion trajectory memory feature extraction network, feature extraction processing is performed on the paddle motion trajectory sequence data to obtain first paddle motion trajectory sequence feature data;

[0023] Based on the first long short-term memory network of the motion trajectory memory feature extraction network, performing time series feature extraction processing on the paddle motion trajectory sequence data and the first paddle motion trajectory sequence feature data to obtain the first paddle motion trajectory time series feature data;

[0024] The first paddle motion trajectory sequence feature data is combined with the first paddle motion trajectory time series feature data and input into the second convolution layer of the motion trajectory memory feature extraction network for feature extraction processing to obtain the second paddle motion trajectory sequence feature data;

[0025] Based on the second long short-term memory network of the motion trajectory memory feature extraction network, the time series feature data of the first paddle motion trajectory and the sequence feature data of the second paddle motion trajectory are subjected to time series feature extraction processing to obtain the time series feature data of the second paddle motion trajectory;

[0026] The second paddle motion trajectory sequence feature data is combined with the second paddle motion trajectory temporal feature data and input into the third convolutional layer of the motion trajectory memory feature extraction network for feature extraction processing to obtain the third paddle motion trajectory sequence feature data;

[0027] Based on the third long short-term memory network of the motion trajectory memory feature extraction network, time series feature extraction processing is performed on the third paddle motion trajectory sequence feature data and the second paddle motion trajectory time series feature data to obtain the paddle motion trajectory time series feature data.

[0028] Furthermore, the Gram image feature extraction network based on the temporal memory enhanced network model performs feature extraction processing on the Gram two-dimensional image of the paddle motion trajectory to obtain the Gram two-dimensional feature data of the paddle motion trajectory, which specifically includes:

[0029] Inputting the Gram 2D ​​image of the paddle motion trajectory into a Gram image feature extraction network of the temporal memory augmented network model, wherein the Gram image feature extraction network includes a fourth convolution layer, a fifth convolution layer, and a sixth convolution layer;

[0030] Based on the fourth convolutional layer of the Gram image feature extraction network, feature extraction processing is performed on the Gram two-dimensional image of the paddle motion trajectory to obtain a first Gram two-dimensional feature image of the paddle motion trajectory;

[0031] The Gram 2D ​​feature image of the first paddle motion trajectory is combined with the Gram 2D ​​image of the paddle motion trajectory and input into the fifth convolutional layer of the Gram 2D ​​image feature extraction network for feature extraction processing to obtain the Gram 2D ​​feature image of the second paddle motion trajectory;

[0032] The Grammar two-dimensional feature image of the first paddle motion trajectory is combined with the Grammar two-dimensional feature image of the second paddle motion trajectory and input into the sixth convolutional layer of the Grammar image feature extraction network for feature extraction processing to obtain the Grammar two-dimensional feature data of the paddle motion trajectory.

[0033] Furthermore, the spatiotemporal correlation fusion network based on the temporal memory enhanced network model performs feature fusion processing on the paddle motion trajectory temporal feature data and the Gram 2D ​​feature data of the paddle motion trajectory to obtain a multi-scale paddle motion trajectory feature fusion graph, which specifically includes:

[0034] Inputting the paddle motion trajectory temporal feature data and the paddle motion trajectory Gram two-dimensional feature data into a spatiotemporal association fusion network of a temporal memory enhanced network model, wherein the spatiotemporal association fusion network includes a first fully connected network, a second fully connected network, and a fourth long short-term memory network;

[0035] Based on the first fully connected network of the spatiotemporal correlation fusion network, feature mapping is performed on the time series feature data of the paddle motion trajectory to obtain a time series feature vector of the paddle motion trajectory;

[0036] The second fully connected network based on the spatiotemporal correlation fusion network performs feature mapping processing on the Gram 2D ​​feature data of the paddle motion trajectory to obtain the Gram 2D ​​feature vector of the paddle motion trajectory;

[0037] Performing inner product calculation on the time series feature vector of the paddle motion trajectory and the Gram's two-dimensional feature vector of the paddle motion trajectory to obtain associated feature points of the paddle motion trajectory;

[0038] The fourth long short-term memory network based on the spatiotemporal correlation fusion network encodes the temporal feature data of the paddle motion trajectory to obtain the spatiotemporal trajectory memory offset;

[0039] The associated feature points of the paddle motion trajectory and the spatiotemporal trajectory memory offset are weightedly calculated to obtain a multi-scale paddle motion trajectory feature fusion map.

[0040] Furthermore, the feature joint attention network based on the temporal memory enhanced network model performs feature recognition processing on the multi-scale paddle motion trajectory feature fusion map to obtain the paddle motion trajectory recognition result, which specifically includes:

[0041] Inputting the multi-scale paddle motion trajectory feature fusion map into a feature joint attention network of a temporal memory enhanced network model, wherein the feature joint attention network includes a first convolutional attention module, a second convolutional attention module, a third convolutional attention module and a joint attention weight calculation module;

[0042] Based on the first convolutional attention module of the feature joint attention network, a convolutional attention calculation process is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the first multi-scale paddle motion trajectory channel attention feature and the first multi-scale paddle motion trajectory spatial attention feature;

[0043] Based on the second convolutional attention module of the feature joint attention network, the multi-scale paddle motion trajectory feature fusion map is subjected to convolutional attention calculation processing to obtain the second multi-scale paddle motion trajectory channel attention feature and the second multi-scale paddle motion trajectory spatial attention feature;

[0044] Based on the third convolutional attention module of the feature joint attention network, convolutional attention calculation processing is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the third multi-scale paddle motion trajectory channel attention feature and the third multi-scale paddle motion trajectory spatial attention feature;

[0045] The first multi-scale paddle motion trajectory channel attention feature, the first multi-scale paddle motion trajectory spatial attention feature, the second multi-scale paddle motion trajectory channel attention feature, the second multi-scale paddle motion trajectory spatial attention feature, the third multi-scale paddle motion trajectory channel attention feature and the third multi-scale paddle motion trajectory spatial attention feature are spliced ​​to obtain the multi-scale paddle motion trajectory global feature;

[0046] Based on the joint attention weight calculation module of the feature joint attention network, the feature joint calculation is performed on the global features of the multi-scale paddle motion trajectory to obtain the global attention joint features of the multi-scale paddle motion trajectory;

[0047] The multi-scale paddle motion trajectory global features and the multi-scale paddle motion trajectory global attention joint features are combined to output the paddle motion trajectory recognition results.

[0048] The second technical solution adopted by the present invention is: a paddle motion trajectory recognition system based on a temporal memory enhancement network, comprising:

[0049] An acquisition module, used for acquiring paddle motion trajectory sequence data and a paddle motion trajectory Gram two-dimensional image;

[0050] A construction module is used to introduce the motion trajectory memory feature extraction network and the Gram image feature extraction network to construct a temporal memory enhancement network model;

[0051] The recognition module is used to perform paddle motion trajectory feature recognition processing on the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image based on the temporal memory enhanced network model to obtain the paddle motion trajectory recognition result.

[0052] The method and system of the present invention have the following beneficial effects: the present invention can analyze the changes in the oar force of the athlete during the paddling phase, the paddling phase, and the water exit phase by acquiring the paddle motion trajectory sequence data and the Grammar two-dimensional image of the paddle motion trajectory, and effectively solve the periodic paddling problem; the motion trajectory memory feature extraction network is used to extract the spatiotemporal feature information of the paddling motion trajectory; the Grammar image feature extraction network is used to extract feature information of different scales of the Grammar two-dimensional image; the motion trajectory memory feature extraction network and the Grammar image feature extraction network are further introduced to construct a temporal memory enhanced network model; finally, based on the temporal memory enhanced network model, the paddle motion trajectory sequence data and the Grammar two-dimensional image of the paddle motion trajectory are subjected to paddle motion trajectory feature recognition processing, so that more detailed spatiotemporal feature information of the paddle motion trajectory can be extracted, the standardization of the athlete's paddle motion can be improved, and thus the athlete's training efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of steps of a method for identifying paddle motion trajectory based on a temporal memory enhancement network of the present invention;

[0054] Figure 2 It is a structural block diagram of a paddle motion trajectory recognition system based on a temporal memory enhancement network of the present invention;

[0055] Figure 3 It is a structural diagram of a temporal memory enhancement network model provided by a specific embodiment of the present invention;

[0056] Figure 4 is a schematic diagram comparing standard paddling motion data and non-standard paddling motion data provided by a specific embodiment of the present invention;

[0057] Figure 5 It is a structural diagram of an action trajectory memory feature extraction network provided by a specific embodiment of the present invention;

[0058] Figure 6 is a schematic diagram of the structure of a Gram image feature extraction network provided by a specific embodiment of the present invention;

[0059] Figure 7 It is a schematic diagram of the structure of a spatiotemporal correlation fusion network provided by a specific embodiment of the present invention;

[0060] Figure 8 is a schematic diagram of the structure of a feature joint attention network provided by a specific embodiment of the present invention;

[0061] Fig. 9 It is a voltage value-time curve diagram of the paddling motion trajectory provided by a specific embodiment of the present invention.

[0062] Figure numerals: 1, first convolutional layer; 2, second convolutional layer; 3, third convolutional layer; 4, first long short-term memory network; 5, second long short-term memory network; 6, third long short-term memory network; 7, fourth convolutional layer; 8, fifth convolutional layer; 9, sixth convolutional layer; 10, fourth long short-term memory network; 11, first fully connected network; 12, second fully connected network; 13, first convolutional attention module; 14, second convolutional attention module; 15, third convolutional attention module; 16, joint attention weight calculation module; 17 represents the data curve collected by the sensor at the bottom of the paddle; 18 represents the data curve collected by the sensor at the top of the paddle; 19 represents that in a complete round of paddling action, the rising part of the curve represents the process of the paddle entering the water, and the falling part represents the process of the paddle leaving the water; 20 represents the smooth straight line represents the process of the paddle not entering the water. DETAILED DESCRIPTION

[0063] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0064] Reference Figure 1 The present invention provides a method for identifying paddle motion trajectories based on a temporal memory enhancement network, the method comprising the following steps:

[0065] S100, obtaining paddle motion trajectory sequence data and a Gram two-dimensional image of the paddle motion trajectory;

[0066] Specifically, the paddle force change data is obtained through the strain gauge, and the paddle swing angle data is obtained through the ten-axis gyroscope sensor; based on the LoRa wireless signal transceiver module and combined with the terminal data coordinate axis, the time change parameter is introduced to draw the paddle force change-time curve diagram and the paddle swing angle change-time curve diagram; the paddle force change-time curve diagram and the paddle swing angle change-time curve diagram are integrated to construct the paddle motion trajectory sequence data; the paddle motion trajectory sequence data is visualized by two-dimensional image processing to obtain the paddle motion trajectory Gram two-dimensional image.

[0067] In this embodiment, eight upper and lower high-precision strain gauges and a ten-axis gyroscope sensor are installed on each paddle, and then these modules are connected to the LoRa wireless signal transceiver module, and the wireless signal transceiver module is connected to the computer.

[0068] It should be noted that LoRa is a long-distance radio technology. Its biggest feature is that it can transmit farther than other wireless methods under the same power consumption conditions, achieving the unity of low power consumption and long distance. Under the same power consumption, it can extend the communication distance by 3-5 times compared with traditional wireless radio frequency. The collected data is sent to the computer through LoRa.

[0069] Furthermore, high-precision strain gauges and ten-axis gyroscope sensor data are obtained multiple times at the same time interval, and the strain gauges are used to reflect the change data of the paddle force. The paddle's swing angle data is obtained through the ten-axis gyroscope sensor, and the strain gauge data is imported into the two-dimensional coordinate axis, with time as the horizontal axis and the strain gauge data as the vertical axis. The various points are connected together through a smooth curve to obtain a paddle force change-time curve graph.

[0070] like Fig. 9 As shown, Fig. 9 17 in the figure represents the data curve collected by the sensor at the bottom of the paddle. Fig. 9 18 in the figure represents the data curve collected by the sensor on the upper part of the paddle. Fig. 9 The 19 in the figure means that in a complete rowing action, the rising part of the curve represents the process of the paddle entering the water, and the falling part represents the process of the paddle leaving the water. Fig. 9 The 20 in the figure indicates that the smooth straight line represents the process in which the paddle does not enter the water. When the strain gauge sensor detects a sudden change in the conversion value for the first time, it represents the first time the paddle enters the water. Then, when the conversion value suddenly changes for the second time, it represents the first time the paddle leaves the water. When the conversion value suddenly changes for the third time, it represents the second time the paddle enters the water, and so on. The time measured every other sudden change in force is the complete time of the paddling action. According to the time point when the strain gauge sensor detects the sudden change in the paddle force, the time when the paddle enters the water and the time when it exits the water can be obtained for each athlete each time they paddle. By comparing the time when each athlete enters the water and the time when it exits the water, the paddling rhythm of each athlete can be adjusted to ensure the uniformity of paddling and improve the paddling effect. The x, y, and z axis data of the paddle surface entering the water are collected by the ten-axis gyroscope. When it exceeds or is lower than the preset value setting range, it means that the paddling action is not standardized.

[0071] Finally, based on the paddle equipped with a ten-axis gyroscope sensor, professional dragon boat athletes and amateur athletes were arranged to use the paddle for rowing tests, and the paddling motion trajectory curves of the two athletes were collected at the background data collection end. By drawing the trajectory curve, it can be seen that the paddling force application method of professional dragon boat athletes is significantly different from that of amateur athletes. In addition to the difference in the shape of the motion trajectory curve, in order to enable the network model to further explore the difference in the time characteristics between the two trajectories, the embodiment of the present invention uses the Gramian Angular Field to visualize the motion trajectory time characteristics of the original collected motion trajectory curve in a two-dimensional image, providing rich temporal feature information for the temporal memory enhanced network model. In terms of data labels, the embodiment of the present invention marks the paddling motion trajectory of professional dragon boat athletes as a standard paddling motion trajectory, and the paddling motion trajectory of amateur athletes as a non-standard paddling motion trajectory. Specific standard and non-standard paddling motion trajectory curves and corresponding Gramian Angular Field example diagrams are shown in the figure below. Figure 4 shown.

[0072] S200, introduce the action trajectory memory feature extraction network and the Gram image feature extraction network to build a temporal memory enhancement network model;

[0073] Specifically, the temporal memory enhancement network model specifically includes a motion trajectory memory feature extraction network, a Grammar image feature extraction network, a spatiotemporal association fusion network and a feature joint attention network, wherein the output end of the motion trajectory memory feature extraction network is connected to the first input end of the spatiotemporal association fusion network, the output end of the Grammar image feature extraction network is connected to the second input end of the spatiotemporal association fusion network, and the output end of the spatiotemporal association fusion network is connected to the input end of the feature joint attention network.

[0074] In this embodiment, if Figure 3 As shown in the figure, the motion trajectory memory feature extraction network and the Gram image feature extraction network are designed to extract the spatiotemporal features of the paddling motion trajectory and the feature information of the Gram 2D ​​image corresponding to the paddling motion trajectory, respectively. For the above two types of feature information, a spatiotemporal correlation fusion module is designed to achieve the global spatiotemporal correlation fusion of motion trajectory features and image features. A feature joint attention module is proposed to fuse feature information of different scales and enhance the model's attention and correlation to the fused features of each scale.

[0075] S300, based on the temporal memory enhanced network model, performing paddle motion trajectory feature recognition processing on the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image to obtain a paddle motion trajectory recognition result.

[0076] S310, inputting the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image into the temporal memory enhancement network model;

[0077] S320, performing temporal feature extraction processing on the paddle motion trajectory sequence data based on the motion trajectory memory feature extraction network of the temporal memory enhanced network model to obtain the paddle motion trajectory temporal feature data;

[0078] Specifically, the paddle motion trajectory sequence data is input into the motion trajectory memory feature extraction network of the temporal memory enhanced network model, and the motion trajectory memory feature extraction network includes a first convolutional layer 1, a second convolutional layer 2, a third convolutional layer 3, a first long short-term memory network 4, a second long short-term memory network 5 and a third long short-term memory network 6; based on the first convolutional layer of the motion trajectory memory feature extraction network, feature extraction processing is performed on the paddle motion trajectory sequence data to obtain first paddle motion trajectory sequence feature data; based on the first long short-term memory network of the motion trajectory memory feature extraction network, temporal feature extraction processing is performed on the paddle motion trajectory sequence data and the first paddle motion trajectory sequence feature data to obtain first paddle motion trajectory temporal feature data; the first paddle motion trajectory sequence feature data is combined with the first paddle motion trajectory temporal feature data and input into the motion trajectory memory feature extraction network. The second convolutional layer of the trajectory memory feature extraction network is used for feature extraction processing to obtain second paddle motion trajectory sequence feature data; based on the second long short-term memory network of the motion trajectory memory feature extraction network, the first paddle motion trajectory temporal feature data and the second paddle motion trajectory sequence feature data are subjected to temporal feature extraction processing to obtain second paddle motion trajectory temporal feature data; the second paddle motion trajectory sequence feature data and the second paddle motion trajectory temporal feature data are combined and input into the third convolutional layer of the motion trajectory memory feature extraction network for feature extraction processing to obtain third paddle motion trajectory sequence feature data; based on the third long short-term memory network of the motion trajectory memory feature extraction network, the third paddle motion trajectory sequence feature data and the second paddle motion trajectory temporal feature data are subjected to temporal feature extraction processing to obtain paddle motion trajectory temporal feature data.

[0079] In this embodiment, if Figure 5 As shown, based on deep learning action sequence classification, the embodiment of the present invention is an action trajectory sequence and its Gram 2D ​​image, and an action trajectory memory feature extraction network and a Gram 2D ​​image feature extraction network are designed respectively. Among them, the action trajectory memory feature extraction network is used to extract the spatiotemporal feature information of the paddling motion trajectory, and the Gram 2D ​​image feature extraction network is used to extract the different scale feature information of the Gram 2D ​​image.

[0080] Taking into account that the motion trajectory collected by the stress sensor on the paddle during the paddling process has a certain periodicity, and the force and amplitude of the paddling can be expressed in the form of a curve. Therefore, when extracting features from the paddling motion trajectory, it is necessary to simultaneously extract the spatiotemporal feature information contained in the trajectory. The motion trajectory memory feature extraction network designed in the embodiment of the present invention uses a convolutional network to extract the morphological features of the motion trajectory. At the same time, through the multiple input multiple output (MIMO) mode of the long short-term memory network LSTM, the potential temporal features of the temporal features of the motion trajectory and the trajectory morphological features can be fully extracted. In order to avoid the disappearance of feature gradients, a residual structure is used to connect three CNN+LSTM feature extraction parts of different scales.

[0081] S330, based on the Gram image feature extraction network of the temporal memory enhanced network model, performing feature extraction processing on the Gram two-dimensional image of the paddle motion trajectory to obtain Gram two-dimensional feature data of the paddle motion trajectory;

[0082] Specifically, the Grammar two-dimensional image of the paddle motion trajectory is input into the Grammar image feature extraction network of the temporal memory enhanced network model, and the Grammar image feature extraction network includes a fourth convolutional layer 7, a fifth convolutional layer 8 and a sixth convolutional layer 9; based on the fourth convolutional layer of the Grammar image feature extraction network, feature extraction processing is performed on the Grammar two-dimensional image of the paddle motion trajectory to obtain a first Grammar two-dimensional feature image of the paddle motion trajectory; the first Grammar two-dimensional feature image of the paddle motion trajectory is combined with the Grammar two-dimensional image of the paddle motion trajectory and input into the fifth convolutional layer of the Grammar image feature extraction network for feature extraction processing to obtain a second Grammar two-dimensional feature image of the paddle motion trajectory; the first Grammar two-dimensional feature image of the paddle motion trajectory is combined with the second Grammar two-dimensional feature image of the paddle motion trajectory and input into the sixth convolutional layer of the Grammar image feature extraction network for feature extraction processing to obtain Grammar two-dimensional feature data of the paddle motion trajectory.

[0083] In this embodiment, if Figure 6 As shown in the figure, in the design of the Gram image feature extraction network structure, the convolution kernel scale is consistent with the convolution kernel scale of each convolution layer in the motion trajectory memory feature extraction network. After the paddling trajectory curve is transformed into a Gram 2D ​​image, the temporal information of the trajectory curve has been mapped into the distribution information of the pixel value change of the image, so the convolution network layer is required to extract feature information of different scales. Similarly, the residual structure is used to connect each convolution layer, which helps to avoid the disappearance of gradient features while improving the feature extraction capability of image input.

[0084] S340, based on the spatiotemporal correlation fusion network of the temporal memory enhanced network model, performing feature fusion processing on the paddle motion trajectory temporal feature data and the Gram 2D ​​feature data of the paddle motion trajectory to obtain a multi-scale paddle motion trajectory feature fusion graph;

[0085] Specifically, the temporal feature data of the paddle motion trajectory and the Gram's two-dimensional feature data of the paddle motion trajectory are input into the spatiotemporal association fusion network of the temporal memory enhanced network model, wherein the spatiotemporal association fusion network includes a first fully connected network 11, a second fully connected network 12 and a fourth long short-term memory network 10; based on the first fully connected network of the spatiotemporal association fusion network, feature mapping processing is performed on the temporal feature data of the paddle motion trajectory to obtain a temporal feature vector of the paddle motion trajectory; based on the second fully connected network of the spatiotemporal association fusion network, feature mapping processing is performed on the Gram's two-dimensional feature data of the paddle motion trajectory to obtain a Gram's two-dimensional feature vector of the paddle motion trajectory; inner product calculation processing is performed on the temporal feature vector of the paddle motion trajectory and the Gram's two-dimensional feature vector of the paddle motion trajectory to obtain associated feature points of the paddle motion trajectory; based on the fourth long short-term memory network of the spatiotemporal association fusion network, encoding processing is performed on the temporal feature data of the paddle motion trajectory to obtain a spatiotemporal trajectory memory offset; weighted calculation processing is performed on the associated feature points of the paddle motion trajectory and the spatiotemporal trajectory memory offset to obtain a multi-scale paddle motion trajectory feature fusion graph.

[0086] In this embodiment, if Figure 7 As shown, for the feature graphs output by each layer of the action trajectory memory feature extraction network and the Gram image feature extraction network, in order to further extract the spatiotemporal correlation features between the trajectory features and the image features, this patent proposes a spatiotemporal correlation fusion module. First, a fully connected network FC is applied to map the trajectory features and the image features to a vector space with the same dimension. i With b i Respectively represent the spatiotemporal feature points of the trajectory and the spatiotemporal feature points of the image after FC dimension reduction. In order to fully integrate the trajectory features and image features, a i With b i The inner product operation is performed, and the result is the associated feature information of the corresponding trajectory spatiotemporal feature points and the image spatiotemporal feature points. In addition, when the feature points are associated, the original temporal feature information of the trajectory features will be lost due to the inner product calculation. In order to compensate for the loss of temporal feature information, the LSTM layer is used to encode the trajectory features, thereby calculating the spatiotemporal trajectory memory offset y i Finally, by adding the corresponding trajectory memory offset to each associated feature point, the global spatiotemporal association fusion of action trajectory features and image features can be achieved.

[0087] S350, based on the feature joint attention network of the temporal memory enhanced network model, perform feature recognition processing on the multi-scale paddle motion trajectory feature fusion map to obtain the paddle motion trajectory recognition result.

[0088] Specifically, the multi-scale paddle motion trajectory feature fusion map is input into the feature joint attention network of the temporal memory enhanced network model, and the feature joint attention network includes a first convolutional attention module 13, a second convolutional attention module 14, a third convolutional attention module 15 and a joint attention weight calculation module 16; based on the first convolutional attention module of the feature joint attention network, the multi-scale paddle motion trajectory feature fusion map is subjected to convolutional attention calculation processing to obtain the first multi-scale paddle motion trajectory channel attention feature and the first multi-scale paddle motion trajectory spatial attention feature; based on the second convolutional attention module of the feature joint attention network, the multi-scale paddle motion trajectory feature fusion map is subjected to convolutional attention calculation processing to obtain the second multi-scale paddle motion trajectory channel attention feature and the second multi-scale paddle motion trajectory spatial attention feature; based on the third convolutional attention module of the feature joint attention network, the multi-scale paddle motion trajectory feature fusion map is subjected to convolutional attention calculation processing to obtain the second multi-scale paddle motion trajectory channel attention feature and the second multi-scale paddle motion trajectory spatial attention feature; The convolution attention calculation is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the third multi-scale paddle motion trajectory channel attention feature and the third multi-scale paddle motion trajectory spatial attention feature; the first multi-scale paddle motion trajectory channel attention feature, the first multi-scale paddle motion trajectory spatial attention feature, the second multi-scale paddle motion trajectory channel attention feature, the second multi-scale paddle motion trajectory spatial attention feature, the third multi-scale paddle motion trajectory channel attention feature and the third multi-scale paddle motion trajectory spatial attention feature are spliced ​​to obtain the multi-scale paddle motion trajectory global feature; based on the joint attention weight calculation module of the feature joint attention network, the feature joint calculation is performed on the global feature of the multi-scale paddle motion trajectory to obtain the global attention joint feature of the multi-scale paddle motion trajectory; the multi-scale paddle motion trajectory global feature and the multi-scale paddle motion trajectory global attention joint feature are combined to output the paddle motion trajectory recognition result.

[0089] In this embodiment, after the trajectory features and the Gram image features pass through the spatiotemporal correlation fusion module, three fusion feature maps of different scales can be obtained. Considering that the fusion features contain abstract and rich spatiotemporal feature information, this patent proposes a feature integration attention module (FIAM) for fusion features of different scales, which can pay attention to the fusion feature map information of each scale while enhancing the correlation between features. The specific implementation method of the feature integration attention mechanism is:

[0090] First, the convolutional block attention module (CBAM) is calculated for the fused feature map. CBAM is composed of a channel attention module CAM and a spatial attention module SAM in series. For CAM, it can automatically learn the importance of each channel and highlight the feature channels that are important to the current task, which is conducive to improving the generalization ability and robustness of the model; in addition, the channel attention mechanism can help the model suppress unimportant feature channels, reduce the interference of redundant information, and improve the efficiency and performance of the model. The specific calculation formula of CAM is as follows:

[0091]

[0092] ChannelAttention=Sigmoid(fc2(fc1(avg nc )))

[0093] Let the input feature map X i The scale is N×C×H×W, where N is the batch size, C is the number of channels, H is the height, and W is the width. First, the input feature map is globally averaged pooled to obtain the average feature description in the channel dimension. Then, two fully connected layers fc1 and fc2 are used to learn the channel attention weights. Finally, the Sigmoid activation function is used to ensure that the weights are in the range [0,1].

[0094] For SAM, it can focus on the key area information of the feature space, thereby helping the model to better understand the spatial structure and geometric relationship of the image, and improve the model's spatial perception and image understanding capabilities. The specific calculation formula of SAM is as follows:

[0095] Conv = Conv(X)

[0096] SpatialAttention=Sigmoid(conv)

[0097] After applying the channel attention mechanism and spatial attention mechanism to the feature map of each scale, the channel attention weighted feature map Y can be obtained i，channel And the spatial attention weighted feature map Y i，spatial .

[0098] Y i,channel =X i ×ChannelAttention(X i )

[0099] Y i，spatial =X i×SpatialAttention(X i )

[0100] Then, the channel attention weighted feature map and the spatial attention weighted feature map are concatenated, and the concatenated feature map is globally averaged and pooled to obtain the global feature description avg i .

[0101]

[0102] Describe the global features avg i Apply a fully connected layer to generate the joint attention weights W i .

[0103] W i =FC(avg i )

[0104] Finally, the feature joint attention output feature map is:

[0105]

[0106] Among them, the structure diagram of the feature joint attention module FIAM is as follows Figure 8 shown.

[0107] In summary, first of all, in terms of feature extraction of motion trajectory and Gram image, the model first receives the motion trajectory sequence and the corresponding Gram image as input. The motion trajectory memory feature extraction network uses convolutional network (CNN) and long short-term memory network (LSTM) to extract the spatiotemporal feature information of the motion trajectory. At the same time, the Gram image feature extraction network also uses CNN to extract the multi-scale feature information of the image. The two networks are connected by a residual structure to avoid the vanishing of feature gradients and improve the efficiency of feature extraction. For the extracted trajectory features and image features, the spatiotemporal correlation fusion module is used to map the motion trajectory features and Gram image features into a vector space with consistent dimensions, and then the inner product operation is used to calculate the correlation information between the trajectory feature points and the image feature points. In addition, the trajectory features are encoded through the LSTM layer, and the spatiotemporal trajectory memory offset is calculated to make up for the lack of temporal feature information. This step realizes the global spatiotemporal correlation fusion of motion trajectory features and image features. For the fusion features of three different scales, a feature joint attention module (FIAM) is used to comprehensively utilize channel attention and spatial attention to model the global correlation between feature maps of different scales, which helps the model to more accurately capture the key features and structural information of the image, improve the model's attention and perception of feature maps, and thus enhance the model's attention to the fusion feature information of different scales and the correlation between features. Finally, the model will perform global average pooling on the fusion feature map processed by the feature joint attention module, and generate joint attention weights through the fully connected layer to obtain the final output mapping label, that is, the classification of the standardization of paddling motion.

[0108] Reference Figure 2 , a paddle motion trajectory recognition system based on a temporal memory enhancement network, comprising:

[0109] An acquisition module 201 is used to acquire paddle motion trajectory sequence data and a paddle motion trajectory Gram two-dimensional image;

[0110] A construction module 202 is used to introduce an action trajectory memory feature extraction network and a Gram image feature extraction network to construct a temporal memory enhancement network model;

[0111] The recognition module 203 is used to perform paddle motion trajectory feature recognition processing on the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image based on the temporal memory enhanced network model to obtain a paddle motion trajectory recognition result.

[0112] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0113] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for identifying paddle motion trajectories based on a temporal memory enhancement network, characterized in that: The following steps are involved: Obtaining paddle motion trajectory sequence data and paddle motion trajectory Gram two-dimensional image; The motion trajectory memory feature extraction network and the Gram image feature extraction network are introduced to construct a temporal memory enhancement network model; The temporal memory enhancement network model specifically includes an action trajectory memory feature extraction network, a Grammar image feature extraction network, a spatiotemporal association fusion network and a feature joint attention network, wherein the output end of the action trajectory memory feature extraction network is connected to the first input end of the spatiotemporal association fusion network, the output end of the Grammar image feature extraction network is connected to the second input end of the spatiotemporal association fusion network, and the output end of the spatiotemporal association fusion network is connected to the input end of the feature joint attention network; Input the paddle motion trajectory sequence data and the paddle motion trajectory Gram 2D ​​image into the temporal memory augmented network model; Inputting the paddle motion trajectory sequence data into the motion trajectory memory feature extraction network of the temporal memory enhancement network model, the motion trajectory memory feature extraction network comprising a first convolutional layer, a second convolutional layer, a third convolutional layer, a first long short-term memory network, a second long short-term memory network and a third long short-term memory network; Based on the first convolutional layer of the motion trajectory memory feature extraction network, feature extraction processing is performed on the paddle motion trajectory sequence data to obtain first paddle motion trajectory sequence feature data; Based on the first long short-term memory network of the motion trajectory memory feature extraction network, performing time series feature extraction processing on the paddle motion trajectory sequence data and the first paddle motion trajectory sequence feature data to obtain the first paddle motion trajectory time series feature data; The first paddle motion trajectory sequence feature data is combined with the first paddle motion trajectory time series feature data and input into the second convolution layer of the motion trajectory memory feature extraction network for feature extraction processing to obtain the second paddle motion trajectory sequence feature data; Based on the second long short-term memory network of the motion trajectory memory feature extraction network, the time series feature data of the first paddle motion trajectory and the sequence feature data of the second paddle motion trajectory are subjected to time series feature extraction processing to obtain the time series feature data of the second paddle motion trajectory; The second paddle motion trajectory sequence feature data is combined with the second paddle motion trajectory temporal feature data and input into the third convolutional layer of the motion trajectory memory feature extraction network for feature extraction processing to obtain the third paddle motion trajectory sequence feature data; Based on the third long short-term memory network of the motion trajectory memory feature extraction network, performing time series feature extraction processing on the sequence feature data of the third paddle motion trajectory and the time series feature data of the second paddle motion trajectory to obtain the time series feature data of the paddle motion trajectory; Inputting the Gram 2D ​​image of the paddle motion trajectory into a Gram image feature extraction network of the temporal memory augmented network model, wherein the Gram image feature extraction network includes a fourth convolution layer, a fifth convolution layer, and a sixth convolution layer; Based on the fourth convolutional layer of the Gram image feature extraction network, feature extraction processing is performed on the Gram two-dimensional image of the paddle motion trajectory to obtain a first Gram two-dimensional feature image of the paddle motion trajectory; The Gram 2D ​​feature image of the first paddle motion trajectory is combined with the Gram 2D ​​image of the paddle motion trajectory and input into the fifth convolutional layer of the Gram 2D ​​image feature extraction network for feature extraction processing to obtain the Gram 2D ​​feature image of the second paddle motion trajectory; The Gram 2D ​​feature image of the first paddle motion trajectory is combined with the Gram 2D ​​feature image of the second paddle motion trajectory and inputted into the sixth convolutional layer of the Gram image feature extraction network for feature extraction processing to obtain Gram 2D ​​feature data of the paddle motion trajectory; Inputting the paddle motion trajectory temporal feature data and the paddle motion trajectory Gram two-dimensional feature data into a spatiotemporal association fusion network of a temporal memory enhanced network model, wherein the spatiotemporal association fusion network includes a first fully connected network, a second fully connected network, and a fourth long short-term memory network; Based on the first fully connected network of the spatiotemporal correlation fusion network, feature mapping is performed on the time series feature data of the paddle motion trajectory to obtain a time series feature vector of the paddle motion trajectory; The second fully connected network based on the spatiotemporal correlation fusion network performs feature mapping processing on the Gram 2D ​​feature data of the paddle motion trajectory to obtain the Gram 2D ​​feature vector of the paddle motion trajectory; Performing inner product calculation on the time series feature vector of the paddle motion trajectory and the Gram's two-dimensional feature vector of the paddle motion trajectory to obtain associated feature points of the paddle motion trajectory; The fourth long short-term memory network based on the spatiotemporal correlation fusion network encodes the temporal feature data of the paddle motion trajectory to obtain the spatiotemporal trajectory memory offset; The paddle motion trajectory-associated feature points and the spatiotemporal trajectory memory offset are weightedly calculated to obtain a multi-scale paddle motion trajectory feature fusion map; Based on the feature joint attention network of the temporal memory enhanced network model, feature recognition processing is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the paddle motion trajectory recognition result.

2. According to claim 1, a method for identifying paddle motion trajectories based on a temporal memory enhancement network is characterized in that: The step of obtaining the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image specifically includes: The change data of the propeller force is obtained through the strain gauge and the swing angle data of the propeller is obtained through the ten-axis gyroscope sensor; Based on the LoRa wireless signal transceiver module, combined with the terminal data coordinate axis, the time change parameter is introduced to draw the propeller force change-time curve diagram and the propeller swing angle change-time curve diagram; Integrate the paddle force change-time curve graph and the paddle swing angle change-time curve graph to construct the paddle motion trajectory sequence data; The paddle motion trajectory sequence data is processed into a two-dimensional image and a Gram two-dimensional image of the paddle motion trajectory is obtained.

3. According to claim 2, a method for identifying paddle motion trajectories based on a temporal memory enhancement network is characterized in that: The step of performing feature recognition processing on the multi-scale paddle motion trajectory feature fusion graph by using the feature joint attention network based on the temporal memory enhanced network model to obtain the paddle motion trajectory recognition result specifically includes: Inputting the multi-scale paddle motion trajectory feature fusion map into a feature joint attention network of a temporal memory enhanced network model, wherein the feature joint attention network includes a first convolutional attention module, a second convolutional attention module, a third convolutional attention module and a joint attention weight calculation module; Based on the first convolutional attention module of the feature joint attention network, a convolutional attention calculation process is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the first multi-scale paddle motion trajectory channel attention feature and the first multi-scale paddle motion trajectory spatial attention feature; Based on the second convolutional attention module of the feature joint attention network, the multi-scale paddle motion trajectory feature fusion map is subjected to convolutional attention calculation processing to obtain the second multi-scale paddle motion trajectory channel attention feature and the second multi-scale paddle motion trajectory spatial attention feature; Based on the third convolutional attention module of the feature joint attention network, convolutional attention calculation processing is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the third multi-scale paddle motion trajectory channel attention feature and the third multi-scale paddle motion trajectory spatial attention feature; The first multi-scale paddle motion trajectory channel attention feature, the first multi-scale paddle motion trajectory spatial attention feature, the second multi-scale paddle motion trajectory channel attention feature, the second multi-scale paddle motion trajectory spatial attention feature, the third multi-scale paddle motion trajectory channel attention feature and the third multi-scale paddle motion trajectory spatial attention feature are spliced ​​to obtain the multi-scale paddle motion trajectory global feature; Based on the joint attention weight calculation module of the feature joint attention network, the feature joint calculation is performed on the global features of the multi-scale paddle motion trajectory to obtain the global attention joint features of the multi-scale paddle motion trajectory; The multi-scale paddle motion trajectory global features and the multi-scale paddle motion trajectory global attention joint features are combined to output the paddle motion trajectory recognition results.

4. A paddle motion trajectory recognition system based on a temporal memory enhancement network, characterized in that: Includes the following modules: An acquisition module, used for acquiring paddle motion trajectory sequence data and a paddle motion trajectory Gram two-dimensional image; A construction module is used to introduce the motion trajectory memory feature extraction network and the Gram image feature extraction network to construct a temporal memory enhancement network model; The temporal memory enhancement network model specifically includes an action trajectory memory feature extraction network, a Grammar image feature extraction network, a spatiotemporal association fusion network and a feature joint attention network, wherein the output end of the action trajectory memory feature extraction network is connected to the first input end of the spatiotemporal association fusion network, the output end of the Grammar image feature extraction network is connected to the second input end of the spatiotemporal association fusion network, and the output end of the spatiotemporal association fusion network is connected to the input end of the feature joint attention network; A recognition module, used for inputting the paddle motion trajectory sequence data and the paddle motion trajectory Gram two-dimensional image into the temporal memory enhancement network model; Inputting the paddle motion trajectory sequence data into the motion trajectory memory feature extraction network of the temporal memory enhancement network model, the motion trajectory memory feature extraction network comprising a first convolutional layer, a second convolutional layer, a third convolutional layer, a first long short-term memory network, a second long short-term memory network and a third long short-term memory network; Based on the first convolutional layer of the motion trajectory memory feature extraction network, feature extraction processing is performed on the paddle motion trajectory sequence data to obtain first paddle motion trajectory sequence feature data; Based on the first long short-term memory network of the motion trajectory memory feature extraction network, performing time series feature extraction processing on the paddle motion trajectory sequence data and the first paddle motion trajectory sequence feature data to obtain the first paddle motion trajectory time series feature data; The first paddle motion trajectory sequence feature data is combined with the first paddle motion trajectory time series feature data and input into the second convolution layer of the motion trajectory memory feature extraction network for feature extraction processing to obtain the second paddle motion trajectory sequence feature data; Based on the second long short-term memory network of the motion trajectory memory feature extraction network, the time series feature data of the first paddle motion trajectory and the sequence feature data of the second paddle motion trajectory are subjected to time series feature extraction processing to obtain the time series feature data of the second paddle motion trajectory; The second paddle motion trajectory sequence feature data is combined with the second paddle motion trajectory temporal feature data and input into the third convolutional layer of the motion trajectory memory feature extraction network for feature extraction processing to obtain the third paddle motion trajectory sequence feature data; Based on the third long short-term memory network of the motion trajectory memory feature extraction network, performing time series feature extraction processing on the sequence feature data of the third paddle motion trajectory and the time series feature data of the second paddle motion trajectory to obtain the time series feature data of the paddle motion trajectory; Inputting the Gram 2D ​​image of the paddle motion trajectory into a Gram image feature extraction network of the temporal memory augmented network model, wherein the Gram image feature extraction network includes a fourth convolution layer, a fifth convolution layer, and a sixth convolution layer; Based on the fourth convolutional layer of the Gram image feature extraction network, feature extraction processing is performed on the Gram two-dimensional image of the paddle motion trajectory to obtain a first Gram two-dimensional feature image of the paddle motion trajectory; The Gram 2D ​​feature image of the first paddle motion trajectory is combined with the Gram 2D ​​image of the paddle motion trajectory and input into the fifth convolutional layer of the Gram 2D ​​image feature extraction network for feature extraction processing to obtain the Gram 2D ​​feature image of the second paddle motion trajectory; The Gram 2D ​​feature image of the first paddle motion trajectory is combined with the Gram 2D ​​feature image of the second paddle motion trajectory and inputted into the sixth convolutional layer of the Gram image feature extraction network for feature extraction processing to obtain Gram 2D ​​feature data of the paddle motion trajectory; Inputting the paddle motion trajectory temporal feature data and the paddle motion trajectory Gram two-dimensional feature data into a spatiotemporal association fusion network of a temporal memory enhanced network model, wherein the spatiotemporal association fusion network includes a first fully connected network, a second fully connected network, and a fourth long short-term memory network; Based on the first fully connected network of the spatiotemporal correlation fusion network, feature mapping is performed on the time series feature data of the paddle motion trajectory to obtain a time series feature vector of the paddle motion trajectory; The second fully connected network based on the spatiotemporal correlation fusion network performs feature mapping processing on the Gram 2D ​​feature data of the paddle motion trajectory to obtain the Gram 2D ​​feature vector of the paddle motion trajectory; Performing inner product calculation on the time series feature vector of the paddle motion trajectory and the Gram's two-dimensional feature vector of the paddle motion trajectory to obtain associated feature points of the paddle motion trajectory; The fourth long short-term memory network based on the spatiotemporal correlation fusion network encodes the temporal feature data of the paddle motion trajectory to obtain the spatiotemporal trajectory memory offset; The paddle motion trajectory-associated feature points and the spatiotemporal trajectory memory offset are weightedly calculated to obtain a multi-scale paddle motion trajectory feature fusion map; Based on the feature joint attention network of the temporal memory enhanced network model, feature recognition processing is performed on the multi-scale paddle motion trajectory feature fusion map to obtain the paddle motion trajectory recognition result.

Images