Multi-person rope skipping analysis method based on video-audio multi-mode deep learning

Abstract

The invention discloses a multi-person rope skipping analysis method based on video-audio multi-mode deep learning, and the method comprises the steps: obtaining an audio-video file in a rope skipping process, separating a video from an audio, carrying out the portrait detection and extraction of a video image signal, tracking a target portrait, extracting the coordinates of skeleton feature points of the target portrait, and carrying out the preprocessing, the method comprises the following steps: acquiring a single-channel audio signal, slicing and intercepting the single-channel audio signal, carrying out time-frequency transformation to obtain a frequency spectrum signal, preprocessing the frequency spectrum signal, and fusing the preprocessed video signal and audio signal to obtain a video-audio fusion signal; and enabling the video-audio fusion signal to pass through a bidirectional long-short time memory cyclic convolutional neural network and a cascaded full-connection network to obtain an output signal flow, converting the output signal flow into a square wave signal, filtering the square wave signal, and carrying out statistical analysis on a rising edge or a falling edge. Interference of non-rope skipping testers can be effectively filtered, and more accurate rope skipping statistical analysis is achieved.

Description

technical field [0001] The present invention relates to the technical field of deep learning, and more specifically relates to a multi-person rope skipping analysis method based on video-audio multimodal deep learning. Background technique [0002] Rope skipping has a long history and is a very ancient sports game. Nowadays, it has also become a very popular fast and effective exercise method in the fast-paced life. [0003] At present, there are mainly two solutions for intelligent counting of rope skipping on the market, one is built into the rope skipping equipment, and the other is using video surveillance. The built-in intelligence in the skipping rope requires the purchase of additional equipment for intelligent hardware, which is not easy to promote, and video capture equipment is already very popular. At present, most of the schemes that use video to analyze the performance of rope skippers use a single image to analyze the fluctuation of the portrait's up and down...

AI Technical Summary

Problems solved by technology

At present, most of the solutions that use video to analyze the performance of rope skippers use a single image to analyze the fluctuation of the portrait's up and down jumps. This kind of failed attempt during the rope skipping process cannot be ruled out, and it is difficult to achieve accurate records of rope skipping results.

Images