Target tracking and semantic segmentation method, device and plug-in for sports video

Abstract

The invention discloses a method, device and plug-in for target tracking and semantic segmentation of sports video. The method includes: extracting a feature map according to input video frames and target initialization position information to obtain feature map information; Perform positioning to obtain target location information; fuse feature map information and target location information, and filter the background information of non-target areas; perform decoding operations according to fused features and feature map information, and finally form a semantic mask of the target. membrane. The device includes: a backbone network encoder, a prediction network unit, a feature fusion network unit and a decoder. The plug-in includes: video information component, video preview component, video playback component, new target component, tracking adjustment component and special effect component. Through the present invention, the tracking accuracy can be improved in sports videos with problems such as target deformation, rotation, and occlusion.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a method, device and plug-in for target tracking and semantic segmentation of sports video. Background technique [0002] In recent years, the rapid development of the mobile Internet, especially the 5G industry, has spawned a wide range of digital entertainment application scenarios. Among them, target tracking algorithms are widely used in video special effects editing, human-computer interaction, video surveillance, TV broadcasting, autonomous driving and scientific analysis. [0003] Object tracking algorithm is a basic topic in the field of computer vision. Early tracking algorithms often search based on manually specified target features, or calculate optical flow to determine target displacement. Tracking algorithms based on this idea often have high time complexity and poor robustness, so their application scope is narrow. After that, the target tracking algori...

AI Technical Summary

Problems solved by technology

[0006] Different from the data for general target tracking and semantic segmentation, sports videos have the following characteristics: the camera position of the video often overlooks the entire sports field from an oblique downward angle, and the length of the player is about 100 pixels, so low The resolution is easy to cause the failure of tracking and semantic segmentation; secondly, athletes on the field often wear two kinds of uniforms, and there will be a large number of similar-looking instances around the tracking target, which is a test of the ability of the tracking algorithm to distinguish; thirdly, due to the When playing and running on the field, there will be severe body movement, deformation, rotation, occlusion, camera lenses will also appear blurred, shaking, etc., the performance of existing algorithms will be significantly degraded in these scenarios

[0007] It is worth mentioning that existing algorithms often regard object tracking and video semantic segmentation as two independent modules, which limits their scope of use and also affects the speed of the entire process

Images