A video group behavior recognition method based on cascaded transformers

A recognition method and group technology, applied in the field of computer vision and deep learning, can solve the problem of not being able to extract video sequence features well, avoid manual feature extraction and offline training, have robustness, and improve recognition accuracy Effect

Active Publication Date: 2022-04-08
ZHEJIANG LAB
View PDF13 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Its disadvantage is that when the graph convolutional network extracts the spatial features of a single-frame group, it does not highlight the discriminative individual features in the group, and only performs simple weighted fusion in the video time feature dimension, which cannot extract video timing features well.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A video group behavior recognition method based on cascaded transformers
  • A video group behavior recognition method based on cascaded transformers
  • A video group behavior recognition method based on cascaded transformers

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0046] In order to make the object, technical solution and technical effect of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0047] Such as figure 2 As shown, a video group behavior recognition method based on cascaded Transformer, firstly, collect and generate a video data set, extract the 3D spatio-temporal features from the video data set through a 3D backbone network, and select the key frame image space feature map; for the key frame image space After the feature map is preprocessed, it is sent to the human target detection Transformer to output the human target frame in the key frame image; then, after mapping and filtering, the sub-feature map corresponding to the human target frame on the key frame image feature map is combined with the surrounding key frame image The frame feature map calculates the query / key / value, inputs the group behavior recognition T...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention relates to the field of computer vision and deep learning, and in particular to a video group behavior recognition method based on cascaded Transformers. First, a video data set is collected and generated, and the video data set is extracted through a three-dimensional backbone network to extract three-dimensional spatio-temporal features, and a key frame image space is selected. Feature map; preprocess the key frame image space feature map and send it to the human target detection Transformer to output the human target frame in the key frame image; then, map the sub The feature map is combined with the frame feature map around the key frame image to calculate the query / key / value, input the group behavior recognition Transformer, and output the group-level spatio-temporal encoding feature map; finally, the group behavior is classified by the multi-layer perceptron. The invention has the effect of effectively improving the accuracy of group behavior recognition.

Description

technical field [0001] The present invention relates to the field of computer vision and deep learning, in particular to a video group behavior recognition method based on cascade Transformer Background technique [0002] Nowadays, surveillance video has been widely used in social public places and plays an extremely important role in maintaining social public safety. Effective identification of abnormal behaviors and events in surveillance video can better play the role of surveillance video. Group behavior is the most common human behavior in videos. Group behavior recognition can effectively prevent dangerous events by automatically identifying group behavior in videos, and has a wide range of application values. [0003] In natural scenes, video group behavior recognition mainly faces two major challenges. One is that the scene is relatively complex, mainly manifested in the large scale change of the human body, background lighting, mutual occlusion between groups, etc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06V20/40G06V20/52G06V10/764G06V10/774G06V10/80G06V10/82G06K9/62G06N3/04G06N3/08
CPCG06N3/084G06N3/045G06F18/24G06F18/253G06F18/214
Inventor 李玲徐晓刚王军祝敏航曹卫强朱亚光
Owner ZHEJIANG LAB
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap