Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Video action recognition method and device

A technology of action recognition and video recognition, applied in the fields of computer vision and machine learning, can solve the problems of difficult to obtain data training network, lack of quality control of video, and difficult to expand to large-scale data sets, etc., to achieve good and effective recognition Effect

Inactive Publication Date: 2018-12-07
ZHONGAN INFORMATION TECH SERVICES CO LTD
View PDF8 Cites 32 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The encoding algorithm based on the hidden Markov model has a good effect on these data, but it is difficult to extend to large-scale data sets
[0007] However, in reality, videos are generally shot by non-professionals and lack quality control, and the labeling work is more complicated. It is difficult to obtain a large amount of labeled data to train the network for new tasks.
Some existing migration learning algorithms use pre-trained image models in video recognition. Some researchers use spatial attention maps to map videos to low-dimensional feature spaces. These domain adaptive algorithms require a large amount of images or other data to transfer knowledge. , it is difficult to achieve few-sample video recognition only by modifying the network structure

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Video action recognition method and device
  • Video action recognition method and device
  • Video action recognition method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] figure 1 is a schematic flow chart of the video action recognition method provided in Embodiment 1 of the present invention, as figure 1 as shown,

[0049] The video action recognition method that the embodiment of the present invention provides, comprises the following steps:

[0050] 101. Extract the spatio-temporal features of the video.

[0051] Specifically, the time-series segmentation network is used to extract the spatio-temporal features of the video. The process includes:

[0052] The spatial convolutional network and temporal convolutional network included in the time series segmentation network extract static image features and moving optical flow features respectively, and generate corresponding feature vectors. Temporal SegmentNetwork (TSN) is used to extract the spatio-temporal features of each segment (reference [2]. Limin Wang, Yuanjun Xiong, Zhe Wang, YuQiao, DahuaLin, Xiaoou Tang, and Luc Van Gool. 2016. Temporal segmentnetworks : Towards good pra...

Embodiment 2

[0070] Figure 5 It is a flow chart of the video action recognition method provided by Embodiment 2 of the present invention, such as figure 2 As shown, the video action recognition method provided by the embodiment of the present invention includes the following steps:

[0071] 201. Perform video preprocessing on the video and the video to be recognized, where the video preprocessing includes video segment segmentation and key frame extraction.

[0072] Specifically, the video for training and recognition and the video to be recognized will be required, and the pictures of RGB static frames and the optical flow pictures of motion will be extracted.

[0073] It should be noted that the process implemented in step 201 can be implemented in other ways besides the ways described in the above steps, and the embodiment of the present invention does not limit the specific way.

[0074] 202. Extract the spatio-temporal features of the video through a temporal segmentation network....

Embodiment 3

[0088] Figure 6 is a schematic structural diagram of a video action recognition device provided by an embodiment of the present invention, as shown in Figure 6 As shown, the video action recognition device provided by the embodiment of the present invention mainly includes an extraction module 31 , a training module 32 and a recognition module 33 .

[0089] Specifically, the extraction module 31 is used to extract the spatio-temporal features of the video, specifically extracting static image features and moving optical flow features through the spatial convolutional network and temporal convolutional network included in the temporal segmentation network, and generating corresponding features vector. The time node for the extraction module to extract the spatio-temporal features of the video is to extract the spatio-temporal features of the video to be identified, or, before using the target dense expansion network model to identify the video to be identified, the extractio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention, which belongs to the technical field of computer vision and machine learning, discloses a video action recognition method and device. The method comprises: spatiotemporal features of avideo are extracted; according to the extracted features, a preset dense expansion network model is trained to obtain a target dense expansion network model; and a to-be-identified video is identifiedby using the target dense expansion network model to obtain a video recognition result. Therefore, the video action can be identified well and effectively; a new type of dense expansion network modelcan be generated by using a few of data; the video action recognition method and device are improved by being compared with the existing video recognition technologies; and a problem of difficult identification due to a few of new task samples is solved. The video action recognition method and device can be applied to video search, vehicle accident detection, medical imaging and other fields widely.

Description

technical field [0001] The invention relates to the technical fields of computer vision and machine learning, in particular to a video action recognition method and device. Background technique [0002] Video action recognition has been widely studied in recent years. Early research mainly focused on traditional artificial features, such as visual features such as spatial interest points (Space-time interest points, STIP), gradient histogram (Histogram of gradient, HoG), optical flow Histogram of optical flow (HOF) has been studied. In addition, the information from the image is extended to capture time information, and the dense trajectory (Dense trajectory) densely tracks and samples the local information of each block of optical flow, but these artificially designed feature representation capabilities are relatively limited, which limits these methods for complex , Large-scale video classification capabilities. [0003] In recent years, convolutional neural networks hav...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00G06N3/08
CPCG06N3/08G06V20/41G06V20/46
Inventor 徐宝函叶浩郑莹斌陆王天宇王恒姜育刚孙谷飞
Owner ZHONGAN INFORMATION TECH SERVICES CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com