Human interaction behavior detection method based on self-attention mechanism

A detection method and attention technology, applied in neural learning methods, computer components, instruments, etc., can solve abstract and complex problems

Pending Publication Date: 2022-07-22
ZHEJIANG UNIV OF TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Because the behavior of interaction is relatively abstract and complex, it is often a

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Human interaction behavior detection method based on self-attention mechanism
  • Human interaction behavior detection method based on self-attention mechanism
  • Human interaction behavior detection method based on self-attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0026] In one embodiment, as figure 1 As shown, a method for detecting human interaction behavior based on a self-attention mechanism is provided, including:

[0027] Step S1: For an image frame to be detected, a preset number of image frames before and after it are taken to form a video segment, and the video segment is preprocessed.

[0028] For the image frame to be detected, that is, the target frame of interest, it is necessary to perform human interaction behavior detection on the frame. When the recognition network model constructed by the present application ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a human interaction behavior detection method based on a self-attention mechanism, and the method comprises the steps: inputting a preprocessed video clip into a constructed recognition network model, extracting spatial-temporal features through a backbone network, carrying out the dimension reduction of the extracted spatial-temporal features, flattening the spatial-temporal features after the dimension reduction, and carrying out the recognition of human interaction behaviors. And then position coding is carried out to obtain a feature vector containing position information, and the feature vector containing the position information is input into an encoder to obtain a shared feature. The method comprises the following steps: firstly, obtaining instance embedding and interaction relation embedding through two decoders, finally obtaining instance interaction characteristics between each instance and an interaction relation through a similarity decoder, obtaining the similarity between each instance and the interaction relation through a classification operation, and determining the interaction relation to which each instance belongs.

Description

technical field [0001] The present application belongs to the technical field of human interaction behavior detection, and in particular relates to a human behavior interaction detection method based on a self-attention mechanism. Background technique [0002] Understanding human interaction behavior is a very basic task in the field of computer vision, and it is helpful for downstream tasks, such as video surveillance, key event retrieval, overall behavior understanding, and sports analysis. The task of human interaction behavior understanding is to predict human interaction in visual signals, which requires positioning and action category prediction for each person in the scene, as well as finding out the interaction between people. Because the behavior of interaction is relatively abstract and complex, it is often a challenge for computers to complete accurate interaction behavior prediction. [0003] At present, there are two main directions for human interaction behavi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06V40/10G06V40/20G06V20/40G06V10/32G06V10/44G06V10/764G06V10/82G06V10/74G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/047G06N3/048G06N3/045G06F18/22G06F18/241G06F18/2415
Inventor 应凯宁王振华
Owner ZHEJIANG UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap