Picture/video important person detection method combining deep learning and relation modeling

An important person, deep learning technology, applied in the field of image processing, can solve the problems of ignoring the relationship between pedestrians, not taking into account the relationship between people and the scene, unable to express information well, to achieve iterability, demand less effect

Pending Publication Date: 2020-04-14
SUN YAT SEN UNIV
View PDF0 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This ignores the role of the relationship between pedestrians for the importance analysis
In addition, this method also ignores the role of spatial information and attention information
The features used in the technology based on multi-layer hybrid relationship graphs are pre-trained from other tasks, which cannot express information well at the high-level semantic level, and only consider the relationship between people, and Does not take into account the relationship between people and the scene

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Picture/video important person detection method combining deep learning and relation modeling
  • Picture/video important person detection method combining deep learning and relation modeling
  • Picture/video important person detection method combining deep learning and relation modeling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0063] Such as figure 1 As shown, the method of the present invention is POINT (deep importance relatIons NeTworks). First, the present invention detects all pedestrians in the picture through a face detector, and then (a) uses the feature expression module to extract personal features and global features, and then inputs these features into the relationship calculation module (b), and the whole relationship module consists of r sub- In each sub-relationship module, we will construct a person-to-person (p2p) relationship graph and a person-to-event (p2e) relationship graph, and then estimate the importance relationship from these two graphs. Then the relational features are encoded and concatenated with the original personal features pfeat to obtain the characteristic features. Finally, the importance features are input into (c) the importance classification module, and the importance of each person is scored.

[0064] Specifically, a method for detecting important people in...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a picture / video important person detection method combining deep learning and relation modeling. The method comprises the following steps: S1, carrying out the feature extraction of the appearance information and the geometric information of a portrait in a picture / video, and carrying out the fusion of the appearance information and geometric information into personal characteristics representing high-level semantics; S2, calculating relation characteristics which cannot be expressed or cannot be highly expressed by independently depending on personal characteristics bymining relations between people and relations between people and scenes in the scenes; and S3, performing importance classification, performing important or unimportant dichotomy on the final featureexpression extracted from the relation calculation model of each portrait, wherein the probability that each portrait is classified into the important category serves as an importance score, and theportrait with the highest score is an important person identified by the relation calculation model. According to the method, the relation between the persons in the picture / video and the relation between the persons and the events in the picture can be autonomously constructed through learning, and the importance degree of the persons can be automatically deduced.

Description

technical field [0001] The invention belongs to the technical field of image processing, and in particular relates to a method for detecting important persons in pictures / videos combined with deep learning and relationship modeling. Background technique [0002] Image / video important person detection refers to finding the important person in this photo / video based on the person's clothing, action, position, interaction information and scene in a given photo containing many portraits. This technology can help with scene understanding and contribute to the development of industries such as text live broadcast, film and television shooting, and security monitoring. For example, in a text live broadcast, it is possible to judge what happened in the scene according to the behavior of the central character in the video, and directly generate a text description. In sports live broadcasting, this method is applied to detect important people in sports scenes, such as the ball holder...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/00G06K9/46G06K9/62G06N3/08
CPCG06N3/08G06V40/168G06V40/10G06V40/172G06V10/454G06F18/24G06F18/214
Inventor 郑伟诗洪发挺
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap