A small sample behavior recognition method and system based on multi-dimensional prototype reconstruction and reinforcement learning

A technology of enhanced learning and recognition methods, applied in the field of computer vision, can solve the problems of weak identification, significant deviation of timing information distribution, and inability to obtain general prototypes, etc., to reduce data deviation, improve discrimination ability, and improve classification accuracy degree of effect

Active Publication Date: 2022-07-26
JIANGNAN UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the prototype network has the following limitations: (1) It is impossible to obtain a general prototype to better represent the average level of the category
(2) There are subtle differences between some actions, and the discriminability between classes is weak
(3) The action distribution of the training set and the test set is unbalanced, and the timing information distribution deviation between different domains is more significant

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A small sample behavior recognition method and system based on multi-dimensional prototype reconstruction and reinforcement learning
  • A small sample behavior recognition method and system based on multi-dimensional prototype reconstruction and reinforcement learning
  • A small sample behavior recognition method and system based on multi-dimensional prototype reconstruction and reinforcement learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0051] see figure 1 and 2 As shown, this embodiment provides a small sample behavior recognition method based on multi-dimensional prototype reconstruction and reinforcement learning, including the following steps:

[0052] S10: Calculate the support set time series feature and the query set time series feature based on the obtained support set sample and the query set sample, and use the support set time series feature and the query set time series feature to calculate to obtain the support set video descriptor and the video descriptor of the query set;

[0053] S20: Solve the original time series attention based on the support set time series features, apply a random shuffling and rearrangement operation on the support set time series features, obtain supplementary features of the support set time series features, and calculate the scrambled time series attention of the supplementary features force, perform an inverse operation on the disrupted time sequence attention to ob...

Embodiment 2

[0085] The following describes a small-sample behavior recognition system based on multi-dimensional prototype reconstruction and reinforcement learning disclosed in Embodiment 2 of the present invention. The small-sample behavior recognition system based on multi-dimensional prototype reconstruction and reinforcement learning described below is the same as the one described above. A few-shot behavior recognition method based on multi-dimensional prototype reconstruction reinforcement learning can be referred to each other.

[0086] see Image 6 As shown, the second embodiment of the present invention discloses a small sample behavior recognition system based on multi-dimensional prototype reconstruction and reinforcement learning, including:

[0087] A video descriptor calculation module 100, the video descriptor calculation module 100 is configured to calculate the support set time series features and the query set time series features based on the obtained support set sampl...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a small sample behavior recognition method based on multi-dimensional prototype reconstruction and reinforcement learning. The average prototype of the class, and the re-weighted similarity attention is used to calculate the similarity between the query set sample and the support set sample and the class average prototype, and re-weight the support set sample and the query set sample according to their corresponding similarity, and get two Prototype, the two prototypes are weighted and summed to obtain a cross-enhanced prototype, and a double triplet optimization classification feature space is constructed to enhance the discriminative ability of the cross-enhanced prototype for different categories, and the optimized cross-enhanced prototype is used to identify all categories. The video in the query set sample is classified, which greatly improves the classification accuracy.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a small sample behavior recognition method and system based on multi-dimensional prototype reconstruction and reinforcement learning. Background technique [0002] With the continuous research of machine vision in theory and practice, human behavior recognition has gradually become an important branch. Traditional action recognition methods can be generalized into RGB image-based and video-based methods, but these methods all have serious limitations, namely, they require a large amount of annotated data to train the model to correctly recognize actions, which brings very expensive Calculate the cost. Small-sample learning aims to classify new samples by learning a small number of samples. Small-sample behavior recognition includes two inputs: support set video representation and query set video representation. The model is trained on the support set and uses the support...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06V40/20G06V20/40G06K9/62G06F16/73G06V10/74G06V10/764
Inventor 蒋敏刘姝雯孔军
Owner JIANGNAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap