Action recognition system for action recognition in unlabeled videos with domain adversarial learning and knowledge distillation
Active Publication Date: 2018-09-20
NEC CORP
View PDF0 Cites 28 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Benefits of technology
The present invention provides an action recognition system that can capture video sequences and recognize actions performed by objects in the video. The system uses a set of unlabeled testing video frames and a reference set of convolutional neural networks (CNNs) to pre-train a recognition engine. The engine is then adapted to a video domain by applying a non-reference set of CNNs to a set of domains that includes the still image domain, a synthetically degraded image domain, and the video domain. The system can recognize an action performed by at least one object in the video sequence and control a hardware device to perform a response action in response to the identified action type. The technical effect of the invention is to provide an effective and efficient solution for action recognition in videos.
Problems solved by technology
In machine learning, there exists the fundamental problem of domain adaptation when the source domain has abundant labeled training data and the target domain has no or little labeled training data but a massive amount of unlabeled data.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0021]The present invention is directed to action recognition system for action recognition in unlabeled videos with domain adversarial learning and knowledge distillation.
[0022]In an embodiment, the present invention solves the fundamental machine learning problem of domain adaptation where the source domain has abundant labeled training data and the target domain has no or only a few numbers of labeled training data but a massive amount of unlabeled data.
[0023]In an embodiment, the present invention utilizes unlabeled video data to train a recognition engine together with labeled image data.
[0024]In an embodiment, the present invention is applied to video face recognition. Of course, the present invention is not limited to solely video face recognition and can be applied to other types of recognition, as readily appreciated by one of ordinary skill in the art given the teachings of the present invention provided herein, while maintaining the spirit of the present invention.
[0025]I...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
An action recognition system is provided that includes a device configured to capture a video sequence formed from a set of unlabeled testing video frames. The system further includes a processor configured to pre-train a recognition engine formed from a reference set of CNNs on a still image domain that includes labeled training still image frames. The processor adapts the recognition engine to a video domain to form an adapted engine, by applying non-reference CNNs to domains that include the still image and video domains and a degraded image domain that includes labeled synthetically degraded versions of the frames in the still image domain. The video domain includes random unlabeled training video frames. The processor recognizes, using the adapted engine, an action performed by at least one object in the sequence, and controls a device to perform a response action in response to an action type of the action.
Description
RELATED APPLICATION INFORMATION[0001]This application claims priority to provisional application Ser. No. 62 / 472,896, filed on Mar. 17, 2017, incorporated herein by reference. This application is related to an application entitled “Recognition In Unlabeled Videos With Domain Adversarial Learning and Knowledge Distillation”, having attorney docket number 16098A, and which is incorporated by reference herein in its entirety. This application is related to an application entitled “Face Recognition System For Face Recognition In Unlabeled Videos With Domain Adversarial Learning And Knowledge Distillation”, having attorney docket number 16098B, and which is incorporated by reference herein in its entirety. This application is related to an application entitled “Surveillance System For Recognition In Unlabeled Videos With Domain Adversarial Learning And Knowledge Distillation”, having attorney docket number 16098C, and which is incorporated by reference herein in its entirety.BACKGROUNDTe...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06K9/00G06K9/46G06K9/62G06N3/02G06N99/00
CPCG06K9/00718G06K9/00744G06K9/4628G06K9/6262G06N3/02G06N99/005G06K2009/00738G08B13/19613G06N3/088G06T9/002G06V40/172G06V20/52G06V10/454G06V10/82G06V30/19173G06N3/045G08B13/196G06N3/08G06N20/00G06V20/41G06V20/46G06V40/168G06V20/44G06F18/21G06F18/22G06F18/217G06F18/24143G06T7/70G06T2207/20081
InventorSOHN, KIHYUKYU, XIANGCHANDRAKER, MANMOHAN
OwnerNEC CORP