Neonatal pain expression recognition method based on double-flow convolutional neural network

Abstract

The invention discloses a neonatal pain expression recognition method based on a double-flow convolutional neural network, and the method comprises the steps: firstly carrying out the framing of a video, and then adding an optical flow displacement field among a plurality of continuous frames to obtain corresponding optical flow information, so as to obtain an optical flow graph; constructing a shared Attention double-flow convolutional neural network, wherein a shared Attention module is added to the network on the basis of the double-flow convolutional neural network, and the network is mainly composed of two pre-trained VGG16 networks and the shared Attention module. In the network, firstly, a frame with the largest expression change is selected from each frame of image sequence to serve as input of one VGG16 network, the network is called as a spatial information network, then an optical flow graph serves as input of the other VGG16 network, and the network is called as a time information network. Finally, by cascading the feature maps passing through the two networks and inputting the feature maps into a full connection layer, neonatal pain expression classification is realized.

Description

technical field [0001] The invention relates to a newborn pain expression recognition method based on a dual-stream convolutional neural network, which belongs to the direction of deep learning and pattern recognition. Background technique [0002] Facial expression recognition has been an active research area, but facial expression recognition is not an easy problem for machine learning methods. Human beings have strong cognitive abilities, and can generally obtain a person's emotional state through facial expressions and body movements, and then take corresponding actions according to the person's emotions. However, it is very difficult for machines to recognize human emotions. With the development of science and technology, the anthropomorphism of machines has become a research hotspot. [0003] In 2014, Karen Simonyan and Andrew Zisserman proposed a two-stream convolutional neural network, using two identical convolutional neural networks to identify behaviors in videos...

AI Technical Summary

Problems solved by technology

Although the convolutional neural network can simplify the process of traditional emotion recognition, it only extracts features from each still frame in the video,

Images