FPGA-based binary neural network real-time facial emotion recognition method

Abstract

The invention discloses a binary neural network real-time facial emotion recognition method based on an FPGA (Field Programmable Gate Array). The method comprises the following steps: 1) preprocessing a collected video image; 2) based on a Harr-like feature template, a window sliding mechanism is adopted to detect and cut a human face in the preprocessed image, a part containing a human face image is intercepted, and a circular LBP operator is adopted to extract local binary information of the intercepted image; 3) inputting the extracted local binary information into a pre-established and trained facial emotion recognition model to obtain an emotion recognition result; the facial emotion recognition model is a binary neural network, and forward reasoning is carried out by adopting an XNOR gate design, bit accumulation operation and threshold comparison method in an FPGA (Field Programmable Gate Array) to realize emotion recognition.

Description

technical field [0001] The invention belongs to the field of emotional computing and heterogeneous computing in the field of FPGA, and in particular relates to a real-time facial emotion recognition method based on a binary neural network based on FPGA. Background technique [0002] Today, computers play an extremely important role in industry and daily life, and are rapidly becoming an integral part of social development. Therefore, the need to study the interaction between humans and computers is increasing day by day. Of course, the premise of human-computer interaction is that a computer can accurately and quickly analyze people's emotions and intentions, and make appropriate responses accordingly. The variety of emotions expressed in facial expressions is a universally valid way of expressing human intent. Analyzing a person's intent through the emotions revealed by the face is called facial emotion recognition technology, and this technology is used in many fields, s...

AI Technical Summary

Problems solved by technology

Such a large number of parameters and calculations cause CNN to rely heavily on high-performance hardware computing platforms such as GPUs.

However, in practical applications, they are usually devices with limited computing resources. For example, embedded devices based on Field Programmable Gate Arrays (Field Programmable GateArray, FPGA) usually have only a few thousand computing units, which are far from being able to handle the data in common deep models. Millions of floating point operations, there is a serious contradiction between complex models and limited computing resources

This directly causes facial emotion detection to be greatly limited in applications that require real-time processing, such as the Advanced Driving Assistance System (Advanced Driving Assistance System, ADAS), which collects the driver's facial information through the camera and transmits it to the embedded system for further processing. However, in practical applications, it is difficult for the embedded computing platform to accurately judge the driver's state in real time, so that it cannot assist driving correctly; in the baby emotion recognition comfort system, the image recognition system recognizes the baby's The server starts the voice pacification module to pacify the crying baby to achieve the effect of remote care. In this system, the real-time detection of the baby's emotions is particularly critical. If the baby's emotional state cannot be conveyed in real time, then remote care cannot be realized.

Images