Real-time Lip Movement Capture via Pretrained Average Model

The patent applies preliminary action by pre-training a lip average model offline using extensive lip sample data. This model is then deployed for real-time inference, separating the time-consuming training phase from the real-time capturing phase. The pre-trained model contains learned lip feature representations that can quickly classify lip movements without requiring extensive training during actual use.

The patent creates a simplified copy of the complex deep learning model by extracting key lip feature points and their relationships from the trained model. This copy consists of geometric relationships between landmark points that can be quickly computed in real-time, maintaining classification accuracy while dramatically reducing computational requirements for real-time operation.

The patent extracts essential lip feature information from extensive training samples by identifying and tracking key landmark points on the lips (such as corners, contours, and central points). Instead of processing entire lip images or complex feature vectors, the system extracts only the necessary geometric coordinates of these landmark points, significantly reducing data complexity while preserving the essential information needed for classification.

The patent transforms the lip movement classification problem from analyzing complex image data to analyzing geometric parameters of landmark points. By changing the representation from pixel-based or feature-vector-based data to simple coordinate geometry (x, y positions of landmark points), the system achieves efficient real-time processing while maintaining classification capability.