158 results about "Facial expression recognition" patented technology

The embodiment of the invention provides a model training method, a facial expression recognition method and device and electronic equipment, and relates to the technical field of video processing. The model training method comprises the following steps: acquiring a sample video and a first sample label; extracting a time feature and a space feature of the sample video by using a time-space feature extraction network in the facial expression recognition model of the initial structure; calculating an attention weight representing the correlation between the spatial feature and the time feature by using a mapping network; performing weighted aggregation on the time features by using the attention weight to obtain fused spatio-temporal features; inputting the fused spatial-temporal features into a classification network to obtain a first recognition result; and performing model training based on the difference between the first recognition result and the first sample label to obtain a trained facial expression recognition model with higher accuracy.

A facial expression recognition method based on grid attention and pyramid segmentation attention belongs to the technical field of deep learning image processing and emotion calculation, and comprises the following steps: introducing a grid attention mechanism and a pyramid segmentation attention mechanism on a ResNet101 large model, capturing local expression detail features through a grid attention module, establishing multi-scale global feature association by using a pyramid segmentation attention mechanism; hierarchical segmentation is carried out on the backbone network, and information from multiple hierarchies is effectively fused; and dynamically balancing the contribution degree of each level of features through learnable parameters, and integrating the extracted features to judge the expression category. Experiments show that the model achieves the recognition accuracy superior to that of a traditional model on a public data set in natural scenes such as complex illumination and posture change. According to the method, a solution with high robustness is provided for facial expression recognition in a complex environment, and the method has important application value in the fields of intelligent human-computer interaction, mental health assessment and the like.

The invention discloses a sign language synthesis service method based on multi-modality, and the method comprises the following steps: S10, carrying out end-cloud collaborative rendering setting: deploying an edge end as a lightweight model to generate a basic action, and deploying a cloud end as operating a MoMask and 3D rendering engine; s20, performing multi-modal data acquisition: acquiring a voice signal through a microphone, acquiring a 48 * 48 pixel face grayscale image through a camera, and acquiring action posture data through an IMU sensor; s30, performing emotion feature extraction on the collected multi-modal data: performing voice emotion recognition to output six types of emotion probability distributions, and performing facial expression recognition to output seven types of expression probability distributions; s40, performing cross-modal feature fusion: dynamically fusing the voice and facial features based on a confidence weighting strategy, and generating a three-dimensional emotion intensity vector; and S50, sign language action generation is carried out, and a 3D skeleton sequence matched with emotion is generated through RVQ layering quantification and MoMask Transformer.

The invention provides a classroom student concentration degree monitoring system and method based on computer vision. The system comprises a vision collection module, an edge calculation unit and a cloud analysis platform. A multi-dimensional sensing mode combining head posture analysis, eye state tracking, facial expression recognition and body movement detection is adopted to evaluate the concentration degree, through a dynamic feature fusion mechanism, the system can intelligently adjust the weight ratio of different modal features, the limitation of single-dimensional analysis in a traditional method is effectively solved, and the accuracy of the system is improved. The system adopts edge calculation and cloud analysis, ensures high efficiency of data processing, ensures reliability of complex algorithm operation, and has important value for improving classroom teaching quality and learning efficiency. According to the invention, through multi-modal visual analysis, behavior characteristics and physiological reactions of students in class are monitored in real time, the concentration level of the students is comprehensively evaluated, and objective teaching feedback and improvement basis are provided for teachers.

The invention discloses a multi-mode mental health assessment system based on artificial intelligence, and belongs to the technical field of mental health assessment. A multi-modal mental health assessment system based on artificial intelligence comprises a data acquisition module, a multi-modal fusion module and a health assessment module. The psychological health assessment method and device solve the problem that in the prior art, in the process of collecting user information, the collection dimension is single, so that the user information is not comprehensive, and then the accuracy of psychological health assessment is affected, and the accuracy of psychological health assessment is improved by integrating multi-source data such as facial expressions, voices, texts and videos. The method effectively improves the objectivity, accuracy and personalized intervention capability of evaluation, greatly improves the overall evaluation accuracy through the fusion of facial expression recognition, voice pressure signal detection, text emotion feature recognition and physiological parameter monitoring technologies, effectively reduces the single-modal error through multi-modal data complementation, enables the user information to be more comprehensive, and improves the user experience. And the accuracy of mental health assessment is improved.

The invention provides a facial expression recognition method based on multi-module collaborative optimization, which has an efficient network structure and can realize high-precision and strong-robustness facial expression recognition in a complex environment. The method comprises the following steps: receiving an RGB face image and preprocessing the RGB face image to obtain a preprocessed image; inputting the preprocessed image into a mixed feature network module, and outputting to obtain a first feature map; inputting the first feature map into an efficient local attention mechanism module, and outputting to obtain a second feature map; and inputting the second feature map into a classifier module to realize facial expression recognition.

The invention provides a GAN-based vehicle-mounted expression data construction method, system and device and a vehicle, and relates to the technical field of intelligent driving. A key point detection technology is utilized to analyze and extract a facial action unit and expression intensity and duration thereof to generate a structured action description. And then, calling a generative adversarial network model to generate a synthetic video corresponding to the target expression type, and carrying out dynamic cutting and frame extraction according to a preset interval on the videos to obtain a multi-frame synthetic image. And finally, performing dual-model filtering on the synthesized image to ensure data quality, and executing privacy anonymization processing, thereby generating a high-quality vehicle-mounted expression data set. According to the method, the data acquisition cost is reduced, the diversity and accuracy of a data set are improved, and the performance and reliability of an expression recognition algorithm in an actual vehicle-mounted environment are effectively enhanced.

The invention discloses a multi-modal cross-domain small sample facial expression recognition method based on relational distillation self-paced learning, and relates to a computer vision technology. Constructing a multi-modal semantic enhancement module, generating semantic descriptions of expressions by using a large language model, performing CLIP coding, and performing alignment and fusion with image visual features in the multi-modal semantic enhancement module to construct a multi-modal prototype; a self-paced learning mechanism based on relational distillation is designed, visual and semantic structural errors are calculated, and progressive training from easy to difficult is realized through a soft and hard mixed sample selection strategy and a mixed sample selection mechanism regulated and controlled by a dynamic threshold value. And cross-domain migration of emotional knowledge from basic expressions to fine-grained composite expressions can be effectively realized. And under the condition that only a small number of labeled samples are provided, rapid adaptation and accurate recognition of new expressions can be realized. The method is remarkably superior to a traditional supervised learning method, has higher practicability and expansibility, and can better meet the requirement for efficient recognition of new expressions in practical application.

The invention discloses a verification method and device based on facial expression recognition and a storage medium. Relates to the field of artificial intelligence, and the method comprises the steps: obtaining a face image of a target customer in a target transaction process of the target customer under the condition of obtaining the authorization of the target customer; extracting facial features from the facial image, inputting the facial features into the expression recognition model, and outputting an emotion category corresponding to the facial expression on the facial image; calculating a transaction risk value corresponding to the facial expression based on the emotion category corresponding to the facial expression; calculating a transaction risk value corresponding to the target transaction based on the transaction information of the target transaction; and determining a target verification mode according to the transaction risk value corresponding to the facial expression and the transaction risk value corresponding to the target transaction, and executing the target verification mode on the target customer. Through application of the method and the device, the problem of relatively low verification accuracy caused by relatively single verification mode for the client identity in the transaction process in related technologies is solved.

The invention relates to the technical field of cross-domain facial expression recognition, in particular to a cross-domain facial expression recognition method and system based on intelligent learning. A shared backbone network is adopted to extract global depth features of the face image, and a coarse-fine branch network is constructed to realize collaborative learning of attitude analysis and emotional interpretation; a coarse and fine feature interaction mechanism is introduced in the training process to relieve subdivision branch too fast convergence, and uncertainty distribution of attitude analysis and emotional interpretation is obtained through similarity measurement and a Gaussian mixture model; modeling a potential feature linear dependency relationship, performing multi-source domain feature alignment, inhibiting tag noise and extracting robust semantic features; and performing cross-domain coupling calculation on the label uncertainty distribution and the semantic features to generate a final recognition result of the cross-domain facial expression. According to the method, the influence of cross-domain difference and label noise of expression recognition from coarse attitude analysis to fine emotion interpretation can be effectively relieved, and the generalization ability of cross-domain facial expression recognition and emotion interpretation precision are improved.

The application discloses a self-supervised facial expression recognition method combining contrast learning and mask image modeling, and the method comprises the following steps: constructing a self-supervised facial expression recognition model combining contrast learning and mask image modeling; performing unsupervised pre-training on the model; performing linear probe evaluation and fine-tuning evaluation on the pre-trained model on a labeled verification set; using the fine-tuned model to perform facial expression recognition on an input facial image; the method learns facial expression representation through a convolution-free twin network, pre-trains the twin network by using contrast loss and mask image modeling loss, so as to simultaneously understand high-level visual semantics and image internal structure, and maximize the consistency between the outputs of the student network and the teacher network. The application can learn rich visual information, is robust to various interference unrelated to facial expression, and simultaneously achieves excellent results in linear probe and fine-tuning evaluation on a facial expression dataset.

The invention discloses an expression recognition method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring an image pair comprising a target user face; the image pair comprises a natural expression image of the target user and a to-be-recognized image; performing fusion processing on the image pair, and inputting the image pair after fusion processing into a trained expression recognition model to obtain an expression recognition result of the to-be-recognized image generated by the expression recognition model; wherein the expression recognition model comprises an expression change extraction module and a decoder; the expression change extraction module is used for extracting target expression change features of the fused image pair; and the decoder is used for outputting an expression recognition result based on the target expression change feature. According to the embodiment of the invention, inherent differences of different individuals in emotional expression are fully considered, and the problem of identification deviation caused by neglecting individual expression characteristics in a traditional expression identification method is reduced, so that the accuracy of expression identification is improved.

The invention discloses a risk control identity authentication system and method based on facial expression recognition, and relates to the technical field of image recognition. The system comprises an identity information acquisition module, a facial feature extraction module, a dynamic feature extraction accuracy analysis module, a multi-dimensional comparison module and a multi-dimensional comparison rationality analysis module. Dynamic feature extraction accuracy evaluation is obtained through feature extraction parameter quantification, whether dynamic feature extraction accuracy optimization is carried out or not is judged, and if yes, a facial expression feature comparison link is carried out after optimization; otherwise, a facial expression feature comparison link is directly carried out, multi-dimensional comparison parameter quantification is obtained to obtain multi-dimensional comparison rationality evaluation, and whether multi-dimensional comparison rationality optimization is carried out is judged, so that the risk control identity authentication reliability is improved, and the problem that in the prior art, the risk control identity authentication efficiency is high is solved. The problem of low risk control identity authentication reliability caused by inaccurate expression dynamic feature extraction exists.

The application provides a network model and method for improving the facial expression recognition accuracy in a video, the method comprising: inputting an initial video image, and performing feature extraction through a 3D convolution network; fusing an AU perception attention module in the 3D convolution network, paying attention to the key area of facial emotional expression through the AU perception attention module, and learning the features of the key parts of the face; encoding the obtained feature map through a capsule network, and encoding the enhanced features through the dynamic routing between the capsules; decoding through three fully connected layers, and realizing the final expression classification through a nonlinear squeeze function. The application realizes higher-precision facial expression recognition.

A vehicle is provided that can mitigate the effects of high ambient noise levels. Using image data and / or other sensor data, the vehicle can determine when the convertible top is in use. The vehicle also determines when the ambient noise level inside the vehicle exceeds a predefined threshold. The vehicle then switches its operating mode to clearly receive and interpret audio input. The vehicle can activate a lip-reading mode and / or a gesture and facial expression recognition mode to identify words spoken by a user. The vehicle can also store mapping information between a gesture and a corresponding verbal command associated with the vehicle. In noisy environments, the vehicle may suggest using gestures / facial expressions instead of verbal commands.

The invention relates to the crossing field of target detection and emotion recognition, in particular to an autistic child facial expression recognition method based on deep learning. Comprising the following steps of: embedding three-dimensional attention (Tri-attention) behind a C3k2 module of a YOLOv11 backbone network so as to model an interactive relationship among three dimensions of height, width and channel in parallel and enhance response to key expression areas such as eyes, mouth corners and the like; three-dimensional attention (Tri-attention) is introduced into the multi-scale feature output end of the neck network again, and effective positioning and background noise suppression of a small-scale face under a complex background are achieved; respectively training and verifying the YOLOv11-TA model by using the public data set and a self-built autistic child data set, and outputting a detection frame of each face and an expression category thereof; by comparing various indexes of the original YOLOv11 and YOLOv11-TA on various data sets, the accuracy and robustness of expression recognition of the autistic children are remarkably improved by the method according to actual verification.

The invention discloses a facial expression detection method based on a Transform dual-channel network and related equipment, and can be applied to the technical field of deep learning. Static features of a video sequence to be processed are extracted through a video sequence feature extractor to form a video sequence feature map carrying global information and local information, and dynamic features of an optical flow sequence to be processed are extracted through an optical flow sequence feature extractor to form an optical flow sequence feature map. Fusing features of different levels in the video sequence feature map through a multi-level feature fusion module to obtain multi-level static fusion features, and carrying out feature refinement on the optical flow sequence feature map through a self-adaptive convolution attention module to obtain space channel refinement dynamic features; the two features are fused through a cross attention fusion module to obtain a cross fusion feature, and then facial expression recognition is performed according to the cross fusion feature, so that the accuracy of expression detection can be effectively improved.

The present application relates to a kind of facial expression recognition methods based on attention mechanism and self-distillation, comprising the following steps: step 1, constructing facial expression recognition dataset;Step 2, constructing facial expression recognition model, the model is composed of feature extraction module, adaptive channel attention module, remodeling module and self-distillation network;Step 3, the input facial expression image is preprocessed;Step 4, using the training set image in the above facial expression recognition dataset trains facial expression recognition model;Step 5, using the facial expression recognition model trained to the test set and validation set in dataset are inferred classification.The method designs a kind of reasonable and efficient facial expression recognition scheme, it is well guided model to pay attention to those important feature information by the mode of attention mechanism;By a new self-distillation form, refine compressed network knowledge, improve the robustness of shallow network output feature and reduce the complexity of model in inference stage.

The application provides an expression recognition model training and expression recognition method and device, electronic equipment and storage medium. The expression recognition model training method comprises obtaining a first expression representation model and a second expression representation model through self-supervised pre-training; fine-tuning the first expression representation model to obtain a first expression recognition model, and performing knowledge distillation on the first expression representation model based on the first expression recognition model to obtain a second expression recognition model; performing knowledge distillation on the second expression representation model based on the second expression recognition model to obtain a third expression recognition model. Through self-supervised pre-training, a large amount of unlabeled sample data is used to reduce the cost of manual labeling. Through self-distillation on the same expression representation model, the accuracy of the second expression recognition model is improved. And through knowledge distillation of the second expression representation model based on the second expression recognition model, the accuracy of the third expression recognition model is further ensured.

The application relates to a facial expression recognition-based method and system for evaluating anxiety of children in dental treatment. The method comprises the following steps: acquiring a three-dimensional facial point cloud sequence, a two-dimensional micro-expression texture image sequence and a bone conduction audio signal sequence in the process of dental treatment of children; performing point cloud cleaning and three-dimensional reference grid reconstruction on the three-dimensional facial point cloud sequence to obtain a geometric base grid sequence, and performing texture mapping to obtain a texture-mapped three-dimensional face grid sequence; extracting an acoustic feature sequence, a facial action unit intensity sequence and a facial micro-motion intensity sequence; normalizing the data before treatment as a calibration baseline to obtain a relative action unit intensity sequence and a relative micro-motion intensity sequence; performing multi-modal fusion to obtain a continuous emotional disturbance index; grading to obtain a dynamic Frankel behavior scale result, calculating an individualized fear survey score, and fusing to obtain a dental treatment anxiety index of children. The method can realize dynamic evaluation of dental treatment anxiety of children and improve the accuracy of anxiety evaluation.

This invention discloses a neurophysiological behavior-driven multimodal intelligent learning guidance system and method. The system achieves closed-loop control by constructing a cognitive-emotional-task adaptive adjustment space: 1) In the cognitive dimension, it integrates EEG neural oscillations and frontoparietal network connections, fNIRS cerebral oxygen metabolism indicators, and eye-tracking data to construct a multimodal cognitive load assessment model; 2) In the emotional dimension, it combines the nonlinear characteristics of skin conductance signals with facial expression recognition to quantify learners' motivation levels and emotional valence in real time; 3) In the task dimension, it parameterizes task difficulty and interaction form as moderating variables. During system operation, the system locates the learner's position on the cognitive-emotional state plane in real time and dynamically adjusts task dimension parameters through a nonlinear mapping strategy. This invention effectively utilizes the decoupling ability of multimodal signals and the interaction of intention to achieve precise adaptive delivery of teaching content.

The invention provides a facial expression recognition method based on multi-level feature extraction and fusion in a natural environment. The method is characterized by comprising the following steps: (1) a feature extraction module: carrying out convolution calculation by using three different convolution kernel scales and dense blocks with DenseNet as a backbone network to generate facial expression feature maps (Feature Maps) at a low level, a middle level and a high level; different dense blocks are connected through transition layers; (2) a feature fusion module which performs global attention and local attention processing on the processed high-level feature map and low-level feature map after element addition calculation so as to generate a fused facial expression feature map; and (3) optimizing a strategy: adopting a strategy of combining a smooth label (Label Smoothing) and L2 regularization to avoid over-fitting.

The invention discloses a multi-view evidence fusion large-angle facial expression recognition method and system, and the method mainly comprises the steps: directly transmitting a read image into a face detection model based on a YOLOv8 architecture, returning all detected face targets in the image, extracting key information, and calculating the in-plane rotation angle of a face; when the absolute value of the rotation angle exceeds a threshold value, activating the GAN model to generate a view 2, and cutting a face region from the original image by using the returned face target all the time regardless of the rotation angle to serve as a view 1; extracting the view 1 and the view 2 in parallel to obtain two groups of independent evidence vectors, and calculating subjective opinions corresponding to the evidence vectors; if the two views are processed, evidence fusion is started, otherwise fusion is not needed, an index of the maximum value is searched in a final belief vector to determine the most credible expression category, and final uncertain expressions are output together. Compared with any single view or simple fusion method, the method provided by the invention can obtain more accurate and more stable recognition performance under various attitude angles.

The invention discloses an intelligent accompanying robot based on visual perception and recognition and a control method, and the robot comprises a visual perception and recognition module which is used for collecting indoor and outdoor environment images in real time, and constructing a three-dimensional environment map; verification and visual positioning of identity recognition of the elderly; recognizing a corresponding emotional state through a facial expression recognition algorithm; recognizing obstacles in the environment through the deployed target detection model; the real-time accompanying control module is used for starting real-time identification and accompanying based on the three-dimensional environment map and the visual positioning of the elderly, and dynamically adjusting following parameters according to the moving state of the elderly to avoid obstacles in the environment; the emotional accompanying and interaction module is used for calling a preset emotional interaction strategy to realize emotional accompanying after visually identifying the emotional state of the elderly; the method has the advantages that visual perception and recognition serve as the core, gait recognition and emotion accompanying and interaction are adopted, and accompanying precision and service initiative are improved.

Facial expression recognition method and apparatus via label distribution learning are disclosed. The facial expression recognition method through label distribution learning, comprising: (a) in the training process, preprocessing an input sample to generate a plurality of augmented samples, creating a target label distribution for the input sample using the plurality of augmented samples, and training a model using supervised learning; and (b) after the training is completed, during the inference process, outputting a facial expression recognition result by inputting a single facial sample into the trained model without using the augmented samples.

This invention discloses an emotion recognition method based on a global and local feature fusion network, relating to the field of face recognition technology. The method includes: constructing an emotion recognition model based on a global and local feature fusion network (GLFNet); inputting a face image to be recognized into the emotion recognition model, extracting intermediate feature maps from the face image through a ResNet-18 backbone network, and feeding these intermediate feature maps into the global and local modules respectively to obtain global and local feature maps; fusing the global and local feature maps using a feature fusion module to obtain fused features; establishing residual connections between the intermediate features and the fused features to obtain the final fused features; and processing the final fused features through a fully connected (FC) layer to obtain the emotion recognition result. This invention can improve the accuracy of facial image emotion recognition results, obtain richer fused feature information, and provide more powerful performance for facial expression recognition tasks.

The application provides a face expression recognition model training method and a face expression recognition method, relates to the picture recognition technical field, and the training method flips the obtained face image, utilizes the natural symmetry feature of the face, calculates the consistency between the features of the face image and the features of the face flipped image, obtains a first loss value, and concentrates the model attention on the key features of the face; the face image is divided into a clean image subset and a noise image subset, the clean image is trained by using a supervised learning method to obtain a second loss value, and the noise image is trained by using a self-supervised learning method to obtain a third loss value; finally, the three loss values are fused, a total loss value and a preset loss value are combined, and the initial recognition model is adjusted to improve the efficiency of the recognition model training and enhance the robustness and accuracy of the face expression recognition model.

The system according to the embodiment aims to improve the accuracy of incident detection while ensuring the privacy of users. [Solution] A system according to an embodiment includes a collection unit, a monitoring unit, a recognition unit, a determination unit, and a notification unit. The collection unit collects biometric information. The monitoring unit monitors the user's movements or facial expressions based on the biometric information collected by the collection unit. The recognition unit detects abnormalities detected by the monitoring unit and recognizes conversations based on the abnormalities. The determination unit determines whether or not an incident has occurred based on the conversation recognized by the recognition unit. The notification unit notifies the user's family of the results of the incident determined by the determination unit.

The application discloses a kind of light driver facial expression recognition methods based on multi-scale convolutional neural network, which realizes face positioning and alignment by multi-task convolutional neural network;Then the enhanced facial image is transmitted into the improved multi-scale convolutional neural network to recognize human face expression;The present application is different from the traditional expression recognition method based on convolutional neural network, a novel multi-scale residual attention module is designed, and a light and efficient network is constructed, the driver's facial image is non-contact to identify the driver's expression state and take corresponding measures, realize safe and intelligent driving, and the method extracts facial image features from multiple scales on one hand, effectively improves the accuracy of facial expression recognition;On the other hand, light network is conducive to deployment in car intelligent cockpit domain controller.

The invention discloses a video facial expression recognition method based on a fuzzy time refinement network, is applied to the technical field of computer vision and artificial intelligence, and aims to solve the problems that in the prior art, the expression time sequence dynamic state is difficult to capture, the processing feature fuzziness is low, and the refinement category recognition precision is low. The method comprises the following steps: preprocessing video frames; extracting spatial features by adopting ResNet-18 (ResNet-18); modeling inter-frame time sequence dependence through LSTM; dynamically weighting key frame features by using a fuzzy attention module (FAN) in combination with Gaussian, triangle and Sigmoid membership functions; carrying out the optimization of feature clustering through the combination of cross entropy loss and TriCenter Loss; outputting an expression category; by adopting the method, the video facial expression recognition accuracy can be effectively improved.