253 results about "Facial expression recognition" patented technology

The invention discloses a facial expression recognition method fusing spatial channel features and fast regional convolution. The method comprises a fast regional convolution module, a spatial channel feature fusion module and a multi-scale attention module. The method comprises the following steps: firstly, extracting multi-scale local features by using depth separable convolution, and constructing a feature extractor by stacking a fast regional convolution module; then, on the basis of a space channel feature fusion module, feature fusion is realized by using deep convolution and a cross-channel self-attention mechanism, so that the calculation complexity and the memory requirement are remarkably reduced; finally, through a multi-scale attention module, feature extraction and spatial information aggregation of different scales are utilized, and the feature representation capacity of the expression change key area is enhanced. According to the invention, a lightweight facial expression recognition network is designed by using respective advantages of a full convolution model and a fast regional convolution model. A large number of experiments show that compared with other methods, the method provided by the invention has better performance, and meanwhile, the calculation cost and the network scale are far smaller than those of the same type of methods.

The invention relates to the technical field of medical electronic equipment, and discloses a face emotion recognition method and system based on generative adversarial separation and expression exchange. The training process of the adopted facial expression recognition model comprises the following steps: pre-training a visual converter based on a self-supervised mask auto-encoder; extracting face feature vectors corresponding to the two face expression images through an image encoder in a visual converter; performing feature decoupling on the face feature vector to separate emotion-related features and emotion-unrelated features, and performing feature exchange to generate exchanged face features; fusing the face key point features with the exchange face features to generate fused features; generating a reconstructed facial expression image and a synthesized exchange facial expression image by using a generator based on the fusion features, and constraining identity consistency of the reconstructed image and the original facial expression image; a discriminator is used for discriminating the authenticity of a reconstructed image, and meanwhile, a classifier is used for performing facial emotion classification on the basis of emotion related features, so that parameters of a facial expression recognition model are optimized. The core problems of poor model generalization, low generation quality and difficulty in non-paired data training in the prior art are effectively solved, and the training cost is reduced.

The invention discloses a multi-modal dynamic facial expression recognition method based on CLIP, and the method comprises the following steps: constructing a label enhancement module, generating positive-negative text supervision, and obtaining positive text features and negative text features; constructing a multi-modal data mining module, and mining different levels of feature information from the video; fusing the facial expression features, the audio features and the fine-grained text description features by using an adaptive fusion strategy to obtain fused feature representation; and performing cosine similarity calculation on the fused feature representation, the positive text features and the negative text features to obtain final emotion classification. According to the method, class label enhancement is introduced, class labels are converted into positive-negative text supervision, and label enhancement is performed through P-N descriptors, so that fuzzy categories which are originally difficult to distinguish can be distinguished; and the similarity between correct image-text pairs is maximized by utilizing a CLIP contrast learning mechanism, so that the classification and retrieval precision is improved.

The invention discloses a facial action unit recognition method and system based on multi-modal fusion and text enhancement. The method comprises the following steps: extracting a facial image attribute set by a visual language model, constructing an adaptive text cue word based on the facial image attribute set, and processing the adaptive text cue word by adopting CLIP to obtain a text semantic feature; extracting facial key point features by the facial key point detection model; the facial expression recognition model extracts facial expression features; the image semantic feature fusion module processes the facial key point features and the facial expression features to obtain multi-modal image semantic features; inputting the multi-modal image semantic features and the text semantic features into a multi-modal feature fusion module to obtain multi-modal fusion features after text information enhancement; the recognition of the facial action unit of the facial image is realized. According to the method, the complementarity of cross-modal features and the robustness of joint representation can be enhanced, and the recognition precision of facial actions in a complex scene is effectively improved.

The embodiment of the invention provides a model training method, a facial expression recognition method and device and electronic equipment, and relates to the technical field of video processing. The model training method comprises the following steps: acquiring a sample video and a first sample label; extracting a time feature and a space feature of the sample video by using a time-space feature extraction network in the facial expression recognition model of the initial structure; calculating an attention weight representing the correlation between the spatial feature and the time feature by using a mapping network; performing weighted aggregation on the time features by using the attention weight to obtain fused spatio-temporal features; inputting the fused spatial-temporal features into a classification network to obtain a first recognition result; and performing model training based on the difference between the first recognition result and the first sample label to obtain a trained facial expression recognition model with higher accuracy.

The present invention provides a facial expression recognition method and system based on multi-cue associative learning, belonging to the technical field of computer vision. The recognition method comprises: inputting a pre-recognized facial image into a student model and / or a teacher model for facial expression recognition. A training method comprises: cropping a global facial sample image to obtain an upper half facial sample image and a lower half facial sample image; extracting cue features; acquiring adjacency matrices corresponding to the upper half facial sample image, the lower half facial sample image and the global facial sample image; fusing associated semantics by using a feature-level attention mechanism, so as to acquire the teacher model; supervising training of the teacher model by using a cross-entropy loss; and supervising training of the student model by using label distillation, KL divergence and a cross-entropy loss.

The invention discloses a medical training evaluation method and system based on an intelligent simulation patient, and relates to the field of artificial intelligence technology and medical simulation, and the method comprises a virtual patient model construction step, a multi-modal perception step, an intelligent interaction step and an intelligent evaluation step. By integrating voice recognition, natural language understanding, facial expression recognition, gesture recognition and virtual reality technologies, a high-simulation and high-interactivity intelligent patient model is constructed for medical students or clinical medical staff to perform diagnosis and treatment training and skill operation assessment. The virtual patient model supports custom information such as age, gender, race, medical history, living habits and the like, different virtual patient images and different pathophysiological states are generated, and personalized learning requirements of users are met. In addition, targeted training suggestions are provided, the training difficulty and content can be adjusted according to the performance of the user, and different training requirements are met.

A facial expression recognition method based on grid attention and pyramid segmentation attention belongs to the technical field of deep learning image processing and emotion calculation, and comprises the following steps: introducing a grid attention mechanism and a pyramid segmentation attention mechanism on a ResNet101 large model, capturing local expression detail features through a grid attention module, establishing multi-scale global feature association by using a pyramid segmentation attention mechanism; hierarchical segmentation is carried out on the backbone network, and information from multiple hierarchies is effectively fused; and dynamically balancing the contribution degree of each level of features through learnable parameters, and integrating the extracted features to judge the expression category. Experiments show that the model achieves the recognition accuracy superior to that of a traditional model on a public data set in natural scenes such as complex illumination and posture change. According to the method, a solution with high robustness is provided for facial expression recognition in a complex environment, and the method has important application value in the fields of intelligent human-computer interaction, mental health assessment and the like.

This invention belongs to the field of facial expression recognition and provides a micro-expression recognition method based on convolutional neural networks and optical flow features, which is used to improve the adaptability and accuracy of micro-expression recognition. The method first constructs a face detection module to perform face detection on each image frame in a video segment, outputs facial key point information, and extracts facial regions of interest (ROIs) based on the facial key point information: nose, mouth, left eye and eyebrow, right eye and eyebrow. Then, optical flow information and optical strain information are introduced to characterize the spatiotemporal information of facial movement and the intensity information of facial deformation. The optical flow eigenvalue of each image frame is calculated, and a binary search algorithm is used to search for the image frame with the maximum optical flow eigenvalue, which is used as the vertex frame for micro-expression recognition. Finally, a micro-expression recognition model based on a convolutional neural network is constructed, which uses the horizontal and vertical components of the optical flow and optical strain of the vertex frame as input. The micro-expression recognition model outputs a micro-expression category prediction result.

The invention discloses a high-precision facial expression emotion detection method based on improved YOLOv8n, which optimizes a facial expression recognition task while retaining a basic framework of the YOLOv8n, and comprises the following steps of: firstly, replacing a standard convolution module in a backbone network and a neck structure of the YOLOv8n with a self-adaptive multi-branch convolution module; a large-scale separable nuclear attention module is integrated to the seventh layer and the tenth layer of a YOLOv8n backbone network, dynamic attention on key facial features is enhanced, interlayer feature injection connection is applied to the YOLOv8n backbone network and a neck structure, cross-layer connection between the sixth layer and the seventeenth layer and cross-layer connection between the ninth layer and the nineteenth layer are established, and therefore the dynamic attention on the key facial features is enhanced. By establishing the direct connection between the shallow and deep features, the information loss is reduced, the multi-scale feature fusion is optimized, and the precision and efficiency of facial expression emotion detection are significantly improved.

In a facial expression recognition method, facial key points are identified as graph nodes. A facial graph structure for a first image is constructed with edges between pairs of the graph nodes based on relationships between the facial key points corresponding to the graph nodes. A first feature of a facial texture is extracted from color information of pixels in the first image. A second feature of the first image is extracted by processing the facial graph structure using a graph neural network (GNN). The first feature and the second feature are combined, to obtain a fused feature. A first expression type of a face in the first image that corresponds to the fused feature is determined. The first expression type is determined from a plurality of facial expression types.

The invention discloses a sign language synthesis service method based on multi-modality, and the method comprises the following steps: S10, carrying out end-cloud collaborative rendering setting: deploying an edge end as a lightweight model to generate a basic action, and deploying a cloud end as operating a MoMask and 3D rendering engine; s20, performing multi-modal data acquisition: acquiring a voice signal through a microphone, acquiring a 48 * 48 pixel face grayscale image through a camera, and acquiring action posture data through an IMU sensor; s30, performing emotion feature extraction on the collected multi-modal data: performing voice emotion recognition to output six types of emotion probability distributions, and performing facial expression recognition to output seven types of expression probability distributions; s40, performing cross-modal feature fusion: dynamically fusing the voice and facial features based on a confidence weighting strategy, and generating a three-dimensional emotion intensity vector; and S50, sign language action generation is carried out, and a 3D skeleton sequence matched with emotion is generated through RVQ layering quantification and MoMask Transformer.

The invention provides a classroom student concentration degree monitoring system and method based on computer vision. The system comprises a vision collection module, an edge calculation unit and a cloud analysis platform. A multi-dimensional sensing mode combining head posture analysis, eye state tracking, facial expression recognition and body movement detection is adopted to evaluate the concentration degree, through a dynamic feature fusion mechanism, the system can intelligently adjust the weight ratio of different modal features, the limitation of single-dimensional analysis in a traditional method is effectively solved, and the accuracy of the system is improved. The system adopts edge calculation and cloud analysis, ensures high efficiency of data processing, ensures reliability of complex algorithm operation, and has important value for improving classroom teaching quality and learning efficiency. According to the invention, through multi-modal visual analysis, behavior characteristics and physiological reactions of students in class are monitored in real time, the concentration level of the students is comprehensively evaluated, and objective teaching feedback and improvement basis are provided for teachers.

Embodiments of the present disclosure disclose a dynamic facial expression recognition method, a device, and product. The method includes: segmenting a video to be recognized into video segments; extracting and aggregating, for each video segment, features of face image frames in the video segment by using a 3D convolutional network, to obtain target feature data of the video segment; aggregating target feature data of the plurality of video segments, to obtain target feature data of the video to be recognized; and determining an expression label of the video to be recognized based on the target feature data of the video to be recognized, to obtain an expression label recognition result of the video to be recognized.

The invention provides a facial expression recognition system and method based on a multi-dimensional mixed attention mechanism, and relates to the technical field of facial data processing, and the method comprises the steps: obtaining a human face image; inputting the preprocessed facial image into a facial expression recognition model, and outputting to obtain a facial expression category; wherein the facial expression recognition model is an enhanced MobileNet V3 neural network based on a feature pyramid and a mixed attention mechanism, after a facial image is subjected to depth separable convolution operation, multi-scale features of the facial image are extracted through the feature pyramid network, high-level semantic features and low-level detail features are obtained and fused, and the facial expression recognition model is obtained. And through a mixed attention mechanism of frequency domain and time domain fusion and space channel fusion, guiding a model to pay attention to effective features in an image by analyzing energy distribution in a frequency domain and extracting space information and channel information, obtaining a final feature vector, and realizing facial expression classification based on the final feature vector.

The invention discloses a multi-mode mental health assessment system based on artificial intelligence, and belongs to the technical field of mental health assessment. A multi-modal mental health assessment system based on artificial intelligence comprises a data acquisition module, a multi-modal fusion module and a health assessment module. The psychological health assessment method and device solve the problem that in the prior art, in the process of collecting user information, the collection dimension is single, so that the user information is not comprehensive, and then the accuracy of psychological health assessment is affected, and the accuracy of psychological health assessment is improved by integrating multi-source data such as facial expressions, voices, texts and videos. The method effectively improves the objectivity, accuracy and personalized intervention capability of evaluation, greatly improves the overall evaluation accuracy through the fusion of facial expression recognition, voice pressure signal detection, text emotion feature recognition and physiological parameter monitoring technologies, effectively reduces the single-modal error through multi-modal data complementation, enables the user information to be more comprehensive, and improves the user experience. And the accuracy of mental health assessment is improved.

The invention provides a facial expression recognition method based on multi-module collaborative optimization, which has an efficient network structure and can realize high-precision and strong-robustness facial expression recognition in a complex environment. The method comprises the following steps: receiving an RGB face image and preprocessing the RGB face image to obtain a preprocessed image; inputting the preprocessed image into a mixed feature network module, and outputting to obtain a first feature map; inputting the first feature map into an efficient local attention mechanism module, and outputting to obtain a second feature map; and inputting the second feature map into a classifier module to realize facial expression recognition.

The invention relates to the technical field of expression recognition, in particular to a virtual digital human expression recognition method and system based on multi-scale expansion convolution. The method comprises the following steps: preprocessing acquired image data; constructing a neural network model based on multi-scale expansion convolution based on the preprocessed data, and training the neural network model by using the preprocessed image data; applying the trained model to image processing, and outputting an expression recognition result; and driving the virtual digital human to generate an expression animation according to an identification result. According to the method, efficient extraction and fusion of multi-level expression features are realized through a multi-scale expansion convolution fusion attention module (MDFA) and a coordinate attention collaboration mechanism, and through global average pooling and a 1 * 1 convolution dimension reduction strategy, the recognition accuracy of the surprising expression with low-resolution input is improved by 7.8%.

The invention provides a GAN-based vehicle-mounted expression data construction method, system and device and a vehicle, and relates to the technical field of intelligent driving. A key point detection technology is utilized to analyze and extract a facial action unit and expression intensity and duration thereof to generate a structured action description. And then, calling a generative adversarial network model to generate a synthetic video corresponding to the target expression type, and carrying out dynamic cutting and frame extraction according to a preset interval on the videos to obtain a multi-frame synthetic image. And finally, performing dual-model filtering on the synthesized image to ensure data quality, and executing privacy anonymization processing, thereby generating a high-quality vehicle-mounted expression data set. According to the method, the data acquisition cost is reduced, the diversity and accuracy of a data set are improved, and the performance and reliability of an expression recognition algorithm in an actual vehicle-mounted environment are effectively enhanced.

The invention discloses a multi-modal cross-domain small sample facial expression recognition method based on relational distillation self-paced learning, and relates to a computer vision technology. Constructing a multi-modal semantic enhancement module, generating semantic descriptions of expressions by using a large language model, performing CLIP coding, and performing alignment and fusion with image visual features in the multi-modal semantic enhancement module to construct a multi-modal prototype; a self-paced learning mechanism based on relational distillation is designed, visual and semantic structural errors are calculated, and progressive training from easy to difficult is realized through a soft and hard mixed sample selection strategy and a mixed sample selection mechanism regulated and controlled by a dynamic threshold value. And cross-domain migration of emotional knowledge from basic expressions to fine-grained composite expressions can be effectively realized. And under the condition that only a small number of labeled samples are provided, rapid adaptation and accurate recognition of new expressions can be realized. The method is remarkably superior to a traditional supervised learning method, has higher practicability and expansibility, and can better meet the requirement for efficient recognition of new expressions in practical application.

The invention discloses a verification method and device based on facial expression recognition and a storage medium. Relates to the field of artificial intelligence, and the method comprises the steps: obtaining a face image of a target customer in a target transaction process of the target customer under the condition of obtaining the authorization of the target customer; extracting facial features from the facial image, inputting the facial features into the expression recognition model, and outputting an emotion category corresponding to the facial expression on the facial image; calculating a transaction risk value corresponding to the facial expression based on the emotion category corresponding to the facial expression; calculating a transaction risk value corresponding to the target transaction based on the transaction information of the target transaction; and determining a target verification mode according to the transaction risk value corresponding to the facial expression and the transaction risk value corresponding to the target transaction, and executing the target verification mode on the target customer. Through application of the method and the device, the problem of relatively low verification accuracy caused by relatively single verification mode for the client identity in the transaction process in related technologies is solved.

The invention relates to the technical field of cross-domain facial expression recognition, in particular to a cross-domain facial expression recognition method and system based on intelligent learning. A shared backbone network is adopted to extract global depth features of the face image, and a coarse-fine branch network is constructed to realize collaborative learning of attitude analysis and emotional interpretation; a coarse and fine feature interaction mechanism is introduced in the training process to relieve subdivision branch too fast convergence, and uncertainty distribution of attitude analysis and emotional interpretation is obtained through similarity measurement and a Gaussian mixture model; modeling a potential feature linear dependency relationship, performing multi-source domain feature alignment, inhibiting tag noise and extracting robust semantic features; and performing cross-domain coupling calculation on the label uncertainty distribution and the semantic features to generate a final recognition result of the cross-domain facial expression. According to the method, the influence of cross-domain difference and label noise of expression recognition from coarse attitude analysis to fine emotion interpretation can be effectively relieved, and the generalization ability of cross-domain facial expression recognition and emotion interpretation precision are improved.

The present invention belongs to the field of computer vision and discloses a facial expression recognition method based on an enhanced self-attention Transformer, which includes the following steps. Step 1: Obtain a facial expression training data set and perform preprocessing. Step 2: Establish a facial expression recognition network model composed of an IR50 convolutional neural network and an enhanced self-attention Transformer. Step 3: Perform preliminary feature extraction by the IR50 convolutional neural network and splice the features in its intermediate stage. Step 4: Input the spliced features into the enhanced self-attention Transformer model, and perform similar feature fusion and key feature screening in sequence. Step 5: Input the result after the Transformer execution into the fully connected layer to obtain the expression classification result. The present invention can improve the inference speed while ensuring the recognition accuracy.

The invention relates to a facial expression recognition method based on space channel convolution and enhanced compression incentive attention, which comprises the following steps: S1, expanding a receptive field by using the space channel convolution, and capturing global information of an image; s2, performing feature fusion on feature information of different scales and different levels; and S3, adaptively adjusting a feature mapping weight by using a global attention compression excitation module, and enhancing the attention of the model on important features. According to the invention, the space channel convolution can enhance the perception capability of multi-scale and cross-level features; the feature fusion module integrates feature information of different scales and hierarchies in a channel dimension, and realizes optimal combination of the feature information through an adaptive weight acquisition mechanism; the global attention compression excitation module can relieve the information loss problem and improve the recognition accuracy. According to the method, the accuracy of facial expression recognition is remarkably improved, meanwhile, the calculation cost and the network scale are far smaller than those of the prior art of the same type, and higher accuracy and robustness are achieved.

The application discloses a self-supervised facial expression recognition method combining contrast learning and mask image modeling, and the method comprises the following steps: constructing a self-supervised facial expression recognition model combining contrast learning and mask image modeling; performing unsupervised pre-training on the model; performing linear probe evaluation and fine-tuning evaluation on the pre-trained model on a labeled verification set; using the fine-tuned model to perform facial expression recognition on an input facial image; the method learns facial expression representation through a convolution-free twin network, pre-trains the twin network by using contrast loss and mask image modeling loss, so as to simultaneously understand high-level visual semantics and image internal structure, and maximize the consistency between the outputs of the student network and the teacher network. The application can learn rich visual information, is robust to various interference unrelated to facial expression, and simultaneously achieves excellent results in linear probe and fine-tuning evaluation on a facial expression dataset.

The invention discloses an expression recognition method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring an image pair comprising a target user face; the image pair comprises a natural expression image of the target user and a to-be-recognized image; performing fusion processing on the image pair, and inputting the image pair after fusion processing into a trained expression recognition model to obtain an expression recognition result of the to-be-recognized image generated by the expression recognition model; wherein the expression recognition model comprises an expression change extraction module and a decoder; the expression change extraction module is used for extracting target expression change features of the fused image pair; and the decoder is used for outputting an expression recognition result based on the target expression change feature. According to the embodiment of the invention, inherent differences of different individuals in emotional expression are fully considered, and the problem of identification deviation caused by neglecting individual expression characteristics in a traditional expression identification method is reduced, so that the accuracy of expression identification is improved.

The invention discloses a risk control identity authentication system and method based on facial expression recognition, and relates to the technical field of image recognition. The system comprises an identity information acquisition module, a facial feature extraction module, a dynamic feature extraction accuracy analysis module, a multi-dimensional comparison module and a multi-dimensional comparison rationality analysis module. Dynamic feature extraction accuracy evaluation is obtained through feature extraction parameter quantification, whether dynamic feature extraction accuracy optimization is carried out or not is judged, and if yes, a facial expression feature comparison link is carried out after optimization; otherwise, a facial expression feature comparison link is directly carried out, multi-dimensional comparison parameter quantification is obtained to obtain multi-dimensional comparison rationality evaluation, and whether multi-dimensional comparison rationality optimization is carried out is judged, so that the risk control identity authentication reliability is improved, and the problem that in the prior art, the risk control identity authentication efficiency is high is solved. The problem of low risk control identity authentication reliability caused by inaccurate expression dynamic feature extraction exists.

The invention provides a deep psychological support intelligent device based on a multi-mode technology. The equipment integrates facial expression recognition, sound and intonation recognition, psychological professional knowledge, deep semantic understanding, multi-modal emotion and physiological signal capture, virtual reality (VR) and augmented reality (AR) technologies, block chain data privacy guarantee, biofeedback technology and other frontier technologies, and has self-learning and optimization functions. The working mode of a professional real psychological counselor can be simulated, and deep psychological support and personalized psychological counseling service are provided for the user.

The present disclosure provides a method, an apparatus and a system for adaptively regulating a surrounding temperature of an area using facial expression recognition, the method comprising: computing a core affective state level of a person in the area based on a detected facial expression of the person, wherein the core affective state level comprises a valence level and an arousal level of the person; determining if the core affective state level matches a target core affective state level set for the area; and in response to determining that the core affective state level does not match the target core affective state level set for the area, deriving an adjustment to the surrounding temperature based on a first adjustment required to match the core affective state level to the target core affective state level.

The application provides a network model and method for improving the facial expression recognition accuracy in a video, the method comprising: inputting an initial video image, and performing feature extraction through a 3D convolution network; fusing an AU perception attention module in the 3D convolution network, paying attention to the key area of facial emotional expression through the AU perception attention module, and learning the features of the key parts of the face; encoding the obtained feature map through a capsule network, and encoding the enhanced features through the dynamic routing between the capsules; decoding through three fully connected layers, and realizing the final expression classification through a nonlinear squeeze function. The application realizes higher-precision facial expression recognition.