103 results about "Facial expression recognition" patented technology

The embodiment of the invention provides a model training method, a facial expression recognition method and device and electronic equipment, and relates to the technical field of video processing. The model training method comprises the following steps: acquiring a sample video and a first sample label; extracting a time feature and a space feature of the sample video by using a time-space feature extraction network in the facial expression recognition model of the initial structure; calculating an attention weight representing the correlation between the spatial feature and the time feature by using a mapping network; performing weighted aggregation on the time features by using the attention weight to obtain fused spatio-temporal features; inputting the fused spatial-temporal features into a classification network to obtain a first recognition result; and performing model training based on the difference between the first recognition result and the first sample label to obtain a trained facial expression recognition model with higher accuracy.

The invention discloses a multi-modal cross-domain small sample facial expression recognition method based on relational distillation self-paced learning, and relates to a computer vision technology. Constructing a multi-modal semantic enhancement module, generating semantic descriptions of expressions by using a large language model, performing CLIP coding, and performing alignment and fusion with image visual features in the multi-modal semantic enhancement module to construct a multi-modal prototype; a self-paced learning mechanism based on relational distillation is designed, visual and semantic structural errors are calculated, and progressive training from easy to difficult is realized through a soft and hard mixed sample selection strategy and a mixed sample selection mechanism regulated and controlled by a dynamic threshold value. And cross-domain migration of emotional knowledge from basic expressions to fine-grained composite expressions can be effectively realized. And under the condition that only a small number of labeled samples are provided, rapid adaptation and accurate recognition of new expressions can be realized. The method is remarkably superior to a traditional supervised learning method, has higher practicability and expansibility, and can better meet the requirement for efficient recognition of new expressions in practical application.

The invention relates to the technical field of cross-domain facial expression recognition, in particular to a cross-domain facial expression recognition method and system based on intelligent learning. A shared backbone network is adopted to extract global depth features of the face image, and a coarse-fine branch network is constructed to realize collaborative learning of attitude analysis and emotional interpretation; a coarse and fine feature interaction mechanism is introduced in the training process to relieve subdivision branch too fast convergence, and uncertainty distribution of attitude analysis and emotional interpretation is obtained through similarity measurement and a Gaussian mixture model; modeling a potential feature linear dependency relationship, performing multi-source domain feature alignment, inhibiting tag noise and extracting robust semantic features; and performing cross-domain coupling calculation on the label uncertainty distribution and the semantic features to generate a final recognition result of the cross-domain facial expression. According to the method, the influence of cross-domain difference and label noise of expression recognition from coarse attitude analysis to fine emotion interpretation can be effectively relieved, and the generalization ability of cross-domain facial expression recognition and emotion interpretation precision are improved.

The invention discloses an expression recognition method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring an image pair comprising a target user face; the image pair comprises a natural expression image of the target user and a to-be-recognized image; performing fusion processing on the image pair, and inputting the image pair after fusion processing into a trained expression recognition model to obtain an expression recognition result of the to-be-recognized image generated by the expression recognition model; wherein the expression recognition model comprises an expression change extraction module and a decoder; the expression change extraction module is used for extracting target expression change features of the fused image pair; and the decoder is used for outputting an expression recognition result based on the target expression change feature. According to the embodiment of the invention, inherent differences of different individuals in emotional expression are fully considered, and the problem of identification deviation caused by neglecting individual expression characteristics in a traditional expression identification method is reduced, so that the accuracy of expression identification is improved.

A vehicle is provided that can mitigate the effects of high ambient noise levels. Using image data and / or other sensor data, the vehicle can determine when the convertible top is in use. The vehicle also determines when the ambient noise level inside the vehicle exceeds a predefined threshold. The vehicle then switches its operating mode to clearly receive and interpret audio input. The vehicle can activate a lip-reading mode and / or a gesture and facial expression recognition mode to identify words spoken by a user. The vehicle can also store mapping information between a gesture and a corresponding verbal command associated with the vehicle. In noisy environments, the vehicle may suggest using gestures / facial expressions instead of verbal commands.

The invention discloses a facial expression detection method based on a Transform dual-channel network and related equipment, and can be applied to the technical field of deep learning. Static features of a video sequence to be processed are extracted through a video sequence feature extractor to form a video sequence feature map carrying global information and local information, and dynamic features of an optical flow sequence to be processed are extracted through an optical flow sequence feature extractor to form an optical flow sequence feature map. Fusing features of different levels in the video sequence feature map through a multi-level feature fusion module to obtain multi-level static fusion features, and carrying out feature refinement on the optical flow sequence feature map through a self-adaptive convolution attention module to obtain space channel refinement dynamic features; the two features are fused through a cross attention fusion module to obtain a cross fusion feature, and then facial expression recognition is performed according to the cross fusion feature, so that the accuracy of expression detection can be effectively improved.

The present application relates to a kind of facial expression recognition methods based on attention mechanism and self-distillation, comprising the following steps: step 1, constructing facial expression recognition dataset;Step 2, constructing facial expression recognition model, the model is composed of feature extraction module, adaptive channel attention module, remodeling module and self-distillation network;Step 3, the input facial expression image is preprocessed;Step 4, using the training set image in the above facial expression recognition dataset trains facial expression recognition model;Step 5, using the facial expression recognition model trained to the test set and validation set in dataset are inferred classification.The method designs a kind of reasonable and efficient facial expression recognition scheme, it is well guided model to pay attention to those important feature information by the mode of attention mechanism;By a new self-distillation form, refine compressed network knowledge, improve the robustness of shallow network output feature and reduce the complexity of model in inference stage.

The application provides an expression recognition model training and expression recognition method and device, electronic equipment and storage medium. The expression recognition model training method comprises obtaining a first expression representation model and a second expression representation model through self-supervised pre-training; fine-tuning the first expression representation model to obtain a first expression recognition model, and performing knowledge distillation on the first expression representation model based on the first expression recognition model to obtain a second expression recognition model; performing knowledge distillation on the second expression representation model based on the second expression recognition model to obtain a third expression recognition model. Through self-supervised pre-training, a large amount of unlabeled sample data is used to reduce the cost of manual labeling. Through self-distillation on the same expression representation model, the accuracy of the second expression recognition model is improved. And through knowledge distillation of the second expression representation model based on the second expression recognition model, the accuracy of the third expression recognition model is further ensured.

This invention discloses a neurophysiological behavior-driven multimodal intelligent learning guidance system and method. The system achieves closed-loop control by constructing a cognitive-emotional-task adaptive adjustment space: 1) In the cognitive dimension, it integrates EEG neural oscillations and frontoparietal network connections, fNIRS cerebral oxygen metabolism indicators, and eye-tracking data to construct a multimodal cognitive load assessment model; 2) In the emotional dimension, it combines the nonlinear characteristics of skin conductance signals with facial expression recognition to quantify learners' motivation levels and emotional valence in real time; 3) In the task dimension, it parameterizes task difficulty and interaction form as moderating variables. During system operation, the system locates the learner's position on the cognitive-emotional state plane in real time and dynamically adjusts task dimension parameters through a nonlinear mapping strategy. This invention effectively utilizes the decoupling ability of multimodal signals and the interaction of intention to achieve precise adaptive delivery of teaching content.

The invention provides a facial expression recognition method based on multi-level feature extraction and fusion in a natural environment. The method is characterized by comprising the following steps: (1) a feature extraction module: carrying out convolution calculation by using three different convolution kernel scales and dense blocks with DenseNet as a backbone network to generate facial expression feature maps (Feature Maps) at a low level, a middle level and a high level; different dense blocks are connected through transition layers; (2) a feature fusion module which performs global attention and local attention processing on the processed high-level feature map and low-level feature map after element addition calculation so as to generate a fused facial expression feature map; and (3) optimizing a strategy: adopting a strategy of combining a smooth label (Label Smoothing) and L2 regularization to avoid over-fitting.

The invention discloses a multi-view evidence fusion large-angle facial expression recognition method and system, and the method mainly comprises the steps: directly transmitting a read image into a face detection model based on a YOLOv8 architecture, returning all detected face targets in the image, extracting key information, and calculating the in-plane rotation angle of a face; when the absolute value of the rotation angle exceeds a threshold value, activating the GAN model to generate a view 2, and cutting a face region from the original image by using the returned face target all the time regardless of the rotation angle to serve as a view 1; extracting the view 1 and the view 2 in parallel to obtain two groups of independent evidence vectors, and calculating subjective opinions corresponding to the evidence vectors; if the two views are processed, evidence fusion is started, otherwise fusion is not needed, an index of the maximum value is searched in a final belief vector to determine the most credible expression category, and final uncertain expressions are output together. Compared with any single view or simple fusion method, the method provided by the invention can obtain more accurate and more stable recognition performance under various attitude angles.

The invention discloses an intelligent accompanying robot based on visual perception and recognition and a control method, and the robot comprises a visual perception and recognition module which is used for collecting indoor and outdoor environment images in real time, and constructing a three-dimensional environment map; verification and visual positioning of identity recognition of the elderly; recognizing a corresponding emotional state through a facial expression recognition algorithm; recognizing obstacles in the environment through the deployed target detection model; the real-time accompanying control module is used for starting real-time identification and accompanying based on the three-dimensional environment map and the visual positioning of the elderly, and dynamically adjusting following parameters according to the moving state of the elderly to avoid obstacles in the environment; the emotional accompanying and interaction module is used for calling a preset emotional interaction strategy to realize emotional accompanying after visually identifying the emotional state of the elderly; the method has the advantages that visual perception and recognition serve as the core, gait recognition and emotion accompanying and interaction are adopted, and accompanying precision and service initiative are improved.

Facial expression recognition method and apparatus via label distribution learning are disclosed. The facial expression recognition method through label distribution learning, comprising: (a) in the training process, preprocessing an input sample to generate a plurality of augmented samples, creating a target label distribution for the input sample using the plurality of augmented samples, and training a model using supervised learning; and (b) after the training is completed, during the inference process, outputting a facial expression recognition result by inputting a single facial sample into the trained model without using the augmented samples.

The system according to the embodiment aims to improve the accuracy of incident detection while ensuring the privacy of users. [Solution] A system according to an embodiment includes a collection unit, a monitoring unit, a recognition unit, a determination unit, and a notification unit. The collection unit collects biometric information. The monitoring unit monitors the user's movements or facial expressions based on the biometric information collected by the collection unit. The recognition unit detects abnormalities detected by the monitoring unit and recognizes conversations based on the abnormalities. The determination unit determines whether or not an incident has occurred based on the conversation recognized by the recognition unit. The notification unit notifies the user's family of the results of the incident determined by the determination unit.

The application discloses a kind of light driver facial expression recognition methods based on multi-scale convolutional neural network, which realizes face positioning and alignment by multi-task convolutional neural network;Then the enhanced facial image is transmitted into the improved multi-scale convolutional neural network to recognize human face expression;The present application is different from the traditional expression recognition method based on convolutional neural network, a novel multi-scale residual attention module is designed, and a light and efficient network is constructed, the driver's facial image is non-contact to identify the driver's expression state and take corresponding measures, realize safe and intelligent driving, and the method extracts facial image features from multiple scales on one hand, effectively improves the accuracy of facial expression recognition;On the other hand, light network is conducive to deployment in car intelligent cockpit domain controller.

The invention discloses a video facial expression recognition method based on a fuzzy time refinement network, is applied to the technical field of computer vision and artificial intelligence, and aims to solve the problems that in the prior art, the expression time sequence dynamic state is difficult to capture, the processing feature fuzziness is low, and the refinement category recognition precision is low. The method comprises the following steps: preprocessing video frames; extracting spatial features by adopting ResNet-18 (ResNet-18); modeling inter-frame time sequence dependence through LSTM; dynamically weighting key frame features by using a fuzzy attention module (FAN) in combination with Gaussian, triangle and Sigmoid membership functions; carrying out the optimization of feature clustering through the combination of cross entropy loss and TriCenter Loss; outputting an expression category; by adopting the method, the video facial expression recognition accuracy can be effectively improved.

This invention discloses an AI-based physical education exam scoring method, belonging to the field of exam scoring methods, including the following steps: (1) activating the cameras deployed in the exam room to collect motion images and facial expression images of the examinee during pull-ups; (2) analyzing the collected facial expression images using AI image recognition technology to divide the exam process into three stages; (3) for each stage, analyzing the examinee's motion images using AI technology to determine whether each pull-up motion is a qualified motion; (4) counting the number of qualified motions in each stage and scoring them separately, adding the scores of the three stages to obtain the examinee's final exam score. This invention can divide the exam into three stages based on facial expression recognition, perform differentiated analysis on the motion characteristics of each stage, better fit the examinee's physical fitness change patterns, and make the evaluation results more targeted.

The application discloses an expression recognition method and system based on cross-scale local difference deep subspace features, which obtains images in different scale spaces through a plurality of Gaussian filters, then blocks the images in different scale spaces, extracts local differences of the images, thereby obtains a cross-scale local difference matrix of a training set, trains a plurality of convolution kernels in a first stage, and extracts again a cross-scale local difference matrix of the images after the first stage of convolution, trains a plurality of convolution kernels in a second stage, thereby learns the convolution kernels in the two stages. After the image to be recognized is convolved in the two stages, the image features are obtained through nonlinear processing and histogram statistics, so that classification and recognition are carried out. The method fuses local differences of different scales of images to extract features, and the network structure is simple, a large number of training samples are not needed, and the hardware requirement is low.

The invention relates to the technical field of robot control, in particular to a comfort assessment method based on multi-modal data, which comprises the following steps: acquiring multi-modal data of a target monitoring user in a robot massage scene, and preprocessing the multi-modal data; performing expression recognition on the face region image features according to a facial expression recognition model; recognizing the fusion features according to a speech emotion recognition model; analyzing the pressure data according to the force perception model; performing basic probability assignment on the facial expression modal features, the final voice emotion modal features and the pressure modal features based on a user comfort discrimination framework; fusing the maximum historical frame recognition result and the pressure basic probability assignment according to a Dempster synthesis rule; and carrying out comparative analysis on the support degree of the comfort state of the user in the Dempster-Shafer evidence theory fusion result and a preset threshold value. According to the invention, multi-modal information and user emotion feedback can be fully utilized, and the accuracy of the comfort of the user in a massage scene is remarkably improved.

The present application relates to the cross field of target detection and emotion recognition, and particularly relates to a facial expression recognition method for autistic children based on deep learning. The method comprises the following steps: embedding a three-dimensional attention (Tri-Attention) after a C3k2 module of a YOLOv11 backbone network to model the interaction relationship among height, width and channel dimensions in parallel, and enhancing the response to key expression regions such as eyes and corners of the mouth; introducing the three-dimensional attention (Tri-Attention) again at the multi-scale feature output end of the neck network to realize effective positioning of small-scale faces in a complex background and suppression of background noise; training and verifying the YOLOv11-TA model by using a public data set and a self-built autistic children data set respectively, and outputting the detection frame and expression category of each face; and comparing various indexes of the original YOLOv11 and the YOLOv11-TA on multiple data sets, and actually verifying that the method significantly improves the accuracy and robustness of the expression recognition of autistic children.

The invention discloses a multi-source data fusion hospitalized patient psychological state intelligent monitoring system, which relates to the technical field of health monitoring and comprises a data acquisition module, a multi-source data fusion module, an environment correction module, a psychological state evaluation module and a predictive analysis module. According to the method, multi-source data of electrocardiogram, electroencephalogram, facial expression recognition and speech emotion analysis are integrated, so that comprehensive and objective assessment of the psychological state of the patient is realized, and the accuracy and reliability of psychological state recognition are remarkably improved; according to the method, environmental parameters of noise and illumination are collected in real time, the comprehensive feature vector is dynamically corrected in combination with a treatment program feature value, interference of the special environment of a hospital on a monitoring result is greatly reduced, in addition, a psychological state reference model based on a convolutional neural network and an LSTM deep learning prediction algorithm are adopted, and the monitoring accuracy is improved. Not only can the current psychological state be accurately identified, but also psychological crisis can be early warned by analyzing the historical data trend.

The invention provides an expression recognition method based on facial region prior guidance. The method comprises the following steps: acquiring a face image data set containing expression category labels; the method comprises the following steps: constructing an expression recognition model based on facial region prior guidance, wherein the model comprises a data processing layer, a two-way token generation module, a sequence fusion and position coding layer, a hierarchical prior guidance Transform encoder module, a global feature fusion layer and a classification layer; using samples in the facial expression data set to train an expression recognition model based on facial area prior guidance; and performing facial expression recognition on a newly input test picture by using the trained model, and outputting an expression category. According to the invention, through a layered learnable attention guiding mechanism, the prior depth of the face area is fused into the model reasoning process, and the recognition accuracy and robustness in a complex scene are significantly improved.

The system according to the embodiment aims to provide real-time support to people who have difficulty controlling their emotions. [Solution] A system according to an embodiment includes a detection unit, an advice unit, and an intervention unit. The detection unit detects the emotion of a subject using voice recognition or facial expression recognition. The advice unit provides advice based on the emotion detected by the detection unit. The intervention unit notifies the subject of the advice provided by the advice unit in real time.

The invention relates to the field of facial expression recognition methods, in particular to a vehicle driving state detection method based on facial expressions, which comprises the following steps: step 1, acquiring acquired driver images and voice information in a driving process; 2, text features in the voice information are extracted through a pre-trained and parameter-frozen large language model, and facial expression features in the driver image are extracted through a singular value decomposition feature fusion model; 3, performing low-rank decomposition on the facial expression features obtained in the step 2 through a low-rank expert model to obtain visual features, and judging whether a driving emotion state is an extreme emotion or not from the visual features and the text features; 4, if the emotion is not the extreme emotion, continuing monitoring, and if the emotion is the extreme emotion, triggering an intervention process; and 5, judging whether the emotion is relieved or not again after the intervention process is triggered, if so, continuing monitoring, and if not, performing vehicle flexible control. According to the invention, the calculation complexity is reduced and the processing efficiency is improved.

The present application relates to a kind of lightweight facial expression recognition method and system based on attention mechanism, comprising the following steps: establishing convolution model, the picture of data set is cropped, and the picture is preprocessed, and the picture after pre-processing is input into convolution model;Picture is extracted in convolution model, attention mechanism is recalibrated and down-sampling operation is carried out, and the feature map of final output is obtained;Vector in feature map is classified into expression, and recognition result is obtained;Establish loss function model, use recognition result to train model parameter and test, complete facial expression recognition.Ghost convolution in GhostNet network is introduced to reduce the parameter amount of pointwise convolution, in order to eliminate the interference of expression irrelevant factors, the improved coordinate attention mechanism can simultaneously focus on the position information and context information of expression image, increase the weight of key area of facial expression image, improve the performance of identification, compared with conventional expression recognition method, model parameter amount is less, and recognition accuracy is higher.

The invention provides an online live broadcast interaction system based on AI glasses, and relates to the technical field of online live broadcast, and the system comprises the AI glasses and a live broadcast background. The AI glasses and the live broadcast background carry out data transmission through Wi-Fi and Bluetooth 5.0, and the live broadcast background synchronizes online live broadcast interaction information to the AI glasses in real time; the AI glasses upload the collected anchor voice information, gesture operation data, eye movement tracking data and facial expression data to a live broadcast background; multi-dimensional real-time interaction between an anchor and fans is achieved through cooperation of the AI glasses and the live broadcast background, the system integrates four interaction modules of voice, gestures, eye movement tracking and facial expression recognition, the limitation of a single interaction mode of traditional live broadcast is broken through, a user can complete instruction input through natural actions, the interaction intuition and immersion are improved, and the user experience is improved. The anchor can trigger the specific operation through eye staring or quickly respond to the interaction requirement through the custom gesture, and the operation delay is reduced.

This invention belongs to the field of image classification in computer vision, specifically relating to a facial expression recognition method based on the fusion of visual Transformer and convolutional network. The method has the following features and includes the following steps: Step 1, preprocessing the image to be trained to obtain a preprocessed image; Step 2, inputting the preprocessed image into a model based on the fusion of visual Transformer and convolutional network for training, thereby obtaining the model's weight file. This model includes a convolutional module, an encoder, and an attention mechanism. The convolutional layer includes associating the positional information of image features; the encoder includes multiple residual modules, which use the encoder's input and final output as the encoder's final output. The encoder consists of multiple residual modules, and these outputs are used as inputs for the fusion attention mechanism; the fusion attention mechanism uses the output of the pooling layer as input to the attention mechanism module, and uses an adaptive attention mechanism to find different weight responses in the features of the input feature map, finally inputting it into the visual Transformer for training; Step 3, loading the model weight file, inputting the test facial expression image into the model to obtain the expression prediction result. Furthermore, the facial expression recognition cutting model of the present invention can better separate features between different categories, thereby improving the accuracy of the expression recognition model prediction.

The invention discloses a visual SOP management system, which belongs to the field of computer systems, integrates three dimensions of facial expression recognition, voice emotion analysis and operation fluency detection, realizes quantitative scoring of the confusion degree of an operator through a multi-modal fusion analysis model, and is higher in recognition accuracy compared with single data monitoring. Abnormal states such as hesitation and lagging can be accurately captured; when the confusion degree reaches a preset threshold value, three similar operation case videos with the highest matching degree are automatically called and pushed according to a standard operation priority sequence, manual intervention is not needed, the response speed is greatly improved compared with traditional manual guidance, meanwhile, case matching is combined with an SOP step number, the operation scene similarity and the difficulty coefficient, and the operation efficiency is improved. The matching accuracy is guaranteed through a cosine similarity algorithm, key corresponding nodes are marked at the same time, operators are helped to quickly understand standard operation key points, the learning cost is reduced, and the working efficiency is improved.

The application provides a marketing video effect evaluation method combining eye movement tracking and emotion recognition, and relates to the technical field of marketing effect evaluation, and is characterized in that the method comprises the following steps: multi-modal data synchronous acquisition, eye movement data processing and visual attention area extraction, facial expression recognition and emotion state analysis, multi-modal data space-time alignment and feature fusion, multi-dimensional evaluation index calculation, video segment level effect evaluation, comprehensive effect evaluation, generation of an evaluation report and optimization suggestions.The application has the following advantages: multi-dimensional evaluation indexes are constructed, the marketing video effect can be more objectively, comprehensively and accurately evaluated, multi-dimensional, fine and objective evaluation of the marketing video effect is realized, targeted and personalized optimization suggestions are given and an effect improvement value is predicted, and a scientific basis is provided for optimization and iteration of the marketing video.

This invention relates to the field of computer vision and discloses a facial expression recognition method based on enhanced action units-guided causal inference. The method includes extracting high-dimensional semantic feature maps from the original image; performing reparameterized sampling to output a sequence of local visual feature blocks; calculating the geometric relationship between the feature sequence and facial key points to generate a position-enhanced feature sequence with superimposed embeddings; predicting the activation intensity and uncertainty of action units, and combining a static prior association matrix to perform causal intervention to generate a corrected feature sequence and a counterfactual feature sequence; and aggregating the corrected sequences to output the expression classification result. By dynamically adjusting the sampling distribution, the method focuses on high-discriminative regions and reduces noise; combines spatial structure and uncertainty estimation to suppress low-confidence features; and utilizes prior knowledge to perform causal inference to remove spurious correlations, verify causal sufficiency, and improve the model's generalization performance in complex scenarios.