88 results about "Expression Feature" patented technology

The invention discloses a doctor-patient speech communication model training method and system based on multi-modal corpus analysis, and belongs to the crossing field of artificial intelligence and medical treatment, and the method comprises the steps: carrying out the extraction according to an OpenPose algorithm to obtain motion features, carrying out the muscle activity intensity detection to obtain expression features, carrying out the speech recognition to obtain text features, and carrying out the recognition of the text features; speech segmentation is carried out based on the time domain energy parameters, and a segmentation result is subjected to modal analysis to obtain acoustic features; performing feature fusion based on an attention mechanism to obtain fusion features, and inputting an obstacle recognition model to obtain an obstacle type; the method comprises the steps of obtaining an obstacle type, obtaining an intervention strategy and an intervention identity according to a solution mapping relation, taking the obstacle type, the intervention strategy and the intervention identity as multi-dimensional labels, carrying out time domain alignment and structured packaging to obtain target data, and training according to the target data to obtain a communication model used for providing dialogue prompt information. Obstacle recognition accuracy can be improved, and the intelligent agent application effect can be improved.

A methodology for auditing the visual diversity of unlabeled human face image datasets uses a set of core human interpretable dimensions derived from human similarity judgments. Given a face image, a model can output dimensional values aligned with the human mental representational space of faces, where values not only express the presence of a feature, but also its extent. Since the model can be learned entirely from human behavior, the learned dimensions are not biased toward features that are easier to verbalize or quantify.

The invention relates to the technical field of image processing, and discloses a facial health state assessment method and system based on layered optical modeling. The objective of the invention is to solve the problems of poor face health state evaluation precision and robustness caused by rough optical modeling, mixed reflection components, weak environmental adaptability and insufficient multi-dimensional health index fusion capability in the prior art. The method comprises the following steps: synchronously acquiring a user face video sequence comprising at least two different spectral bands through a multispectral imaging device; constructing a multilayer optical transmission model of the skin tissue, and separating a specular reflection component and a diffuse reflection component from the face video sequence; extracting a physiological time sequence signal related to subcutaneous blood flow activity from the separated diffuse reflection component, and calculating heart rate, heart rate variability and blood oxygen saturation parameters; and fusing the physiological parameters and expression and micro-expression features extracted from the facial video sequence, and outputting quantitative evaluation results of fatigue degree, pressure level and emotional state through a pre-trained deep neural network model. The system comprises a multispectral image acquisition module, a layered optical modeling module, a reflection component separation module, a physiological parameter inversion module and a health state evaluation module. According to the technical scheme, the signal-to-noise ratio and stability of physiological signal extraction in a complex illumination environment can be remarkably improved, the adaptability to individuals with different skin colors and dynamic illumination conditions is enhanced, and more accurate and more robust judgment of the high-order health state is achieved.

The invention provides an emotional feature recognition system and method for classroom teaching. The method comprises the following steps: acquiring facial image data and behavior time sequence data of a student in a classroom teaching process through a camera after the student clearly knows and agrees to authorize; determining dynamic expression features of the student through a preset facial analysis model according to the facial image data; on the basis of a head posture change sequence and an eye fixation focus track extracted from behavior time sequence data, constructing concentration indexes and confusion indexes of students in different teaching links through multi-feature fusion; and when it is detected that a preset condition is met, generating an emotional feature tag of the student in classroom teaching based on the dynamic expression feature, the concentration index and the confusion index, and pushing the emotional feature tag to a teacher terminal for visual display. By adopting the scheme of the invention, multi-modal feature fusion analysis of the classroom state of the student can be realized, so that the accuracy of classroom teaching regulation and control is improved.

The invention relates to an interactive expression training method and device, equipment and a medium. The method comprises the following steps: acquiring voice data of an expression subject; performing voice recognition on the voice data to obtain text data corresponding to the voice data; extracting emotion expression features of the voice data; and generating expression feedback information of the expression subject based on the text data and the emotion expression features through a multi-modal large model. Therefore, the expression feedback information of the expression subject can be generated in combination with the text data and the emotion expression features, so that the dimensionality of expression training feedback can be enriched, and the effectiveness of expression training is improved.

The invention relates to the technical field of education information, in particular to a programming education home-school feedback generation method and system based on multi-modal fusion. The method comprises the following steps: firstly, collecting multi-modal data of a student in a programming learning process, wherein the multi-modal data at least comprises expression image data with timestamps and behavior log data; extracting an expression feature vector and a behavior-time correlation feature vector; based on scene judgment, dynamically distributing fusion weights and carrying out weighted fusion to generate multi-modal fusion features; and determining the technical shortages, the learning state and the time efficiency of the student, matching specific suggestions from the feedback suggestion library, generating a personalized feedback report, and pushing the personalized feedback report to the terminal. According to the method, multi-dimensional process data such as expressions, behaviors and time are fused, accurate learning condition diagnosis and personalized guidance are achieved through a scenarized weight distribution and conflict correction mechanism, and the problems of feedback lagging, general and insufficient precision in the prior art are effectively solved.

The invention relates to the technical field of online advertisement putting, in particular to an online advertisement putting optimization system and method based on big data analysis. An emotion category detection unit extracts video space-time micro-expression features and image-text semantic emotion features according to video image-text content browsed by a user and generates cross-modal emotion vectors; a fine-grained emotion label with intensity grading is output, a dynamic risk value is calculated through a cumulative overload prediction model based on the obtained continuous emotion sequence and user equipment use duration real-time data, early warning is judged and triggered, and after the advertisement putting optimization unit receives the early warning, similar advertisements are matched and shielded through emotion fingerprints, and then the advertisement putting optimization unit is started. The opposite advertisements are selected according to the emotion wheel polarity mapping space and the user acceptance thermodynamic diagram, and the low-intensity advertisements are retrieved and generated by using the intensity attenuation algorithm when no alternative advertisements exist, so that the problem that emotion accumulation is neglected in traditional putting is solved, and the advertisement putting and user experience are balanced.

This invention discloses a method and system for pain threshold determination based on multimodal automatic laser stimulation and animal behavior analysis. The method includes: using an RVC-Pose convolutional neural network model to detect key points on the animal's foot and outputting the foot's spatial coordinates in real time; positioning a laser spot on the foot and outputting continuously adjustable laser stimulation according to a preset gradient; extracting facial key points and micro-expression features using a Light-Face facial recognition algorithm, and / or reconstructing the animal's three-dimensional skeleton using a multi-view geometric reconstruction algorithm and extracting pain-related behavioral features; inputting the multimodal behavioral data into a multi-parameter fusion judgment model to automatically identify pain responses, adaptively adjusting the stimulation intensity under gradient stimulation mode, terminating stimulation and recording the pain threshold when an effective pain response is detected. This invention achieves fully automatic closed-loop control for pain threshold determination, solving the problems of inaccurate positioning, imprecise stimulation, and subjective judgment in existing technologies, significantly improving the objectivity and accuracy of pain assessment.

The invention discloses an information recommendation method, electronic equipment and a computer readable storage medium. The method comprises the following steps: in response to a received first data request, based on environmental parameters and position transformation information of a target object, controlling expression acquisition equipment to acquire the target object to view a target expression image of current recommendation information; performing feature extraction on the target expression image to obtain expression features of multiple dimensions; analyzing the expression of the target object for viewing the current recommendation information based on the expression features of the multiple dimensions to obtain an expression analysis result; in response to a received second data request sent by the target client, obtaining object behavior data of the target object for the current recommendation information; and adjusting the current recommendation information based on the expression analysis result and the object behavior data to obtain target recommendation information. According to the invention, the technical problem of poor information recommendation effect in related technologies is solved.

The invention provides an expression recognition method and system based on cross attention and depth center loss, and relates to the technical field of computer vision, and the method comprises the following steps: obtaining a facial expression data set, and carrying out the preprocessing of the facial expression data set, and obtaining a preprocessed expression data set; constructing a facial expression recognition model, and training the facial expression recognition model based on the preprocessed expression data set to obtain a trained facial expression recognition model; and performing expression recognition by using the trained facial expression recognition model to obtain an expression recognition result. According to the method, double-branch feature optimization is performed through depth feature weighting and multi-head cross attention, and joint loss is constructed to supervise and train the model, so that the problems of insufficient local expression feature capture, serious feature redundancy, fuzzy classification boundary and weak model generalization ability in the existing expression recognition technology are effectively solved; and cooperative improvement of expression recognition precision, robustness and training efficiency is realized.

This application discloses an expression recognition method and apparatus, belonging to the field of artificial intelligence technology. The method includes: extracting features from a first set of video frames corresponding to a first video to obtain a first image feature set corresponding to the first set of video frames, wherein all video frames in the first set of video frames include facial images; inputting the first image feature set into a first expression recognition model, and performing denoising processing on the image features in the image feature set through a denoising module in the first expression recognition model to obtain a first expression feature set, wherein the first expression feature set includes expression features of facial images in video frames and expression change features of facial images between every two video frames; and reconstructing expressions based on the first expression feature set through the first expression recognition model to recognize facial expressions in the first video.

This invention proposes a text detection method generated by a large language model, comprising: segmenting the input text into multiple basic text units; extracting the rhetorical relationship types between the multiple basic text units and constructing a multi-relationship graph; extracting the first semantic expression features of the text segment corresponding to the first node and the second semantic expression features corresponding to the second node through a pre-trained language model; inputting the multi-relationship graph into a multi-relationship graph neural network model for feature learning, updating the first and second semantic expression features; and performing graph readout processing on the learned and updated first and second semantic expression features to generate probability distributions of different fine-grained categories corresponding to the input text. This method achieves fine-grained text detection.

The invention discloses a facial expression recognition method based on feature deentanglement and self-distillation. The method comprises the following steps: S1, extracting an initial feature map from an input facial image through a backbone network; s2, generating a weight map based on the initial feature map, and decomposing the initial feature map into expression-related features and expression-independent features by using the weight map; s3, extracting supplementary features related to expressions from the expression irrelevant features, and fusing the supplementary features with the expression related features to obtain enhanced expression features; and S4, guiding the backbone network to carry out optimization training through a self-distillation mechanism by utilizing the enhanced expression features, so that the trained backbone network can directly output robust expression recognition features. According to the method, through characteristic decomposition, supplementary fusion and self-distillation optimization, the model is effectively separated and irrelevant interference information is inhibited, so that the accuracy and robustness of expression recognition are improved.

The invention discloses a virtual image construction method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring target text data and first video data; performing emotion recognition processing on the target text data to obtain a corresponding expression tag; performing voice conversion processing on the target text data based on the expression label to obtain target voice data; driving the lip of the target user in the first video data to act by taking the target voice data as a driving signal to obtain second video data; extracting a first mouth opening and closing feature in the first video data, and determining a first speaking style feature index; adjusting the general expression feature data based on the first speaking style feature index and the general speaking style feature index to obtain expression feature data corresponding to the target user; and performing expression editing processing on the second video data by using the expression feature data to obtain the target video data, thereby enhancing the reality sense of the virtual image and the immersion sense of the user.

The invention relates to the technical field of artificial intelligence, and discloses a micro-expression analysis method based on facial key point recognition, and the method comprises the steps: positioning a facial region through a high-precision face detection model; acquiring a face key point sequence by using a lightweight key point extraction network; constructing a micro-expression feature vector based on the dynamic change of the time sequence key points; and micro expressions are classified and identified by combining an attention mechanism and a multi-layer perceptron. The system comprises an image acquisition module, a key point extraction module, a time sequence feature modeling module and a micro expression classification module. According to the method, the accuracy and the environmental adaptability of micro-expression recognition are remarkably improved by fusing the space-time key point dynamic information and the attention mechanism.

The application discloses a micro-expression recognition method based on optical flow features, first selects micro-expression data sets: CASME, CASME II and CAS(ME) 2 , and respectively maps emotions of all video frame sequences in the three data sets into three categories of "Negative", "Positive" and "Surprise"; all video frame sequences of the selected data set are preprocessed to obtain video frame sequences with a resolution of 128*128; then, a TV-L1 energy functional is used to extract optical flow features of the video frame sequences, and the optical flow features are spliced by a channel superposition method to serve as inputs of a neural network RSCANet; finally, the neural network RSCANet is used to extract micro-expression features and obtain a classification result. The application solves the problem that it is difficult to extract subtle motion changes of a face in a video frame due to the characteristics of micro-expression in the prior art.

This application discloses a method, apparatus, device, medium, and program product for identifying student psychological states, aiming to address the problems in existing technologies such as the difficulty in real-time, comprehensive, and dynamic identification of students' psychological states, and the lack of effective early detection and warning capabilities. The method includes: acquiring a multi-dimensional behavioral feature sequence of the target student, which includes at least an expression feature sequence, a behavioral trajectory feature sequence, and a voice emotion feature sequence; inputting the multi-dimensional behavioral feature sequence into a trained psychological state identification model to output psychological state representation information for characterizing the target student's psychological state; and identifying the target student's psychological state based on the psychological state representation information.

This invention relates to the field of health data analysis technology, and particularly to a method and application for analyzing the correlation between eating micro-expressions and food satisfaction based on deep learning. The method includes the following steps: S1, capturing facial video images of users during the eating process using a camera device, and extracting eating facial feature sequences from them; S2, separating chewing actions and facial muscle movements from the eating facial feature sequences based on the spatial displacement differences of facial key points, and obtaining target micro-expression feature vectors. In this invention, by cross-evaluating the captured eating micro-expression emotional state with the user's physiological health indicators, nutritional threshold constraints and weight corrections are applied to the initial food preferences generated based on emotions. This allows for strict control of the intake of core risk nutrients while catering to the user's personal taste satisfaction, ultimately generating personalized recommended recipes that balance emotional experience and medical health standards.

The invention relates to an insurance verbal skill optimization system and method based on multi-modal emotional feedback, and the system comprises a multi-source data collection module which is used for collecting the video data, audio data and explanation content of a customer in real time in the process of a sales session between an insurance salesman and the customer; the feature extraction module performs feature extraction on the video data, the audio data and the explanation content to obtain an expression feature vector, a voice feature vector and a content feature; an emotion-insurance verbal skill association analysis module performs emotion-insurance verbal skill association analysis on the expression feature vector, the voice feature vector and the content feature according to a preset verbal skill effectiveness real-time scoring model to obtain a verbal skill effectiveness score value; and the real-time feedback optimization module is used for comparing the verbal skill effectiveness score value with a preset threshold value, and when the verbal skill effectiveness score value is lower than the preset threshold value, a preset optimized verbal skill is pushed to an insurance salesman. And the real-time optimization of the insurance sales verbal skill and the personalized adaptation of different customers are met.

The application relates to the technical field of artificial intelligence, affective computing and computer-assisted teaching, in particular to a sign language emotion recognition and teaching feedback method based on machine learning, which collects learner sign language videos and extracts face, hand and posture holographic key points; action semantic features and emotion expression features are obtained in parallel by using a double-branch time sequence coding network; the sign language content and the emotion state containing continuous values of valence-arousal-dominance and discrete categories are obtained by a semantic recognition subnetwork and an emotion recognition subnetwork respectively; the recognition result is compared with standard semantics and emotion labels, and emotion intensity, naturalness, semantic matching degree scores and comprehensive quality scores are calculated; emotion correction instructions, reinforcement learning training sequence planning and key point trajectory visualization feedback are generated based on the scores; and teaching strategies are iteratively improved through closed-loop optimization and emotion resonance models. The application realizes synchronous recognition and quantitative evaluation of sign language semantics and emotions, and provides an adaptive feedback means for sign language emotion expression ability training.

This invention relates to a method and system for identifying mental health abnormalities based on multimodal feature consistency, comprising the following steps: acquiring raw data files containing both audio and video files from the same scene; preprocessing the video and audio data within the acquired raw data files; normalizing the micro-expression severity scores of each frame in a continuous frame sequence to obtain micro-expression keyframe vectors; and inputting the frequency features extracted from the audio data into an audio stream deep feature extraction network to obtain deep speech features F. A And audio feature prediction results y A A continuous sequence of frames is simultaneously fed into a video stream depth feature extraction network to obtain depth video features F. V And video feature prediction results y V ; after that, F A and F V Feature fusion is performed, and then the data is fed into a mental health classification network for final prediction. This invention effectively utilizes facial micro-expression features and voice features to identify mental health abnormalities.

This invention relates to the field of artificial intelligence technology and discloses a method for generating instant empathetic responses for robots based on micro-expression triggers. The method includes collecting facial micro-expression data of the interacting object, generating standardized micro-expression data through preprocessing, generating feature vector data based on micro-expression feature extraction, generating classification data by combining emotional state analysis, generating trigger signals for empathetic responses, generating empathetic response text data using natural language generation technology, and outputting instant empathetic responses through speech synthesis. Simultaneously, the micro-expression analysis model is optimized based on interaction feedback data. This invention ensures the reliability of the data foundation for generating empathetic response trigger signals, guarantees the real-time performance and reliability of interactive responses, and improves the friendliness and overall effect of robot interaction.

The invention relates to a children rehabilitation outdoor scene interaction system based on adaptive SLAM and multi-modal time sequence fusion, and belongs to the technical field of children training, and the system comprises an adaptive projection interaction subsystem which is used for realizing + / -2mm precision three-dimensional motion capture through a distributed sensor, adaptively adjusting the projection brightness in combination with ambient light, and outputting the projection brightness; simulating tactile feedback according to the light and shadow intensity change; the multi-source data acquisition and time sequence processing subsystem is used for acquiring physiological signals, expression features and clinical scale data of children, calculating optimal lag time tau of physiological and expression data through cross correlation coefficients to dynamically align the data, adjusting data window duration T based on emotional feature variance, and constructing an emotional specificity lag correlation model; the dynamic adjustment subsystem is used for calculating a scene adjustment coefficient K by taking the normalized value S'of the rehabilitation stage index S as input, and dynamically adjusting the complexity, excitation intensity and interaction speed of a training scene; and a family-institution cooperation subsystem. According to the invention, behaviors and actions of children can be captured through the infrared sensor.

The application provides an image processing method and device, electronic equipment and computer readable storage medium; the method comprises: acquiring image data of a facial expression; extracting a first expression feature and a first identity feature from the image data; performing feature splicing on the first expression feature and the first identity feature to obtain a first joint feature, and extracting joint information between the first expression feature and the first identity feature from the first joint feature; estimating mutual information between the first expression feature and the first identity feature, adjusting the first expression feature to a second expression feature with the mutual information minimized as the target; and predicting a classification result of the facial expression based on the second expression feature and the joint information; in this way, expression and identity collaborative modeling is realized.

The invention discloses a facial paralysis recognition and grade evaluation system and method based on an expression analysis technology, and relates to the technical field of medical image intelligent analysis, and the method comprises the steps: carrying out the virtual correction of a face image in an action segment set based on a three-dimensional muscle group activation field, generating a healthy twin face, and carrying out the recognition and grade evaluation. Calculating a feature residual error between the face image and the healthy twin face to form a residual error severity index; performing feature fusion on the residual severity index, the three-dimensional muscle group activation field and the two-dimensional expression feature vector to form a muscle group associated residual feature graph, performing graph-level feature aggregation and multi-task reasoning through a graph neural network, outputting a facial paralysis recognition result and a graph-level feature vector, and performing multi-classification prediction on the graph-level feature vector to obtain a facial paralysis recognition result. And outputting the facial paralysis severity level. The facial paralysis identification and severity level evaluation method realizes more accurate and explainable facial paralysis identification and severity level evaluation, and improves the reliability and practicability of clinical auxiliary diagnosis.

The invention relates to the technical field of robot emotion accompanying, and discloses a robot emotion accompanying system based on deep learning. The system comprises an emotional state recognition module, an emotional demand analysis module and an emotional interaction strategy generation module. The emotional state recognition module extracts facial micro-expression features, voice spectrum features and body movement track features based on the user interaction data flow and performs multi-modal feature fusion to generate a user emotional state vector; the emotion demand analysis module retrieves an emotion demand knowledge graph based on the vector, matches an emotion demand tag, calculates an emotion demand compactness index, and generates a demand analysis result; and the emotion interaction strategy generation module calls the basic strategy template, adjusts the strategy parameter weight according to the demand closeness index, and generates a personalized emotion interaction strategy instruction set. The system can comprehensively and accurately identify the user emotion, accurately analyze the emotion demand, generate a personalized interaction strategy, and improve the emotion accompanying quality and the user experience.

The invention relates to the technical field of artificial intelligence, can be applied to the fields of financial science and technology and medical health, and discloses an emotional speaking head video generation method, device, equipment and medium, and the method comprises the steps: obtaining a driving audio and an identity image, and employing a pre-trained audio encoder to encode the driving audio to obtain an audio emotion vector; processing the identity image and the audio emotion vector based on an emotion face representation model to obtain a face identity feature, an expression feature and an emotion intensity value; generating a speaking head video clip through a pre-trained generation model according to the facial identity feature, the expression feature and the emotion intensity value; and generating an emotional speaking head video by adopting a time extrapolation strategy according to the speaking head video clip. The emotion expression accuracy and reality of emotion speaking head video generation are improved.

The invention is suitable for the field of privacy protection, and provides a face image anonymization processing method and system, and the method comprises the following steps: obtaining a to-be-processed image of a user, and employing a pre-trained face detection model to position and cut out a single face region image; selecting a target artistic style template from a predefined intelligent style library; inputting the face region image into a pre-trained fast style migration model to generate an anonymized face image block; and fusing the anonymized face image block to an original position in a to-be-processed image to obtain an anonymized image. According to the anonymized image output by the method, the natural contour and expression features of the face are kept, and the aesthetic value of the image is improved through an artistic expression form.

The application relates to the technical field of artificial intelligence and psychological interviews, in particular to an emotional conversation intelligent interview method and system, which comprises the following steps: collecting voice, text and micro-expression features through a multi-modal sensing unit; constructing a semantic-emotion coupling feature model, monitoring the mapping relationship between semantic logic and emotional fluctuation and calculating a conflict coefficient; when the semantic feedback and the emotional valence are negatively correlated, triggering a contradiction seeking logic; using a heterogeneous graph converter to construct interview multi-dimensional features into a heterogeneous graph, generating a context vector through an attention mechanism; generating a guiding prompt word with an emotional regulation factor based on the vector, guiding the interviewee to release real psychological motivation. The application can accurately identify the real intention hidden under the disguise, eliminate individual expression differences, balance deep mining and empathy injection, and realize self-adaptive and high-precision collection of interview information.

Embodiments of the present application provide a video generation method and device, electronic equipment and storage medium, input a target video frame containing a face image of a target object in a to-be-processed video into a pre-trained face recognition model, determine the face feature of the target object in the target video frame as a target face feature; determine the expression feature vector of the to-be-processed video based on the target face feature in each target video frame; for each to-be-processed audio, process the to-be-processed audio based on a pre-trained beat point prediction model to obtain a target beat feature vector of the to-be-processed audio; calculate the similarity of the expression feature vector and the target beat feature vector as the matching degree of the to-be-processed video and the to-be-processed audio; and perform synthesis processing on the to-be-processed video and the target audio in each to-be-processed audio to obtain a target video; the matching degree of the target audio and the to-be-processed video is maximum. Based on this, the generation efficiency of the video can be improved.