274 results about "Expression Feature" patented technology

The invention provides an AI-based marketing video auditing method, and relates to the technical field of AI marketing video auditing, and the method comprises the steps: obtaining a multi-modal data original structure set, and extracting image semantic features, voice expression features, text semantic features and scene label information, and obtaining an image semantic feature set, a visual rhythm feature set, a voice expression feature set, a voice and picture synchronous association vector structure, a text semantic feature set and a subtitle semantic and image main body linkage relation graph. By constructing an image semantic feature set, a voice expression feature set, a text semantic feature set and a visual rhythm feature set and fusing the image semantic feature set, the voice expression feature set, the text semantic feature set and the visual rhythm feature set into a multi-modal content fusion feature tensor, unified modeling of an AI marketing video at visual, auditory and semantic levels can be realized; and subsequent microscopic consistency detection, compliance knowledge graph and emotion semantic conflict identification are effectively performed, so that full-link risk perception and accurate auditing of video contents are realized.

The invention provides a theft behavior recognition method and system based on computer vision, and relates to the technical field of machine vision, and the method comprises the steps: preprocessing a video stream, detecting a target through a YOLOv11 algorithm, extracting features through a feature pyramid network and a path aggregation network, extracting a posture skeleton and micro expression features through an Openpose algorithm and wavelet transform, and carrying out the recognition of a theft behavior through a computer. And inputting a deep learning model to generate a behavior feature vector. And inputting the feature vector into an analysis engine fusing a convolutional neural network and a Transform structure, calculating the similarity with a preset template and generating a confidence score, thereby realizing accurate identification and timely early warning of the theft behavior.

The invention relates to the field of intelligent interaction, and discloses an intelligent hardware dynamic interaction system based on voice semantic fusion and multi-modal perception, which comprises the following steps of: constructing a context model of continuous operation by collecting continuous voice instructions, gesture actions and expression information of a user; semantic analysis and feature fusion are carried out on currently collected voice, gesture and expression features, meanwhile, credibility indexes of all modes are calculated through a weighting or deep learning model, weighting correction is carried out on a fusion result, a real-time feedback algorithm is adopted for weight adjustment for continuous optimization, the next operation intention of a user is predicted through deep learning, and the user experience is improved. And in combination with historical interaction data, online feedback and prediction errors, context management, modal weight and intention prediction strategies are adaptively optimized, and the updated strategies are used for next-round context acquisition and multi-modal fusion. The method has the advantage of improving the recognition accuracy in the continuous interaction scene.

The invention provides an interactive classroom student dynamic portrait generation method, a medium and a system, and belongs to the technical field of student dynamic portrays.The method comprises the steps that a student facial image sequence is collected, facial feature points are extracted to construct an expression feature matrix, and sight line focus data are obtained by combining an eye movement tracking technology to calculate an eye light concentration degree index; and establishing an individual association model, identifying specific expressions and analyzing attention fluctuation characteristics. And fusing multi-dimensional features such as an expression similarity vector and an eye light concentration index into a dynamic attention state score. Learning state evaluation is realized based on a multilayer bidirectional converter network, wherein multi-head attention mechanism parameters are determined by expression change frequency, an eye light concentration threshold and an attention fluctuation rate. And finally, generating a student dynamic portrait feature map, and forming a classroom interaction effect evaluation index through clustering analysis, thereby solving the technical problem that the dynamic change of the student classroom attention state is difficult to accurately evaluate based on a single feature.

The invention discloses an AI classroom behavior collaborative analysis method and system. The method comprises the following steps: acquiring classroom environment data in real time; preprocessing the classroom environment data to respectively obtain target student video data, target student audio data, target student action data and target student expression data; performing feature extraction on the target student video data, the target student audio data, the target student action data and the target student expression data to respectively obtain expression features, voice content features, action features and posture features; performing comprehensive processing on the expression features, the voice content features, the action features and the posture features to obtain comprehensive features; performing multiple recognition on the comprehensive features, and performing post-processing on an initial recognition result obtained by recognition to obtain a target recognition result; and performing quantitative evaluation on the target identification result to generate a behavior analysis report. According to the invention, behaviors of students can be accurately identified and evaluated in real time, and classroom management and teaching optimization levels are effectively improved.

The invention relates to an interactive multi-round dialogue digital human modeling method, which comprises the following steps of: extracting double-person multi-modal characteristics in an interactive multi-round dialogue scene, including voice characteristics and expression characteristics of a speaker in a current dialogue round and voice characteristics of a listener in a previous dialogue round in the current dialogue round; performing time sequence alignment and reinforcement on the extracted double-person multi-modal features based on a time dimension to obtain a combined feature sequence; according to the joint feature sequence, generating a voice text of the listener in the current dialogue round and synchronous expression parameters based on a codec fused with an attention mechanism; and generating a corresponding 3D facial animation frame sequence according to the expression parameters of the listener in the current dialogue round.

The invention relates to a micro-expression recognition method based on staged adaptive course learning. The method comprises the following steps: A, preprocessing a micro-expression video sequence and a macro-expression video sequence; b, constructing a spatial-temporal feature fusion model, performing deep feature extraction on the micro-expression data set obtained by preprocessing, and pre-training a macro-expression recognition teacher model; c, constructing a deep learning algorithm based on staged adaptive curriculum learning, and introducing a micro-expression recognition-oriented deep learning algorithm based on staged adaptive curriculum learning in the training process of the constructed spatio-temporal feature fusion model to optimize the training process; and D, carrying out classification identification on the macro expression identification teacher model obtained by training on a test set. According to the method, more effective and discriminative micro-expression features are obtained, the generalization ability of the model is improved, and the problems that in the existing micro-expression recognition field, available data sets are lacked, redundant information contained in the data sets is large, and the recognition accuracy is not high are further solved.

The present invention provides a single-image-based three-dimensional face reconstruction and editing method, comprising: extracting features by means of a pre-trained EG3D network, so as to generate a preliminary three-dimensional face; using an inversion module to map a monocular image to a latent code space, and optimizing a latent code to generate a realistic three-dimensional face image; making use of a depth estimation technology and multi-view projection to create a pseudo multi-view image; then, by means of semantic segmentation and expression feature extraction, fusing the features to generate a comprehensive representation; according to editing requirements, optimizing and adjusting the latent code, so as to implement customized editing; and finally, a generator outputting an edited face image. The present invention can enable more efficient, flexible and high-quality three-dimensional face reconstruction and editing, thereby effectively overcoming the defects of high cost, low universality and lack of customized editing in existing face reconstruction technologies.

The invention relates to a head-up display adjusting method, system and device based on emotion recognition and a medium. The method comprises the following steps: acquiring a face video frame sequence of a driver and a physiological signal sequence corresponding to the face video frame sequence; inputting the face video frame sequence into an expression recognition model, and extracting an expression feature sequence of the face video frame sequence; extracting a physiological feature sequence from the physiological signal sequence according to a physiological feature extraction model; inputting the expression feature sequence and the physiological feature sequence into a multi-modal emotion recognition model to generate an emotion category; and dynamically adjusting the head-up display content on the vehicle according to a preset adjustment strategy based on the emotion category. The head-up display content is dynamically adjusted according to the emotion of the driver, and the driving safety is improved.

The invention relates to the technical field of artificial intelligence and medical health, and discloses a pressure detection model training method and device, electronic equipment and a storage medium, and the method comprises the steps: constructing a pressure detection model which comprises a feature extraction layer, a feature fusion layer and a result prediction layer which are connected in sequence; wherein the feature extraction layer comprises an action unit feature recognition network, an emotion feature recognition network, an expression feature recognition network and a recessive feature recognition network; a training set is constructed, each data sample in the training set is a face image of the recognition object, and the data samples comprise marks of face action unit features, face emotion features, face expression features and face image coding features; and inputting the training set into a pressure detection model for training to obtain the pressure detection model. The method greatly reduces the pressure detection cost, is high in detection result efficiency, improves the operability, and is convenient for a user to use. And the practicability is high.

The invention relates to a device and a method for assisting in detecting depression degree by utilizing emotional picture cognitive deviation, and relates to the technical field of biomedical engineering and medical instruments, and the device comprises an emotional picture library module which is used for storing a pre-selected emotional picture set; the display interaction module is used for displaying pictures to the user and recording classification selection and reaction time of the user; the data acquisition module is used for acquiring user facial micro-expressions and synchronously storing the facial micro-expressions and classification results; the data processing module is used for calculating positive picture error rate related features and normalizing behavior features and micro expression features into feature vectors; the depression index calculation module is used for outputting depression indexes; and the result output module is used for generating a visual report. The depression index is calculated based on behavior data, deviation is avoided, multi-modal features are fused to improve quantization precision, low-cost non-intrusive screening is achieved through a touch screen and a camera, and an embedded processor is integrated to improve the integration level and generalizability.

The invention discloses a student state real-time analysis method and device based on deep learning, and the method comprises the steps: obtaining and preprocessing image data of students in a classroom, and carrying out the multi-scale face detection and facial feature extraction of the preprocessed image data; expression features are extracted based on standardized facial feature data, fusion is carried out in combination with attention indexes and attitude features, a time sequence feature sequence is constructed, a heavy-tailed recurrent neural network is applied to carry out time sequence modeling, an evaluation standard is established, evaluation parameters are adjusted through a self-adaptive threshold value, and finally a student state evaluation result is obtained. Through the heavy-tailed recurrent neural network and a slow transition mechanism to low-dimensional chaos, subtle changes and long-term trends of student states can be accurately captured, and the technical problems that a traditional student state monitoring method is poor in real-time performance, limited in coverage and insufficient in individuation are solved.

The invention relates to the technical field of facial dynamic video image pain assessment, in particular to a facial dynamic video image pain assessment method based on a dynamic fusion module, and the method comprises the steps: collecting data through a multi-modal sensor, and generating a three-dimensional feature mapping map; activating an adaptive weight adjustment module to generate a configuration parameter set; executing micro-expression feature extraction and dynamic fusion operation to output a high-precision feature vector; inputting a grading evaluation model to complete pain degree quantitative grading. According to the method, facial micro-expression changes can be accurately captured, feature distortion can be dynamically recovered, nonlinear feature components can be mined, feature extraction integrity and evaluation accuracy can be improved, meanwhile, the feature capture capability in a complex environment can be enhanced through a controlled gradient optimization dynamic fusion technology, and the calculation efficiency can be optimized.

The invention discloses a facial expression recognition method and a facial expression recognition device, which are used for solving the technical problem of poor recognition accuracy caused by the existing facial expression recognition method. The method comprises the following steps: acquiring a to-be-detected video frame face image, and generating a global face image and an initial key point set based on the to-be-detected video frame face image; generating a plurality of face local area images according to the initial key point set by adopting a local candidate module based on an area relation algorithm; inputting each face local area image into a preset local expression feature extraction network for local feature extraction, and outputting multiple pieces of local expression feature data; performing global feature extraction on the global face image through a global expression feature extraction network based on a continuous attention algorithm, and generating global expression feature data corresponding to the global face image; and a preset feature classifier is adopted to perform facial expression recognition according to the global expression feature data and the local expression feature data, and a facial expression recognition result is generated.

The invention discloses an image facial feature extraction and fusion method and system and a memory, and the method comprises the steps: carrying out the feature extraction of a source role facial image, and generating a basic feature map and an identity feature vector; performing decoupling processing on the basic feature map by taking the identity feature vector as a control condition, and separating an expression feature channel and an illumination feature channel; inputting the decoupled expression feature channel and illumination feature channel into a scene perception gating network, and generating an adaptive fusion feature map in combination with the target scene image; executing Poisson fusion and generative adversarial refinement on the adaptive fusion feature map, and outputting an optimized fusion image; through a decoupling mechanism guided by an identity feature vector, texture distortion and illumination sensitivity are avoided, a scene perception gating network dynamically adjusts a feature fusion weight, cross-scene adaptability is remarkably improved, poisson-generative adversarial two-stage fusion introduces antagonism refinement on the basis of gradient domain optimization, boundary splicing traces are eliminated, and the fusion precision is improved. And high-fidelity and cross-scene facial feature fusion is realized.

The invention provides an intelligent computing platform abnormal root cause positioning method fused with a large language model and related equipment, and relates to the technical field of anomaly detection. The method comprises the steps of firstly obtaining semantic expression features of interaction between a target student and a large language model, and judging whether the interaction obstacle degree exceeds a preset value or not; if yes, the feedback type of the obstacle is determined firstly, then dialogues in a preset window before the obstacle appears are searched, the starting point of the obstacle is positioned, and key semantics of the content of the obstacle in the period are extracted. And then based on key semantics, searching a target learning group and a matching dialogue of users in a set area, identifying a plurality of error types and determining a basic score, matching the error types with an obstacle feedback type to obtain a matching score, combining the error types with the obstacle feedback type to determine a basic error type, and finally modifying and generating a reply and feeding back the reply to students. By implementing the method, a closed loop from obstacle identification to root cause positioning to generation of targeted reply is realized, and the accuracy and efficiency of abnormal root cause positioning are greatly improved.

The invention provides an image processing method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring image data of a facial expression; extracting a first expression feature and a first identity feature from the image data; performing feature splicing on the first expression feature and the first identity feature to obtain a first joint feature, and extracting joint information between the first expression feature and the first identity feature from the first joint feature; estimating mutual information between the first expression feature and the first identity feature, and adjusting the first expression feature into a second expression feature by taking minimization of the mutual information as a target; predicting a classification result of the facial expression based on the second expression feature and the joint information; therefore, expression and identity collaborative modeling is realized.

A method of training a deep learning model and a method of synthesizing a speech are provided, which relate to a field of artificial intelligence technology, in particular to fields of large model, large language model, generative model, deep learning, and speech processing technologies. The method of training a deep learning model includes: determining a reference speech feature of a sample speech, the reference speech feature being associated with a prosodic feature of the sample speech; retrieving a speech library using a sample text corresponding to the sample speech, so as to obtain a pronunciation expression feature of the sample text; inputting the pronunciation expression feature into the deep learning model to obtain an output speech feature; determining a loss of the deep learning model according to the reference speech feature and the output speech feature; and adjusting a parameter of the deep learning model according to the loss.

The invention relates to the technical field of image recognition and biological feature authentication, and particularly discloses a client identity real-time checking method based on image recognition. The method comprises the following steps: synchronously acquiring a certificate image, a real-time video stream and environmental parameters, and preprocessing to generate an environmental parameter matrix and a biological pulse sequence; certificate static texture features and video dynamic micro-expression features are separated and extracted through a double-flow network, and feature weights are dynamically adjusted and dimensionality reduction is carried out based on environmental parameters; the historical weight and the illumination factor are fused to generate a dynamic fusion coefficient, and feedback optimization feature extraction is carried out; decoding the iris tremor frequency to generate a living body confidence coefficient, combining the dynamic weight fusion features to obtain an initial similarity, and carrying out multiplication fusion on the initial similarity and the living body confidence coefficient; a micro-expression periodic jitter mode is detected, and risk factors are output after fraud punishment is applied; and triggering model re-calibration based on the risk factor. According to the method, the verification precision and the anti-counterfeiting capability under the scenes of complex illumination, head deflection and screen attack can be remarkably improved, and high-robustness identity authentication is realized.

The invention relates to the technical field of psychological state assessment, and particularly discloses a psychological state assessment method and system based on environmental perception, and a readable storage medium, and the method comprises the following steps: collecting an environment scene and an assessed object behavior image through an RGB camera, and extracting an object category, a spatial layout and a dynamic event through a semantic segmentation model; acquiring an environment voiceprint through a microphone, and recognizing a voice category by using a voice recognition model; synchronously acquiring facial micro-expression feature points, voice rhythm features and head, neck and shoulder vibration frequencies of the evaluated object; and generating a semantic vector from the object / scene label through a semantic embedding model. According to the psychological state assessment method based on environmental perception, through three core innovations of quantitative modeling of environmental semantics, causal reasoning and a dynamic attention mechanism and personalized adaptive calibration, the neglect limitation of traditional psychological assessment on environmental factors is broken through, and the accuracy, robustness and practicability of psychological state assessment are remarkably improved.

The invention discloses a full-process closed-loop nursing assistant robot control system based on multi-modal interaction. The system comprises a multi-modal sensing module, a facial expression analysis module, an action evaluation module, a closed-loop control module, an execution driving module and a data interaction module. The multi-modal sensing module integrates multi-source information to realize accurate feature fusion; the facial expression analysis module generates expression feature vectors; the action evaluation module calculates action normative parameters; the closed-loop control module generates a dynamic task queue and distributes instructions; the execution driving module responds to the instruction to complete operation; and the data interaction module realizes data transmission and storage. The system overcomes the defects that in the prior art, multi-modal information fusion is insufficient, and control closed-loop real-time performance and adaptability are poor, nursing accuracy and efficiency are improved, accurate and personalized nursing requirements are met, manual nursing defects are made up, and operation normalization is unified.

The embodiment of the invention provides a multi-mode mental health prediction method, electronic equipment and a medium, and belongs to the technical field of medical science and technology and artificial intelligence. The method comprises the following steps: performing psychological health detection on target user psychological features of fused user micro-expression features, user micro-action features, user facial features, user emotion categories and user micro-mood categories to obtain psychological health prediction data of a target user. According to the embodiment of the invention, psychological health detection is carried out on the target user psychological features fused with the user micro-expression features, the user micro-action features, the user facial features, the user emotion categories and the user micro-mood categories, so that the psychological activity state of the user can be comprehensively captured; the problem that detection is inaccurate due to the fact that psychological health detection is carried out only through user micro-expressions as single basis features is solved, and the accuracy of psychological health detection is remarkably improved.

The invention aims to provide a multi-modal cancer prognosis evaluation method based on Transform, which comprises the following steps: S1, pre-processing multi-modal data, carrying out foreground segmentation and patch segmentation on pathology to obtain patch features, and clustering to obtain a phenotype cluster expressed by morphological specificity; performing expression significance analysis on the multi-omics data, and screening significant expression features; s2, single-mode modeling is carried out, strong pathological representation is obtained based on Transform and a two-stage attention convergence layer, and multi-omics modeling is realized based on a self-normalization network (SNN); s3, cross-modal interaction is carried out, and dense omics-phenotype cluster common attention mapping is realized by using a cross-modal Transform; s4, performing multi-modal fusion, and integrating the cross-modal interaction feature, each single-modal feature and clinical information by using a unified Transform coding layer to obtain a final multi-modal representation; and S5, risk prediction: inputting the final multi-modal representation into a risk prediction module to obtain a risk prediction result. Pathology and multi-omics characteristics are effectively fused, and clinical decision making is promoted.

The invention provides a digital human auxiliary diagnosis method and device, electronic equipment and a readable storage medium, and is applied to the technical field of medical data processing. The method comprises the following steps: processing question and answer audio information of a patient to generate question semantic information and sound feature information of the patient; processing the real-time facial video information and the physiological status information of the patient to generate facial micro-expression feature information and physiological index feature information; processing the sound feature information, the facial micro-expression feature information and the physiological index feature information based on a multi-modal fusion network to generate real-time emotional state information of the patient; processing the real-time emotional state information of the patient, the question semantic information of the patient and the preference information of the patient to generate target reply style information of the digital person; and processing the real-time operation information of the hospital department of the target area and the historical case record information of the patient based on the question and answer planning network, generating resource allocation result information, and generating digital human reply information.

The invention relates to the technical field of movie and television animation production, in particular to a movie and television animation character expression generation method and system combined with motion capture, and the method comprises the steps: firstly collecting a real-time motion data set of a target actor through motion capture equipment, including face and body motion data subsets; the facial action data is composed of facial key point displacement data with multiple continuous timestamps, then calling a preset expression mapping model to decouple the facial action data to obtain a basic and personalized expression feature set, then adjusting the basic expression feature set according to preset emotion parameters of an animation role, generating an emotion adaptive expression feature set, and generating a personalized expression feature set according to the emotion adaptive expression feature set; the method comprises the steps of obtaining a personalized expression feature set, fusing with the personalized expression feature set to obtain a target expression feature set, finally driving a three-dimensional face model to generate an expression animation by using the target expression feature set, and binding body action data to a three-dimensional body model to generate a complete character animation, thereby improving the authenticity and expressive force of animation character expressions.

The invention relates to the field of image detection, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a learning state monitoring method, device, equipment and medium, the method comprises the following steps: obtaining a learning video of a target student, and carrying out face area identification on the learning video to obtain a target face area; performing expression recognition and eye expression recognition on the target face region to obtain a face expression feature and an eye expression region feature; performing human skeleton recognition on the learning video to obtain target human skeleton key points, and recognizing posture angle features according to the target human skeleton key points; acquiring a learning audio, and performing voice semantic analysis on the learning audio to obtain voice semantic features; determining a learning state concentration degree according to the facial expression feature, the eye expression region feature, the attitude angle feature and the voice semantic feature; and performing learning state monitoring according to the learning state concentration degree to obtain a state monitoring result. According to the invention, the state monitoring efficiency and accuracy can be improved.

The invention relates to a three-dimensional dynamic face reconstruction method based on mixed features and regional expressions, and belongs to the technical field of computer vision, and the method comprises the following steps: S1, collecting monocular dynamic face video data, and carrying out the preprocessing; s2, acquiring a multi-resolution hash feature and a two-dimensional plane hash feature of the three-dimensional space point, and splicing the two features to construct a mixed feature; s3, key facial expression dimensions are screened out through variance analysis, and multi-resolution hash features are fused to construct regional expression features; s4, inputting the mixed features and the regional expression features into a density network, and calculating volume density and geometric features; s5, color feature branches irrelevant to the visual angle are introduced, and colors of the three-dimensional space points are generated in combination with the color feature branches relevant to the visual angle. According to the method, the accuracy and reality of three-dimensional face reconstruction can be effectively improved, and meanwhile, the training speed is increased, so that rapid and high-fidelity three-dimensional face reconstruction is realized.

The invention discloses a facial expression migration method and device based on an adversarial auto-encoder, equipment and a medium, and relates to the technical field of expression migration. The facial expression migration method comprises the following steps: S1, acquiring a target image for providing appearance features and a source image for providing expression features; s2, extracting an expression feature vector according to the source image; and S3, encoding the target image and the source image by using an encoder to obtain depth features. And S4, adjusting the depth feature in combination with the expression feature vector so as to apply the expression of the source image to the target image, and obtaining an expression fusion feature. And S5, performing decoding processing on the expression fusion features, and performing reconstruction to generate a preliminary image. And S6, performing multi-scale optimization on the preliminary image through a depth facial semantic module to obtain an optimized image. And S7, performing super-resolution reconstruction processing on the optimized image to generate an output image. Wherein the output image migrates the expression dynamic state of the source image while keeping the identity characteristics of the target image.

The invention provides a smile image recognition system, which comprises a smile display module, and the smile display module comprises a smile acquisition sub-module, a smile recognition sub-module, a smile evaluation sub-module and a display sub-module. Wherein the smile acquisition sub-module is configured to acquire a target object; the smile recognition sub-module is configured to recognize the collected smile image so as to obtain expression features in the smile image; the smile evaluation module is configured to perform scoring test on each smile image according to a preset method; and the display sub-module is configured to provide the scored smile image pair to the display device according to a preset rule. The smiling condition of the student can be displayed, so that the psychological condition of the student is reflected laterally, a school can observe the smiling display condition of the student in the system through the system, psychological counseling can be carried out in a targeted mode, and the psychological health level of the student is promoted to be improved.

The invention provides a customer service platform based on an AI digital human, a service method, equipment and a medium, and relates to the field of intelligent customer service, and the platform comprises a front-end display layer which is used for generating a chat interface for interaction between a user and the AI digital human; the digital human customer service interaction layer is used for performing sentiment analysis on the query content to obtain a sentiment category corresponding to the query content; matching a target virtual image and a target expression feature corresponding to the emotion category from a configured comparison table of different emotion categories, different virtual images and different expression features; generating a target virtual image of the AI digital human according to the target virtual image and the target expression feature; outputting response content by using the target virtual image voice; and the business processing layer is used for matching response content corresponding to the query content from the configured customer service knowledge base after receiving the query request. According to the application, the AI digital human image can be combined to realize multi-terminal efficient interaction intelligent customer service.