173 results about "Expression Feature" patented technology

The invention relates to the field of intelligent interaction, and discloses an intelligent hardware dynamic interaction system based on voice semantic fusion and multi-modal perception, which comprises the following steps of: constructing a context model of continuous operation by collecting continuous voice instructions, gesture actions and expression information of a user; semantic analysis and feature fusion are carried out on currently collected voice, gesture and expression features, meanwhile, credibility indexes of all modes are calculated through a weighting or deep learning model, weighting correction is carried out on a fusion result, a real-time feedback algorithm is adopted for weight adjustment for continuous optimization, the next operation intention of a user is predicted through deep learning, and the user experience is improved. And in combination with historical interaction data, online feedback and prediction errors, context management, modal weight and intention prediction strategies are adaptively optimized, and the updated strategies are used for next-round context acquisition and multi-modal fusion. The method has the advantage of improving the recognition accuracy in the continuous interaction scene.

The present invention provides a single-image-based three-dimensional face reconstruction and editing method, comprising: extracting features by means of a pre-trained EG3D network, so as to generate a preliminary three-dimensional face; using an inversion module to map a monocular image to a latent code space, and optimizing a latent code to generate a realistic three-dimensional face image; making use of a depth estimation technology and multi-view projection to create a pseudo multi-view image; then, by means of semantic segmentation and expression feature extraction, fusing the features to generate a comprehensive representation; according to editing requirements, optimizing and adjusting the latent code, so as to implement customized editing; and finally, a generator outputting an edited face image. The present invention can enable more efficient, flexible and high-quality three-dimensional face reconstruction and editing, thereby effectively overcoming the defects of high cost, low universality and lack of customized editing in existing face reconstruction technologies.

The invention relates to a device and a method for assisting in detecting depression degree by utilizing emotional picture cognitive deviation, and relates to the technical field of biomedical engineering and medical instruments, and the device comprises an emotional picture library module which is used for storing a pre-selected emotional picture set; the display interaction module is used for displaying pictures to the user and recording classification selection and reaction time of the user; the data acquisition module is used for acquiring user facial micro-expressions and synchronously storing the facial micro-expressions and classification results; the data processing module is used for calculating positive picture error rate related features and normalizing behavior features and micro expression features into feature vectors; the depression index calculation module is used for outputting depression indexes; and the result output module is used for generating a visual report. The depression index is calculated based on behavior data, deviation is avoided, multi-modal features are fused to improve quantization precision, low-cost non-intrusive screening is achieved through a touch screen and a camera, and an embedded processor is integrated to improve the integration level and generalizability.

The invention relates to the technical field of facial dynamic video image pain assessment, in particular to a facial dynamic video image pain assessment method based on a dynamic fusion module, and the method comprises the steps: collecting data through a multi-modal sensor, and generating a three-dimensional feature mapping map; activating an adaptive weight adjustment module to generate a configuration parameter set; executing micro-expression feature extraction and dynamic fusion operation to output a high-precision feature vector; inputting a grading evaluation model to complete pain degree quantitative grading. According to the method, facial micro-expression changes can be accurately captured, feature distortion can be dynamically recovered, nonlinear feature components can be mined, feature extraction integrity and evaluation accuracy can be improved, meanwhile, the feature capture capability in a complex environment can be enhanced through a controlled gradient optimization dynamic fusion technology, and the calculation efficiency can be optimized.

The invention discloses an image facial feature extraction and fusion method and system and a memory, and the method comprises the steps: carrying out the feature extraction of a source role facial image, and generating a basic feature map and an identity feature vector; performing decoupling processing on the basic feature map by taking the identity feature vector as a control condition, and separating an expression feature channel and an illumination feature channel; inputting the decoupled expression feature channel and illumination feature channel into a scene perception gating network, and generating an adaptive fusion feature map in combination with the target scene image; executing Poisson fusion and generative adversarial refinement on the adaptive fusion feature map, and outputting an optimized fusion image; through a decoupling mechanism guided by an identity feature vector, texture distortion and illumination sensitivity are avoided, a scene perception gating network dynamically adjusts a feature fusion weight, cross-scene adaptability is remarkably improved, poisson-generative adversarial two-stage fusion introduces antagonism refinement on the basis of gradient domain optimization, boundary splicing traces are eliminated, and the fusion precision is improved. And high-fidelity and cross-scene facial feature fusion is realized.

The invention provides an intelligent computing platform abnormal root cause positioning method fused with a large language model and related equipment, and relates to the technical field of anomaly detection. The method comprises the steps of firstly obtaining semantic expression features of interaction between a target student and a large language model, and judging whether the interaction obstacle degree exceeds a preset value or not; if yes, the feedback type of the obstacle is determined firstly, then dialogues in a preset window before the obstacle appears are searched, the starting point of the obstacle is positioned, and key semantics of the content of the obstacle in the period are extracted. And then based on key semantics, searching a target learning group and a matching dialogue of users in a set area, identifying a plurality of error types and determining a basic score, matching the error types with an obstacle feedback type to obtain a matching score, combining the error types with the obstacle feedback type to determine a basic error type, and finally modifying and generating a reply and feeding back the reply to students. By implementing the method, a closed loop from obstacle identification to root cause positioning to generation of targeted reply is realized, and the accuracy and efficiency of abnormal root cause positioning are greatly improved.

The invention relates to the technical field of image recognition and biological feature authentication, and particularly discloses a client identity real-time checking method based on image recognition. The method comprises the following steps: synchronously acquiring a certificate image, a real-time video stream and environmental parameters, and preprocessing to generate an environmental parameter matrix and a biological pulse sequence; certificate static texture features and video dynamic micro-expression features are separated and extracted through a double-flow network, and feature weights are dynamically adjusted and dimensionality reduction is carried out based on environmental parameters; the historical weight and the illumination factor are fused to generate a dynamic fusion coefficient, and feedback optimization feature extraction is carried out; decoding the iris tremor frequency to generate a living body confidence coefficient, combining the dynamic weight fusion features to obtain an initial similarity, and carrying out multiplication fusion on the initial similarity and the living body confidence coefficient; a micro-expression periodic jitter mode is detected, and risk factors are output after fraud punishment is applied; and triggering model re-calibration based on the risk factor. According to the method, the verification precision and the anti-counterfeiting capability under the scenes of complex illumination, head deflection and screen attack can be remarkably improved, and high-robustness identity authentication is realized.

The invention discloses a full-process closed-loop nursing assistant robot control system based on multi-modal interaction. The system comprises a multi-modal sensing module, a facial expression analysis module, an action evaluation module, a closed-loop control module, an execution driving module and a data interaction module. The multi-modal sensing module integrates multi-source information to realize accurate feature fusion; the facial expression analysis module generates expression feature vectors; the action evaluation module calculates action normative parameters; the closed-loop control module generates a dynamic task queue and distributes instructions; the execution driving module responds to the instruction to complete operation; and the data interaction module realizes data transmission and storage. The system overcomes the defects that in the prior art, multi-modal information fusion is insufficient, and control closed-loop real-time performance and adaptability are poor, nursing accuracy and efficiency are improved, accurate and personalized nursing requirements are met, manual nursing defects are made up, and operation normalization is unified.

The invention relates to the field of image detection, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a learning state monitoring method, device, equipment and medium, the method comprises the following steps: obtaining a learning video of a target student, and carrying out face area identification on the learning video to obtain a target face area; performing expression recognition and eye expression recognition on the target face region to obtain a face expression feature and an eye expression region feature; performing human skeleton recognition on the learning video to obtain target human skeleton key points, and recognizing posture angle features according to the target human skeleton key points; acquiring a learning audio, and performing voice semantic analysis on the learning audio to obtain voice semantic features; determining a learning state concentration degree according to the facial expression feature, the eye expression region feature, the attitude angle feature and the voice semantic feature; and performing learning state monitoring according to the learning state concentration degree to obtain a state monitoring result. According to the invention, the state monitoring efficiency and accuracy can be improved.

The invention relates to an interview auxiliary method based on multi-modal data identification, and the method comprises the steps: collecting multi-modal data of a job seeker in an interview process, carrying out the alignment time sequence processing of the multi-modal data, and enabling the multi-modal data to comprise a video stream, an audio stream and text data; judging whether the job seeker is in an abnormal state or not based on micro-expression features, voice features and text features extracted from the multi-modal data of the aligned time sequence; when the job seeker is in an abnormal state, dynamically adjusting feature weights corresponding to the micro-expression features, the voice features and the text features based on an abnormal state probability; calculating an abnormal state probability value based on the adjusted feature weight, the micro-expression feature, the voice feature and the text feature; calculating an abnormal state type according to the abnormal state probability value and a type threshold value; and displaying interview prompt information to an interviewer according to the abnormal state type. According to the method, the misjudgment rate of abnormal states is reduced, and the accuracy and effectiveness of interview analysis are improved.

The invention discloses a doctor-patient speech communication model training method and system based on multi-modal corpus analysis, and belongs to the crossing field of artificial intelligence and medical treatment, and the method comprises the steps: carrying out the extraction according to an OpenPose algorithm to obtain motion features, carrying out the muscle activity intensity detection to obtain expression features, carrying out the speech recognition to obtain text features, and carrying out the recognition of the text features; speech segmentation is carried out based on the time domain energy parameters, and a segmentation result is subjected to modal analysis to obtain acoustic features; performing feature fusion based on an attention mechanism to obtain fusion features, and inputting an obstacle recognition model to obtain an obstacle type; the method comprises the steps of obtaining an obstacle type, obtaining an intervention strategy and an intervention identity according to a solution mapping relation, taking the obstacle type, the intervention strategy and the intervention identity as multi-dimensional labels, carrying out time domain alignment and structured packaging to obtain target data, and training according to the target data to obtain a communication model used for providing dialogue prompt information. Obstacle recognition accuracy can be improved, and the intelligent agent application effect can be improved.

A methodology for auditing the visual diversity of unlabeled human face image datasets uses a set of core human interpretable dimensions derived from human similarity judgments. Given a face image, a model can output dimensional values aligned with the human mental representational space of faces, where values not only express the presence of a feature, but also its extent. Since the model can be learned entirely from human behavior, the learned dimensions are not biased toward features that are easier to verbalize or quantify.

A model processing method includes, obtaining a first source head model and a plurality of pieces of expression feature data; performing deformation matching on the first source head model according to a first model feature of a target head model, to obtain a second source head model; determining a deformation parameter according to a first deformation relationship between the first and second source head models and a second deformation relationship from a first model expression to a second model expression, the first model expression being determined according to neutral feature data of the first source head model, and the second model expression being determined according to expression feature data indicated by an expression movement instruction in the plurality of pieces of expression feature data; and performing expression transfer on the target head model according to the deformation parameter, to obtain a target head model having the second model expression.

The invention relates to the technical field of image processing, and discloses a facial health state assessment method and system based on layered optical modeling. The objective of the invention is to solve the problems of poor face health state evaluation precision and robustness caused by rough optical modeling, mixed reflection components, weak environmental adaptability and insufficient multi-dimensional health index fusion capability in the prior art. The method comprises the following steps: synchronously acquiring a user face video sequence comprising at least two different spectral bands through a multispectral imaging device; constructing a multilayer optical transmission model of the skin tissue, and separating a specular reflection component and a diffuse reflection component from the face video sequence; extracting a physiological time sequence signal related to subcutaneous blood flow activity from the separated diffuse reflection component, and calculating heart rate, heart rate variability and blood oxygen saturation parameters; and fusing the physiological parameters and expression and micro-expression features extracted from the facial video sequence, and outputting quantitative evaluation results of fatigue degree, pressure level and emotional state through a pre-trained deep neural network model. The system comprises a multispectral image acquisition module, a layered optical modeling module, a reflection component separation module, a physiological parameter inversion module and a health state evaluation module. According to the technical scheme, the signal-to-noise ratio and stability of physiological signal extraction in a complex illumination environment can be remarkably improved, the adaptability to individuals with different skin colors and dynamic illumination conditions is enhanced, and more accurate and more robust judgment of the high-order health state is achieved.

A multimedia data processing method and apparatus, and a computer-readable storage medium are disclosed. The method may include: acquiring an audio stream and a video stream of multimedia data; parsing the audio stream to obtain text feature data, and matching the text feature data according to a preset mapping relationship to determine topic feature data; parsing the video stream to obtain expression feature data, and matching the expression feature data according to a preset mapping relationship to determine an emotion index; and rendering the multimedia data based on the text feature data, the emotion index, and the topic feature data.

The invention relates to the technical field of computer vision and artificial intelligence, in particular to a shielded facial expression recognition method and system based on a dynamic double-flow network. The method comprises the following steps: acquiring a to-be-recognized face image; inputting the face image into a parallel face expression flow and a shielding perception flow at the same time to extract a global expression feature vector and a shielding robust expression feature vector; inputting the global expression feature vector and the shielding robust expression feature vector into a dynamic weight generator to generate a dynamic fusion weight; performing weighted fusion on the global expression feature vector and the shielding robust expression feature vector to obtain a final feature vector; and inputting the final feature vector into a classifier to obtain an expression classification result. According to the method, the shielding information is used as an effective feature instead of being simply ignored or repaired, the recognition accuracy and robustness in the presence of shielding objects such as masks and glasses are remarkably improved, and the method can be widely applied to the fields of intelligent security and human-computer interaction.

The invention provides an emotional feature recognition system and method for classroom teaching. The method comprises the following steps: acquiring facial image data and behavior time sequence data of a student in a classroom teaching process through a camera after the student clearly knows and agrees to authorize; determining dynamic expression features of the student through a preset facial analysis model according to the facial image data; on the basis of a head posture change sequence and an eye fixation focus track extracted from behavior time sequence data, constructing concentration indexes and confusion indexes of students in different teaching links through multi-feature fusion; and when it is detected that a preset condition is met, generating an emotional feature tag of the student in classroom teaching based on the dynamic expression feature, the concentration index and the confusion index, and pushing the emotional feature tag to a teacher terminal for visual display. By adopting the scheme of the invention, multi-modal feature fusion analysis of the classroom state of the student can be realized, so that the accuracy of classroom teaching regulation and control is improved.

The invention relates to an interactive expression training method and device, equipment and a medium. The method comprises the following steps: acquiring voice data of an expression subject; performing voice recognition on the voice data to obtain text data corresponding to the voice data; extracting emotion expression features of the voice data; and generating expression feedback information of the expression subject based on the text data and the emotion expression features through a multi-modal large model. Therefore, the expression feedback information of the expression subject can be generated in combination with the text data and the emotion expression features, so that the dimensionality of expression training feedback can be enriched, and the effectiveness of expression training is improved.

The invention relates to the technical field of education information, in particular to a programming education home-school feedback generation method and system based on multi-modal fusion. The method comprises the following steps: firstly, collecting multi-modal data of a student in a programming learning process, wherein the multi-modal data at least comprises expression image data with timestamps and behavior log data; extracting an expression feature vector and a behavior-time correlation feature vector; based on scene judgment, dynamically distributing fusion weights and carrying out weighted fusion to generate multi-modal fusion features; and determining the technical shortages, the learning state and the time efficiency of the student, matching specific suggestions from the feedback suggestion library, generating a personalized feedback report, and pushing the personalized feedback report to the terminal. According to the method, multi-dimensional process data such as expressions, behaviors and time are fused, accurate learning condition diagnosis and personalized guidance are achieved through a scenarized weight distribution and conflict correction mechanism, and the problems of feedback lagging, general and insufficient precision in the prior art are effectively solved.

The application discloses a milk preference degree prediction method and device based on micro-expression recognition and a medium, and relates to the technical field of image processing and behavior analysis. The method comprises the following steps: decomposing facial video data of a target when the target tastes milk into an image sequence and performing pretreatment; adopting a multi-scale optical flow method to capture facial micro-expression changes and extract facial space-time features; constructing a cross-modal time sequence module, processing horizontal and vertical optical flow features by using an encoder, and fusing the processed horizontal and vertical optical flow features to obtain fused features; constructing and training a micro-expression recognition model; constructing a milk preference degree prediction model based on the cross-modal time sequence module, and combining micro-expression features and the fused features to perform prediction; training the milk preference degree prediction model; pretreating a video to be recognized, inputting the pretreated video into the trained milk preference degree prediction model, and finally outputting a milk preference degree score of the target. The application can realize high-automation and high-accuracy milk preference degree prediction.

The application relates to the technical field of computer vision, and discloses a ViT-based multi-target tracking system and method for behaviors and expressions of teenagers. The method comprises the following steps: acquiring time-series aligned multi-modal visual data; obtaining semantically matched body region images and calibrated face images through spatial correlation constraint and appearance feature auxiliary matching; constructing a double ViT submodel, wherein a behavior feature submodel adopts a space-time attention mechanism, and an expression feature submodel adopts a mask enhancement mechanism; generating a joint feature vector and a fusion quality score by fusing cross-modal information through a cross-attention mechanism; dynamically adjusting a similarity threshold for hierarchical matching through the fusion quality score, and realizing multi-target tracking in combination with Kalman filtering; querying a behavior-expression correlation rule base to calculate a state evaluation confidence, and outputting a behavior category, an expression category and a state evaluation result. The application can realize accurate tracking and behavior-expression collaborative analysis of a teenager group.

A system of fatigue detection includes an image sensor, a voice sensor, a memory and a processor. The image sensor is configured to capture at least one facial image of a driver. The voice sensor is configured to collect a voice of the driver. The memory is configured to store the at least one facial image and the voice. The processor is configured to extract at least one micro-expression feature from the at least one facial image, and establish a fatigue detection model based on the at least one micro-expression feature, and utilize the fatigue detection model to obtain a fatigue detection result. The processor is further configured to utilize a voice detection algorithm to recognize the voice to obtain a voice recognition result. The processor is further configured to determine a mental state of the driver based on the fatigue detection result and the voice recognition result.

The application discloses an electronic photo frame interface adjustment method and system based on emotion recognition, which acquires user expression images in real time, extracts micro-expression features to generate emotion labels, and uses a pre-established color mapping database to obtain a preliminary color matching scheme. A cache mechanism is used to store the correspondence between high-frequency emotion labels and color matching schemes, improving the retrieval efficiency. For scenes with frequent emotional fluctuations, the stored correspondence between high-frequency emotion labels and color matching schemes is used to quickly search whether the emotion label data in the cache has a matching high-frequency emotion label, improving the retrieval efficiency. For emotion label data that does not hit the cache, compression processing is performed to obtain second color matching scheme data, achieving optimization of database query efficiency when emotion data fluctuates frequently, and ensuring the accuracy and consistency of color adjustment in a fuzzy classification and multi-device environment.

The invention relates to the crossing field of computer vision and emotion calculation, and particularly provides a micro-expression real-time emotion research and judgment method and device, and the method comprises the following steps: S1, multi-mode data collection; s2, micro-expression feature extraction; s3, multi-modal fusion optimization is carried out; s4, real-time emotion tracking; and S5, edge-cloud collaborative study and judgment. Compared with the prior art, the method has the advantages that the dynamic compensation algorithm can effectively improve the micro-expression detection rate, the edge calculation can greatly reduce the response time, the feature compression algorithm can save the bandwidth, and meanwhile, the method supports recognition of various basic emotions and composite emotions, and a self-adaptive adjustment mechanism can switch analysis modes in different scenes such as medical treatment and security.

The invention discloses a light pressure relieving method based on face micro-expression recognition, and relates to the technical field of intelligent driving, and the method comprises the steps: collecting a driver face image through a vehicle-mounted camera, carrying out the preprocessing of the driver face image, carrying out the standardization processing of the preprocessed driver face image, and obtaining key micro-expression feature data; performing pressure state evaluation based on the key micro-expression feature data and the driving context data to generate a pressure index; triggering a hierarchical soothing intervention strategy according to the pressure index, verifying the soothing effect after the hierarchical soothing intervention strategy is executed, and updating an intervention triggering threshold value based on a verification result. The method can improve the depiction capability of a real driving situation, keeps continuous effectiveness along with individual and scene evolution, and has the engineering advantages of maintainability and expandability.

The invention relates to the technical field of online advertisement putting, in particular to an online advertisement putting optimization system and method based on big data analysis. An emotion category detection unit extracts video space-time micro-expression features and image-text semantic emotion features according to video image-text content browsed by a user and generates cross-modal emotion vectors; a fine-grained emotion label with intensity grading is output, a dynamic risk value is calculated through a cumulative overload prediction model based on the obtained continuous emotion sequence and user equipment use duration real-time data, early warning is judged and triggered, and after the advertisement putting optimization unit receives the early warning, similar advertisements are matched and shielded through emotion fingerprints, and then the advertisement putting optimization unit is started. The opposite advertisements are selected according to the emotion wheel polarity mapping space and the user acceptance thermodynamic diagram, and the low-intensity advertisements are retrieved and generated by using the intensity attenuation algorithm when no alternative advertisements exist, so that the problem that emotion accumulation is neglected in traditional putting is solved, and the advertisement putting and user experience are balanced.

The application relates to the technical field of image processing, and provides a face micro-expression recognition method and device, electronic equipment and a storage medium, the method comprising the following steps: acquiring a face micro-expression image sequence to be recognized; pre-processing the face micro-expression image sequence; extracting HCTP features of three orthogonal planes of the pre-processed face micro-expression image sequence; determining feature histograms in three dimensions based on the HCTP features; the HCTP features are image features obtained by combining CTP features on the basis of Haar features; the feature histograms in the three dimensions are standardized; the three feature histograms after the standardization are concatenated into a histogram vector; the histogram vector is input into a trained neural network model based on deep learning for classification; and a corresponding expression category is obtained. In this way, by considering the face micro-expression feature information of the three orthogonal planes and the central pixel information, comprehensive features are extracted, and the accuracy of face micro-expression recognition is improved.

The application discloses a system and method for dynamically switching multiple interactive subjects in a cooperative live broadcast, which comprises a subject feature modeling module, a cooperative semantic perception module, a multi-subject switching module, a context migration module, a multi-source context perception module, and a candidate reply generation engine. The system learns representations based on historical text interaction samples of multiple interactive subjects to obtain expression feature models of each interactive subject. The user's text interaction information and cooperative operation information are semantically encoded to obtain cooperative interaction features. The target interactive subject identity is output by a pre-set identity matching neural network. When the interactive subject identity is switched, the inherited context features are generated, and the fused context features are obtained through attention fusion, and then the candidate replies conforming to the target interactive subject identity are generated. The system can reduce the problems of identity confusion and context break in cooperative live broadcast interaction.

The invention provides an expression recognition method, and relates to the technical field of artificial intelligence. The method comprises the following steps: extracting basic features of an expression image; processing the basic feature by using a residual network and / or a capsule network to obtain a first expression feature; using a channel and space attention mechanism network and / or a lightweight attention mechanism for the basic feature to obtain a second expression feature; and fusing the first expression feature and the second expression feature, and performing classification to obtain an expression recognition result. The invention further provides an expression recognition device and equipment, a storage medium and a program product.

This invention discloses a method and system for pain threshold determination based on multimodal automatic laser stimulation and animal behavior analysis. The method includes: using an RVC-Pose convolutional neural network model to detect key points on the animal's foot and outputting the foot's spatial coordinates in real time; positioning a laser spot on the foot and outputting continuously adjustable laser stimulation according to a preset gradient; extracting facial key points and micro-expression features using a Light-Face facial recognition algorithm, and / or reconstructing the animal's three-dimensional skeleton using a multi-view geometric reconstruction algorithm and extracting pain-related behavioral features; inputting the multimodal behavioral data into a multi-parameter fusion judgment model to automatically identify pain responses, adaptively adjusting the stimulation intensity under gradient stimulation mode, terminating stimulation and recording the pain threshold when an effective pain response is detected. This invention achieves fully automatic closed-loop control for pain threshold determination, solving the problems of inaccurate positioning, imprecise stimulation, and subjective judgment in existing technologies, significantly improving the objectivity and accuracy of pain assessment.