1467 results about "Facial characteristic" patented technology

A multi-modal articulation system translates human gestures into control commands for digital and automation devices. The system integrates a camera module, which captures facial and hand movements, with an image processing unit that extracts facial features, such as eyebrow movement, lip curvature, eyelid motion, and eye trajectory. These extracted features are analyzed in real time by an articulation recognition module that compares them against a database to recognize specific gestures. The system tracks and processes air-drawn hand gestures using a gesture trajectory tracking unit, which converts the motions into digital representations and classifies them using deep learning algorithms. The recognized gestures are converted into machine-readable control signals and executed on various connected devices via a relay control unit. The system incorporates adaptive algorithms to improve accuracy, account for environmental conditions, and stabilize hand tremors, providing a gesture-based control interface for applications ranging from smart home systems to multimedia devices.

The invention relates to the field of road traffic, in particular to a multi-feature fusion fatigue driving detection method and a fatigue driving detection system. The method comprises the following steps: extracting facial features from a face image of a driver; extracting vehicle features from the vehicle driving parameters of the vehicle driven by the driver; and fusing the facial features and the vehicle features to judge whether the driver is in fatigue driving. Extracting facial features by designing a CNN model; extracting basic convolution features; extracting local convolution features; extracting global convolution features; performing pooling operation; aggregating global features; and carrying out dimensionality reduction mapping. A self-encoder is designed to extract vehicle characteristics; a symmetric deep neural network structure is adopted, and high-dimensional time sequence data is compressed to a low-dimensional potential space through nonlinear mapping; through combination and matching of the CNN model and the auto-encoder, the technical defects of feature redundancy, noise interference, information loss and suboptimal decision existing in an existing multi-feature fusion fatigue driving detection system are thoroughly solved.

The invention provides a multi-mode-based man-machine interaction method and system, and relates to the technical field of man-machine interaction. Voice input of a user is received, environment information is collected, facial expressions of the user are captured, and voice features, environment features and facial features are obtained based on the input; utilizing a pre-trained emotion recognition model to generate emotion features of the user, and utilizing a pre-trained user behavior model to generate behavior features according to the current operation information of the user; after the features are obtained, the first weights and the second weights of the features are dynamically adjusted, feature weighted fusion is carried out, and comprehensive features are obtained; the multiple candidate responses are generated based on the comprehensive characteristics, the final response is screened out through a preset scoring mechanism and output, comprehensive understanding of the intention and the emotional state of the user is achieved through comprehensive utilization of multiple data sources, and meanwhile the weight of each characteristic can be dynamically adjusted so that more intelligent and humanized interaction experience can be generated.

The invention discloses a fatigue driving monitoring and early warning method based on multi-dimensional feature fusion, and relates to the field of intelligent traffic and safe driving, and the method comprises the steps: carrying out the feature extraction of a collected face image of a vehicle driver through a feature extraction model based on a convolutional neural network, and obtaining a face key point; then calculating the eye closing rate, the mouth opening rate, the head posture data and the facial muscle data to finally obtain multi-dimensional features, and fusing the multi-dimensional features by using a multi-layer perceptron to obtain fused facial features; and then, a fatigue state detection model obtained based on recurrent neural network training is utilized to detect the facial feature time sequence at a plurality of continuous sampling moments so as to detect whether a fatigue driving state occurs and trigger alarm feedback. According to the method, the fatigue state of the driver can be identified and judged more accurately through combination of multi-dimensional data fusion and a time sequence analysis algorithm, and the monitoring and early warning accuracy and reliability are improved. And the system can keep high robustness even in an illumination change or complex driving environment.

Systems and techniques for generating and animating non-player characters (NPCs) within virtual digital environments are provided. Multimodal input data is received that comprises a plurality of input modalities for interaction with an NPC having a set of body features and a set of facial features. The multimodal input data is processed through one or more neural networks to generate animation sequences for both the body features and facial features of the NPC. Generating such animation sequences includes disentangling the multimodal input data to generate substantially disentangled latent representations, combining these representations with the multimodal input data, and using a large-language model (LLM) to generate speech data for the NPC. Further processing using reverse diffusion generates face vertex displacement data and joint trajectory data based on the combined representation and generated speech data. The face vertex displacement data, joint trajectory data, and speech data are used to produce an animated representation of the NPC, which is then provided to environment-specific adapters to animate the NPC within a virtual digital environment.

The invention discloses a pig individual identification and health monitoring system based on biological characteristics, and relates to the technical field of medical treatment, and the system comprises a multi-mode biological characteristic collection module which is used for obtaining pig 3D facial characteristics, ear blood vessel distribution characteristics, gait dynamic characteristics and physiological data; the biological feature fusion recognition module is used for fusing multi-modal features based on a Transform architecture and realizing individual unique identification and feature template dynamic updating; and the digital twinborn health monitoring module is used for constructing a pig full-life-cycle digital twinborn model and realizing health state evaluation and disease early warning in combination with an LSTM neural network. According to the pig individual recognition and health monitoring system based on the biological characteristics, 1024-dimensional standardized feature vectors are constructed through cooperation of multiple devices such as an RGB-D camera, a near-infrared camera and a binocular vision sensor, modal weights are dynamically distributed in combination with a Transform network, and high-precision recognition in a complex scene is achieved.

Systems and techniques for generating and animating non-player characters (NPCs) within virtual digital environments are provided. Multimodal input data is received that comprises a plurality of input modalities for interaction with an NPC having a set of body features and a set of facial features. The multimodal input data is processed through one or more neural networks to generate animation sequences for both the body features and facial features of the NPC. Generating such animation sequences includes disentangling the multimodal input data to generate substantially disentangled latent representations, combining these representations with the multimodal input data, and using a large-language model (LLM) to generate speech data for the NPC. Further processing using reverse diffusion generates face vertex displacement data and joint trajectory data based on the combined representation and generated speech data. The face vertex displacement data, joint trajectory data, and speech data are used to produce an animated representation of the NPC, which is then provided to environment-specific adapters to animate the NPC within a virtual digital environment.

An object emotion analysis method and apparatus and an electronic device are provided. The method includes: extracting a static facial feature and a dynamic feature from multimedia data associated with a target object, wherein the dynamic feature includes one or more of an expression change feature, a sound feature and a language content feature: inputting the static facial feature and the dynamic feature into a pre-trained object emotion analysis model, fusing the static facial feature and the dynamic feature by the object emotion analysis model, and outputting an emotion analysis result.

The invention relates to the technical field of audio and video identity recognition, in particular to an audio and video identity recognition system based on multimode clue driving. The audio and video identity recognition system comprises an audio feature extraction module, a video feature extraction module, a multimode clue fusion module, a living body detection module and an identity recognition and verification module. The audio feature extraction module is used for capturing a voice signal of a user and extracting key voiceprint features, the video feature extraction module is used for acquiring facial features or limb features of the user and extracting related features, and the multi-mode clue fusion module is used for performing intelligent weighted fusion on the features of audio and video modes. The method comprises the following steps: firstly, using a living body detection module to comprehensively analyze the dynamic characteristics of audio and video, judging whether a user is a real individual, and finally, completing the final verification of the identity of the user through an identity recognition and verification module. The problem of feature extraction and modal fusion in the prior art is effectively solved.

System, methods, and instrumentalities are disclosed for encoding semantic face parameters. A device may receive video data from an encoding device. The video data may include an indication of a value of a rig parameter associated with a video frame. The device may determine a three-dimensional mesh associated with a human face in the video frame based on the value of the rig parameter and a local three-dimensional rig. The value of the rig parameter may indicate a semantic facial feature of the human face in the video frame. The device may generate a two-dimensional image of the human face in the video frame based on the three-dimensional mesh of the human face.

The invention belongs to the technical field of face recognition, and particularly relates to a face recognition algorithm adaptive to illumination variation, which uses Gaussian filtering to reduce noise caused by illumination variation, convert a color image into a grey-scale image, reduce calculation complexity, perform adaptive histogram equalization, estimate global illumination conditions in the image and improve the face recognition accuracy. Calculating the main direction and intensity of illumination in the image, predicting the current illumination condition, and enabling the image brightness to adapt to different environment illumination; identifying a face region in the image, using a deep learning method to position face related points, using an LBP to extract local features insensitive to illumination variation, and using a deep convolutional neural network to extract global features; introducing a multi-illumination data set; the Euclidean distance is used for comparing the extracted feature vectors, the similarity between the extracted feature vectors and known faces in a database is recognized, according to the similarity score, threshold judgment is adopted for AQW to obtain a final recognition decision, and the method has the effect of being capable of accurately adapting to facial features under different illumination conditions in real time.

The invention discloses a campus face recognition abnormal behavior monitoring method and system based on artificial intelligence, and the method comprises the steps: collecting the video stream data of a person in a campus in real time, and recognizing the behavior characteristics of the person in a video frame in real time through employing a lightweight target detection algorithm; extracting facial features of the personnel by using a pre-trained face recognition model, and recognizing identity information of the personnel; generating a personnel behavior semantic description vector with a spatio-temporal context based on the behavior characteristics and the identity information; inputting the personnel behavior semantic description vector into a dynamic early warning threshold engine, and outputting an abnormal behavior probability and risk level evaluation result; and triggering an early warning information pushing mechanism according to the abnormal behavior probability and the risk level evaluation result, and sending early warning information to a corresponding responsible person terminal. By using the embodiment of the invention, the accurate and automatic association of the abnormal behavior and the personnel identity can be realized, and the early warning accuracy, the handling response speed and the system adaptive ability of campus safety monitoring are improved.

The invention provides an interactive classroom student dynamic portrait generation method, a medium and a system, and belongs to the technical field of student dynamic portrays.The method comprises the steps that a student facial image sequence is collected, facial feature points are extracted to construct an expression feature matrix, and sight line focus data are obtained by combining an eye movement tracking technology to calculate an eye light concentration degree index; and establishing an individual association model, identifying specific expressions and analyzing attention fluctuation characteristics. And fusing multi-dimensional features such as an expression similarity vector and an eye light concentration index into a dynamic attention state score. Learning state evaluation is realized based on a multilayer bidirectional converter network, wherein multi-head attention mechanism parameters are determined by expression change frequency, an eye light concentration threshold and an attention fluctuation rate. And finally, generating a student dynamic portrait feature map, and forming a classroom interaction effect evaluation index through clustering analysis, thereby solving the technical problem that the dynamic change of the student classroom attention state is difficult to accurately evaluate based on a single feature.

An apparatus for virtual avatar generation is provided, including a face encoder, a face fusion engine, a voice encoder, and a voice fusion engine. The face encoder extracts first facial features from the first facial image and then encodes they into M first facial feature vectors and extracts second facial features from the second facial image and then encodes they into M second facial feature vectors. The face fusion engine generates a facial fused feature vector from the facial feature vectors, thereby generating a fused image. The voice encoder extracts first voice features from the first audio clip and then encodes they into N first voice feature vectors and extracts second voice features from the second audio clip and then encodes they into N second voice feature vectors. The voice fusion engine synthesizes a voice fused feature vector from the voice feature vectors, thereby generating a fused voice.

The invention discloses a face recognition system based on computer vision and an application method thereof, and relates to the technical field of face recognition. The system comprises a fine-grained shielding classification module, a multi-modal feature fusion module, a cross-modal confrontation identification module and a compensation verification module. By analyzing the depth data and the polarized light parameters, an occlusion thermodynamic diagram is generated, and an identifiable area and an occlusion area are marked. And according to the occlusion label of the thermodynamic diagram, activating infrared and sonic sensor data, extracting the relevance between the temperature change and the vibration frequency, and generating a facial occlusion area feature map. And calculating the similarity between the sheltered region feature map and the non-sheltered region feature, and detecting a forgery attack. And analyzing the difference rate of adjacent frames through a sliding window, marking compensation distortion, rejecting low-credibility data in combination with confidence and attack marks, and outputting a complete facial feature vector and a risk level. And the recognition accuracy and the system security in a complex environment are remarkably improved.

In one embodiment, a method includes receiving a voice input having acoustic features from a first client system associated with a first user, determining emotions associated with the voice input based on one or more of the acoustic features by machine-learning models, determining facial features for a first extended-reality (XR) avatar representing the first user based on the emotions, and sending instructions for rendering the first XR avatar representing the first user to a second client system associated with a second user, wherein the first XR avatar is rendered with the determined facial features.

The invention discloses an automobile data recorder fatigue driving detection and reminding method based on multi-modal data fusion, and the method comprises the following steps: S1, collecting and preprocessing a face image through a vehicle-mounted camera, and carrying out the face detection and region positioning; s2, dynamically adjusting the size and the position of an image block by adopting a dynamic image blocking technology; s3, performing facial image feature extraction by using a visual converter; s4, acquiring vehicle multi-modal data and fusing the vehicle multi-modal data with the facial features; s5, optimizing a visual converter network structure and training hyper-parameters through a wolf pack optimization algorithm; s6, performing fatigue state analysis based on the optimized visual converter; and S7, when the fatigue state is detected, reminding the driver to have a rest through sound-light alarm. The fatigue driving detection and reminding method of the automobile data recorder is high in precision and high in real-time performance through a deep learning algorithm of multi-modal data fusion and optimization, and the recognition accuracy of fatigue driving and the robustness of the system are effectively improved.

The invention discloses an automatic conference summary generation method based on multi-source heterogeneous information fusion. The method comprises the following steps: extracting facial features of a speaker by using a face recognition technology and recognizing a face identity of the speaker; acquiring an audio signal through a microphone array, and recognizing a voiceprint identity of a speaker by using a voiceprint recognition technology; in combination with video and audio information, through a multi-source heterogeneous information fusion technology, alignment of video and audio data is carried out in time; through a sound source positioning technology, a voiceprint identity and a face identity are matched and aligned in space, and the identity of a speaker is accurately positioned and recognized; the spokesman and the speaking content are calibrated and separated, and it is ensured that the identity of the spokesman is accurately matched with the speaking content; and generating a spokesman abstract and a conference summary according to the calibrated speech content by using natural language processing and a deep learning model. The method is suitable for various conference scenes, the speaking content of different spokesmen can be recognized, the spokesman abstract is generated, and the efficiency and accuracy of conference summary generation are improved.

The invention discloses an intelligent digital human training method and system based on multi-modal interaction, and relates to the technical field of digital human training, and the method comprises the steps: designing a touch feedback strategy, and defining the output corresponding to a digital human according to different touch information; converting the voice input into a text; generating a natural language reply; converting into digital human voice output by using a voice synthesis technology; capturing dynamic characteristics of the motion and converting the dynamic characteristics into actions of a digital human; identifying and extracting facial feature points, and constructing a facial model of the digital human; analyzing relative positions and motions of the facial feature points, and mapping the relative positions and motions to a facial model of the digital human; aligning the time sequence of the audio with the lip movement track of the digital human; constructing a basic character model; analyzing intention and emotion input by the user in real time; a comprehensive corresponding strategy is generated, and actions, facial expressions, voices and appearance feedback which should be taken by the digital person in the current situation are determined.

The invention relates to the technical field of medical diagnosis systems, and discloses an AI physique diagnosis software analysis system which comprises an MCU module used for sending commands to execution modules and receiving and processing data signals of the execution modules to achieve diagnosis operation of the analysis system; the data acquisition module is used for collecting facial images and pulse information physiological indexes of the patient through an acquisition medium; the image processing module is used for analyzing facial features of the patient, the facial features comprise complexion, tongue coating and tongue quality, and corresponding evaluation data are generated through the obtained facial feature information. According to the AI physique diagnosis software analysis system, the accuracy and efficiency of traditional Chinese medicine physique identification are remarkably improved, the subjectivity and professional threshold are reduced, remarkable social and economic benefits are generated, and the system can more accurately evaluate the physique and display experimental data by using a deep learning algorithm and multi-modal data analysis.

The invention discloses a meta universe-based conference interaction method, and relates to the technical field of virtual conferences, and the method comprises the steps: extracting the facial features of participants, comparing the facial features with a pre-established facial feature library, obtaining the unique identity of each participant, and recognizing the identities of the participants; after the identities of the participants are determined, position changes of key body joint points of the participants are detected in real time, and specific actions of the participants are judged by analyzing relative positions and movement tracks between the joint points; the method comprises the following steps: receiving the acoustic characteristics of the participant language, converting the voice signal of the participant into a text, and analyzing the semantics and emotion of the text in combination with natural language processing to obtain the language expression intention of the participant. According to the method and the device, the optimal interaction mode can be dynamically selected from the preset library according to different interaction intentions, and the audio and video parameters and the interface layout are adjusted in real time, so that intelligent interaction in a multi-user scene is realized, the interaction experience of a conference is optimized, and the interaction effect can also be improved.

The invention discloses a multi-modal emotion recognition method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring text, video and audio data of a user and respectively performing feature extraction to obtain text features, audio features and facial features; the three features are input into a pre-trained multi-modal emotion recognition model, the multi-modal emotion recognition model comprises a first fusion module, a second fusion module, a third fusion module, a fourth fusion module and a classification module, the audio features and the text features are fused through the first fusion module, and audio text features are obtained; fusing the facial features and the text features by using a second fusion module to obtain facial text features; performing feature enhancement on the text features by using a third fusion module to obtain enhanced text features; fusing the audio text features, the face text features and the enhanced text features by using a fourth fusion module to obtain multi-modal features; and classifying the multi-modal features by using a classification module to obtain a sentiment classification result of the user.

The invention relates to the technical field of identity authentication, and discloses an identity authentication method and system based on a large-model multi-mode venue, and the method comprises the steps: obtaining a fingerprint image, a face image and behavior interaction data of a user; performing fingerprint feature extraction according to the fingerprint image to obtain a fingerprint feature vector; performing facial feature extraction according to the facial image to obtain a facial feature vector; performing feature fusion on the fingerprint feature vector and the face feature vector, and generating a fusion feature vector after data dimension reduction; performing behavior feature extraction according to the behavior interaction data to obtain a behavior feature vector; and after cross-modal mapping is carried out according to the fusion feature vector and the behavior feature vector, matching degree analysis is carried out in a preset unified semantic space, and an identity authentication result is obtained. The method has the following effect that the identification security of identity authentication can be improved.

A system for providing privacy measures for a computing device comprising: a memory a display configured to display images and / or data which may be subject to privacy concerns; at least one image sensor configured to provide image data of a user operating the computing device and image data of the environment surrounding the user within an image sensor field of view; at least one processor configured to: receive the image data to segment the image data to isolate the facial features of a person comprising a user and generate a user facial vector representative of characteristics of the segmented facial image data; and compare the user facial vector with facial vector data associated with one or more authorised device users stored in the memory; and a privacy controller module configured to determine a privacy threat alert based on the facial vector comparison; wherein, on receipt of a privacy threat alert from the privacy controller, the processor is configured to modify the display of the electronic device to preserve the privacy of the displayed image and / or data.

The invention provides a screen interaction method and system based on face recognition, and the method comprises the steps: obtaining a face image sequence triggered by a target user in a screen interaction scene, carrying out the matching of a target interaction mode from a preset interaction mode library according to the face dynamic behavior characteristics, and carrying out the matching of a target interaction mode based on the coordinates of a screen fixation region and the target interaction mode. Generating dynamic interaction content corresponding to the sight focus of the user, calling a pre-trained facial feature fusion model, performing multi-level feature extraction on a facial region in the facial image sequence, generating a user identity verification vector and an emotional state vector, and performing real-time adjustment on presentation parameters of the dynamic interaction content based on the multi-level feature extraction. And outputting the adjusted screen interaction signal. According to the invention, the user experience can be optimized while the interaction efficiency is improved.

The invention discloses an access control face recognition system and method based on a three-dimensional technology, and relates to the technical field of access control face recognition, and the recognition system comprises a face data acquisition module which is used for capturing three-dimensional face data of a target person by using a 3D camera; the multi-modal fusion module is used for fusing the three-dimensional face data and the traditional two-dimensional image data; the modeling module is used for generating an accurate three-dimensional face model; the dynamic adjustment module is used for creating a real-time dynamic adjustment model; the matching module is used for extracting facial features from the three-dimensional face model and matching the facial features with the three-dimensional face model in the database; and the identity verification and authorization module is used for carrying out identity verification according to the matching result. According to the method, three-dimensional face data and traditional two-dimensional image data are fused, the recognition accuracy and reliability can be improved, a real-time dynamic adjustment function is introduced, recognition parameters can be automatically optimized according to different environment light rays and angles, and better use prospects are brought.

A method includes obtaining image frames capturing one or more people in at least one scene and identifying features of the image frames. The method also includes providing the identified features to a trained spatiotemporal transformer machine learning model configured to generate a set of features for each of the one or more people. The set of features for each person includes facial features of the person and pose features of the person over time. The method further includes performing face identification using the facial features to generate one or more first embeddings representing at least one face of at least one person and performing gait identification using the pose features to generate one or more second embeddings representing at least one gait of at least one person. In addition, the method includes identifying at least one of the one or more people based on the first and second embeddings.

The invention provides a mouth shape generation method and device for voice driving, and relates to the technical field of natural language processing. The method comprises the following steps: based on an ffmpeg tool, carrying out image processing according to original video data to obtain spliced frame image data and facial feature points; according to the original audio data, performing feature extraction through a deep audio feature extractor to obtain audio features; performing feature fusion through an audio-video sequence feature fusion device to obtain fusion features; performing video generation through a lip action generator according to the facial feature points and the fusion features to obtain synthesized video data; optimizing a lip action generator according to the original video data and the synthesized video data; and performing video generation based on the depth audio feature extractor, the audio-video sequence feature fusion device and the optimized lip action generator to obtain target synthesized video data. The invention relates to a mouth shape generation method aiming at voice driving, which is high in video resolution and sufficient in facial texture detail reservation.

The invention discloses an angry emotion recognition method based on a video stream, and belongs to the technical field of emotion recognition. The method comprises the following steps: preprocessing a video stream of a visitor to obtain a time-space aligned standardized facial image sequence, a 3D skeleton sequence and a voice segment; performing feature extraction on the standardized facial image sequence by using a 3D-ResNet34 network and a micro-expression optical flow enhancement algorithm to obtain optical flow facial features; performing feature extraction on the 3D skeleton sequence by using a space-time diagram convolutional network to obtain limb action features; performing feature extraction on the voice segments to obtain voice features; and performing fusion processing on the facial features, the body movement features and the voice features by using a multi-head space-time cross attention mechanism and an expansion time sequence convolution algorithm, and outputting an angry emotion recognition result through dynamic weight distribution classification. According to the method, the angry emotion recognition accuracy of the system is greatly improved on the whole.

The invention relates to the technical field of identity continuous verification, and particularly discloses an Internet of Things identity continuous verification method based on a multi-mode technology, which comprises the following steps: continuously collecting face image data and keyboard input mode data of a user in an interaction process of the user and Internet of Things equipment; the deep learning technology is adopted to process the face image data and the keyboard input mode data of the user to extract the face semantic features and the keyboard use behavior mode features of the user, and then cross-modal association learning interaction fusion is carried out on the face features and the keyboard use behavior mode features of the user, so that the user experience is improved. Therefore, the potential association between the two is captured, and the continuous verification of the user identity is realized. Through the mode, the behavior of the user can be continuously monitored, and an alarm is given out in time when the identity of the user is abnormal, so that the safety of the Internet of Things equipment is effectively improved, and meanwhile, the problem that the user experience is reduced possibly caused by a traditional continuous verification method is avoided.