134 results about "Affective computing" patented technology

An end-side-cloud three-in-one active chat robot system for high-emotional merchants comprises an end side, a local edge server and a cloud end, and the end side is used for collecting multi-modal data of a user and performing local lightweight real-time processing and response execution; the local edge server is used for receiving and fusing the multi-modal features and the context information from the end side, and carrying out sentiment calculation, dialogue management and active trigger decision making with medium complexity; the cloud end is used for operating a super-large-scale model and providing global knowledge management, long-term user portrait storage and model training optimization; and the end side, the local edge server and the cloud end carry out cooperative communication through an encrypted channel to form a distributed intelligent processing architecture. According to the method, global optimization is realized by integrating end-side lightweight sensing, edge multi-modal fusion and cloud long-term memory. A composite finite state machine (FSM) active questioning mechanism is combined with sentiment calculation, and is different from traditional rule type triggering. Off-line and on-line fusion scheduling and multi-agent role playing are combined to be applied to a high-emotional-quotient interaction scene, and the simulation is improved. Multi-modal emotion perception circulation is introduced, and the problem of single text emotion misjudgment is solved.

A facial expression recognition method based on grid attention and pyramid segmentation attention belongs to the technical field of deep learning image processing and emotion calculation, and comprises the following steps: introducing a grid attention mechanism and a pyramid segmentation attention mechanism on a ResNet101 large model, capturing local expression detail features through a grid attention module, establishing multi-scale global feature association by using a pyramid segmentation attention mechanism; hierarchical segmentation is carried out on the backbone network, and information from multiple hierarchies is effectively fused; and dynamically balancing the contribution degree of each level of features through learnable parameters, and integrating the extracted features to judge the expression category. Experiments show that the model achieves the recognition accuracy superior to that of a traditional model on a public data set in natural scenes such as complex illumination and posture change. According to the method, a solution with high robustness is provided for facial expression recognition in a complex environment, and the method has important application value in the fields of intelligent human-computer interaction, mental health assessment and the like.

The invention relates to the technical field of artificial intelligence and emotion calculation, and discloses an emotion state evaluation method based on multi-modal data, and the method comprises the steps: obtaining and decoupling an implicit signal, an explicit signal and situation data of an evaluated object; and respectively generating implicit features, explicit features and context vectors through double-flow decoupling coding and context knowledge graph coding. And the core engine determines the internal emotional state and the extrinsic emotional representation in parallel, and dynamically regulates and controls a bridging conversion process from the internal state to the predicted extrinsic representation by utilizing the situation vector. Finally, the emotion regulation intensity is calculated by comparing actual and predicted extrinsic representations, and the emotion regulation intensity, the internal state and the extrinsic representations are jointly used as a multi-dimensional evaluation result to be output. According to the method, the dynamic relationship and the adjustment strategy between the internal feeling and the external expression can be disclosed, a deeper and more three-dimensional analysis view angle is provided for the fields of mental health, human-computer interaction and the like, and the method has remarkable application value.

The invention relates to the technical field of computer vision and emotion calculation, in particular to an emotion real-time recognition and long-term monitoring method based on facial expressions. According to the method, an emotion recognition result is obtained by recognizing a high-definition facial image, and the emotion recognition result, environment information and physiological state data are fused to obtain time-space aligned multi-modal data; performing emotional causal analysis based on the multi-modal data, and judging emotional causes by combining a rule engine and a machine learning model: outputting a real-time emotional state recognition result and a periodic emotional report according to the emotional causes, and performing differentiated feedback according to the emotional causes. According to the method, through multi-source data fusion and a causal inference mechanism, the accuracy and interpretability of emotion recognition are effectively improved, and the technical span from passive recognition to personalized active intervention is realized.

The invention discloses a multi-modal emotion calculation method and system, and mainly relates to the technical field of artificial intelligence and human-computer interaction. Comprising the following steps: synchronously collecting voice, visual and tactile multi-modal data of a user, and carrying out feature extraction on each modal data; performing cross-modal fusion on the extracted multi-modal features, including time sequence alignment and semantic association modeling, and generating a global emotion feature vector; performing dynamic emotion reasoning based on the global emotion feature vector, and outputting an emotion category and emotion intensity; generating a tactile feedback signal according to the emotion category and the emotion intensity, and driving an actuator to output; and updating the emotion memory graph based on user interaction data to complete personalized model self-adaption. The method has the beneficial effects that high-precision and low-delay emotion recognition and natural tactile feedback in a cross-culture scene are realized, and the core bottleneck of dynamic drift adaptation failure and emotion-behavior feedback decoupling in the prior art is solved.

The invention relates to the technical field of handwriting analysis, and discloses a handwriting analysis method and system based on emotion calculation multi-modal recognition, and the method comprises the steps: obtaining handwriting data of a writer, and obtaining handwriting features through employing a multi-modal collection technology, specifically, dividing a writing process into a plurality of writing time periods according to time, and each time period does not exceed 5 minutes; collecting pen point pressure data by using a pressure sensor, collecting writing speed data by using a speed sensor, and collecting font structure data by using an image sensor; for each section of handwriting, executing a feature extraction strategy, fusing dynamic and static features, and generating a multi-modal feature set; inputting the feature set into a pre-trained emotion calculation model; multi-modal fusion breaks through the limitation of a traditional single mode, handwriting features are comprehensively described through a 200 + index, dynamic and static dimensions are combined, and the comprehensiveness of analysis is improved; the 300 + submodel and the million-level sample library ensure the analysis precision, the deep learning architecture captures feature association, and the generalization ability of the model is enhanced through cross validation and online learning.

The invention discloses a student psychological risk perception method based on multiple modes, and relates to the technical field of emotion calculation and intelligent education. The method comprises the following steps: firstly, extracting a facial expression feature vector and a voice intonation feature vector respectively by using a convolutional neural network and Fourier transform through a collected video stream and an audio stream; then adaptive denoising processing is carried out on environmental interference, timestamp alignment and dynamic time warping are carried out on the denoised multi-modal data, time sequence synchronization is ensured, and corrected multi-modal sequence data are formed; then, dynamic emotion track features are extracted from the sequence data, a preliminary emotion state label is generated by comparing the dynamic emotion track features with a baseline threshold value, and the threshold value is adaptively updated in combination with historical data so as to improve the judgment accuracy; and finally, aggregating the emotional state labels of a plurality of students to generate a visual group emotional thermodynamic diagram so as to realize macroscopic perception of group psychological risks. The accuracy, robustness and visualization degree of student psychological state analysis are effectively improved, and an efficient technical means is provided for campus psychological early warning.

The invention provides an emotion and cognition driven children education interaction system and method, and relates to the technical field of man-machine interaction. Emotion is quantified through an emotion calculation module by using a titer awakening degree two-dimensional continuous model, and an emotion migration slope is predicted in combination with a time sequence convolutional network; accurate description and prospective prediction of the emotion dynamics of children are realized, and the accuracy and initiative of emotion guidance are remarkably improved; secondly, the constructed knowledge graph endows the nodes with three-dimensional dynamic attributes of mastery degree, interest intensity and associated confusion degree, and real-time updating is carried out according to interaction data, so that the cognitive development trajectory of children is dynamically described, and a solid foundation is provided for personalized teaching; and finally, proposals are actively pushed by setting collaborative windows of interest, cognitive correction and the like, and a structured knowledge injection interface is provided, so that parents are converted from passive supervisors to common constructors of education contents, and deep fusion of family intelligence and artificial intelligence is realized.

The invention relates to the field of intelligent speech synthesis and emotion calculation, and discloses a digital human tour guide speech generation method, system and device and a storage medium, the digital human tour guide speech generation method comprises the following steps: S1, constructing a user mental model comprising an initial knowledge state of a user for a knowledge graph; s2, on the basis of the model, predicting and evaluating candidate narrative paths, planning an optimal path and determining an expected mental state; s3, multi-modal explanation content is generated and broadcasted according to the optimal path; s4, collecting real-time feedback of the user to obtain a real mental state; and S5, comparing the real state with the expected state, calculating a prediction deviation, and dynamically calibrating the user mental model for subsequent planning according to the prediction deviation. According to the method, prospective path planning is carried out by constructing the mental model, closed-loop calibration and robustness evaluation are combined, and personalized explanation which is accurate, stable and free of lag adjustment is achieved.

The invention discloses a multi-user emotion recognition and digital human feedback method based on behavior data, and belongs to the field of artificial intelligence emotion calculation. Sensing user approaching, identifying and confirming the identity of a registered user, distributing an identifier, distributing an identifier for a visitor, and establishing an independent session channel in a multi-user scene; capturing time sequence dialogue behavior data containing voice rhythm features, dialogue interaction modes and linguistic features in real time; inputting the data into a sequence information processing model based on a state space theory and provided with a data dependence selection mechanism and an emotion analysis model spliced by static context features, and outputting multi-dimensional emotion and cognitive state tags; generating an emotion support strategy through a decision-making model in combination with the personal file of the user; and generating a multi-mode control instruction, and driving the digital human to synchronously present special effects such as expressions and the like. According to the method, privacy concerns are eliminated through non-intrusive collection, the method is adaptive to multi-user scenes, deep cognitive states can be recognized, and interaction effectiveness is improved for a long time.

The invention relates to the technical field of digital human voice interaction, and discloses a digital human live voice interaction system fused with emotion calculation. The system constructs an emotional response time window by acquiring a user voice input data stream, the starting point of the emotional response time window is the end moment of the user voice input data stream, and the end point is obtained by subtracting the necessary duration of voice response synthesis from the preset maximum response cut-off moment. The system obtains an emotional state vector of the user in real time and judges whether the emotional state vector reaches an emotional intensity threshold value or not; and if so, predicting the total generation duration of the digital human voice response. And when the residual duration of the emotional response time window is equal to the total generation duration, taking the window starting point as a voice response starting moment, and controlling the digital human to start voice response generation. According to the system, the voice response matched with the emotional state of the user is generated through refined time window management and emotional state perception, invalid response and interaction delay are reduced, and the naturalness of digital human live broadcast voice interaction and the user experience are improved.

The invention provides a customer service response system and method based on artificial intelligence. The customer service response system comprises a client, an intelligent access network, a core AI engine system, a multi-modal understanding system, a federated learning coordinator, a knowledge management system, an emotion calculation engine system, a dynamic decision system and a digital ethical arbitration system. According to the method, multi-dimensional data such as texts, voices and vision are fused through a multi-modal understanding system, more accurate user intention recognition is achieved in combination with a cross-modal entanglement network and intention conflict detection, and the misjudgment problem caused by traditional single-modal analysis is reduced; the emotion calculation engine is combined with physiological signals, voice emotion analysis and micro-expression recognition to dynamically sense the emotion state of the user, so that customer service interaction is more similar.

The invention provides an intelligent triage method and system for outpatient and emergency treatment, and relates to the technical field of medical information. The method comprises the steps that physiological data, symptom description text data, facial image data and voice audio data of a patient are collected through a multi-mode sensor; analyzing the image and audio data by using an emotion calculation engine to generate emotion state data; processing the physiological and text data through an analysis model to generate illness state emergency degree data; fusing emotion and illness state data to form a preliminary triage result; in combination with the real-time medical resource state data, a triage decision-making scheme including department allocation, processing priority and personalized pacifying strategies is generated through a dynamic decision-making device; finally, guiding and pacifying are executed through the execution terminal and the interaction robot, automation, intelligence and humanization of the triage process are achieved, the triage efficiency and accuracy are remarkably improved, medical resource configuration is optimized, and care for psychological needs of patients is enhanced.

The invention relates to the technical field of vocational ability evaluation, in particular to a student-based vocational ability evaluation system and method, and the system comprises a data collection module which is used for collecting multi-source data and evaluation data of students; the vocational ability modeling module is used for establishing a vocational ability index system based on a vocational ability model comprising profession, method and social ability dimensions; and the data processing and calculating module is used for preprocessing the multi-source data, calculating ability scores of professional and method dimensions according to the vocational ability index system, and calculating ability scores of social ability dimensions through natural language processing, emotion calculation or behavior log analysis. In the invention, the multi-source data and the evaluation data of the trainees are collected, and the vocational ability model including profession, method and social ability dimensions is established, so that the problem that the comprehensive vocational ability of the trainees is difficult to comprehensively reflect due to the fact that most traditional vocational ability evaluation adopts questionnaires or single evaluation is solved.

The invention provides a mental health risk early warning and intervention system based on multi-modal emotion calculation, and belongs to the field of mental health analysis, and the system comprises a multi-modal fusion module which is used for obtaining multi-modal data, carrying out the constraint and fusion of the multi-modal data, and generating a fused emotion feature; the risk prediction module is used for calculating a mental health risk probability and an emotional state according to the fused emotional characteristics through a two-stage accumulation risk prediction algorithm, and performing mental health risk early warning based on the emotional state; and the strategy recommendation module is used for extracting emotion track characteristics based on the fused emotion characteristics and the mental health risk probability, and carrying out intervention strategy recommendation to obtain an intervention strategy of the current mental health risk. According to the invention, through the multi-modal fusion module, the risk prediction module and the strategy recommendation module, four modal data of video, audio, text and physiology are processed at the same time, a nonlinear accumulation process of mental health risks is accurately modeled, and risk early warning and intervention capabilities are improved.

The invention relates to the technical field of brain-computer interfaces and emotion calculation, in particular to an electroencephalogram emotion recognition method based on adaptive multi-stream graph fusion and gated space-time Transform, which comprises the following steps: constructing and training an electroencephalogram emotion recognition model, and inputting to-be-detected signal data into the trained electroencephalogram emotion recognition model to obtain a detection result; the electroencephalogram emotion recognition model comprises a self-adaptive multi-stream graph fusion network, a gating enhanced time sequence block and an emotion classification head; the average accuracy and the F1 score of the method are superior to those of the existing mainstream model, and the superiority of the method in the aspect of improving the personalized emotion recognition performance is proved.

The invention discloses a multi-modal sentiment analysis system and method based on dynamic routing and feature decoupling, and relates to the field of artificial intelligence and sentiment calculation. The system comprises a feature decoupling module, a decoupling guide routing module and a dynamic attention fusion module. The method comprises the following steps: firstly, decomposing an original feature of each input mode into a mode sharing feature and a mode specific feature through a feature decoupling module; aggregating the modal sharing features to obtain enhanced modal sharing features; a decoupling-guided routing module generates a group of modal weights for each sample in a self-adaptive manner on the basis of the decoupled modal importance weights; the dynamic attention fusion module performs dynamic weighting and cross-modal attention fusion on the modal specific features by using the modal weight to obtain enhanced specific features; and finally, fusing the enhanced specific features and the enhanced modal sharing features to carry out emotion prediction. According to the method, the defect that an existing dynamic fusion method makes a decision on entanglement features is overcome, and more accurate modal importance evaluation and fusion are achieved.

The invention relates to the technical field of intelligent education, in particular to a learning resource recommendation method, system and device, a medium and a product. According to the method, basic information and historical learning behavior data of a target object are obtained, an initial interest map is constructed in combination with a graph neural network algorithm, emotion calculation is performed according to multi-modal emotion data of the target object, a comprehensive emotion vector is generated, and therefore the target object is obtained according to a first interest vector of the initial interest map and the comprehensive emotion vector. And determining a learning resource recommendation result so as to improve the accuracy and adaptability of learning resource recommendation and further improve the learning efficiency of target objects such as children.

The invention discloses a mode-missing-oriented multi-mode small sample emotion recognition method and device, and relates to the technical field of emotion computation.The method comprises the steps that S1, electroencephalogram data and eye movement data are obtained; s2, constructing a multi-modal small sample emotion recognition model comprising a feature extraction module, a missing modal reconstruction module, an uncertainty perception adaptive fusion module and a small sample classification module; s3, training the multi-modal small sample emotion recognition model by using the electroencephalogram data and the eye movement data to obtain a trained multi-modal small sample emotion recognition model; and S4, performing emotion recognition by using the trained multi-modal small sample emotion recognition model. According to the method, the multi-modal small sample emotion recognition model is constructed, and the technical effect of remarkably improving the accuracy, robustness and generalization ability of multi-modal emotion recognition in complex reality scenes such as modal deficiency, labeled sample scarcity and cross-subject distribution offset is achieved.

The invention discloses an intelligent music recommendation method based on emotional perception and acoustic features, and relates to the technical field of intelligent recommendation systems and emotional computation.The method comprises the steps that physiological signals, music acoustic features and historical behavior data of a user are collected; preprocessing the multi-source features and mapping the multi-source features to the same dimension to construct a fusion matrix; building a double-branch deep learning model, fusing features through an attention mechanism and training parameters; an evolutionary algorithm is adopted to optimize hyper-parameter screening optimal combination; generating a recommendation list matched with the real-time emotion and preference; and continuously collecting user interaction data, and regularly and incrementally training the dynamic update model. According to the method, emotion and behavior dual-drive recommendation is achieved by fusing physiological signals and acoustic features, emotion perception is accurate, recommended content fits the real-time mood, model optimization is efficient, recommendation precision and diversity are remarkably improved, and the music consumption experience of a user is greatly improved.

The invention discloses an emotion classification method and system for heart electromagnetic signals of heart rate variability based on topology analysis, and relates to the technical field of biomedical signal processing and emotion calculation, and the emotion classification method comprises the following steps: preprocessing magnetocardiogram signals or electrocardiogram signals; positioning R waves, extracting an RR interval sequence, and extracting statistics of topological feature points of the persistent graph as topological features; and performing fusion through the time sequence branch, the topological branch and the fusion layer to generate a fusion vector, and classifying the emotional state based on the fusion vector. According to the method, the signal quality and the anti-interference capability are remarkably improved, topological data analysis (TDA) is innovatively introduced, quantitative characterization of a heart rate variability nonlinear dynamic structure is achieved, and the capturing capability of the model to a key physiological mode is enhanced; the problem of overfitting under small samples is effectively relieved, the accuracy, robustness and cross-individual generalization ability of emotion classification are remarkably improved, and a new method is provided for emotion calculation.

The invention discloses a rehabilitation nursing system for treating autism spectrum disorder children based on AI intelligent accompanying. The rehabilitation nursing system comprises an AI intelligent accompanying robot, a multi-mode interaction module, a personalized intervention scheme generation module, a real-time monitoring module and a cloud data platform. The system obtains facial expressions, voices, actions and electroencephalogram signals of a child patient in real time through a camera, a sensor, electroencephalogram equipment and other multi-mode data acquisition modules, emotion recognition is conducted in combination with a convolutional neural network, the emotion state is analyzed through an emotion calculation model, and an autism diagnosis and treatment knowledge base is constructed through a knowledge graph. The system generates a personalized intervention scheme based on a deep learning algorithm, and dynamically adjusts the training difficulty through real-time intervention means such as virtual coaches, action correction, dialogue guidance and the like in combination with reinforcement learning. The system provides an efficient and personalized home rehabilitation solution for children with autism, the social ability is improved, and the rehabilitation period is shortened.

The invention relates to the technical field of artificial intelligence, and discloses a child emotion calculation device and method, a storage medium and an accompanying toy, the child emotion calculation device comprises a sensing module configured to collect modal data including facial expressions, voices, contact behaviors, motion postures and environment information; a calculation module configured to receive the modal data and convert the modal data into digital features; the emotion understanding module is configured to identify a user emotion state and an interaction intention according to the digital characteristics; the decision reasoning module is configured to generate a personalized behavior strategy according to the emotional state and the interaction intention; and the interactive execution module is configured to output a physical feedback signal to an execution terminal according to the personalized behavior strategy. According to the method, simultaneous analysis of modes such as facial expressions, voices, intonations and tactile behaviors of children is realized through a multi-mode cooperation mechanism, the emotion recognition accuracy is greatly improved, and professional psychological accompanying support can be provided for the children.

The application discloses a kind of based on bidirectional attention mechanism and multi-level fusion adolescent emotion recognition method and system, it is related to artificial intelligence and affective computing technical field.The method includes: obtaining and preprocessing the audio and video data of adolescent, extracts initial feature vector;Characteristics are input into the multi-level fusion model based on the architecture of Transformer, sequentially early, mid and late fusion are carried out by bidirectional cross attention mechanism, to capture subtle signs, semantic conflict and decision weighting respectively;Dynamic weighting is carried out to anti-interference by introducing gate mechanism to fusion characteristics;Finally, output emotion category.Corresponding system contains data acquisition, feature processing, dynamic gate and output module.The application solves the problem of inconsistent expression of adolescent emotion and modal heterogeneity by multi-level progressive fusion and dynamic loss optimization, significantly improves the recognition accuracy, robustness and generalization ability of complex emotion.

The invention relates to the technical field of child emotion recognition, in particular to an emotion calculation method and device for child expressions, and the method comprises the steps: collecting and marking the multi-modal dynamic expression data of a plurality of Asian children, and constructing an enhanced mixed expression database according to the multi-modal dynamic expression data of the Asian children; performing iterative training on a pre-constructed deep convolutional neural network model by using the enhanced mixed expression database until a preset iterative period is reached, so as to obtain a deep time sequence emotion calculation model; and inputting the multi-modal dynamic expression of the child to be recognized into the depth time sequence emotion calculation model to calculate the current emotion of the child. Therefore, the problems that an existing child emotion recognition model is mostly based on European and American adult database training and mostly adopts single-frame static image analysis, and the complete dynamic process of expressions cannot be captured, so that the accuracy is remarkably reduced, the model robustness is poor, and effective application in real and continuous interaction scenes is difficult are solved.

This application provides a joint emotion recognition method and system based on action unit-driven attention, relating to the fields of computer vision and affective computing. The method includes: inputting a feature map into an action unit (AU)-driven attention branch, processing it through a sigmoid activation function to obtain an activation vector, and inputting this vector into a learnable mapping layer; projecting the AU information back into the spatial geometric space to generate a single-channel spatial attention map, which is then multiplied element-wise with the feature map to obtain a weighted feature map; determining continuous values ​​for discrete emotion category probabilities, valence, and arousal; constructing a total loss function; and training the model using the acquired AffectNet and Aff-wild2 datasets to determine the target model for emotion recognition. This application achieves end-to-end joint optimization of AU detection and emotion recognition, significantly enhancing the interpretability and generalization ability of the model while improving emotion recognition accuracy, enabling it to better adapt to the emotion recognition needs in complex scenarios.

The invention provides a personalized voice cloning accompanying robot system and method based on a large model, and relates to the technical field of artificial intelligence voice interaction and emotion computing, the system comprises an end side and a cloud end, and the method comprises the following steps: collecting a user voice sample, and carrying out preliminary feature extraction; finely adjusting the TTS model by adopting a feed-shot learning algorithm to generate personalized sound; inputting multi-modal data to an emotion fusion LLM engine, and outputting a response text with an emotion tag; voice response is synthesized through local reasoning; and outputting a voice response. According to the method, the problems of low cloning accuracy, insufficient emotion fusion, high privacy risk, high response delay and the like of an existing voice cloning technology in an accompanying robot are solved, and high accuracy (gt; 95%) few sample cloning, emotional response generation, end-side privacy protection and low latency (lt; and the applicability of the accompanying robot in children education and elderly nursing scenes is improved.

The invention relates to the crossing field of computer vision and emotion calculation, and particularly provides a micro-expression real-time emotion research and judgment method and device, and the method comprises the following steps: S1, multi-mode data collection; s2, micro-expression feature extraction; s3, multi-modal fusion optimization is carried out; s4, real-time emotion tracking; and S5, edge-cloud collaborative study and judgment. Compared with the prior art, the method has the advantages that the dynamic compensation algorithm can effectively improve the micro-expression detection rate, the edge calculation can greatly reduce the response time, the feature compression algorithm can save the bandwidth, and meanwhile, the method supports recognition of various basic emotions and composite emotions, and a self-adaptive adjustment mechanism can switch analysis modes in different scenes such as medical treatment and security.

The invention discloses intelligent teaching equipment integrating emotion calculation and feedback adjustment, and relates to the technical field of intelligent teaching. The data acquisition module is used for acquiring multi-modal emotion related data of the learner; the emotion calculation module is used for receiving the multi-modal emotion related data, preprocessing the multi-modal emotion related data, performing emotion state recognition through a preset emotion recognition model, and outputting an emotion recognition result; the teaching control module comprises a teaching content database and a feedback adjustment strategy library, and is used for receiving the emotion recognition result, matching a corresponding teaching adjustment parameter from the feedback adjustment strategy library according to the emotion recognition result, adjusting the teaching content in the teaching content database based on the teaching adjustment parameter, and generating an adjusted teaching instruction; and the feedback output module is used for receiving the teaching instruction and displaying the adjusted teaching content to the learner in a preset output form. According to the method, the accuracy and comprehensiveness of emotion recognition are remarkably improved.

The invention provides a multi-modal personality recognition method based on a multi-scale network model, and belongs to the technical field of emotion calculation. The method comprises the following steps: acquiring to-be-detected video data and to-be-detected audio data; extracting image features of the video data through discrete wavelet transform in combination with a convolutional neural network; extracting audio features of the video data through a multi-layer perceptron; splicing the image features and the audio features to obtain comprehensive features; and obtaining a personality identification result by using the comprehensive features. According to the method, audio features and video features are respectively extracted and spliced, information from different modes is combined, the complementarity among the modes is utilized, so that the prediction accuracy and robustness are improved, meanwhile, the model is allowed to independently process data of each mode, then the features of the data are flexibly combined, and the prediction accuracy and robustness are improved. And greater flexibility is provided for designing and optimizing the model.