Emotion recognition interaction method and device based on AI intelligent analysis and electronic equipment

Abstract

The invention relates to the technical field of emotion recognition, in particular to an emotion recognition interaction method and device based on AI intelligent analysis and electronic equipment, and the method comprises the steps: obtaining video, audio and electric signal data synchronously collected by a camera, a microphone, an EEG and a galvanic skin sensor, and extracting first emotion feature data, inputting the first emotional feature data into a preset AI reinforcement learning strategy network to obtain emotional uncertainty output by the AI reinforcement learning strategy network, obtaining second emotional feature data, sequentially performing feature weighting and feature fusion on the second emotional feature data, and inputting a feature fusion result into a dual-channel convolutional neural network to obtain a second emotional feature data; the emotion recognition result feature vector is obtained, the emotion recognition result feature vector and a label of a preset emotion feature information database are compared and analyzed based on an increment correction algorithm, the accuracy is calculated, correction is conducted, and the problems that in the prior art, emotion recognition is low in accuracy and real-time performance are solved.

Description

Technical Field

[0001] This invention relates to the field of emotion recognition technology, and in particular to an emotion recognition interaction method, device, and electronic device based on AI intelligent analysis. Background Technology

[0002] Emotion recognition has evolved from single-modal (video only or only physiological signals) to multimodal fusion. A common approach is to extract features from signals such as camera, microphone, EEG, and electrodermal signals, then concatenate or vote them, and finally use CNN / RNN / Transformer for classification. Some studies have introduced reinforcement learning to regulate dialogue strategies and incremental learning to alleviate model aging. However, the sensor clocks are independent, the acquisition frame rate / sampling rate is fixed, and the feature fusion and model update processes are disconnected.

[0003] However, the existing technology still has the following shortcomings: due to different clock sources for each module, cross-modal data is difficult to align at the millisecond level, resulting in poor real-time performance; fixed sampling strategies lead to redundancy or missed detections, and emotion inflection points are easily lost, resulting in reduced accuracy of emotion recognition. Summary of the Invention

[0004] The purpose of this invention is to provide an AI-based intelligent analysis-based emotion recognition interaction method, device, and electronic device to solve the problems of low accuracy and real-time performance in existing emotion recognition technologies.

[0005] To achieve the above objectives, this invention provides an AI-based intelligent analysis-based emotion recognition interaction method, comprising the following steps: S1. Construct a common clock pulse for the camera, microphone, EEG, and electrodermal sensor; acquire video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor; analyze the video, audio, and electrical signal data; and extract the first emotion feature data. S2. Input the first emotion feature data into the preset AI reinforcement learning strategy network to obtain the emotion uncertainty output by the AI ​​reinforcement learning strategy network. Adjust the working step size of the camera, microphone, EEG, and skin conductance sensor based on the emotion uncertainty to obtain the second emotion feature data. S3. Perform feature weighting and feature fusion on the second emotion feature data in sequence, and input the feature fusion result into a dual-channel convolutional neural network to obtain the emotion recognition result feature vector. S4. Based on the incremental correction algorithm, the feature vector of the emotion recognition result is compared and analyzed with the labels of the preset emotion feature information database to calculate the accuracy and make corrections.

[0006] In some embodiments of this application, in S1, acquiring video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor includes: Synchronous pulse markers are implanted in the physical layer of the sensor, and a self-supervised alignment network is used to use the EEG signal of the next 1 second as the prediction target of the speech / video segment to perform modality alignment with zero manual annotation. The modal alignment results are subjected to noise reduction, fusion, and compression sequentially using tensor fusion and information bottleneck algorithms to obtain synchronized video, audio, and electrical signal data.

[0007] In some embodiments of this application, in S1, analyzing video, audio, and electrical signal data to extract first emotional feature data includes: Video modal features include facial motion unit intensity temporal sequence, global / local optical flow field, 3D head pose angle, and eye gaze vector; Audio modal features include 128-dimensional log-Mel spectrum, fundamental frequency F0, normalized energy, speech rate, and first and second order MFCC differences; EEG electrical signal modal characteristics, including 8-channel 5-band power spectral density, differential entropy, and inter-channel phase lock-in value; Skin conductance modal characteristics, including mean tonic value, phasic peak count, mean rise slope, and recovery half-life.

[0008] In some embodiments of this application, in step S2, the first emotion feature data is input into a preset AI reinforcement learning policy network, and the emotion uncertainty output by the AI ​​reinforcement learning policy network is obtained, including: The four modality features are concatenated, as shown in the expression: ; in, To concatenate vectors, For video modal features, For audio modal features, EEG electrical signal modal characteristics, Characteristics of skin conductance modes; The AI ​​reinforcement learning policy network first outputs the emotion probability distribution through one forward inference, expressed as: ; in, This is the normalized exponentiation function. To introduce a nonlinear activation function for sparsity computation, The learnable parameters for the second layer of the AI ​​reinforcement learning policy network. The first layer of learnable parameters for the AI ​​reinforcement learning policy network. , All are preset process parameters; The sentiment uncertainty u is calculated using Shannon entropy, and the expression is: ; in, The probability estimation bias for the i-th emotion type.

[0009] In some embodiments of this application, in S2, adjusting the working step size of the camera, microphone, EEG, and electrodermal sensor based on emotion uncertainty to obtain second emotion feature data includes: Normalize the emotional uncertainty u to Within the range, mapping is performed according to three threshold segments; when When, maintain a low power step size; when When, increase to the preset middle step size; when At that time, increase to the preset high-resolution step size; Data on the second emotion feature was obtained by collecting data based on the adjusted work step size.

[0010] In some embodiments of this application, in step S3, the sequential feature weighting and feature fusion of the second emotion feature data includes: The four modal features of the second emotion feature data are weighted and fused, and the expression is as follows: ; Where z is the 128-dimensional emotion vector after feature fusion. To output the linear transformation matrix, This is the weight matrix of the fusion layer. The weighted results of the video modality, audio modality-EEG electrical signal modality of the second emotion feature data. For the second emotion feature data, the skin conductance modal features, This is the bias vector for the fusion layer. This is the output layer bias vector.

[0011] In some embodiments of this application, in step S3, the feature fusion result is input into a dual-channel convolutional neural network to obtain an emotion recognition result feature vector, including: For the spatial channels of a two-channel convolutional neural network, the expression is: ; in, The spatial channel output feature map is then subjected to ReLU nonlinear activation. The convolution kernel weight matrix for the spatial channels. The matrix offset for the spatial channel; For the temporal channels of a two-channel convolutional neural network, the expression is: ; in, The time-series channel output feature map is then subjected to ReLU nonlinear activation. For the depthwise separable convolution parameters of the temporal channels, This is the bias vector for the timing channel; Will and The vectors are concatenated and flattened along the channel dimension to generate a unified feature vector. The expression is: ; Perform full-connection dimensionality reduction: ; in, This is the feature vector of the emotion recognition result. For the final fully connected dimensionality reduction matrix, For the final fully connected dimensionality reduction matrix bias, This represents the feature dimension of the final emotion recognition result.

[0012] In some embodiments of this application, an emotion recognition interactive device based on AI intelligent analysis is also disclosed, including: The data extraction module is used to construct a common clock pulse for the camera, microphone, EEG, and electrodermal sensor, acquire video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor, analyze the video, audio, and electrical signal data, and extract the first emotion feature data. The step size adjustment module is used to input the first emotion feature data into the preset AI reinforcement learning strategy network, obtain the emotion uncertainty output by the AI ​​reinforcement learning strategy network, and adjust the working step size of the camera, microphone, EEG, and skin conductance sensor based on the emotion uncertainty to obtain the second emotion feature data. The emotion recognition module is used to perform feature weighting and feature fusion on the second emotion feature data in sequence, and input the feature fusion result into a dual-channel convolutional neural network to obtain the emotion recognition result feature vector. The correction module is used to compare and analyze the feature vector of the emotion recognition result with the labels of the preset emotion feature information database based on the incremental correction algorithm, calculate the accuracy and make corrections.

[0013] In some embodiments of this application, an electronic device is also disclosed, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements an emotion recognition interaction method based on AI intelligent analysis.

[0014] The advantages and beneficial effects of this invention compared to the prior art are: 1. This invention achieves synchronous acquisition of data from a camera, microphone, EEG, and electrodermal sensor using a co-current clock pulse. It also utilizes an AI reinforcement learning strategy network to map emotional uncertainty into sensor working step size, automatically reducing the frame rate / sampling rate during periods of emotional stability and increasing the acquisition density at millisecond levels during moments of emotional transition. This reduces redundant data and transmission bandwidth, while capturing micro-expressions and differential electrical responses at inflection points, thus reducing end-to-end latency and improving the real-time performance of data processing.

[0015] 2. This invention uses feature weighting and a dual-channel convolutional neural network to deeply mine complementary information between modalities, and uses an incremental correction algorithm to compare the recognition result vector with the database labels online and update the network weights by backpropagating the error. This can continuously suppress errors caused by individual, environmental and label drift, and achieve long-term stable high-precision emotion recognition in various application scenarios.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 This is a flowchart of the emotion recognition and interaction method based on AI intelligent analysis in an embodiment of the present invention; Figure 2 This is a block diagram of an AI-based intelligent analysis-based emotion recognition interactive device according to an embodiment of the present invention. Detailed Implementation

[0018] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," and "connect" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] like Figure 1 As shown, this invention provides an emotion recognition interaction method based on AI intelligent analysis, including the following steps: S1. Construct a common clock pulse for the camera, microphone, EEG, and electrodermal sensor, acquire video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor, analyze the video, audio, and electrical signal data, and extract the first emotion feature data.

[0021] EEG (electroencephalography) is a technique that non-invasively and in real-time records the electrical activity of brain neurons by placing electrodes on the scalp. It plots the weak voltage fluctuations generated by the synchronous firing of brain cells as waveforms along a time axis. Skin conductance represents the changes in skin electrical conduction when the body is stimulated, generally expressed as resistance and its logarithm or conductance and its square root. It can serve as an indicator of the function of the sympathetic nervous system. Using the same clock source to simultaneously power four sensors—camera, microphone, EEG, and skin conductance—ensuring complete alignment of all sampling times eliminates microsecond-level drift caused by individual crystal oscillators, guaranteeing zero-time-difference synchronization of video frames, audio frames, brain waves, and skin conductance at millisecond or even sub-millisecond levels, providing a precise time reference for subsequent cross-modal fusion.

[0022] S2. Input the first emotion feature data into the preset AI reinforcement learning strategy network to obtain the emotion uncertainty output by the AI ​​reinforcement learning strategy network. Adjust the working step size of the camera, microphone, EEG, and skin conductance sensor based on the emotion uncertainty to obtain the second emotion feature data.

[0023] The pre-set AI reinforcement learning policy network is an intelligent network model that has been trained offline with large-scale multimodal emotion data before formal deployment. It can use the first emotion features extracted synchronously at the current moment, such as facial expressions, speech prosody, EEG power spectrum, and skin conductance level, as the state vector, output an "emotion uncertainty" value, and give the optimal sampling frequency (action) of each sensor for the next step. The goal is to minimize the amount of data and power consumption while ensuring recognition accuracy. During training, deep deterministic policy gradient (DDPG) or proximal policy optimization (PPO) is used to drive the policy with a compound reward of "small recognition error + low sampling cost" until the policy converges.

[0024] S3. Perform feature weighting and feature fusion on the second emotion feature data in sequence, and input the feature fusion result into a dual-channel convolutional neural network to obtain the emotion recognition result feature vector.

[0025] After each of the two channels of a dual-channel convolutional neural network undergoes convolution, pooling, and full connection, the output high-dimensional feature vectors are concatenated, weighted, or fused with attention, and then fed into Softmax / MLP to obtain the final emotion category or regress the emotion dimension value.

[0026] S4. Based on the incremental correction algorithm, the feature vector of the emotion recognition result is compared and analyzed with the labels of the preset emotion feature information database to calculate the accuracy and make corrections.

[0027] The emotion feature vector just inferred by the dual-channel CNN is quickly compared with the original labels of the corresponding user / scene in the local emotion feature database to calculate the current recognition accuracy. If the deviation exceeds the threshold, the incremental correction algorithm immediately uses a small amount of new data to make small, local updates to the network weights and writes the corrected feature vector back to the database to ensure that the model is continuously corrected and adjusted according to the user's emotion changes.

[0028] In some embodiments of this application, in S1, acquiring video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor includes: Synchronous pulse markers are implanted in the physical layer of the sensor, and a self-supervised alignment network is used to use the EEG signal of the next 1 second as the prediction target of the speech / video segment to perform modality alignment with zero manual annotation. The modal alignment results are subjected to noise reduction, fusion, and compression sequentially using tensor fusion and information bottleneck algorithms to obtain synchronized video, audio, and electrical signal data.

[0029] In some embodiments of this application, in S1, analyzing video, audio, and electrical signal data to extract first emotional feature data includes: Video modal features include facial motion unit intensity temporal sequence, global / local optical flow field, 3D head pose angle, and eye gaze vector; Audio modal features include 128-dimensional log-Mel spectrum, fundamental frequency F0, normalized energy, speech rate, and first and second order MFCC differences; EEG electrical signal modal characteristics, including 8-channel 5-band power spectral density, differential entropy, and inter-channel phase lock-in value; Skin conductance modal characteristics, including mean tonic value, phasic peak count, mean rise slope, and recovery half-life.

[0030] It's important to understand that facial motion unit intensity timing encodes facial muscle movements (such as raising eyebrows or upturning the corners of the mouth) into a continuous intensity of 0-5; three-dimensional head posture angles refer to describing head orientation using pitch, yaw, and roll angles, with each angle encoded; eye gaze vector refers to the direction vector of the eye's gaze in 3D space, quantifying the direction and speed of attention shift; fundamental frequency F0 refers to the lowest resonant frequency of each frame of speech, corresponding to the speed of vocal cord vibration; normalized energy refers to the volume of the entire sentence or each frame, which, after eliminating differences in recording gain, reveals emotional fluctuations. Differential entropy is a measure of "information entropy" for each frequency band; a higher value indicates more disordered neuronal firing, usually corresponding to high arousal or emotional confusion; mean tonic value refers to the baseline conductivity of the skin in a resting state, reflecting chronic stress or overall arousal baseline; pulse count refers to the number of "peaks" caused by a sudden increase in sweating within a short period; recovery half-life refers to the time required for the peak to return to half its peak value, with a shorter recovery half-life indicating better emotional regulation and a longer recovery half-life indicating sustained tension.

[0031] In some embodiments of this application, in step S2, the first emotion feature data is input into a preset AI reinforcement learning policy network, and the emotion uncertainty output by the AI ​​reinforcement learning policy network is obtained, including: The four modality features are concatenated, as shown in the expression: ; in, To concatenate vectors, For video modal features, For audio modal features, EEG electrical signal modal characteristics, Characteristics of skin conductance modes; The AI ​​reinforcement learning policy network first outputs the emotion probability distribution through one forward inference, expressed as: ; in, This is the normalized exponentiation function. To introduce a nonlinear activation function for sparsity computation, The learnable parameters for the second layer of the AI ​​reinforcement learning policy network. The first layer of learnable parameters for the AI ​​reinforcement learning policy network. , All are preset process parameters; The sentiment uncertainty u is calculated using Shannon entropy, and the expression is: ; in, The probability estimation bias for the i-th emotion type.

[0032] In some embodiments of this application, in S2, adjusting the working step size of the camera, microphone, EEG, and electrodermal sensor based on emotion uncertainty to obtain second emotion feature data includes: Normalize the emotional uncertainty u to Within the range, mapping is performed according to three threshold segments; when When, maintain a low power step size; when When, increase to the preset middle step size; when At that time, increase to the preset high-resolution step size; Data on the second emotion feature was obtained by collecting data based on the adjusted work step size.

[0033] This invention achieves synchronous acquisition of data from a camera, microphone, EEG, and electrodermal sensor using a co-current clock pulse. It also utilizes an AI reinforcement learning strategy network to map emotional uncertainty into sensor working step size, automatically reducing the frame rate / sampling rate during periods of emotional stability and increasing the acquisition density at millisecond levels during moments of emotional transition. This reduces redundant data and transmission bandwidth, while capturing micro-expressions and differential electrical responses at inflection points, thus reducing end-to-end latency and improving the real-time performance of data processing.

[0034] In some embodiments of this application, in step S3, the sequential feature weighting and feature fusion of the second emotion feature data includes: The four modal features of the second emotion feature data are weighted and fused, and the expression is as follows: ; Where z is the 128-dimensional emotion vector after feature fusion. To output the linear transformation matrix, This is the weight matrix of the fusion layer. The weighted results of the video modality, audio modality-EEG electrical signal modality of the second emotion feature data. For the second emotion feature data, the skin conductance modal features, This is the bias vector for the fusion layer. This is the output layer bias vector.

[0035] In some embodiments of this application, in step S3, the feature fusion result is input into a dual-channel convolutional neural network to obtain an emotion recognition result feature vector, including: For the spatial channels of a two-channel convolutional neural network, the expression is: ; in, The spatial channel output feature map is then subjected to ReLU nonlinear activation. The convolution kernel weight matrix for the spatial channels. The matrix offset for the spatial channel; For the temporal channels of a two-channel convolutional neural network, the expression is: ; in, The time-series channel output feature map is then subjected to ReLU nonlinear activation. For the depthwise separable convolution parameters of the temporal channels, This is the bias vector for the timing channel; Will and The vectors are concatenated and flattened along the channel dimension to generate a unified feature vector. The expression is: ; Perform full-connection dimensionality reduction: ; in, This is the feature vector of the emotion recognition result. For the final fully connected dimensionality reduction matrix, For the final fully connected dimensionality reduction matrix bias, This represents the feature dimension of the final emotion recognition result.

[0036] This invention utilizes feature weighting and a dual-channel convolutional neural network to deeply mine complementary information between modalities. It also employs an incremental correction algorithm to compare the recognition result vector with database labels online and update the network weights by backpropagating errors. This continuously suppresses errors caused by individual, environmental, and label drift, achieving long-term stable high-precision emotion recognition in various application scenarios.

[0037] In some embodiments of this application, such as Figure 2 As shown, an AI-based emotion recognition interactive device was also disclosed, including: The data extraction module is used to construct a common clock pulse for the camera, microphone, EEG, and electrodermal sensor, acquire video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor, analyze the video, audio, and electrical signal data, and extract the first emotion feature data. The step size adjustment module is used to input the first emotion feature data into the preset AI reinforcement learning strategy network, obtain the emotion uncertainty output by the AI ​​reinforcement learning strategy network, and adjust the working step size of the camera, microphone, EEG, and skin conductance sensor based on the emotion uncertainty to obtain the second emotion feature data. The emotion recognition module is used to perform feature weighting and feature fusion on the second emotion feature data in sequence, and input the feature fusion result into a dual-channel convolutional neural network to obtain the emotion recognition result feature vector. The correction module is used to compare and analyze the feature vector of the emotion recognition result with the labels of the preset emotion feature information database based on the incremental correction algorithm, calculate the accuracy and make corrections.

[0038] In some embodiments of this application, an electronic device is also disclosed, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements an emotion recognition interaction method based on AI intelligent analysis.

[0039] In this application, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. In case of any inconsistency, the meaning set forth in this specification or derived from the content described herein shall prevail. Furthermore, the terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit the scope of this application.

[0040] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An AI-based intelligent analysis-based emotion recognition interaction method, characterized in that, Includes the following steps: S1. Construct a common clock pulse for the camera, microphone, EEG, and electrodermal sensor; acquire video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor; analyze the video, audio, and electrical signal data; and extract the first emotion feature data. S2. Input the first emotion feature data into the preset AI reinforcement learning strategy network to obtain the emotion uncertainty output by the AI ​​reinforcement learning strategy network. Adjust the working step size of the camera, microphone, EEG, and skin conductance sensor based on the emotion uncertainty to obtain the second emotion feature data. S3. Perform feature weighting and feature fusion on the second emotion feature data in sequence, and input the feature fusion result into a dual-channel convolutional neural network to obtain the emotion recognition result feature vector. S4. Based on the incremental correction algorithm, the feature vector of the emotion recognition result is compared and analyzed with the labels of the preset emotion feature information database to calculate the accuracy and make corrections.

2. The AI-based intelligent analysis-based emotion recognition interaction method according to claim 1, characterized in that, In step S1, acquiring video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor includes: Synchronous pulse markers are implanted in the physical layer of the sensor, and a self-supervised alignment network is used to use the EEG signal of the next 1 second as the prediction target of the speech / video segment to perform modality alignment with zero manual annotation. The modal alignment results are subjected to noise reduction, fusion, and compression sequentially using tensor fusion and information bottleneck algorithms to obtain synchronized video, audio, and electrical signal data.

3. The AI-based intelligent analysis-based emotion recognition interaction method according to claim 2, characterized in that, In step S1, the analysis of video, audio, and electrical signal data to extract the first emotional feature data includes: Video modal features include facial motion unit intensity temporal sequence, global / local optical flow field, 3D head pose angle, and eye gaze vector; Audio modal features include 128-dimensional log-Mel spectrum, fundamental frequency F0, normalized energy, speech rate, and first and second order MFCC differences; EEG electrical signal modal characteristics, including 8-channel 5-band power spectral density, differential entropy, and inter-channel phase lock-in value; Skin conductance modal characteristics, including mean tonic value, phasic peak count, mean rise slope, and recovery half-life.

4. The AI-based intelligent analysis-based emotion recognition interaction method according to claim 3, characterized in that, In step S2, the first emotion feature data is input into a preset AI reinforcement learning policy network, and the emotion uncertainty output by the AI ​​reinforcement learning policy network includes: The four modality features are concatenated, as shown in the expression: ； in, To concatenate vectors, For video modal features, For audio modal features, EEG electrical signal modal characteristics, Characteristics of skin conductance modes; The AI ​​reinforcement learning policy network first outputs the emotion probability distribution through one forward inference, expressed as: ； in, This is the normalized exponentiation function. To introduce a nonlinear activation function for sparsity computation, The learnable parameters for the second layer of the AI ​​reinforcement learning policy network. The first layer of learnable parameters for the AI ​​reinforcement learning policy network. , All are preset process parameters; The sentiment uncertainty u is calculated using Shannon entropy, and the expression is: ； in, The probability estimation bias for the i-th emotion type.

5. The AI-based intelligent analysis-based emotion recognition interaction method according to claim 4, characterized in that, In step S2, the working step size of the camera, microphone, EEG, and electrodermal sensor is adjusted based on the emotion uncertainty to obtain the second emotion feature data, including: Normalize the emotional uncertainty u to Within the range, mapping is performed according to three threshold segments; when When, maintain a low-power step size; when When, increase to the preset middle step size; when At that time, increase to the preset high-resolution step size; Data on the second emotion feature was obtained by collecting data based on the adjusted work step size.

6. The AI-based intelligent analysis-based emotion recognition interaction method according to claim 5, characterized in that, In step S3, the second emotion feature data is sequentially subjected to feature weighting and feature fusion, including: The four modal features of the second emotion feature data are weighted and fused, and the expression is as follows: ； Where z is the 128-dimensional emotion vector after feature fusion. To output the linear transformation matrix, This is the weight matrix of the fusion layer. The weighted results of the video modality, audio modality-EEG electrical signal modality of the second emotion feature data. For the second emotion feature data, the skin conductance modal features, This is the bias vector for the fusion layer. This is the output layer bias vector.

7. The AI-based intelligent analysis-based emotion recognition interaction method according to claim 6, characterized in that, In step S3, the feature fusion result is input into a dual-channel convolutional neural network to obtain the emotion recognition result feature vector, which includes: For the spatial channels of a two-channel convolutional neural network, the expression is: ； in, The spatial channel output feature map is then subjected to ReLU nonlinear activation. The convolution kernel weight matrix for the spatial channels. The matrix offset for the spatial channel; For the temporal channels of a two-channel convolutional neural network, the expression is: ； in, The time-series channel output feature map is then subjected to ReLU nonlinear activation. For the depthwise separable convolution parameters of the temporal channels, This is the bias vector for the timing channel; Will and The vectors are concatenated and flattened along the channel dimension to generate a unified feature vector. The expression is: ； Perform full-connection dimensionality reduction: ； in, This is the feature vector of the emotion recognition result. For the final fully connected dimensionality reduction matrix, For the final fully connected dimensionality reduction matrix bias, This represents the feature dimension of the final emotion recognition result.

8. An AI-based intelligent analysis-based emotion recognition interactive device, characterized in that: include: The data extraction module is used to construct a common clock pulse for the camera, microphone, EEG, and electrodermal sensor, acquire video, audio, and electrical signal data synchronously collected by the camera, microphone, EEG, and electrodermal sensor, analyze the video, audio, and electrical signal data, and extract the first emotion feature data. The step size adjustment module is used to input the first emotion feature data into the preset AI reinforcement learning strategy network, obtain the emotion uncertainty output by the AI ​​reinforcement learning strategy network, and adjust the working step size of the camera, microphone, EEG, and skin conductance sensor based on the emotion uncertainty to obtain the second emotion feature data. The emotion recognition module is used to perform feature weighting and feature fusion on the second emotion feature data in sequence, and input the feature fusion result into a dual-channel convolutional neural network to obtain the emotion recognition result feature vector. The correction module is used to compare and analyze the feature vector of the emotion recognition result with the labels of the preset emotion feature information database based on the incremental correction algorithm, calculate the accuracy and make corrections.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the AI-based intelligent analysis-based emotion recognition interaction method as described in any one of claims 1-7.

Images