Emotion recognition system and method based on artificial intelligence

Abstract

The invention discloses an emotion recognition system and method based on artificial intelligence, relates to the technical field of artificial intelligence, and solves the problems that in the prior art, different emotion results of people with different cultures under the same expression or voice state are not considered, and the influence degree of external noise on emotion judgment is different. The technical problem that the emotion recognition accuracy is low and the efficiency of an emotion recognition system is low is solved. The emotion category is generated according to the culture label and the personnel data; generating a predicted personnel behavior according to the emotion category and the environment state; generating an alarm signal according to the predicted personnel behavior; according to the method and the system, a prompt is made according to an alarm signal, the cultural label of a person and the noise of the environment where the person is located are considered, the influence degree between multi-modal data is dynamically adjusted, and the person behavior is predicted according to the environment state and the emotion category, so that the emotion recognition accuracy and efficiency are improved, and the emotion recognition comprehensiveness is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically an emotion recognition system and method based on artificial intelligence. Background Art

[0002] In recent years, computer and communications technologies have advanced rapidly, with cutting-edge technologies such as artificial intelligence (AI) experiencing a booming development. Amidst this wave of technological innovation, a wide range of intelligent systems powered by AI have mushroomed. Against this backdrop, the efficient and accurate dynamic analysis and identification of the emotional states of participants within these systems has become a highly sought-after research topic. Emotion recognition refers to the process of perceiving, understanding, and classifying the emotional states expressed by individuals through a range of methods and techniques. It has broad applications in fields such as human-computer interaction, mental health, education, and entertainment.

[0003] The prior art (invention patent application with publication number CN116189669A) discloses a multimodal emotion recognition method and system, which includes: a multimodal information preprocessing step, collecting the user's voice data and image data to obtain corresponding voice segment data and facial image data; a multimodal emotion recognition step, performing voice, text and expression visual emotion analysis to obtain the corresponding emotional state results; a first HMM processing step, combining the results of the multimodal emotional state into a set of features, inputting them into the first HMM as observation values, and outputting the corresponding probability of each emotion; a correlation calculation step, using the cross-correlation function CCF for the results of the multimodal emotional state to calculate the correlation between the results; and a second HMM processing step, inputting the correlation results and the results of the multimodal emotional state as observation values into the second HMM, and taking the emotion with the largest output value as the final recognition result.

[0004] The above patent calculates the corresponding emotional state of multimodal data separately, and then calculates the correlation of the results to obtain the final emotional state. However, it fails to take into account that people with different cultures may have different emotional states for the same expression or even voice data, and external noise will cause the degree of influence of multimodal data in emotion discrimination to be different, resulting in low accuracy of emotion recognition. At the same time, it fails to consider the prediction and protection of abnormal behavior after emotion recognition, resulting in low comprehensiveness of the emotion recognition system, which in turn leads to low efficiency of the emotion recognition system. Therefore, the emotion recognition system still needs further improvement. Summary of the Invention

[0005] The present application aims to solve at least one of the technical problems existing in the prior art; to this end, the present application proposes an emotion recognition system and method based on artificial intelligence, which is used to solve the technical problems that the prior art lacks consideration of the fact that people of different cultures have different emotional results under the same expression or voice state, and the degree of influence of external noise on emotion judgment varies, resulting in low accuracy of emotion recognition and low efficiency of the emotion recognition system.

[0006] To achieve the above-mentioned objectives, the first aspect of the present application provides an artificial intelligence-based emotion recognition system, comprising: a data acquisition module, a data analysis module, an early warning module, and a database; the data module is connected to the data analysis module; the data analysis module is connected to the early warning module; the database is connected to the data acquisition module, the data analysis module, and the early warning module, respectively; The data acquisition module acquires personnel data and environmental data through data acquisition equipment; the personnel data includes personnel ID, face detection image, voice data and personnel behavior; the environmental data includes geographic location and environmental status; The data analysis module generates cultural labels based on face detection images, geographic locations, and voice data; generates emotion categories based on cultural labels and personnel data; generates predicted personnel behavior based on emotion categories and environmental conditions; and generates an alarm signal based on the predicted personnel behavior. The early warning module: makes prompts according to the alarm signal and contacts the management personnel; The database is used to store data of each module and store historical data required for training the model.

[0007] Through the above steps, this application fully considers the cultural background characteristics of personnel and the noise factors of the environment in which they are located, and dynamically adjusts the interaction influence weights between multimodal data; based on environmental situation perception and emotion category discrimination, it constructs a personnel behavior prediction model, which effectively enhances the accuracy and timeliness of emotion recognition, while expanding the dimension of emotion recognition and achieving more comprehensive and accurate emotion perception analysis.

[0008] Furthermore, generating cultural labels based on the face detection image, geographic location, and voice data includes: Obtain face detection images, geographic location, and voice data; Integrate facial detection images, geographic location, and voice data into cultural analysis data; The cultural analysis data is input into a cultural label generation model to obtain a cultural label; the cultural label generation model is constructed through an artificial intelligence model.

[0009] Furthermore, the cultural label generation model is constructed through an artificial intelligence model, including: Obtain some historical and cultural analysis data and their corresponding historical and cultural labels; Divide a number of historical and cultural analysis data and their corresponding historical and cultural labels into training data, verification data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select an artificial intelligence model as the base model; Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, we finally obtain a cultural label generation model whose input is cultural analysis data and output is cultural labels.

[0010] Furthermore, generating emotion categories based on cultural labels and personnel data includes: Acquire cultural labels and personnel data; the personnel data includes face detection images and voice data; the voice data includes voice content; the voice content is integrated with environmental noise; Determining a multimodal suppression coefficient based on the face detection image and the speech content; the multimodal suppression coefficient represents the degree to which the face detection image, the speech content, and the text data are suppressed, including a face suppression coefficient, a speech suppression coefficient, and a text suppression coefficient; Obtaining initial weights from an initial weight table according to the environmental state; the initial weight table is set by experts based on the degree to which multimodal parameters affect emotions under different environmental states; Inputting the multimodal suppression coefficient and the initial weight into a weight estimation model to obtain a decision weight; the weight estimation model is constructed using a machine learning model; the decision weight refers to the degree of influence of the face detection image, voice content, and text data on the emotion category recognition; Generate emotion categories based on decision weights, cultural labels, face detection images, and speech content.

[0011] Furthermore, determining the multimodal suppression coefficient based on the face detection image and speech content includes: Obtain face detection images and voice content; The face detection image is recognized by the multi-task cascade convolutional neural network model MTCNN to obtain several key point detection results and the occlusion rate ZL is obtained by the ratio between the area of the occluder and the total area of the face; The key point confidence GDZD is calculated by outputting the variance of several key point coordinates output by the output network model O-Net; ; Where i represents the number of the key point, They are respectively represented as the coordinate variance of the i-th key point; TK and TG are respectively represented as the image width and image height, and k is represented as the vector dimension; The face suppression coefficient is calculated by weighted fusion of the occlusion rate and key point confidence; The WebRTC voice activity detection model is used to segment the speech content into speech segments and noise segments. The spectral energy is integrated to obtain the speech energy YN and the noise energy ZN. The signal-to-noise ratio degradation rate XBSL is obtained by performing nonlinear calculations on the speech energy and noise energy. The fundamental frequency sequence in the speech content is extracted by the YIN algorithm, and the formula is used to calculate Calculate the fundamental frequency fluctuation index JBZ; where PJP represents the fundamental frequency mean, t represents the fundamental frequency number, represents the fundamental frequency value of the tth fundamental frequency; max() represents the maximum value operation; the YIN algorithm is an efficient algorithm for estimating the fundamental frequency of an audio signal; The speech suppression coefficient is calculated by weighted fusion of the signal-to-noise ratio attenuation rate and the fundamental frequency fluctuation index; The text data is obtained by converting the speech segment into text through the large language model, and the expected number of emotional words YCS is obtained through the neural regulation network model NRC; the actual number of emotional words SCS is obtained through the bidirectional transformer model BERT model; Calculate the text suppression coefficient WYX; The multimodal suppression coefficient includes a face suppression coefficient, a speech suppression coefficient and a text suppression coefficient.

[0012] This application uses multimodal data of people to identify important influencing factors of emotions. Due to external noise, some factors may not provide sufficient effective data when performing emotion recognition. Therefore, it is necessary to consider the degree of suppression of multimodal data by external noise. Different degrees of influence can be assigned to multimodal data in subsequent emotion recognition, so that the final decision data is more in line with reality and the accuracy of emotion recognition is improved.

[0013] Furthermore, the weight estimation model is constructed through a machine learning model, including: Obtaining a number of historical multimodal suppression coefficients and historical initial weights, as well as historical decision weights corresponding to the historical multimodal suppression coefficients and historical initial weights; Dividing a number of historical multimodal suppression coefficients and historical initial weights, as well as historical decision weights corresponding to the historical multimodal suppression coefficients and historical initial weights, into training data, validation data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select a machine learning model as the base model; train the base model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain a pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, a weight estimation model is finally obtained, whose input is the multimodal suppression coefficient and initial weight and the output is the decision weight.

[0014] Furthermore, generating emotion categories based on decision weights, cultural labels, face detection images, and speech content includes: Obtain decision weights, cultural labels, face detection images and voice content, and their corresponding text data; Assigning decision weights to the corresponding face detection images, voice content, and text data to obtain decision multimodal data; the decision multimodal data is multimodal data after the decision weights are assigned; Input cultural labels and decision-making multimodal data into the emotion assessment model to obtain emotion categories; The emotion assessment model is obtained by training an artificial intelligence model on historical assessment data, including: Acquiring historical evaluation data; the historical evaluation data includes a number of historical cultural labels and historical decision multimodal data and historical emotion categories; Divide historical evaluation data into training data, validation data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select an AI model as the base model; train the base model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain a pre-trained model; By validating the pre-trained model on the test set until the preset conditions are met, we finally obtain an emotion assessment model whose input is cultural labels and decision multimodal data and output is emotion categories.

[0015] Furthermore, generating predicted human behavior based on emotion categories and environmental conditions includes: Obtain environmental status, several historical emotion categories, and historical personnel behaviors; Integrate several historical emotion categories and historical personnel behaviors into several behavior prediction sequences; Inputting the environmental state and several behavior prediction sequences into the behavior prediction model to obtain the predicted personnel behavior; The behavior prediction model is constructed using the long short-term memory network model LSTM, including: Obtain several historical behavior prediction sequences and their corresponding historical personnel behaviors; Divide several historical behavior prediction sequences and their corresponding historical personnel behaviors into training data, verification data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select the long short-term memory network model LSTM as the basic model; Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, we finally obtain a behavior prediction model whose input is a behavior prediction sequence and whose output is the predicted human behavior.

[0016] Furthermore, generating an alarm signal based on the predicted personnel behavior includes: Get predicted personnel behavior; When the predicted behavior of the person falls within the normal range, no action is taken; When the predicted personnel behavior falls within the range of abnormal behavior, an abnormal personnel behavior warning signal is generated.

[0017] Another aspect of the present invention provides an emotion recognition method based on artificial intelligence, comprising: S0: Acquire personnel data and environmental data; the personnel data includes personnel ID, face detection image and voice data, and personnel behavior; the environmental data includes geographic location and environmental status; S1: Generate cultural labels based on face detection images, geographic location and voice data; S2: Generate emotion categories based on cultural labels and personnel data; S3: Generate predicted human behavior based on emotion categories and environmental states; S4: Generate an alarm signal based on predicted personnel behavior; S5: Prompt according to the alarm signal and contact the management.

[0018] Compared with the prior art, the present invention has the following advantages: 1. This application generates cultural labels based on face detection images, geographic location and voice data; generates emotion categories based on cultural labels and personnel data; generates predicted personnel behavior based on emotion categories and environmental conditions; generates alarm signals based on predicted personnel behavior, takes personnel cultural labels and the noise of the environment into consideration, dynamically adjusts the degree of influence between multimodal data, and predicts personnel behavior based on environmental conditions and emotion categories, thereby improving the accuracy and efficiency of emotion recognition and enhancing the comprehensiveness of emotion recognition.

[0019] 2. This application considers the person's facial detection image, geographic location and voice data during emotion recognition to determine the person's culture. Since people of different cultures may express different emotion categories under the same facial expression and voice state, cultural factors need to be considered when performing accurate emotion recognition to make emotion recognition more accurate.

[0020] 3. This application obtains the initial weight of multimodal data through environmental status, and obtains decision weight through weight estimation model and combination of multimodal suppression coefficient and initial weight, so that the degree of influence between multimodal data can be adaptively adjusted according to the multimodal suppression coefficient, providing strong data support for judgment during emotion recognition.

[0021] 4. This application predicts the behavior of a person in the future based on the environmental status, historical emotional categories and their corresponding human behaviors after identifying the person's emotional category. The person's future behavior may be affected by the environmental status of the person. By predicting future behavior, early warning can be given when dangerous behavior occurs, reducing the probability of dangerous events, while improving the comprehensiveness of the emotion recognition system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 This is a schematic diagram of the principle of an artificial intelligence-based emotion recognition system in this application; Figure 2 Generate a flow chart for the emotion categories of this application; Figure 3 This is a flow chart of an emotion recognition method based on artificial intelligence in this application. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions of this application in conjunction with the embodiments. Obviously, the embodiments described are only a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] See also Figure 1The first embodiment of the present application provides an emotion recognition system based on artificial intelligence, comprising: a data acquisition module, a data analysis module, an early warning module and a database; the data module and the data analysis module are connected; the data analysis module and the early warning module are connected; the database is respectively connected to the data acquisition module, the data analysis module and the early warning module; Data acquisition module: This module acquires personnel data and environmental data through data acquisition equipment, which includes several sensors. Personnel data includes personnel ID, facial detection images, voice data, and personnel behavior. Environmental data includes geographic location and environmental status. Environmental status refers to the current state, such as driving scenes or meeting scenes. Data Analysis Module: Generates cultural labels based on face detection images, geographic location, and voice data. Cultural labels refer to labels that indicate a person's cultural affiliation. Emotional categories, such as happiness and anger, are generated based on cultural labels and person data. Based on emotion categories and environmental conditions, predicted person behavior is generated. Predicted person behavior refers to the behavior a person is likely to perform in the future under the current emotion category. Alarm signals are generated based on the predicted person behavior. Early warning module: issues prompts based on alarm signals and contacts management personnel; alarm signals include those indicating abnormal personnel behavior, etc. The database is used to store data for each module and store historical data required for training the model.

[0026] In this embodiment, the cultural label is generated based on the face detection image, geographic location, and voice data, including: Obtain face detection images, geographic location, and voice data; Integrate facial detection images, geographic location, and voice data into cultural analysis data; cultural analysis data refers to data about the culture of the analyst; The cultural analysis data is input into the cultural label generation model to obtain the cultural label; the cultural label generation model is constructed through an artificial intelligence model; the cultural labels include East Asian culture, Western European culture, etc.

[0027] The cultural label generation model in this embodiment is constructed using an artificial intelligence model, including: Obtain some historical and cultural analysis data and their corresponding historical and cultural labels; Divide a number of historical and cultural analysis data and their corresponding historical and cultural labels into training data, verification data, and test data; Data preprocessing is performed on the training data, validation data, and test data to obtain the training set, validation set, and test set; the ratio between the training set, test set, and validation set is 7:2:1; Select an artificial intelligence model as the basic model; artificial intelligence models include convolutional neural network models, etc. Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, a cultural label generation model is finally obtained, whose input is cultural analysis data and output is cultural labels; the preset condition for all models in this embodiment during training is that the model accuracy reaches 95%, that is, model training is stopped when the model accuracy exceeds 95%.

[0028] This embodiment takes cultural labels into consideration in emotion recognition because the same facial expression may produce different emotional states due to cultural differences.

[0029] This embodiment constructs a multimodal cultural perception model, which achieves accurate identification of individual cultural attributes by integrating the micro-expression features of face detection images, the spatiotemporal semantic information of geographic location, and the rhythmic features of voice data. Given that different cultural groups have significant emotional semantic differences under the same facial expressions and voice modalities, such as a subtle smile in East Asian culture may correspond to a polite response, while similar expressions in Western culture tend to express joy, dynamic calibration of multimodal feature weights effectively improves the robustness and accuracy of emotion recognition in cross-cultural contexts.

[0030] See also Figure 2 In this embodiment, the emotion categories are generated based on cultural labels and personnel data, including: Acquire cultural labels and person data; person data includes face detection images and voice data; voice data includes voice content; voice content incorporates environmental noise; if noise is present, it is directly mapped to the voice data; and if a person in the face detection image is wearing a mask, the mask acts as external noise, which will suppress the face detection image modal data. Determining a multimodal suppression coefficient based on the face detection image and speech content; the multimodal suppression coefficient represents the degree to which the face detection image, speech content, and text data are suppressed, including a face suppression coefficient, a speech suppression coefficient, and a text suppression coefficient; Initial weights are obtained from an initial weight table based on the environmental state; the initial weight table is set by experts based on the degree to which multimodal parameters affect emotions under different environmental states; The multimodal suppression coefficient and initial weight are input into the weight estimation model to obtain the decision weight; the weight estimation model is constructed through a machine learning model; the decision weight refers to the degree of influence of the face detection image, voice content and text data on the emotion category recognition; Generate emotion categories based on decision weights, cultural labels, face detection images, and speech content.

[0031] In this embodiment, determining the multimodal suppression coefficient based on the face detection image and the speech content includes: Obtain face detection images and voice content; The face detection image is recognized by the multi-task cascade convolutional neural network model MTCNN to obtain several key point detection results and the occlusion rate ZL is obtained by the ratio between the area of the occluder and the total area of the face. The specific calculation formula is expressed as follows: ; Among them, ZDM and RLM represent the area of the occluder and the area of the face respectively; The key point confidence GDZD is calculated by outputting the variance of several key point coordinates output by the output network model O-Net; ; Where i represents the number of the key point, They are respectively represented as the coordinate variance of the i-th key point; TK and TG are respectively represented as the image width and image height, and k is represented as the vector dimension. The specific value is set according to experience. In this embodiment, k is set to 10; The face suppression coefficient RYX is calculated by weighted fusion of the occlusion rate and key point confidence. The calculation formula is: ;in, and Expressed as weight coefficient, and ∈(0,1), the specific value is set according to experience, considering , it is believed that the occlusion rate is more suppressed than the confidence of the key points. In this embodiment, and Set to 0.7 and 0.3 respectively; The WebRTC voice activity detection model is used to segment the speech content into speech segments and noise segments, and the spectral energy is integrated to obtain the speech energy YN and the noise energy ZN. The signal-to-noise ratio attenuation rate XBSL is obtained by performing nonlinear calculations on the speech energy and noise energy. The specific calculation formula is: ; The fundamental frequency sequence in the speech content is extracted by the YIN algorithm, and the formula is used to calculate Calculate the fundamental frequency fluctuation index JBZ; where PJP represents the fundamental frequency mean, t represents the fundamental frequency number, It represents the fundamental frequency value of the tth fundamental frequency; max() represents the maximum value operation; as the fundamental frequency value deviates from the fundamental frequency mean, the basic fluctuation index will gradually increase; The speech suppression coefficient YYX is calculated by weighted fusion of the signal-to-noise ratio attenuation rate and the fundamental frequency fluctuation index; the calculation formula is expressed as: ;in, and Expressed as weight coefficient, and ∈(0,1), the specific value is set according to experience, considering , it is believed that the signal-to-noise ratio attenuation rate is more suppressed than the fundamental frequency fluctuation index. In this embodiment, and Set to 0.6 and 0.4 respectively; The text data is obtained by converting the speech segment into text through the large language model, and the expected number of emotional words YCS is obtained through the neural regulation network model NRC; the actual number of emotional words SCS is obtained through the bidirectional transformer model BERT; and the calculation formula is used to calculate the number of emotional words YCS. Calculate the text suppression coefficient WYX; the smaller the ratio between the actual number of sentiment words and the expected number of sentiment words, the greater the text suppression coefficient will be. The multimodal suppression coefficient includes the face suppression coefficient, the speech suppression coefficient and the text suppression coefficient.

[0032] The weight estimation model in this embodiment is constructed through a machine learning model, including: Obtaining a number of historical multimodal suppression coefficients and historical initial weights, as well as historical decision weights corresponding to the historical multimodal suppression coefficients and historical initial weights; Dividing a number of historical multimodal suppression coefficients and historical initial weights, as well as historical decision weights corresponding to the historical multimodal suppression coefficients and historical initial weights, into training data, validation data, and test data; Data preprocessing is performed on the training data, validation data, and test data to obtain the training set, validation set, and test set; the ratio between the training set, test set, and validation set is 7:2:1; Select a machine learning model as the basic model; machine learning models include BP models, etc. Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, a weight estimation model is finally obtained, whose input is the multimodal suppression coefficient and initial weight and the output is the decision weight.

[0033] In another embodiment, the decision weight is obtained by: Obtain the basic weight JQ corresponding to the multimodal data and its corresponding multimodal suppression coefficient YX; the sum of several basic weights is 1; Determine the decision weight through the decision weight calculation formula , the decision weight calculation formula is:

[0034] Wherein, j, m, and n represent the numbers of the multimodal suppression coefficients and their corresponding basic weights. The values of j, m, and n range from 1 to N, and N represents the total number of multimodal suppression coefficients. It is represented as the maximum value in the multimodal suppression coefficient; h is represented as the adjustment strength coefficient, h∈(0,1); the specific value is set according to experience. In this embodiment, h is set to 0.5. The setting of h is to control the amplitude of the change of the basic weight; through the above calculation formula, the basic weight corresponding to the highest value in the multimodal suppression coefficient can be reduced, while the basic weight values corresponding to other modal data are increased, and the final sum of the several decision weights is 1.

[0035] This embodiment constructs a dynamic weight allocation mechanism driven by environmental perception, builds an initial weight matrix for multimodal data by collecting environmental state parameters in real time, and introduces a weight estimation model based on deep learning. This model generates a decision weight vector with environmental adaptability by fusing the multimodal suppression coefficient with the initial weight parameters, and realizes the dynamic adjustment of the degree of interactive influence of multimodal data, providing strong data support for judgment during emotion recognition.

[0036] In this embodiment, the emotion category is generated based on the decision weight, cultural label, face detection image, and speech content, including: Obtain decision weights, cultural labels, face detection images and voice content, and their corresponding text data; Assigning decision weights to the corresponding face detection images, voice content, and text data to obtain decision multimodal data; the decision multimodal data is the multimodal data after the decision weights are assigned; Input cultural labels and decision-making multimodal data into the emotion assessment model to obtain emotion categories; emotion categories include happiness, sadness, and anger; The sentiment assessment model is obtained by training an artificial intelligence model on historical assessment data, including: Obtain historical evaluation data; historical evaluation data includes several historical cultural labels, historical decision-making multimodal data, and historical sentiment categories; Divide historical evaluation data into training data, validation data, and test data; Data preprocessing is performed on the training data, validation data, and test data to obtain the training set, validation set, and test set; the ratio between the training set, test set, and validation set is 7:2:1; Select an artificial intelligence model as the basic model; artificial intelligence models include convolutional neural network models, etc. Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By validating the pre-trained model on the test set until the preset conditions are met, we finally obtain an emotion assessment model whose input is cultural labels and decision multimodal data and output is emotion categories.

[0037] This embodiment obtains multiple factors that affect emotion recognition, including decision data and cultural labels after the influence of external noise, and can accurately and quickly obtain emotion categories through a pre-trained emotion assessment model, thereby improving the accuracy and efficiency of the emotion recognition system.

[0038] In this embodiment, the process of generating predicted human behavior based on emotion category and environmental status includes: Obtain the state of the environment and several historical emotion categories and historical personnel behaviors; Integrate several historical emotion categories and historical personnel behaviors into several behavior prediction sequences; Inputting the environmental state and several behavior prediction sequences into the behavior prediction model to obtain the predicted personnel behavior; The behavior prediction model is constructed using the long short-term memory network model LSTM, which includes: Obtain several historical behavior prediction sequences and their corresponding historical personnel behaviors; Divide several historical behavior prediction sequences and their corresponding historical personnel behaviors into training data, verification data, and test data; Data preprocessing is performed on the training data, validation data, and test data to obtain the training set, validation set, and test set; the ratio between the training set, test set, and validation set is 7:2:1; Select the long short-term memory network model LSTM as the basic model; Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, we finally obtain a behavior prediction model whose input is a behavior prediction sequence and whose output is the predicted human behavior.

[0039] This embodiment considers that emotional state affects human behavior. Since human behavior has a lag, that is, it occurs after the emotion is generated, there is a certain time difference, so the human behavior that occurs after the current emotion can be expressed as predicted human behavior.

[0040] In this embodiment, generating an alarm signal based on predicted personnel behavior includes: Get predicted personnel behavior; When the predicted behavior of the person falls within the normal range, no action is taken; When the predicted personnel behavior falls within the abnormal range, an abnormal personnel behavior warning signal is generated; the normal behavior range and the abnormal behavior range are set based on experience.

[0041] This embodiment constructs a three-dimensional prediction model of environment-emotion-behavior. On the basis of accurately identifying individual emotion categories, it establishes a dynamic behavior prediction engine by integrating real-time environmental status perception data, historical emotion time series characteristics and corresponding behavior patterns. It fully considers the modulation effect of environmental context on behavioral decision-making, predicts potential behavior trajectories in future time periods, realizes early perception and proactive intervention of abnormal behavior, effectively reduces the accident rate, and improves the comprehensiveness of the emotion recognition system.

[0042] See also Figure 3 Another aspect of the present application provides an artificial intelligence-based emotion recognition method, comprising: S0: Acquire personnel data and environmental data; personnel data includes personnel ID, face detection image and voice data, and personnel behavior; environmental data includes geographic location and environmental status; S1: Generate cultural labels based on face detection images, geographic location and voice data; S2: Generate emotion categories based on cultural labels and personnel data; S3: Generate predicted human behavior based on emotion categories and environmental states; S4: Generate an alarm signal based on predicted personnel behavior; S5: Prompt according to the alarm signal and contact the management.

[0043] Some of the data in the above formula are calculated by removing the dimensions and taking their numerical values. The formula is a formula that is closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0044] The working principle of this application is: obtaining personnel data and environmental data; generating cultural labels based on face detection images, geographic location and voice data; generating emotion categories based on cultural labels and personnel data; generating predicted personnel behavior based on emotion categories and environmental conditions; generating alarm signals based on predicted personnel behavior; making prompts based on alarm signals and contacting management personnel, taking into account personnel cultural labels and the noise of the environment in which they are located, dynamically adjusting the degree of influence between multimodal data, and predicting personnel behavior based on environmental conditions and emotion categories, so that the accuracy and efficiency of emotion recognition are improved, and the comprehensiveness of emotion recognition is improved, avoiding the problem that the existing technology lacks consideration that people of different cultures have different emotional results in the same expression or voice state, and the degree of influence of external noise on emotion judgment is different, resulting in low accuracy of emotion recognition and low efficiency of the emotion recognition system.

[0045] The above embodiments are only used to illustrate the technical method of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present application.

Claims

1. An emotion recognition system based on artificial intelligence, characterized in that: include: Interconnected data acquisition module and data analysis module; The data acquisition module acquires personnel data and environmental data through data acquisition equipment; the personnel data includes face detection images, voice data and personnel behavior; the environmental data includes geographic location and environmental status; The data analysis module generates cultural labels based on face detection images, geographic locations, and voice data; generates emotion categories based on cultural labels and personnel data; generates predicted personnel behavior based on emotion categories and environmental conditions; and generates an alarm signal based on the predicted personnel behavior.

2. The artificial intelligence-based emotion recognition system according to claim 1, characterized in that: Generating cultural labels based on face detection images, geographic locations, and voice data includes: Obtain face detection images, geographic location, and voice data; Integrate facial detection images, geographic location, and voice data into cultural analysis data; The cultural analysis data is input into a cultural label generation model to obtain a cultural label; the cultural label generation model is constructed through an artificial intelligence model.

3. The artificial intelligence-based emotion recognition system according to claim 2, characterized in that: The cultural label generation model is constructed using an artificial intelligence model, including: Obtain some historical and cultural analysis data and their corresponding historical and cultural labels; Divide a number of historical and cultural analysis data and their corresponding historical and cultural labels into training data, verification data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select an artificial intelligence model as the base model; Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, we finally obtain a cultural label generation model whose input is cultural analysis data and output is cultural labels.

4. The artificial intelligence-based emotion recognition system according to claim 1, characterized in that: The speech data includes speech content, and generating emotion categories based on cultural labels and personnel data includes: Obtain cultural labels, face detection images, and voice content; Determining a multimodal suppression coefficient based on the face detection image and speech content; Obtaining initial weights from an initial weight table according to the environmental state; the initial weight table is set by experts based on the degree to which multimodal parameters affect emotions under different environmental states; Inputting the multimodal suppression coefficient and the initial weight into a weight estimation model to obtain a decision weight; the weight estimation model is constructed by a machine learning model; Generate emotion categories based on decision weights, cultural labels, face detection images, and speech content.

5. The artificial intelligence-based emotion recognition system according to claim 4, characterized in that: Determining the multimodal suppression coefficient based on the face detection image and speech content includes: The face detection image is recognized by the multi-task cascade convolutional neural network model MTCNN to obtain several key point detection results and the occlusion rate is obtained by the ratio between the area of the occluder and the total area of the face; The key point confidence GDZD is calculated by outputting the variance of several key point coordinates output by the output network model O-Net; ; Where i represents the number of the key point, They are respectively represented as the coordinate variance of the i-th key point; TK and TG are respectively represented as the image width and image height, and k is represented as the vector dimension; The face suppression coefficient is calculated by weighted fusion of the occlusion rate and key point confidence; The WebRTC voice activity detection model is used to segment the speech content into speech segments and noise segments. The spectral energy is integrated to obtain speech energy and noise energy. The signal-to-noise ratio degradation rate is obtained by performing nonlinear calculations on the speech energy and noise energy. The fundamental frequency sequence in the speech content is extracted by the YIN algorithm, and the formula is used to calculate Calculate the fundamental frequency fluctuation index JBZ; where PJP represents the fundamental frequency mean, t represents the fundamental frequency number, It represents the fundamental frequency value of the tth fundamental frequency; max() represents the maximum value operation; The speech suppression coefficient is calculated by weighted fusion of the signal-to-noise ratio attenuation rate and the fundamental frequency fluctuation index; The text data is obtained by converting the speech segment into text using a large language model, and the expected number of sentiment words is obtained using the neural regulation network model NRC; the actual number of sentiment words is obtained using the bidirectional transformer model BERT; and the text suppression coefficient is calculated based on the ratio between the actual number of sentiment words and the expected number of sentiment words. The multimodal suppression coefficient includes a face suppression coefficient, a speech suppression coefficient and a text suppression coefficient. The multimodal suppression coefficient represents the degree to which the face detection image, speech content and text data are suppressed.

6. The artificial intelligence-based emotion recognition system according to claim 4, characterized in that: The weight estimation model is constructed through a machine learning model, including: Obtaining a number of historical multimodal suppression coefficients and historical initial weights, as well as historical decision weights corresponding to the historical multimodal suppression coefficients and historical initial weights; Divide several historical multimodal suppression coefficients and historical initial weights and their corresponding historical decision weights into training data, verification data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select a machine learning model as the base model; Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, a weight estimation model is finally obtained, whose input is the multimodal suppression coefficient and initial weight and the output is the decision weight.

7. The artificial intelligence-based emotion recognition system according to claim 4, characterized in that: The method of generating emotion categories based on decision weights, cultural labels, face detection images, and speech content includes: Obtain text data corresponding to decision weights and speech content; Assigning decision weights to the corresponding face detection images, voice content, and text data to obtain decision multimodal data; the decision weights refer to the degree of influence of the face detection images, voice content, and text data on the emotion category recognition; the decision multimodal data is the multimodal data after the decision weights are assigned; Input cultural labels and decision-making multimodal data into the emotion assessment model to obtain emotion categories; The emotion assessment model is obtained by training an artificial intelligence model on historical assessment data, including: Acquiring historical evaluation data; the historical evaluation data includes a number of historical cultural labels and historical decision multimodal data and historical emotion categories; Divide historical evaluation data into training data, validation data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select an AI model as the base model; train the base model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain a pre-trained model; By validating the pre-trained model on the test set until the preset conditions are met, we finally obtain an emotion assessment model whose input is cultural labels and decision multimodal data and output is emotion categories.

8. The artificial intelligence-based emotion recognition system according to claim 1, characterized in that: The method of generating predicted human behavior based on emotion categories and environmental states includes: Obtain environmental status, several historical emotion categories, and historical personnel behaviors; Integrate several historical emotion categories and historical personnel behaviors into several behavior prediction sequences; Inputting the environmental state and several behavior prediction sequences into the behavior prediction model to obtain the predicted personnel behavior; The behavior prediction model is constructed using the long short-term memory network model LSTM, including: Obtain several historical behavior prediction sequences and their corresponding historical personnel behaviors; Divide several historical behavior prediction sequences and their corresponding historical personnel behaviors into training data, verification data, and test data; Perform data preprocessing on the training data, verification data and test data to obtain the training set, verification set and test set; Select the long short-term memory network model LSTM model as the basic model; Train the basic model using the training set, and adjust the learning rate and hyperparameters on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set until the preset conditions are met, we finally obtain a behavior prediction model whose input is a behavior prediction sequence and whose output is the predicted human behavior.

9. The artificial intelligence-based emotion recognition system according to claim 1, characterized in that: The generating of an alarm signal according to the predicted personnel behavior includes: Get predicted personnel behavior; When the predicted behavior of the person falls within the normal range, no action is taken; When the predicted personnel behavior falls within the range of abnormal behavior, an abnormal personnel behavior warning signal is generated.

10. An artificial intelligence-based emotion recognition method, applied to an artificial intelligence-based emotion recognition system according to any one of claims 1 to 9, characterized in that: include: S0: Acquire personnel data and environmental data; the personnel data includes personnel ID, face detection image and voice data, and personnel behavior; the environmental data includes geographic location and environmental status; S1: Generate cultural labels based on face detection images, geographic location and voice data; S2: Generate emotion categories based on cultural labels and personnel data; S3: Generate predicted human behavior based on emotion categories and environmental states; S4: Generate an alarm signal based on the predicted personnel behavior.

Images