Artificial intelligence emotion recognition system based on deep learning
Through a deep learning emotion recognition system combining facial and text data, the problem of low accuracy of speech emotion recognition in the prior art is solved, and higher recognition accuracy and effectiveness are achieved.
Patent Information
- Application Number
- CN202411116194.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-15
AI Technical Summary
The existing deep learning-based speech emotion recognition system only recognizes from the sound aspect, cannot accurately recognize emotions, and cannot verify the accuracy of the recognition results, resulting in low recognition accuracy and reduced effectiveness.
Provide an artificial intelligence emotion recognition system based on deep learning, combining facial and text emotional data, and collect and analyze facial, voice and text emotional data through the emotion information analysis module, emotion attribute analysis module and emotion data analysis module to generate accurate Emotional feature vectors and optimize the identification results based on history.
It improves the accuracy and accuracy of emotion recognition, enhances the effectiveness of the recognition system, and ensures the accuracy and effectiveness of emotion recognition through multi-faceted analysis and historical record optimization.
Smart Images

Figure CN118965076A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of emotion recognition technology, and in particular to an artificial intelligence emotion recognition system based on deep learning. Background Art
[0002] Emotion recognition refers to the process of identifying and classifying human emotional states. It has broad application prospects and huge potential. With the continuous development of science and technology, people's demand for human-computer interaction is getting higher and higher. As an important part of human-computer interaction, the accuracy and efficiency of emotion recognition directly affect the user experience. Deep learning can continuously update the recognition system and make the artificial intelligence emotion recognition system more accurate. Generating an artificial intelligence emotion recognition system based on deep learning is an important means to promote scientific and technological progress and improve the efficiency and quality of human-computer interaction. It has broad application prospects and huge development potential.
[0003] Prior art, such as the invention patent application with announcement number CN114387997A, discloses a speech emotion recognition method based on deep learning, the steps of which include: 1. Acquire a conversation speech set and perform frame division and windowing processing; 2. Extract the LLDs feature sequence and spectrogram of the speech frame; 3. Extract the representation of the LLDs sequence; 4. Extract the representation of the spectrogram. 5. Use the strategy of feature fusion and result fusion to obtain the final emotion prediction result. The present invention can effectively predict the emotion category of speech and improve the prediction accuracy.
[0004] With respect to the above scheme, the applicant of the present invention has found that the above technology has at least the following technical problems: 1. The above scheme only recognizes emotions from the aspect of sound, and may not be able to accurately recognize emotions. Emotion recognition also involves facial emotions and text emotions. Recognition of a single aspect may result in a low recognition accuracy rate. On the other hand, by analyzing only the sound, it is impossible to verify whether the recognition result is accurate, and it is impossible to update the emotion recognition system based on the recognition result, which reduces the effectiveness of emotion recognition. Summary of the invention
[0005] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide an artificial intelligence emotion recognition system based on deep learning.
[0006] To solve the above technical problems, the present invention adopts the following technical solutions: The present invention provides an artificial intelligence emotion recognition system based on deep learning, comprising the following modules: an emotion information analysis module, which is used to collect various facial emotion values, various sound emotion values and various emotion text characters of the recognition segment, and then analyze the various facial emotion values, various sound emotion values and various emotion text characters of the recognition segment to obtain the emotion attributes and emotion feature data of the three items of facial collection, sound collection and text collection of the recognition segment.
[0007] The sentiment attribute analysis module is used to judge the sentiment attributes of the three items collected in the recognition segment. If the analysis result is accurate, the sentiment feature data of the three items collected in the recognition segment are analyzed to obtain the sentiment feature vector of the three items collected in the recognition segment with accurate analysis results. If the analysis result is inaccurate, the historical analysis records of the recognition segments in the database are analyzed to obtain other recognition segments, and then the sentiment attributes and feature data of the three items collected in each other recognition segment are collected, and the sentiment attributes and feature data of the three items collected in each other recognition segment are analyzed to obtain the sentiment feature vector of the three items collected in the recognition segment with inaccurate analysis results.
[0008] The sentiment data analysis module is used to collect feature vectors of the three standard emotions set in the database, collect the three sentiment feature vectors of the recognition segment, obtain the sentiment of the recognition segment, and change the feature vector of the standard emotion according to the historical recognition record of the recognition segment in the database.
[0009] Preferably, the emotional information analysis of the recognition segment is specifically performed as follows: the emotional attributes are divided into three categories: positive emotion, neutral emotion and negative emotion, and the data intervals of various facial emotional attributes of positive emotion and negative emotion are obtained from the database. If the various facial emotional values of the recognition segment all belong to the positive emotion facial emotion data interval, it indicates that the facial emotional attribute of the recognition segment is positive; if the various facial emotional values of the recognition segment all belong to the negative emotion facial emotion data interval, it indicates that the facial emotional attribute of the recognition segment is negative; if the facial emotional attribute of the recognition segment is neither positive nor negative, it indicates that the facial emotional attribute of the recognition segment is neutral, thereby obtaining the facial emotional attributes of the recognition segment, and similarly, obtaining the sound emotional attributes of the recognition segment.
[0010] According to the emotional word characters of positive emotions, neutral emotions and negative emotions in the database, the emotional text characters of the recognition segment are divided into positive emotional recognition segment text characters, neutral emotional recognition segment characters and negative emotional recognition segment characters, and then the number of positive, neutral and negative characters in the recognition segment is obtained, and the emotional text characters of the recognition segment are summarized to obtain the total emotional text characters of the recognition segment, and the positive text rate of the recognition segment is obtained by dividing the number of positive characters in the recognition segment by the total emotional text characters. Similarly, the neutral text rate and negative text rate of the recognition segment are obtained, and the attribute with the highest text rate is selected as the text emotional attribute of the recognition segment, so as to obtain the three collected emotional attributes of the recognition segment. At the same time, according to the preset three collected feature data templates, the three collected feature data are extracted from various facial emotion values, various sound emotion values and each emotional text character of the recognition segment.
[0011] Preferably, the emotional attributes of the three items collected in the recognition segment are judged, and the specific judgment process is as follows: if the facial emotional attributes, voice emotional attributes and text emotional attributes of the recognition segment are the same, it indicates that the analysis result is accurate; if the emotional attributes of any one of the three items collected in the recognition segment are different from the emotional attributes of the other two items collected, it indicates that the analysis result is inaccurate. Preferably, the emotional attributes and feature data of the three items collected of each other recognition segment are analyzed, and the specific analysis process is as follows: the emotional attributes of the three items collected of each other recognition segment are counted to obtain the number of other recognition segments of the three types of emotional attributes collected by the face, and the emotional attributes of the other recognition segments with the largest number of facial collection are recorded as the emotional attributes of the recognition segment, so as to obtain the emotional attributes of the facial collection with inaccurate analysis, and from the erroneous emotional feature data of each other recognition segment, the emotional feature data of each other recognition segment corresponding to the emotional attributes of the inaccurate facial collection are extracted, and the average value is calculated to obtain the emotional feature data of the facial collection of the recognition segment with inaccurate analysis results, and so on, the emotional attributes of the sound collection of the inaccurate recognition segment, the emotional feature data of the sound collection, the emotional attributes of the text collection and the emotional feature data of the text collection are obtained, and the emotional attributes of the three items collected of the inaccurate recognition segment are summarized.
[0012] If the sentiment attributes of the three items collected are all the same, the sentiment feature data of the three items collected with inaccurate analysis results are transformed according to the transformation scheme to obtain the sentiment feature vectors of the three items collected with inaccurate analysis results.
[0013] If any two of the three collected emotional attributes are the same and different from the emotional attributes of another collected item, the emotional feature data of the two collected items with the same emotional attributes will be transformed according to the transformation plan, and the emotional feature vector of the other collected item with different emotional attributes will be recorded as zero, thus obtaining the emotional feature vector of the three collected items with inaccurate analysis results.
[0014] If any of the three collected emotional attributes is different from the other two, the analysis accuracy of the three collected items is obtained from the database, the feature data of the item with the highest analysis accuracy is selected and converted according to the conversion plan, and the feature vectors of the other two collected items are recorded as zero, so as to obtain the emotional feature vectors of the three collected items with inaccurate analysis results.
[0015] Preferably, the accurate emotion feature vector of the identified segment is analyzed, and the specific analysis process is as follows: Substitute the feature vector of each standard emotion and the feature vector of the identified segment into the emotion similarity calculation formula In the above example, we get the similarity index of the standard emotion a. , a is the number of the standard emotion set, , , A, B and C are the facial emotion feature vector, voice emotion feature vector and text emotion feature vector of the recognition segment respectively, , and are the facial emotion feature vector, voice emotion feature vector and text emotion feature vector of the standard emotion a in the database, , and are respectively a preset weight factor for the first facial collection, a preset weight factor for the first sound collection, and a preset weight factor for the first text collection, , , , , and are respectively a preset weight factor for the second facial collection and a preset weight factor for the second sound collection, , , , and are the weight factor of the third sound collection and the weight factor of the second text collection, respectively. , , , and are respectively a preset weight factor for the third face collection and a preset weight factor for the third text collection, , , , the standard emotion with the largest similarity index is selected as the emotion of the identified segment.
[0016] Preferably, the specific changing process of changing the characteristic vector of the standard emotion is as follows: obtaining each recognition record from the database, obtaining the emotion recognized each time and the highest similarity index of each recognition, summarizing the highest similarity index of each recognition according to the emotion recognized each time, obtaining the similarity index of each recognition of each standard emotion, calculating the average value of the similarity index of each recognition of each standard emotion, obtaining the precise index of the characteristic vector of each standard emotion, if the precise index is lower than the preset index, it means that the standard emotion needs to be updated, selecting the characteristic vector corresponding to the highest similarity index in the standard emotion recognition record as the updated characteristic vector of the standard emotion, and updating it after each recognition.
[0017] The beneficial effects of the present invention are as follows: 1. The present invention obtains the three collected emotion attributes and emotion feature data of the recognition segment by analyzing various facial emotion values, various sound emotion values and various emotion text characters of the recognition segment, and judges the analysis result through the three collected emotion attributes of the recognition segment. If the analysis is accurate, the three collected emotion feature data are directly converted into an emotion feature vector. If the analysis is inaccurate, the three collected emotion attributes and emotion feature data of each other recognition segment are collected through the collection range optimized by the emotion analysis historical record, and the emotion feature vector of the recognition segment is obtained through the analysis of the three collected emotion attributes and emotion feature data of each other recognition segment, and then the emotion feature vector of the recognition segment is analyzed to obtain the recognition segment emotion, and then the feature vector of each standard emotion is optimized according to the emotion recognition historical record. The present invention increases the accuracy of emotion recognition by collecting and analyzing the three aspects of face, sound and text.
[0018] 2. The present invention increases the accuracy of emotion recognition by optimizing the collection range through analyzing historical records, and increases the effectiveness of emotion recognition by optimizing the feature vectors of standard emotions based on recognition historical records. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0020] Figure 1 It is a schematic diagram of the system structure connection of the present invention. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] according to Figure 1 As shown, the present invention provides an enterprise energy management and control system, including the following modules: a sentiment information analysis module, a sentiment attribute analysis module and a sentiment data analysis module.
[0023] The sentiment information analysis module is connected to the sentiment attribute analysis module and the database, and the sentiment data analysis module is connected to the sentiment attribute analysis module.
[0024] The emotional data analysis module is used to collect various facial emotion values, various sound emotion values and various emotional text characters of the recognition segment, and then analyze the various facial emotion values, various sound emotion values and various emotional text characters of the recognition segment to obtain the emotional attributes and emotional feature data of the three items of facial collection, sound collection and text collection of the recognition segment.
[0025] It should be noted that the "three collections" mentioned in this article refer to facial collection, voice collection and text collection.
[0026] In a specific embodiment, the various facial emotion values, various sound emotion values and various emotion text characters of the collected recognition segment are specifically collected as follows: the facial image of the recognition segment is collected by a camera, and the various facial emotion values of the recognition segment are obtained from the facial image; the sound of the recognition segment is collected by a sound recording device, and the various sound emotion values of the recognition segment are obtained from the sound of the recognition segment; at the same time, the sound is converted into text characters by a speech-to-text device, and various emotion text characters are extracted from the text characters of the recognition segment.
[0027] In a specific embodiment, the emotion information analysis of the recognition segment is specifically performed as follows: the emotion attributes are divided into three categories: positive emotion, neutral emotion and negative emotion, and the data intervals of various facial emotion attributes of positive emotion and negative emotion are obtained from the database. If the various facial emotion values of the recognition segment all belong to the positive emotion facial emotion data interval, it indicates that the facial emotion attribute of the recognition segment is positive; if the various facial emotion values of the recognition segment all belong to the negative emotion facial emotion data interval, it indicates that the facial emotion attribute of the recognition segment is negative; if the facial emotion attribute of the recognition segment is neither positive nor negative, it indicates that the facial emotion attribute of the recognition segment is neutral, thereby obtaining the facial emotion attribute of the recognition segment. Similarly, the sound emotion attribute of the recognition segment is obtained.
[0028] It should be noted that the "three categories" mentioned in this article all refer to positive emotions, neutral emotions and negative emotions.
[0029] According to the emotional word characters of positive emotions, neutral emotions and negative emotions in the database, the emotional text characters of the recognition segment are divided into positive emotional recognition segment text characters, neutral emotional recognition segment characters and negative emotional recognition segment characters, and then the number of positive, neutral and negative characters in the recognition segment is obtained, and the emotional text characters of the recognition segment are summarized to obtain the total emotional text characters of the recognition segment, and the positive text rate of the recognition segment is obtained by dividing the number of positive characters in the recognition segment by the total emotional text characters. Similarly, the neutral text rate and negative text rate of the recognition segment are obtained, and the attribute with the highest text rate is selected as the text emotional attribute of the recognition segment, so as to obtain the three collected emotional attributes of the recognition segment. At the same time, according to the preset three collected feature data templates, the three collected feature data are extracted from various facial emotion values, various sound emotion values and each emotional text character of the recognition segment.
[0030] The sentiment attribute analysis module is used to judge the sentiment attributes of the three items collected in the recognition segment. If the analysis result is accurate, the sentiment feature data of the three items collected in the recognition segment are analyzed to obtain the sentiment feature vector of the three items collected in the recognition segment with accurate analysis results. If the analysis result is inaccurate, the historical analysis records of the recognition segments in the database are analyzed to obtain other recognition segments, and then the sentiment attributes and feature data of the three items collected in each other recognition segment are collected, and the sentiment attributes and feature data of the three items collected in each other recognition segment are analyzed to obtain the sentiment feature vector of the three items collected in the recognition segment with inaccurate analysis results.
[0031] In a specific embodiment, the emotional attributes collected in the three items of the recognition segment are judged, and the specific judgment process is as follows: if the facial emotional attribute, the voice emotional attribute and the text emotional attribute of the recognition segment are the same, it indicates that the analysis result is accurate; if the emotional attribute collected in any one of the three items of the recognition segment is different from the emotional attributes collected in the other two items, it indicates that the analysis result is inaccurate. In a specific embodiment, the three items of collected emotion feature data of the recognition segment are analyzed, and the specific analysis process is as follows: the three items of collected emotion feature data of the recognition segment include facial emotion feature data, voice emotion feature data and text emotion feature data of the recognition segment, and the three items of collected emotion feature data of the recognition segment are converted into three items of collected emotion feature vectors according to a conversion scheme, and the conversion scheme is: by using the CNN model to perform feature representation on the facial emotion feature data of the recognition segment, the facial emotion feature vector of the recognition segment is obtained, the voice emotion feature data of the recognition segment is extracted by the features of the ERMSLM model, and the voice emotion feature vector of the recognition segment is obtained, and the text emotion feature data of the recognition segment is calculated by the Attention mechanism to obtain the text feature vector of the recognition segment.
[0032] In a specific embodiment, the historical analysis records of the identification segments in the database are analyzed. The specific analysis process is as follows: the historical analysis records of the identification segments including the number of successful analyses and the average analysis time of each acquisition range are taken and substituted into the index calculation formula In the example above, we can get the usage index of the collection range b. , b is the number of the collection range, , , and are the number of successful analyses and the average analysis time for acquisition range b, and They are the preset standard number of successful analyses and standard analysis duration, and They are the weight factors of the preset standard successful analysis times and the average analysis time. , , , select the collection range of the maximum usage index as the collection range of the identification segment.
[0033] It should be noted that the collection range is the first i collections and the last j collections of the identification segment, and each collection range corresponds to different other identification segments. For example, a certain collection range is the first two collections and the last one collection of the identification segment. It means that the other identification segments of the collection range include two other identification segments collected twice before the identification segment and one identification segment collected once after the identification segment. The "other" identification segments are distinguished from the identification segments by "other".
[0034] The last identification segment of the past is collected and the first identification segment of the future is collected. The number of successful analyses of each collection range and the analysis duration of each collection range are calculated from the database. The average number of successful analyses of each collection range is calculated to obtain the standard number of successful analyses. The average analysis duration of each collection range is calculated to obtain the standard analysis duration. The maximum number of successful analyses R among the successful analyses of each collection range is obtained from the database. The longest analysis duration S of each analysis duration of each collection range is obtained from the database. , .
[0035] In a specific embodiment, the emotional attributes and feature data of the three items collected of each other recognition segment are analyzed, and the specific analysis process is as follows: the emotional attributes of the three items collected of each other recognition segment are counted to obtain the number of other recognition segments of the three types of emotional attributes collected by the face, and the emotional attribute of the other recognition segments with the largest number of emotional attributes collected by the face is recorded as the emotional attribute of the recognition segment, so as to obtain the emotional attributes of the face collection with inaccurate analysis, and from the erroneous emotional feature data of each other recognition segment, the emotional feature data of each other recognition segment corresponding to the emotional attribute of the inaccurate facial collection is extracted, and the average value is calculated to obtain the emotional feature data of the face collection of the recognition segment with inaccurate analysis results, and so on, the emotional attributes of the sound collection of the inaccurate recognition segment, the emotional feature data of the sound collection, the emotional attributes of the text collection and the emotional feature data of the text collection are obtained, and the emotional attributes of the three items collected of the inaccurate recognition segment are summarized.
[0036] If the sentiment attributes of the three items collected are all the same, the sentiment feature data of the three items collected with inaccurate analysis results are transformed according to the transformation scheme to obtain the sentiment feature vectors of the three items collected with inaccurate analysis results.
[0037] If any two of the three collected emotional attributes are the same and different from the emotional attributes of another collected item, the emotional feature data of the two collected items with the same emotional attributes will be transformed according to the transformation plan, and the emotional feature vector of the other collected item with different emotional attributes will be recorded as zero, thus obtaining the emotional feature vector of the three collected items with inaccurate analysis results.
[0038] If any of the three collected emotional attributes is different from the other two, the analysis accuracy of the three collected items is obtained from the database, the feature data of the item with the highest analysis accuracy is selected and converted according to the conversion plan, and the feature vectors of the other two collected items are recorded as zero, so as to obtain the emotional feature vectors of the three collected items with inaccurate analysis results.
[0039] The sentiment data analysis module is used to collect feature vectors of the three standard emotions set in the database, collect the three sentiment feature vectors of the recognition segment, obtain the sentiment of the recognition segment, and change the feature vector of the standard emotion according to the historical recognition record of the recognition segment in the database.
[0040] In a specific embodiment, the accurate emotion feature vector of the recognition segment is analyzed, and the specific analysis process is as follows: Substitute the feature vector of each standard emotion and the feature vector of the recognition segment into the emotion similarity calculation formula In the above example, we get the similarity index of the standard emotion a. , a is the number of the standard emotion set, , , A, B and C are the facial emotion feature vector, voice emotion feature vector and text emotion feature vector of the recognition segment respectively, , and are the facial emotion feature vector, voice emotion feature vector and text emotion feature vector of the standard emotion a in the database, , and are respectively a preset weight factor for the first facial collection, a preset weight factor for the first sound collection, and a preset weight factor for the first text collection, , , , , and are respectively a preset weight factor for the second facial collection and a preset weight factor for the second sound collection, , , , and are the weight factor of the third sound collection and the weight factor of the second text collection, respectively. , , , and are respectively a preset weight factor for the third face collection and a preset weight factor for the third text collection, , , , the standard emotion with the largest similarity index is selected as the emotion of the identified segment.
[0041] It should be noted that the facial emotion feature vector, voice emotion feature vector and text emotion feature vector of each standard emotion are set by the staff and updated after each recognition. The facial emotion recognition accuracy x, voice emotion recognition accuracy y and text emotion recognition accuracy z are obtained from the database. , , , , , , , , .
[0042] In a specific embodiment, the characteristic vector of the standard emotion is changed, and the specific changing process is as follows: each recognition record is obtained from the database to obtain the emotion recognized each time and the highest similarity index of each recognition; according to the emotion recognized each time, the highest similarity index of each recognition is summarized to obtain the similarity index of each recognition of each standard emotion; the average value of the similarity index of each recognition of each standard emotion is calculated to obtain the precise index of the characteristic vector of each standard emotion; if the precise index is lower than the preset index, it means that the standard emotion needs to be updated; the characteristic vector corresponding to the highest similarity index in the standard emotion recognition record is selected as the updated characteristic vector of the standard emotion, and updated after each recognition.
[0043] A database is used to store historical analysis records of recognition segments, three collection feature vectors of standard emotions, historical recognition records of recognition segments, intervals of various facial emotion attributes of positive emotions, intervals of various facial emotion attributes of negative emotions, various emotion word characters of positive emotions, various emotion word characters of neutral emotions, various emotion word characters of negative emotions, analysis accuracy of the three collections, the number of successful analyses of each collection range, the duration of each analysis of each collection range, facial emotion recognition accuracy x, voice emotion recognition accuracy y and text emotion recognition accuracy z.
[0044] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined in this specification, they shall all fall within the protection scope of the present invention.
Claims
1. An artificial intelligence emotion recognition system based on deep learning, characterized in that: Includes the following modules: The emotional information analysis module is used to collect various facial emotional values, various voice emotional values and various emotional text characters of the recognition segment, and then analyze the various facial emotional values, various voice emotional values and various emotional text characters of the recognition segment to obtain the emotional attributes and emotional feature data of the three items of facial collection, voice collection and text collection of the recognition segment; The emotional attribute analysis module is used to judge the emotional attributes of the three items collected in the recognition segment. If the analysis result is accurate, the emotional feature data of the three items collected in the recognition segment are analyzed to obtain the emotional feature vector of the three items collected in the recognition segment with accurate analysis results. If the analysis result is inaccurate, the historical analysis records of the recognition segments in the database are analyzed to obtain other recognition segments, and then the emotional attributes and feature data of the three items collected in the other recognition segments are collected, and the emotional attributes and feature data of the three items collected in the other recognition segments are analyzed to obtain the emotional feature vector of the three items collected in the recognition segment with inaccurate analysis results. The sentiment data analysis module is used to collect feature vectors of the three standard emotions set in the database, collect the three sentiment feature vectors of the recognition segment, obtain the sentiment of the recognition segment, and change the feature vector of the standard emotion according to the historical recognition record of the recognition segment in the database.
2. The artificial intelligence emotion recognition system based on deep learning according to claim 1, characterized in that: The specific collection process of various facial emotion values, various voice emotion values and various emotion text characters of the collection recognition segment is as follows: The facial image of the recognition segment is collected by the camera, and various facial emotion values of the recognition segment are obtained from the facial image. The sound of the recognition segment is collected by the sound input device, and various sound emotion values of the recognition segment are obtained from the sound of the recognition segment. At the same time, the sound is converted into text characters by the speech-to-text device, and various emotional text characters are extracted from the text characters of the recognition segment.
3. The artificial intelligence emotion recognition system based on deep learning according to claim 1, characterized in that: The specific analysis process of the sentiment information analysis of the recognition segment is as follows: The emotional attributes are divided into three categories: positive emotion, neutral emotion and negative emotion. The data intervals of various facial emotional attributes of positive emotion and negative emotion are obtained from the database. If the various facial emotional values of the recognition segment belong to the positive emotional facial emotional data interval, it indicates that the facial emotional attribute of the recognition segment is positive. If the various facial emotional values of the recognition segment belong to the negative emotional facial emotional data interval, it indicates that the facial emotional attribute of the recognition segment is negative. If the facial emotional attribute of the recognition segment is neither positive nor negative, it indicates that the facial emotional attribute of the recognition segment is neutral. In this way, the facial emotional attribute of the recognition segment is obtained. Similarly, the sound emotional attribute of the recognition segment is obtained. According to the emotional word characters of positive emotions, neutral emotions and negative emotions in the database, the emotional text characters of the recognition segment are divided into positive emotional recognition segment text characters, neutral emotional recognition segment characters and negative emotional recognition segment characters, and then the number of positive, neutral and negative characters in the recognition segment is obtained, and the emotional text characters of the recognition segment are summarized to obtain the total emotional text characters of the recognition segment, and the positive text rate of the recognition segment is obtained by dividing the number of positive characters in the recognition segment by the total emotional text characters. Similarly, the neutral text rate and negative text rate of the recognition segment are obtained, and the attribute with the highest text rate is selected as the text emotional attribute of the recognition segment, so as to obtain the three collected emotional attributes of the recognition segment. At the same time, according to the preset three collected feature data templates, the three collected feature data are extracted from various facial emotion values, various sound emotion values and each emotional text character of the recognition segment.
4. The artificial intelligence emotion recognition system based on deep learning according to claim 3 is characterized in that: The emotional attributes of the three items collected in the recognition segment are judged, and the specific judgment process is as follows: If the facial emotion attributes, voice emotion attributes and text emotion attributes of the recognition segment are the same, it indicates that the analysis result is accurate. If any one of the three emotion attributes collected in the recognition segment is different from the other two emotion attributes collected, it indicates that the analysis result is inaccurate.
5. The artificial intelligence emotion recognition system based on deep learning according to claim 1, characterized in that: The three collected emotional feature data of the recognition segment are analyzed, and the specific analysis process is as follows: The three items of collected emotional feature data of the recognition segment include facial emotional feature data, voice emotional feature data and text emotional feature data of the recognition segment. The three items of collected emotional feature data of the recognition segment are converted into three items of collected emotional feature vectors according to a conversion scheme. The conversion scheme is: the facial emotional feature data of the recognition segment is represented by a CNN model to obtain the facial emotional feature vector of the recognition segment; the voice emotional feature data of the recognition segment is extracted by the features of the ERMSLM model to obtain the voice emotional feature vector of the recognition segment; the text emotional feature data of the recognition segment is calculated by the Attention mechanism to obtain the text feature vector of the recognition segment.
6. The artificial intelligence emotion recognition system based on deep learning according to claim 5, characterized in that: The analysis of the historical analysis records of the identified segments in the database is performed, and the specific analysis process is as follows: Take the historical analysis records of the identification segment, including the number of successful analyses and the average analysis time of each acquisition range, and substitute them into the index calculation formula In the above example, we can get the usage index of the collection range b. , b is the number of the collection range, , , and are the number of successful analyses and the average analysis time for acquisition range b, and They are the preset standard number of successful analyses and standard analysis duration, and They are the weight factors of the preset standard successful analysis times and the average analysis time. , , , select the collection range of the maximum usage index as the collection range of the identification segment.
7. The artificial intelligence emotion recognition system based on deep learning according to claim 6, characterized in that: The three collected emotional attributes and characteristic data of each other recognition segment are analyzed, and the specific analysis process is as follows: Count the emotional attributes of the three items collected in each other recognition segment, and obtain the number of other recognition segments of the three types of emotional attributes collected by the face. Record the emotional attributes of the other recognition segments with the largest number of emotional attributes collected by the face as the emotional attributes of the recognition segment, so as to obtain the emotional attributes of the face collection with inaccurate analysis. From the erroneous emotional feature data of each other recognition segment, extract the emotional feature data of each other recognition segment corresponding to the emotional attributes of the inaccurate facial collection, calculate the average value, and obtain the emotional feature data of the face collection of the recognition segment with inaccurate analysis results. By analogy, obtain the emotional attributes of the sound collection of the inaccurate recognition segment, the emotional feature data of the sound collection, the emotional attributes of the text collection, and the emotional feature data of the text collection, and summarize to obtain the emotional attributes of the three items collected by the inaccurate recognition segment; If the emotional attributes of the three items collected are all the same, the emotional feature data of the three items collected with inaccurate analysis results are converted according to the conversion scheme to obtain the emotional feature vectors of the three items collected with inaccurate analysis results; If any two of the three collected emotional attributes have the same emotional attributes and are different from the emotional attributes of another collected item, the emotional feature data of the two collected items with the same emotional attributes are transformed according to the transformation scheme, and the emotional feature vector of the other collected item with different emotional attributes is recorded as zero, so as to obtain the emotional feature vector of the three collected items with inaccurate analysis results; If any of the three collected emotional attributes is different from the other two, the analysis accuracy of the three collected items is obtained from the database, the feature data of the item with the highest analysis accuracy is selected and converted according to the conversion plan, and the feature vectors of the other two collected items are recorded as zero, so as to obtain the emotional feature vectors of the three collected items with inaccurate analysis results.
8. The artificial intelligence emotion recognition system based on deep learning according to claim 1, characterized in that: The precise emotional feature vector of the identified segment is analyzed, and the specific analysis process is as follows: Substitute the feature vectors of each standard emotion and the feature vectors of the recognition segment into the emotion similarity calculation formula In the above example, we get the similarity index of the standard emotion a. , a is the number of the standard emotion set, , , A, B and C are the facial emotion feature vector, voice emotion feature vector and text emotion feature vector of the recognition segment respectively, , and are the facial emotion feature vector, voice emotion feature vector and text emotion feature vector of the standard emotion a in the database, , and are respectively a preset weight factor for the first facial collection, a preset weight factor for the first sound collection, and a preset weight factor for the first text collection, , , , , and are respectively a preset weight factor for the second facial collection and a preset weight factor for the second sound collection, , , , and are the weight factor of the third sound collection and the weight factor of the second text collection, respectively. , , , and are respectively a preset weight factor for the third face collection and a preset weight factor for the third text collection, , , , the standard emotion with the largest similarity index is selected as the emotion of the identified segment.
9. The artificial intelligence emotion recognition system based on deep learning according to claim 1, characterized in that: The specific process of changing the characteristic vector of the standard emotion is as follows: Obtain each recognition record from the database, obtain the emotion recognized each time and the highest similarity index of each recognition, summarize the highest similarity index of each recognition according to the emotion recognized each time, obtain the similarity index of each recognition of each standard emotion, calculate the average value of the similarity index of each recognition of each standard emotion, and obtain the precise index of the feature vector of each standard emotion. If the precise index is lower than the preset index, it means that the standard emotion needs to be updated, select the feature vector corresponding to the highest similarity index in the standard emotion recognition record as the updated feature vector of the standard emotion, and update it after each recognition.
10. The deep learning-based artificial intelligence emotion recognition system according to claim 1, characterized in that: It also includes a database for storing historical analysis records of recognition segments, three collected feature vectors of standard emotions, historical recognition records of recognition segments, intervals of various facial emotion attributes of positive emotions, intervals of various facial emotion attributes of negative emotions, various emotion word characters of positive emotions, various emotion word characters of neutral emotions, various emotion word characters of negative emotions and the analysis accuracy of the three collections.
