Employee sentiment index evaluation method and device based on multi-modal data

Abstract

The application provides an employee emotion index evaluation method and device based on multi-modal data, which comprises the following steps: collecting multi-modal related data of employees; the related data comprises voice data of employees, text data in enterprise communication tools, video data of visual behavior capture and physiological signals of employees; performing feature extraction on the related data and cross-modal alignment to obtain multi-modal technical features; calculating data quality coefficients and scene correlation degree coefficients of each mode to calculate final dynamic weights of each mode; fusing the multi-modal technical features based on the final dynamic weights of each mode and performing analysis to obtain a comprehensive emotion index of the employees; the comprehensive emotion index comprises emotion tendency, emotion intensity and stress index. In the application, the comprehensive emotion index of the employees is obtained by comprehensively analyzing the data of multiple modes, so that the defects of insufficient comprehensiveness and low accuracy of current employee emotion evaluation are overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to an employee emotion index evaluation method and device based on multi-modal data. BACKGROUND

[0002] Currently, enterprises mostly use questionnaires, interviews and observations to understand the emotions of employees. Traditional methods cannot accurately discover employee emotions in real time, which brings many obstacles to team building. In terms of team cooperation, if the conflicts or estrangement between employees due to emotional problems cannot be timely detected, it may lead to poor internal communication and low cooperation efficiency in the team. Employees in a long-term negative emotional state will have a significant decline in work performance, which will further affect the overall performance of the company. Moreover, if the employee emotional problems spread within the company, it may also lead to talent loss, increase the company's labor costs, weaken the company's market competitiveness, and hinder the smooth realization of the company's goals.

[0003] The prior art has the following disadvantages: strong subjectivity, poor real-time performance, low data collection frequency, and difficulty in fully capturing the complexity of emotions. In addition, the accuracy is limited by only using text analysis for judgment. SUMMARY

[0004] The main purpose of the present application is to provide an employee emotion index evaluation method and device based on multi-modal data, which aims to overcome the defects of current employee emotion evaluation being not comprehensive and low accuracy.

[0005] To achieve the above purpose, the present application provides an employee emotion index evaluation method based on multi-modal data, comprising the following steps:

[0006] Collecting multi-modal related data of employees; the related data includes voice data of employees, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of employees;

[0007] Feature extraction is performed on the related data and cross-modal alignment is performed to obtain multi-modal technical features;

[0008] The data quality coefficients and scene correlation coefficients of each modality are calculated to calculate the final dynamic weight of each modality;

[0009] Based on the final dynamic weight of each modality, the multi-modal technical features are fused and analyzed to obtain the comprehensive emotion index of the employee; the comprehensive emotion index includes emotional tendency, emotional intensity and stress index.

[0010] Further, the feature extraction of the related data includes:

[0011] Extracting acoustic features of speech rate, tone, pause frequency from real-time collected conference / call voice;

[0012] Analyzing the sentiment tendency of text data in enterprise communication tools and identifying semantic features in combination with contextual context;

[0013] Analyzing the implicit emotional features of facial micro-expressions and body language in video data collected by the camera;

[0014] Analyzing the physiological features of physiological signals collected by wearable devices.

[0015] Further, the data quality coefficients of each modality are calculated, including:

[0016] Based on the signal-to-noise ratio of voice data, voice activity detection integrity, the data quality coefficient of voice data modality is calculated;

[0017] Based on the occluded area of the camera, the effective area ratio, the data quality coefficient of the video data modality is calculated;

[0018] The data quality coefficient of the text data modality is calculated by using the BERT model to calculate the text confidence and the special symbol processing state;

[0019] Based on the stability of physiological signals, the data missing rate, the data quality coefficient of physiological signal modality is calculated.

[0020] Further, the calculation of the scene correlation coefficient includes:

[0021] For each modality data in different scene types, different correlation coefficients are preset; wherein the scene types at least include remote conference, independent office, team collaboration discussion;

[0022] According to the current scene type when collecting related data, the scene correlation coefficient of each modality is determined.

[0023] Further, the calculation formula of the final dynamic weight of each modality is: ;

[0024] Wherein, 、 The data quality coefficient of the i, jth modality, 、 The scene correlation coefficient of the i, jth modality, The final dynamic weight of the ith modality.

[0025] Further, after obtaining the comprehensive emotional index of the employee, including:

[0026] determine an emotion type based on the comprehensive emotion index, and generate a coping strategy according to the emotion type, the current overtime length of the employee, and the daily work report content; wherein the coping strategy comprises automatically triggering a workflow to adjust the task priority and recommending a rest interval;

[0027] establish a correlation model between the comprehensive emotion index and the performance indicator, and identify employees with a high risk of turnover tendency.

[0028] Further, the method further comprises:

[0029] real-time removal of identity information from the voice data and the text data;

[0030] The behavior data in the related data is processed by using a k-anonymization technology.

[0031] Further, after obtaining the comprehensive emotion index of the employee, the method further comprises:

[0032] obtaining environmental parameters, including the noise level of the office area, the light intensity, the temperature and humidity parameters, and the team personnel density;

[0033] obtaining the interference coefficients corresponding to each of the environmental parameters; wherein a correlation analysis is performed in advance between the environmental parameters and the emotion recognition error to generate the interference coefficients corresponding to each of the environmental parameters;

[0034] calibrating the comprehensive emotion index based on the interference coefficients of each of the environmental parameters to obtain a final comprehensive emotion index.

[0035] Further, after obtaining the comprehensive emotion index of the employee, the method further comprises:

[0036] constructing a team work relationship graph, calculating the emotion conduction coefficients between employees based on the collaborative work frequency and the communication density of the employees in the team;

[0037] real-time monitoring of employees with abnormal comprehensive emotion indexes in the team, calculating the potential influence value of the abnormal employees on the overall emotion of the team based on the emotion conduction coefficients between the employees;

[0038] when the potential influence value exceeds a preset threshold, automatically generating a differentiated intervention scheme.

[0039] The application also provides an employee emotion index evaluation device based on multi-modal data, comprising:

[0040] a collection unit for collecting multi-modal related data of employees; the related data comprises voice data of employees, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of employees;

[0041] an alignment unit configured to extract features from the related data and perform cross-modal alignment to obtain multi-modal technical features;

[0042] a calculation unit configured to calculate data quality coefficients and scene correlation coefficients of each modality to calculate final dynamic weights of each modality;

[0043] an analysis unit configured to fuse the multi-modal technical features based on the final dynamic weights of each modality and perform analysis to obtain a comprehensive emotion index of the employee; the comprehensive emotion index includes an emotion tendency, an emotion intensity, and a stress index.

[0044] The employee emotion index evaluation method and device based on multi-modal data provided by the application include: collecting multi-modal related data of an employee; the related data includes voice data of the employee, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of the employee; features are extracted from the related data and cross-modal alignment is performed to obtain multi-modal technical features; data quality coefficients and scene correlation coefficients of each modality are calculated to calculate final dynamic weights of each modality; the multi-modal technical features are fused based on the final dynamic weights of each modality and analysis is performed to obtain a comprehensive emotion index of the employee; the comprehensive emotion index includes an emotion tendency, an emotion intensity, and a stress index. In the application, the comprehensive emotion index of the employee is obtained by comprehensively analyzing the data of multiple modalities, which overcomes the defects of current employee emotion evaluation, such as insufficient comprehensiveness and low accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a step schematic diagram of the employee emotion index evaluation method based on multi-modal data in an embodiment of the application;

[0046] Figure 2 is a structural block diagram of the employee emotion index evaluation device based on multi-modal data in an embodiment of the application;

[0047] Figure 3 is a structural schematic block diagram of a computer device in an embodiment of the application.

[0048] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the application more clear, the application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application.

[0050] Reference Figure 1In an embodiment of the present application, a method for evaluating employee emotion index based on multi-modal data is provided, comprising the following steps:

[0051] In step S1, multi-modal data of employees are collected, including voice data of employees, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of employees.

[0052] In step S2, feature extraction is performed on the data, and cross-modal alignment is performed to obtain multi-modal technical features.

[0053] In step S3, data quality coefficients and scene correlation coefficients of each mode are calculated to calculate the final dynamic weight of each mode.

[0054] In step S4, the multi-modal technical features are fused based on the final dynamic weight of each mode, and the comprehensive emotion index of the employees is obtained by analysis, including emotion tendency, emotion intensity, and stress index.

[0055] In this embodiment, as described above in step S1, the basic information reflecting the emotional state of employees is obtained through diversified data collection methods. Specifically, the voice data collection covers the real-time voice stream of employees in conference speeches, work calls and other scenarios, and supports the retrospective acquisition of historical audio files to capture voice expression features at different times. The text data collection in enterprise communication tools covers emails, instant messaging messages and other content, including not only the text itself, but also auxiliary expression elements such as emoticons and special punctuation, and the context association relationship is preserved completely. The visual behavior data records the facial micro-expression changes and body movement trajectories of employees through the image collection device deployed in compliance, focusing on capturing the emotional external manifestations that are difficult to hide. The physiological signals are monitored by the wearable device worn by the employee, including heart rate fluctuations, skin electrical reactions and other physiological indicators, which can directly reflect the activity state of the autonomic nervous system and provide an objective basis for judging the internal emotional changes. In the collection process, the specific scene, time node and other metadata generated by the data are recorded synchronously, and all original data can also be encrypted to ensure data security.

[0056] As described in step S2 above, it is the key link to transform the original data into analyzable features. For voice data, acoustic analysis techniques are used to extract features such as speech rate changes, pitch fluctuations, and pause patterns, which can reflect the changes in speech caused by emotional fluctuations; for text data, natural language processing techniques are used to analyze the emotional tendencies and semantic implications in the text, as well as to identify the emotional features implied by complex expressions such as sarcasm and metaphor; for visual data, computer vision algorithms are used to extract facial key point motion trajectories (such as mouth corner curvature and eyebrow position changes) and body posture features (such as hand gesture amplitude and body tilt angle) from video streams, which can reflect unconscious emotional expressions; for physiological data, signal processing techniques are used to extract physiological features such as heart rate variability and skin conductance intensity that can reflect stress and emotional state. After completing feature extraction, standardized processing is used to map features of different modalities to a unified numerical interval, and a time window division strategy is used to solve the timing difference problem. Finally, a feature mapping model is used to project various features into a shared feature space, effectively aligning cross-modal features and ensuring comparability of different types of features.

[0057] As described in step S3 above, the weight of each modality data in the comprehensive analysis is determined by quantitative evaluation. The calculation of data quality coefficient is based on the characteristics of different modalities: speech data mainly refers to the intelligibility and the proportion of effective speech, text data focuses on the reliability of semantic analysis and the recognition effect of special symbols, visual data pays attention to the degree of image occlusion and the visibility of key areas, and physiological data is based on signal stability and data integrity. The scene correlation coefficient is preset according to the emotional reflection ability of each modality data in different working scenarios: for example, in a remote meeting scenario, the correlation of speech data is higher; in an independent office scenario, the reference value of physiological signals is greater; in a team collaboration scenario, the correlation of speech and visual data is relatively prominent. The final weight is calculated by integrating the data quality coefficient and the scene correlation coefficient, so that the data with high quality and high matching degree with the current scene occupies a higher weight in the fusion analysis, thereby improving the accuracy of evaluation.

[0058] As described in step S4 above, a quantitative indicator that can comprehensively reflect the emotional state of employees is generated by fusing multi-modal features. Based on the weight determined in step S3, the cross-modal aligned features are weighted and fused to finally form three-dimensional comprehensive emotion indexes: emotional tendency is used to distinguish the positive or negative attributes of emotion, emotional intensity reflects the intensity of emotional expression, and stress index measures the current psychological stress level of employees. The generation of the above indexes is not only based on real-time data, but also combines historical features for trend analysis, and is associated with specific working scenarios and task information to trace the potential causes of emotional changes. Through this multi-dimensional quantitative evaluation, the emotional state of employees can be objectively presented.

[0059] In an embodiment, feature extraction is performed on the related data, including:

[0060] Acoustic feature extraction of speech speed, tone, and pause frequency of real-time collected conference / call speech;

[0061] Analyzing the sentiment tendency of text data in enterprise communication tools and identifying semantic features in combination with contextual context;

[0062] Analyzing the implicit emotional features of facial micro-expressions and body language in video data collected by a camera;

[0063] Analyzing the physiological features of physiological signals collected by a wearable device.

[0064] In this embodiment, for real-time collected conference or call speech, core features reflecting emotional state are extracted through acoustic analysis technology. Among them, the speech speed feature is obtained by counting the number of phonemes and the change rate in unit time, which can reflect the emotional tension or relaxation state; the tone feature focuses on the mean, fluctuation range and change trend of the fundamental frequency, and different emotions (such as rising tone when angry and falling tone when depressed) will show significant differences; the pause frequency is calculated by counting the number of speech interruptions and the proportion of pause duration in unit time, which captures the expression rhythm changes caused by hesitation, thinking or emotional fluctuations.

[0065] For text data in enterprise communication tools, natural language processing technology is used for in-depth analysis. First, sentiment analysis is used to identify the positive, negative or neutral emotional tone implied in the text, and the coherence of the semantics is understood in combination with the contextual context to avoid bias caused by isolated interpretation. On this basis, further extraction of semantic features including word sentiment intensity, sentence structure features (such as strong emotions expressed by rhetorical questions), special symbols (such as exclamation marks, emoticons), and other emotional directions, especially focusing on the identification of indirect expression methods such as satire and metaphor, to fully capture the real emotions behind the text.

[0066] For video data collected by a camera, computer vision algorithms are used for emotion feature extraction. Through facial key point detection technology, micro-expression changes such as eyebrow raising or drooping, mouth stretching or shrinking, and eye muscle movement state are captured, which often reflect the true emotions that are difficult to hide; at the same time, through human pose estimation, body language features are analyzed, including the amplitude and frequency of gestures, the forward or backward of the body, and the change of the sitting posture, etc., which converts these non-verbal signals into quantifiable emotional features, supplementing the deficiencies of facial expression analysis.

[0067] For the physiological signals collected by the wearable device, the indicators closely related to the emotional state are extracted through the physiological feature analysis technology. The heart rate variability (the fluctuation of heart rate over time) is focused on, which is directly related to emotions such as stress and anxiety; the intensity and frequency of the galvanic skin response are also analyzed, which can reflect the degree of sympathetic nervous excitement and indirectly represent the arousal level of emotions; in addition, auxiliary physiological features such as body temperature changes and respiratory rates are also included, which together constitute a set of physiological features reflecting the internal emotional fluctuations.

[0068] In an embodiment, the data quality coefficients of each modality are calculated, including:

[0069] The data quality coefficient of the speech data modality is calculated based on the signal-to-noise ratio of the speech data and the speech activity detection integrity;

[0070] The data quality coefficient of the video data modality is calculated based on the occluded area of the camera and the effective area ratio;

[0071] The data quality coefficient of the text data modality is calculated based on the text confidence calculated by the BERT model and the special symbol processing state;

[0072] The data quality coefficient of the physiological signal modality is calculated based on the stability of the physiological signal and the data missing rate.

[0073] In this embodiment, the data quality coefficient of the speech data modality is calculated based on the signal-to-noise ratio and the speech activity detection integrity as the core evaluation indicators. The coefficient value is determined by quantitatively analyzing the performance of these two indicators. The signal-to-noise ratio reflects the ratio of speech signal to background noise, the higher the value, the better the speech clarity, the less the environmental interference, and the stronger the reliability support for emotion feature extraction; on the contrary, a low signal-to-noise ratio means that the speech is overwhelmed by noise, which may lead to distortion of acoustic feature extraction. The speech activity detection integrity is used to measure the proportion of valid speech segments in the entire collection period. The higher the integrity, the lower the proportion of non-speech segments such as invalid silence and noise, and the more sufficient the valid speech information for emotion analysis; if the integrity is insufficient, it means that the valid speech data is scarce, which may affect the comprehensiveness of emotion feature extraction. In the specific calculation, the specific value of the signal-to-noise ratio and the integrity of the speech activity detection are obtained through signal processing technology, and then the quantitative results of these two indicators are fused into a single quality coefficient according to the preset mapping rule. This coefficient can objectively reflect the reliability of speech data for emotion analysis.

[0074] The data quality coefficient calculation of the video data modality is carried out around the camera blocked area and the effective area proportion ratio, and the quality quantization is realized through the comprehensive evaluation of the two indicators. The camera blocked area refers to the area in the collected picture where the face or key part of the body cannot be seen due to obstacles. The higher the proportion of the blocked area, the less effective visual information can be used for analysis, which may cause the loss of micro-expression or body language features. On the contrary, a low blocked area proportion indicates better integrity of visual data. The effective area proportion ratio focuses on the proportion of the key area (such as face, hand, etc.) related to emotional expression in the entire collected picture. The higher the proportion, the more sufficient the capture of key area details, which is more conducive to extracting accurate emotional features. If the effective area proportion ratio is too low, it may cause the blurring or loss of key emotional expression information. When calculating, the computer vision algorithm needs to be used to analyze the video frame to obtain the blocked area proportion and the effective area proportion ratio, respectively. Then, the two are converted into a unified quality coefficient through weighted fusion to reflect the support quality of the video data for emotional analysis.

[0075] The data quality coefficient calculation of the text data modality is based on the text confidence output by the BERT model and the special symbol processing state. The final coefficient is determined by integrating the performance of the two indicators. The text confidence calculated by the BERT model reflects the reliability of the model's understanding of the text semantics and the judgment of the emotional tendency. The higher the confidence, the better the accuracy of text feature extraction, and the greater the reference value for emotional analysis. If the confidence is low, it means that the text semantics may be misread, affecting the effectiveness of emotional features. The special symbol processing state is used to evaluate the recognition and analysis ability of non-text emotional expression elements such as emoticons and special punctuation in the text. The better the processing state, the more complete the capture and conversion of these auxiliary emotional expression information into effective features, which can supplement the emotional expression of the text itself. If the processing state is not good, it may cause the omission of non-text emotional information, affecting the comprehensiveness of text emotional analysis. In the calculation process, the confidence value of the BERT model and the completion degree index of special symbol processing need to be obtained first, and then they are integrated into a single quality coefficient according to the preset fusion rule to quantify the quality level of the text data for emotional analysis.

[0076] The data quality coefficient calculation of the physiological signal modalities takes the stability of the physiological signal and the data loss rate as the core indicators, and realizes quality quantization by comprehensively analyzing the performance of the two indicators. The stability of the physiological signal reflects the fluctuation degree of the signal in the collection process. The higher the stability, the less the signal is affected by equipment interference and other factors, and the more the physiological characteristics (such as heart rate variability and galvanic skin response) extracted can truly reflect the emotional state. If the stability is insufficient, the noise mixed in the signal may cause the physiological characteristics to be distorted, affecting the accuracy of emotional analysis. The data loss rate is used to measure the proportion of the physiological signal collected completely in the total collection period. The lower the loss rate, the better the integrity of the physiological data, which can provide continuous and complete emotional change trajectory. If the loss rate is too high, it means that the physiological information of the key period is lost, which may cause the emotional feature analysis to be broken. In the calculation, the signal processing technology is used to evaluate the stability level and the data loss rate of the physiological signal respectively, and then a preset fusion algorithm is used to convert the two into a unified quality coefficient to objectively reflect the reliability of the physiological data for emotional analysis.

[0077] In an embodiment, the calculation of the scene correlation coefficient includes:

[0078] Different correlation coefficients are preset for each modality data in different scene types; wherein the scene types at least include remote meeting, independent office, team collaboration discussion;

[0079] According to the current scene type when collecting related data, the scene correlation coefficient of each modality is determined.

[0080] In this embodiment, the preset of the scene correlation coefficient is based on the actual contribution difference of each modality data to emotional expression in different working scenes, and a standardized coefficient mapping rule is established. First, the core working scenes of enterprises need to be sorted out, which at least cover three typical scenes of remote meeting, independent office, and team collaboration discussion. These three types of scenes are significantly different in communication methods, interaction frequency, and emotional expression carriers, and are the key scenes of emotional state changes in employees' daily work.

[0081] For each type of scene, the correlation of each modality data (voice, text, video, and physiological signal) with emotional recognition needs to be analyzed: in the remote meeting scene, voice is the main communication carrier for employees to convey opinions and express attitudes, and emotional fluctuations are often directly reflected in the changes of speech speed and tone. Therefore, the correlation coefficient of the voice modality needs to be set to a high level; text is mostly used as a supplementary record of meeting content or short interaction, and its reflection of emotion is weaker than voice, so the correlation coefficient is second; video can capture facial micro-expressions (such as frowning and drooping corners of the mouth), which can assist in judging the emotional state, so the correlation coefficient needs to be higher than text but lower than voice; physiological signals are mainly used to monitor the hidden stress caused by long meetings, and their reflection of immediate emotional expression is relatively indirect, so the correlation coefficient is set to a low level.

[0082] In the independent office scenario, the employee mainly works alone, and the voice communication is very little, so the contribution of the voice modality to the emotion recognition is greatly reduced, and the correlation coefficient is set to the lowest; the text is mostly the work docking message or task record with colleagues, the emotion expression is relatively implicit, and the correlation coefficient is slightly higher than that of the voice; the video can capture the natural state of the employee when he is alone (such as restlessness after sitting for a long time, relaxation after completing a task), and the reflection of the emotion is more real, so the correlation coefficient is higher than that of the text; the physiological signal can monitor the psychological stress when the employee is alone in real time, and is the core carrier of emotion recognition, so the correlation coefficient is set to the highest.

[0083] In the team collaboration discussion scenario, the employee frequently exchanges opinions through the voice, and the emotion often fluctuates with the collision of views, so the correlation coefficient of the voice modality remains at a high level; the text is mostly temporary communication or file sharing instructions in the collaboration process, and the emotion reflection is weak, so the correlation coefficient is lower than that of the voice; the video can capture the body language (such as gesture amplitude and body orientation) in the team interaction, which can assist in judging whether the emotion is positive, so the correlation coefficient is higher than that of the text; the physiological signal is used to monitor the collaboration pressure (such as skin conductance when there is a difference of opinion), but the reflection of the emotion has a lower priority than the voice and the video, so the correlation coefficient is set to a medium level.

[0084] Through the above correlation analysis of the scenarios and modalities, a fixed correlation coefficient is preset for each modality data in each type of scenario, forming a standardized coefficient library to ensure the consistency and rationality of subsequent coefficient calling.

[0085] In the actual employee emotion index evaluation, the specific scenario type during data collection needs to be determined, and the current data corresponding to the scenario (remote meeting, independent office, or team collaboration discussion) is determined through automatic recognition of the system (such as judging whether it is a remote meeting by connecting with the meeting software, or judging whether it is an independent office by the office area camera) or manual auxiliary labeling.

[0086] After the scenario type is determined, the correlation coefficients of the modality data corresponding to the scenario are called from the preset coefficient library: if the current scenario is a remote meeting, the preset coefficients of the voice, text, video, and physiological signal in the remote meeting scenario are directly matched; if the current scenario is an independent office, the corresponding coefficients in the independent office scenario are called; if it is a team collaboration discussion scenario, the preset coefficients in the scenario are called.

[0087] In an embodiment, the calculation formula of the final dynamic weight of each modality is: ;

[0088] wherein, , are the data quality coefficients of the i-th and j-th modalities, , is the scene correlation coefficient of the i-th and j-th modal, respectively, is the final dynamic weight of the i-th modal.

[0089] In this embodiment, the synergy of data reliability and scene adaptability is achieved. The numerator part of the formula multiplies the data quality coefficient of the i-th modal with the scene correlation coefficient, which not only reflects the reliability of the modal data itself (such as the intelligibility of voice data, the accuracy of text analysis), but also reflects its matching degree with the current scene (such as the high correlation of voice data in remote meetings). This product relationship ensures that only the modal data with high quality and high scene adaptability can occupy a dominant position in the maximum weight, avoiding the deviation caused by single-dimensional evaluation. For example, even if the quality of certain modal data is extremely high, but the correlation with the current scene is low (such as voice data in the independent office scene), its weight will be reasonably suppressed.

[0090] Through the above calculation logic, the weight of each modal can objectively reflect its actual information value in the current scene: high-quality and high-correlation modal data obtains higher weight and plays a dominant role in the calculation of the comprehensive emotion index; low-quality or low-correlation modal data weight is reduced, reducing its interference with the overall result. This differentiated weight allocation mechanism makes the multi-modal feature fusion process more consistent with the actual emotion expression rule, ultimately improving the accuracy and reliability of employee emotion index evaluation, and providing a more scientific quantitative basis for the formulation of subsequent emotion intervention strategies.

[0091] In an embodiment, after obtaining the comprehensive emotion index of the employee, the following steps are included:

[0092] Based on the comprehensive emotion index, determine the emotion type, and generate coping strategies according to the emotion type combined with the current overtime length and work daily report content of the employee; wherein the coping strategies include automatically triggering workflows to adjust task priorities, recommending rest intervals;

[0093] Establish a correlation model between the comprehensive emotion index and performance indicators to identify high-risk turnover tendency employees.

[0094] In this embodiment, after obtaining the comprehensive emotion index of the employee, first, according to the combined characteristics of emotion tendency, emotion intensity and stress index, the current emotion type of the employee is accurately defined, for example, when the emotion tendency presents obvious negative features, the emotion intensity is at a high level, and the stress index exceeds the normal threshold, it can be determined as high-pressure negative emotion; if the emotion tendency is positive but the emotion intensity is low, and the stress index is in the normal range, it can be determined as stable positive emotion. Through the synergy analysis of multi-dimensional indexes, the accuracy of emotion type classification is ensured.

[0095] On the basis of identifying the emotion type, further combined with the real-time work state data of the employee, a deep correlation analysis is carried out: on the one hand, the current overtime length information of the employee is called to judge whether the emotion is abnormal due to long-term high-intensity work; if the high-pressure negative emotion is accompanied by overtime work for many consecutive days, it usually points to the emotional problems caused by work overload; on the other hand, the key content in the work diary of the employee is analyzed, the task progress, difficulty feedback and other information are extracted, and it is identified whether the emotional abnormality is related to a specific work task; for example, if the task difficulty exceeds the expectation and the cooperation is blocked in the work diary, and the negative emotion characteristics appear synchronously, the cause of the emotional problem can be locked and the specific work scene is related.

[0096] Based on the correlation analysis results of the above emotion type and work state, a differentiated coping strategy is generated: for the high-pressure negative emotion caused by work overload, the workflow adjustment mechanism is automatically triggered, the task priority of the employee is reduced, the urgent but non-core task is assigned to other collaborative members, and reasonable rest intervals are recommended according to the overtime length and pressure index to avoid further deterioration of the emotion; for the negative emotion caused by task difficulty, in addition to adjusting the task priority, relevant resource support (such as technical guidance documents, collaboration personnel assistance channels) is also pushed synchronously, and the task is decomposed in stages to help the employee gradually relieve stress; for the employee with stable positive emotion, the challenge task can be recommended to be moderately increased, or the current work rhythm can be maintained to continue the good emotional state. The whole coping strategy generation process is centered on data-driven to ensure the pertinence and executability of the strategy, and realizes the deep integration of emotional intervention and work management.

[0097] Then, through data integration, the comprehensive emotion index data (including the time sequence changes of emotion tendency, emotion intensity and pressure index) and the corresponding performance indicators (such as task completion rate, work quality score, project contribution, etc.) of the employee in the historical period are collected, and an analysis data set containing multi-dimensional data is constructed. In the data preprocessing stage, the emotion index and the performance indicator need to be aligned in time dimension to ensure that each set of emotion data can match the performance in the same period, and abnormal data (such as extreme emotion value, performance statistical error data) is eliminated to ensure the reliability of the data set.

[0098] Subsequently, a correlation model between the comprehensive sentiment index and the performance indicators is constructed using data analysis techniques: through correlation analysis, the dimensions of the sentiment index that have the most significant impact on performance are identified. Further, through regression analysis or machine learning algorithms, the degree of influence of changes in the sentiment index on the performance indicators is quantified, forming a predictable correlation model. Based on the above correlation model, combined with the time series variation characteristics of the employee sentiment index, high-risk employees with a tendency to leave are identified: if the comprehensive sentiment index of an employee shows a long-term negative trend (such as the sentiment tendency remaining in the negative interval, and the stress index being higher than the threshold for a long time), and this change in sentiment has been reflected as a significant decline in performance indicators (such as a decrease in task completion rate, a decrease in work quality score), while excluding short-term work task adjustments, personal temporary affairs, and other non-sustained influencing factors, it can be determined that the employee has a high risk of leaving. Such employees are marked as key attention objects, and warning information is pushed to managers, combined with the causes of changes in their emotions and performance, targeted intervention measures (such as one-on-one communication and counseling, work task re-matching, career development path planning, etc.) are implemented to help the enterprise intervene in advance and reduce the risk of loss of core talents.

[0099] In an embodiment, after obtaining the comprehensive sentiment index of the employee, the following steps are included:

[0100] Generate an employee personal sentiment dashboard to display real-time sentiment radar charts, historical trend curves, cross-team comparison analysis, and generate daily employee sentiment reports;

[0101] Display group sentiment distribution by department / project team;

[0102] Set permission hierarchical control, employees can independently view personal data, managers can only see group statistical results and risk warning data, and set sensitive operation audit logs.

[0103] In this embodiment, after obtaining the employee comprehensive emotion index, first, an employee personal emotion dashboard is constructed, which intuitively presents the dynamic changes and comparison information of the employee emotion state in the form of multi-dimensional data visualization. Among them, the real-time emotion radar chart intuitively displays the specific numerical distribution of the current emotion tendency, emotion intensity and stress index through multi-axis coordinates. The axis length of different dimensions corresponds to the index value, helping employees quickly locate the core characteristics of their current emotions. The historical trend curve presents the change trajectory of the employee's emotion tendency, emotion intensity and stress index in the near future with time as the horizontal axis and emotion index as the vertical axis. Through the curve fluctuation, the stability or abnormal change node of the emotion can be clearly identified. For example, if the stress index curve continues to rise in a certain period, it may prompt the employee to pay attention to the impact of the work or life state in that stage on the emotion. The cross-team comparison analysis compares the employee's personal emotion index with the average emotion index of other employees in the same department or same position, and displays the difference in the form of column chart or line chart, helping employees objectively judge their own emotion state in the group and avoid cognitive bias caused by single self-perception.

[0104] On the basis of the personal emotion dashboard, a daily employee emotion report is further generated. The report content focuses on the change of the emotion index on that day, not only including the real-time emotion radar chart and the core data summary of the emotion trend on that day, but also analyzing the possible causes of the emotion fluctuation in combination with the emotion change node associated with the work scene on that day (such as the meeting period, the task-intensive period). At the same time, preliminary adjustment schemes are given for the abnormal emotion on that day (such as the stress index exceeding the regular threshold), so that employees can timely adjust their own state based on the report content and realize self-management and adjustment of emotion.

[0105] To support emotion management decisions at the organizational level, the comprehensive emotion indexes of individual employees are classified and integrated by department or project team to generate a group emotion distribution visualization result. Specifically, first, the average, median and distribution interval of the emotion tendency, emotion intensity and stress index of all employees in the group are calculated by department or project team, and the group emotion characteristics are reflected through the statistical data.

[0106] In the visual presentation, the group emotion distribution can adopt forms such as heat map, box plot or pie chart: the heat map uses different color gradients to represent the average emotion tendency (such as red for negative and green for positive) or the average stress index (such as dark color for high stress and light color for low stress) of the group by department / project team, so that the manager can intuitively judge the differences in the emotional atmosphere of each group through color; the box plot shows the distribution dispersion degree of the emotion index of each group, helping the manager to identify the balance of the emotion state within the group. Through the visual display of the group emotion distribution, the manager can quickly grasp the overall emotion condition of each department / project team from the organizational level, and timely find the groups with abnormal emotional atmosphere, providing direction for optimizing team management and improving organizational efficiency.

[0107] To protect the safety of employee emotion data, a strict permission hierarchical control mechanism is constructed, and different data access permissions are divided according to user roles (employees, managers). For an employee, only his / her emotion data access permission is opened, and the employee can view the personal emotion dashboard, historical trend curve and daily emotion report to ensure that the employee can independently master his / her emotion state, while avoiding personal data being obtained by others; for a manager, the permission range is limited to the group emotion statistics results within the jurisdiction of the manager and the early warning information of high-risk emotion employees in the group, but the manager cannot view the specific personal emotion detail data of the employees, which meets the management decision needs of the manager and prevents individual data leakage. The permission hierarchical control is realized through account role binding and data access interface restriction, and the user role and permission range are automatically checked when data is called, the data access request beyond the permission is rejected, and the first line of defense for data protection is formed.

[0108] At the same time, sensitive operation audit logs are set to record all key operations involving emotion data in real time. The audit log content includes operation time, operation user, operation type, operation object and operation result, ensuring that each sensitive operation can be traced. When data access anomalies occur, the audit log can provide key traceability basis to help administrators promptly investigate security risks; in addition, the audit log is regularly archived to meet data security compliance requirements, further strengthen the safety protection and safety control of emotion data, and ensure that the entire emotion index evaluation system is both compliant and safe in data use, and enhance the trust of employees in data security.

[0109] In an embodiment, the method further comprises:

[0110] Removing identity information from voice data and text data in real time;

[0111] Using k-anonymization technology to process behavior data in the related data.

[0112] In this embodiment, when voice data is collected and processed, information that can be associated with the identity of an employee, such as voiceprint features and names mentioned in the voice, is stripped in real time; when text data is processed, email addresses, usernames, employee numbers and other identity information that can be directly located to individuals are deleted at the same time, so as to avoid identity information leakage from the source and ensure that subsequent data processing is only carried out around emotion analysis and does not involve employee personal identity association.

[0113] The collected behavior data of the employee visual behavior and physiological signal correlation is processed by using k-anonymization technology. By grouping the data and concealing the individual unique features, the processed data needs to be presented together with at least k other individual data, and the specific behavior information of a certain employee cannot be identified alone, thereby ensuring that the employee behavior data cannot be tracked or identified alone while retaining the data for emotional analysis value.

[0114] In an embodiment, after the analysis obtains the comprehensive emotion index of the employee, the method further comprises:

[0115] Obtaining environmental parameters, including office area noise level, light intensity, temperature and humidity parameters, and team personnel density;

[0116] Obtaining the interference coefficient corresponding to each of the environmental parameters; wherein, the correlation analysis between the environmental parameters and the emotion recognition error is carried out in advance, and the interference coefficient corresponding to each environmental parameter is generated;

[0117] Calibrating the comprehensive emotion index based on the interference coefficient of each environmental parameter to obtain the final comprehensive emotion index.

[0118] In this embodiment, the key environmental data affecting the employee's emotional perception and expression is collected in real time by the sensors deployed in the office area, including the noise level (such as equipment operation noise, personnel communication volume) of the office area, the light intensity (such as natural light intensity, indoor light brightness), the temperature and humidity parameters (such as environmental temperature, air humidity), and the team personnel density (such as the number of employees in the office area at the same time, the degree of workstation crowding). The above environmental parameters may indirectly affect the employee's emotional expression or the emotion recognition process, for example, high decibel noise may cause the employee's emotional irritability, and may also interfere with the accurate extraction of voice emotion features, therefore, it needs to be included in the calibration category.

[0119] Secondly, the determination of the environmental interference coefficient, the correlation analysis between the environmental parameters and the emotion recognition error is carried out in advance: through a large amount of experimental data, the deviation degree of the emotion index evaluation result and the real emotional state of the employee under different environmental parameter values is studied. For example, when the noise level exceeds a certain threshold, the emotion feature recognition error of the voice mode will increase significantly; when the light intensity is too weak, the micro-expression recognition accuracy of the visual mode will decrease. Based on these correlation laws, the corresponding interference coefficient is set for each environmental parameter, and the coefficient size is positively correlated with the emotion recognition error caused by the parameter, that is, the stronger the interference of the parameter on the emotion recognition, the higher the corresponding interference coefficient. In actual application, according to the real-time acquired environmental parameters, the preset interference coefficient is directly matched.

[0120] Finally, the calibration of the comprehensive emotion index is performed, the interference coefficients corresponding to each environmental parameter are integrated, and the influence weight of the different modal emotion characteristics is combined to modify the preliminary comprehensive emotion index. For example, if the noise level in the real-time environment is high (corresponding to a high interference coefficient) and the light intensity is insufficient (corresponding to a medium interference coefficient), the speech emotion characteristics affected by the noise and the visual emotion characteristics affected by the light are mainly modified, and the comprehensive emotion index is recalculated to obtain the final comprehensive emotion index after eliminating the environmental interference, so that the evaluation result is more in line with the real emotional state of the employee in the absence of environmental interference.

[0121] In an embodiment, after the analysis obtains the comprehensive emotion index of the employee, it further includes:

[0122] A team work relationship graph is constructed, and the emotion conduction coefficient between employees in the team is calculated based on the collaborative work frequency and communication density of the employees in the team.

[0123] The comprehensive emotion index of the abnormal employee in the team is monitored in real time, and the potential influence value of the abnormal employee on the overall emotion of the team is calculated based on the emotion conduction coefficient between the employees.

[0124] When the potential influence value exceeds a preset threshold, a differentiated intervention scheme is automatically generated.

[0125] In this embodiment, first, a team work relationship graph is constructed, and the interaction information of team members in the work is collected by connecting the data sources of enterprise internal project management tools, collaborative office software, communication systems, etc. The collaborative work frequency takes the number of projects participated by members, the number of tasks processed by members, and the work duration completed by members as core indicators, reflecting the closeness of the work intersection between members. The communication density takes the number of calls between members, the frequency of instant messages, and the duration of joint participation in meetings as the measurement standard, reflecting the frequency of information interaction between members. The above interaction data is converted into nodes and edges in the graph: each employee is an independent node in the graph, the edges between nodes represent the work association between employees, and the weight of the edge is determined by the comprehensive calculation result of the collaborative work frequency and the communication density. The higher the weight, the closer the work association between the two employees.

[0126] On the basis of the team work relationship graph, the mood conduction coefficient between employees is calculated: the mood conduction coefficient is used to quantify the possibility and strength of the influence of the mood state of a certain employee on the mood of another employee, and the numerical value is positively correlated with the closeness of the work association between the two employees. The closer the work association (i.e. the higher the weight of the edge between nodes in the graph), the greater the probability of mutual mood conduction and the stronger the influence, and the higher the corresponding mood conduction coefficient; on the contrary, the mood conduction coefficient between employees with loose work association is lower. In the calculation process, historical mood conduction data need to be corrected: for example, if historical data show that the probability of a negative mood appearing in one of two employees with close work association and another employee also appearing a negative mood in a short time is significantly higher than that of other combinations, the mood conduction coefficient of the two employees should be increased to ensure that the coefficient can reflect the basis of work association and conform to the actual mood conduction rule. The final mood conduction coefficient matrix provides accurate quantitative basis for subsequent analysis of the spread of mood in the team.

[0127] Then, the comprehensive mood index of the employees in the team is monitored in real time. The comprehensive mood index of each employee is updated at a set frequency (such as every hour, every half day), and employees with abnormal mood index are automatically marked through pre-set abnormality determination criteria (such as mood tendency remaining in the negative range, stress index exceeding the conventional threshold by a certain percentage, and mood intensity fluctuating sharply in a short time), and these employees are regarded as potential mood risk spread sources.

[0128] After identifying the employees with mood abnormalities, the potential influence value of the abnormal employees on the overall mood of the team is calculated based on the obtained mood conduction coefficient. The calculation of the potential influence value needs to cover all associated employees of the abnormal employee in the team work relationship graph. For each member associated with the abnormal employee, the mood conduction coefficient of the two is multiplied by the mood abnormality degree of the abnormal employee (such as the magnitude of the mood tendency deviating from the neutral value, the proportion of the stress index exceeding the threshold), to obtain the individual influence value of the abnormal employee on the member; then, the individual influence values of all associated members are summarized and weighted adjusted in combination with the total number of the team and the importance of the associated members in the team (such as core business backbone, team manager), to finally obtain the potential influence value of the abnormal employee on the overall mood of the team. For example, if the abnormal employee is a core member of the team, he has close work association (high mood conduction coefficient) with most members, and his mood abnormality degree is high, then the calculated potential influence value of the team will be significantly higher than that when an ordinary member has mood abnormality, accurately reflecting the spread range and severity of mood risk in the team.

[0129] After determining that the emotional risk has an impact on the team beyond the controllable range, an intervention strategy is developed to curb the spread of emotional risk and improve the team emotional atmosphere. The core of the differentiated intervention plan is to develop a stratified strategy according to the different roles and the degree of influence of the intervention objects. For the source employee with emotional abnormalities, the intervention plan focuses on individual emotional counseling and problem solving, combined with the possible causes of emotional abnormalities (such as excessive work load and excessive task difficulty), and automatically pushes a one-on-one communication appointment reminder to the manager, while recommending suitable emotional regulation resources (such as stress relief courses and psychological counseling channels). If the associated data shows that emotional abnormalities are related to specific work tasks, the task priority of the employee or the allocation of collaborative support will be adjusted; for the associated members affected by emotional conduction, the intervention plan focuses on emotional prevention and atmosphere guidance, and through team announcement, it pushes positive work atmosphere construction content, while sending the list of associated members to the manager, prompting the manager to pay attention to the emotional changes of the associated members, and if necessary, a small team communication meeting is held to timely resolve potential negative emotional conduction; for the whole team, the intervention plan aims to optimize the emotional atmosphere, and if the potential impact value is significantly high, non-urgent high-intensity work tasks are suspended, short-term team relaxation activities are organized, or office environment parameters (such as noise reduction and optimized lighting) are adjusted to reduce the soil for negative emotional spread from the environmental and task arrangement levels. The whole intervention plan generation process does not require human intervention and is seamlessly integrated with the existing work flow of the enterprise, ensuring that the intervention measures can be quickly implemented and effectively reducing the impact of emotional risk on team performance.

[0130] In an embodiment, the method further comprises:

[0131] Mapping the team work relationship graph into a work relationship undirected graph, mapping the fluctuation variance of the comprehensive emotional index of the employee corresponding to each node in the team work relationship graph to the node of the undirected graph, and giving the node a weight to obtain a fluctuation undirected graph;

[0132] Traversing the nodes of the fluctuation undirected graph, concatenating the fluctuation variance of each node with the node weight, and converting it into a 16-bit encoding string;

[0133] Taking the 16-bit encoding string as the initial input, combining the environmental parameter sequence composed of the environmental parameters of the day to form an original seed pool; iteratively processing the original seed pool, and setting the iteration number to be a preset multiple of the number of currently employed employees in the team. In each iteration process, the fluctuation variance of the node of the fluctuation undirected graph is taken as the fine adjustment amount of the chaotic parameter to obtain a chaotic confusion seed;

[0134] Based on the mean value of the team's daily comprehensive emotional index, the chaotic confusion seed is segmented to obtain a segmented confusion seed; after inserting an expansion parameter into each segmented confusion seed, it is hashed respectively and combined into a hash string;

[0135] The comprehensive emotion indexes of the employees in the team are mapped to integers, and a combination of bit positions corresponding to the integers in a hash string is extracted as a key to encrypt and store relevant data.

[0136] In the embodiment, first, the team work relationship graph is mapped to a work relationship undirected graph, so that the collaboration association between employees is presented in a graphical manner. On this basis, the fluctuation variance of the comprehensive emotion index of the employee corresponding to each node is extracted, which reflects the stability of the employee's emotional state, and the greater the fluctuation, the more unstable the emotion. The fluctuation variance is taken as the node attribute of the undirected graph, and each node is assigned a weight, which is usually positively correlated with the collaboration frequency and role importance of the employee in the team. Finally, a fluctuation undirected graph that integrates work relationship and emotional fluctuation characteristics is formed. Through this transformation, the initial data for key generation is deeply bound to the actual business characteristics and employee emotional state of the team.

[0137] Further, the nodes of the fluctuation undirected graph are traversed, and the traversal order can follow the principle of node weight from high to low. In the traversal process, the fluctuation variance of each node and the node weight are spliced in a fixed format to form a continuous feature data chain. Subsequently, the data chain is converted into a 16-bit hexadecimal encoding string, which compresses the original data volume through encoding and converts numerical information into character form suitable for subsequent encryption operations, providing standardized input for constructing the key seed.

[0138] Next, the randomness and complexity of the key seed are enhanced by integrating multi-dimensional dynamic data and chaos processing. First, the generated 16-bit hexadecimal encoding string is taken as the core input and fused with the environmental parameter sequence of the day. The environmental parameter sequence is composed of real-time data such as noise level, light intensity, temperature and humidity, and personnel density in the office area in a fixed order. By introducing dynamic environmental factors, the uncertainty of the seed pool is improved. The two are spliced in an alternating order of feature encoding string and environmental parameter sequence to form an original seed pool with variable length.

[0139] Subsequently, the original seed pool is subjected to chaotic iteration processing: the number of iterations is set to a pre-set multiple (such as 3 times) of the current number of employees in the team, ensuring that the number of iterations is dynamically adapted to the team size; a chaotic mapping algorithm is used for each iteration, and the fluctuation variance of the corresponding node in the fluctuation undirected graph is used as the fine adjustment amount of the chaotic parameter, so that the iteration process dynamically changes with the employee emotional fluctuation characteristics; the greater the emotional fluctuation of the node, the greater the adjustment amplitude of the chaotic parameter. Through multiple rounds of chaotic iteration, the characteristics of the original seed pool are fully confused, generating a chaotic confusion seed with high randomness, breaking the linear correlation between data, and providing a secure foundation for subsequent key generation.

[0140] Further, by combining the overall team emotion features and segmented operations, the complexity of the key material is further improved. Based on the mean value of the team's comprehensive emotion index on that day, the chaotic confusion seed is divided into several segments, and the higher the mean value, the more segments there are, so that the segmented logic is associated with the team's overall emotional state. Each segment corresponds to a segmented confusion seed, and an expansion parameter is inserted in each segment. The expansion parameter is converted from the mean value of the edge weight of the undirected graph, reflecting the overall team collaboration tightness.

[0141] Each segmented confusion seed after inserting the expansion parameter is subjected to a hash operation, and the operation process uses a variant hash algorithm to avoid security risks caused by fixed algorithm patterns by adjusting the initial constant and round function parameters of the standard algorithm. The hash results of all segments are spliced in order to form a complete hash string, so that the final key material integrates team emotion features, collaboration relationships, and chaotic randomness.

[0142] Then, by mapping the individual emotional features of the employees, key information is extracted from the hash string to form the final key. The comprehensive emotion index of each employee in the team is mapped to a specific integer, and the mapping rule is based on the normalization processing and interval division of the index value to ensure that the emotional state of each employee corresponds to a unique integer identifier. According to these integers, the corresponding bits in the hash string are located, and these bits are sorted and combined according to the tightness of the collaboration relationship between employees to form the final encryption key.

[0143] The above key generation process not only integrates team work relationships, employee emotional fluctuations, environmental parameters, and other multi-dimensional business features, but also ensures encryption security through chaotic processing and hash operations, making the key deeply coupled with business logic, effectively improving the security of related data encryption storage, while reducing the risk of being cracked.

[0144] In an embodiment, the method further comprises:

[0145] The characters included in the comprehensive emotion index of each employee in the team are sequentially added to each node of the undirected graph;

[0146] Based on the comprehensive emotion index of each employee in the team, the mean value of the team's emotional tendency and the mean value of the emotional intensity are calculated;

[0147] A preset encoding table is obtained; the preset encoding table includes an index character column and an encoding character column;

[0148] Based on the mean value of the emotional tendency, the index character column of the preset encoding table is adjusted by character exchange; based on the mean value of the emotional intensity, the shift step is calculated, and the encoding character column of the preset encoding table is circularly shifted by the corresponding step to obtain a first encoding table;

[0149] According to the number of edges in the team work relationship graph, a specified character in the character list of the first encoding table is replaced to obtain a second encoding table;

[0150] The undirected graph and the second encoding table are superimposed on the same layer, and characters in the undirected graph that overlap with the character list of the second encoding table are spliced and replaced into the corresponding character list to obtain a third encoding table;

[0151] The related data is encoded and stored based on the third encoding table.

[0152] In this embodiment, first, the employee emotion features are associated with the undirected graph structure to provide a basic carrier for subsequent encoding table variation. The characters contained in the comprehensive emotion index of each employee in the team are added to the corresponding nodes of the undirected graph in order according to the node order of the employee in the team work relationship graph. Each node stores a character of the comprehensive emotion index of the corresponding employee.

[0153] Next, the overall team emotion state is quantified to provide adjustment basis for the preliminary variation of the encoding table. Based on the comprehensive emotion index of each employee in the team, the mean value of the emotional tendency and the mean value of the emotional intensity of the team are calculated respectively. The mean value of the emotional tendency reflects the overall emotional bias of the team (such as biasing positive or negative), and the mean value of the emotional intensity reflects the intensity of the overall emotional expression of the team. Both of them are used as core quantitative indicators of team emotion features to drive the dynamic adjustment of the encoding table, so that the encoding table variation is closely related to the team emotion state.

[0154] Further, a preset encoding table is obtained, which adopts a two-dimensional structure and includes an index character list and an encoding character list: the index character list is composed of a series of reference characters for positioning, and the encoding character list includes encoding mapping characters corresponding to the index characters one by one, and both of them together constitute the basic character mapping relationship to provide an original template for subsequent variation adjustment. The preset encoding table is usually designed based on a general encoding rule (such as Base64), to ensure that the initial mapping relationship has basic encoding functions.

[0155] Then, the preset encoding table is preliminarily mutated by introducing team emotion features. Based on the calculated average emotion tendency, the index character column of the preset encoding table is adjusted by character exchange. If the average emotion tendency is positive, the characters at even positions in the index character column are exchanged with the previous characters; if the average emotion tendency is negative, the characters at odd positions are exchanged with the previous characters, so that the arrangement order of the index characters dynamically changes with the team emotion bias. At the same time, based on the average emotion intensity, the shift step is calculated. The calculation method is that the average emotion intensity is rounded according to a preset ratio (such as 1 / 10), and the encoding character column of the preset encoding table is circularly shifted by the step, so that the position distribution of the encoding characters is associated with the team emotion intensity. After the index character exchange and the encoding character shift, the first encoding table is formed, and the first dynamic adjustment of the encoding table is completed.

[0156] Further, the first encoding table is secondarily mutated in combination with team work relationship features. According to the total edge number in the team work relationship graph, the specified characters in the encoding character column of the first encoding table are determined. The positions of the specified characters are integer multiples of the total edge number (for example, if the edge number is n, the characters at positions n, 2n, 3n, etc. are specified characters). The replacement rule is to add the ASCII code value of the specified character to the corresponding edge number to form the second encoding table.

[0157] Then, the third mutation of the encoding table is realized through the spatial superposition interaction of the undirected graph and the encoding table. The constructed undirected graph with fused emotion index characters is superimposed on the same layer after being scaled by the same ratio as the generated second encoding table, so that the node characters of the undirected graph and the encoding character column of the encoding table form an overlapping relationship in space. The overlapping characters in the node character column of the undirected graph and the encoding character column of the second encoding table are identified, and these overlapping characters are spliced and replaced into the corresponding encoding character column, so that the distribution of the encoding table characters is affected by both the topology of the undirected graph and the emotion characters. After the character exchange, the third encoding table is formed. In other embodiments, the encoding character column of the third encoding table can also be subjected to de-duplication processing and interpolation processing.

[0158] Finally, the employee-related data is segmented into character sequences according to a preset format, and the corresponding position of each character in the index column is found based on the index character column of the third encoding table. Then, the character in the encoding character column corresponding to the position is taken as the encoding result. After the mapping of all characters is completed, the complete encoding data is formed by splicing, and then the encoding data is stored in a specified medium. Since the third encoding table integrates team dynamic features and has undergone multiple mutations, the uniqueness and security of the encoding result are ensured, and the deep coupling between the encoding process and the team business features also improves the anti-cracking ability of data storage.

[0159] Referring to Figure 2 In another embodiment of the present application, an employee emotion index evaluation device based on multi-modal data is also provided, comprising:

[0160] The collection unit is configured to collect multi-modal related data of the employee, wherein the related data comprises voice data of the employee, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of the employee.

[0161] The alignment unit is configured to perform feature extraction on the related data and cross-modal alignment to obtain multi-modal technical features.

[0162] The calculation unit is configured to calculate data quality coefficients and scene correlation coefficients of each modality to calculate final dynamic weights of each modality.

[0163] The analysis unit is configured to fuse the multi-modal technical features based on the final dynamic weights of each modality and perform analysis to obtain a comprehensive emotion index of the employee, wherein the comprehensive emotion index comprises an emotion tendency, an emotion intensity, and a stress index.

[0164] In the embodiment, the specific implementation of each unit in the above device embodiment can refer to the description in the above method embodiment, and will not be repeated here.

[0165] Referring to Figure 3 , the embodiment of the present application further provides a computer device, which can be a server, and the internal structure thereof can be as shown in Figure 3 . The computer device comprises a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide calculation and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store corresponding data in the embodiment. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement the above method.

[0166] Those skilled in the art can understand Figure 3 that the structure shown in the figure is only a block diagram of part of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied.

[0167] The embodiment of the present application further provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by the processor to implement the above method. It can be understood that the computer readable storage medium in the embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0168] In summary, the employee emotion index evaluation method and device based on multi-modal data provided in the embodiments of the present application include: collecting multi-modal data of employees; the data includes voice data of employees, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of employees; performing feature extraction on the data and cross-modal alignment to obtain multi-modal technical features; calculating data quality coefficients and scene correlation coefficients of each mode to calculate the final dynamic weight of each mode; fusing the multi-modal technical features based on the final dynamic weight of each mode and performing analysis to obtain the comprehensive emotion index of the employees; the comprehensive emotion index includes emotion tendency, emotion intensity, and stress index. In the present application, the comprehensive emotion index of employees is obtained by comprehensive analysis of multi-modal data, which overcomes the defects of current employee emotion evaluation, such as insufficient comprehensiveness and low accuracy.

[0169] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium provided by the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0170] It is to be understood that the terminology "including", "comprising", or any other variation thereof, is intended to cover a non-exclusive inclusion such that process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0171] The above description is merely the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made according to the content of the present application specification and drawings, or directly or indirectly applied to other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. An employee sentiment index evaluation method based on multi-modal data, characterized in that, The method comprises the following steps: Collecting multi-modal related data of employees; the related data includes voice data of employees, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of employees; Feature extraction is performed on the related data and cross-modal alignment is performed to obtain multi-modal technical features; The data quality coefficients and scene correlation coefficients of each modality are calculated to calculate the final dynamic weights of each modality; Based on the final dynamic weights of each modality, the multi-modal technical features are fused and analyzed to obtain the comprehensive emotion index of the employee; the comprehensive emotion index includes emotion tendency, emotion intensity, and stress index; A team work relationship graph is constructed, the emotion transmission coefficients between employees are calculated based on the frequency of collaborative work and communication density of employees in the team, the comprehensive emotion index of the abnormal employee in the team is monitored in real time, the potential influence value of the abnormal employee on the overall emotion of the team is calculated based on the emotion transmission coefficients between employees, and when the potential influence value exceeds a preset threshold, a differentiated intervention scheme is automatically generated; The team work relationship graph is mapped to a work relationship undirected graph, the fluctuation variance of the comprehensive emotion index of each node corresponding to the employee in the team work relationship graph is mapped to a node of the undirected graph, and a node weight positively correlated with the collaboration frequency and role importance of the employee in the team is given, to obtain a fluctuation undirected graph; the nodes of the fluctuation undirected graph are traversed, the fluctuation variance of each node is spliced with the node weight, and is converted into a code string; the code string is taken as an initial input, combined with an environment parameter sequence composed of environmental parameters of the day, to form an original seed pool; The environment parameter sequence is composed of noise level, illumination intensity, temperature and humidity, and personnel density in the office area in a fixed order; The original seed pool is iteratively processed, the iteration number is set to a preset multiple of the number of current employees in the team, and in each iteration process, the fluctuation variance of the node of the fluctuation undirected graph is taken as a fine adjustment amount of a chaotic parameter to obtain a chaotic confusion seed; Based on the mean value of the comprehensive emotion index of each employee in the team on the day, the chaotic confusion seed is segmented to obtain a segmented confusion seed; after inserting an expansion parameter in each segmented confusion seed, a hash is performed respectively and combined into a hash string; the comprehensive emotion index of each employee in the team is mapped to an integer, and the bit combination corresponding to each integer in the hash string is extracted as a key to encrypt and store the related data; The security of the encrypted storage of the related data is improved, the risk of being cracked is reduced, and the evaluation accuracy of the comprehensive emotion index of the employee is ensured. 2.The method for evaluating employee sentiment index based on multi-modal data according to claim 1, wherein, Feature extraction is performed on the related data, including: Acoustic feature extraction of speech speed, tone, and pause frequency of the real-time collected conference / call voice; Analyzing the emotional tendency of the text data in the enterprise communication tools, and identifying the semantic features in combination with the context; Analyzing the implicit emotional features of facial micro-expressions and body language in the video data collected by the camera; Analyzing the physiological features of the physiological signals collected by the wearable device. 3.The method for evaluating employee sentiment index based on multi-modal data according to claim 1, wherein, The data quality coefficients of each modality are calculated, including: Based on the signal-to-noise ratio and voice activity detection integrity of the voice data, the data quality coefficient of the voice data modality is calculated; The data quality coefficient of the video data modality is calculated based on the blocked area of the camera, and the proportion of the effective area; The data quality coefficient of the text data modality is calculated by using the text confidence calculated by the BERT model and the special symbol processing state; The data quality coefficient of the physiological signal modality is calculated based on the stability of the physiological signal and the data missing rate. 4.The method for evaluating employee sentiment index based on multi-modal data according to claim 1, wherein, The calculation of the scene correlation coefficient includes: Different correlation coefficients are preset for each modality data in different scene types; wherein the scene types at least include remote meeting, independent office, team collaboration discussion; According to the current scene type when collecting related data, the scene correlation coefficient of each modality is determined. 5.The method for evaluating employee sentiment index based on multi-modal data according to claim 1, wherein, The calculation formula of the final dynamic weight of each modality is: ； wherein, , are the data quality coefficients of the i, j-th modality, respectively, are the scene correlation coefficients of the i, j-th modality, respectively, is the final dynamic weight of the i-th modality.

6. The method for evaluating employee sentiment index based on multi-modal data according to claim 1, characterized in that, After obtaining the comprehensive emotion index of the employee, it includes: Based on the comprehensive emotion index, determine the emotion type, and generate coping strategies according to the emotion type combined with the current overtime length of the employee and the work daily report content; wherein the coping strategies include automatically triggering workflow to adjust task priority, recommending rest interval; An association model between the comprehensive emotion index and the performance indicator is established to identify employees with high risk of turnover.

7. The method for evaluating employee sentiment index based on multi-modal data according to claim 1, characterized in that, The method further includes: Real-time removal of identity information from voice data and text data; The behavior data in the related data is processed by using k-anonymization technology.

8. The method for evaluating employee sentiment index based on multi-modal data according to claim 1, characterized in that, After obtaining the comprehensive emotion index of the employee, it also includes: Obtain environmental parameters, including office area noise level, light intensity, temperature and humidity parameters, and team personnel density; Obtain the interference coefficient corresponding to each of the environmental parameters; wherein the association analysis between the environmental parameters and the emotion recognition error is performed in advance to generate the interference coefficient corresponding to each environmental parameter; Calibrate the comprehensive emotion index based on the interference coefficient of each environmental parameter to obtain the final comprehensive emotion index.

9. An employee sentiment index evaluation device based on multi-modal data, characterized by, It includes: A collection unit for collecting multi-modal related data of employees; The related data includes voice data of employees, text data in enterprise communication tools, video data of visual behavior capture, and physiological signals of employees; An alignment unit for feature extraction and cross-modal alignment of the related data to obtain multi-modal technical features; A calculation unit for calculating the data quality coefficient of each modality and the scene correlation coefficient to calculate the final dynamic weight of each modality; An analysis unit for fusing multi-modal technical features based on the final dynamic weight of each modality and analyzing to obtain the comprehensive emotion index of the employee; the comprehensive emotion index includes emotion tendency, emotion intensity, and stress index; A team work relationship graph is constructed, the emotion transmission coefficient between employees is calculated based on the collaborative work frequency and communication density of employees in the team; the comprehensive emotion index of abnormal employees in the team is monitored in real time, the potential influence value of abnormal employees on the overall emotion of the team is calculated based on the emotion transmission coefficient between employees; when the potential influence value exceeds a preset threshold, a differentiated intervention scheme is automatically generated; The team work relationship graph is mapped to a work relationship undirected graph, the fluctuation variance of the comprehensive emotional index of the employee corresponding to each node in the team work relationship graph is mapped to a node of the undirected graph, and a node weight positively correlated with the collaboration frequency and role importance of the employee in the team is given to obtain a fluctuation undirected graph; the nodes of the fluctuation undirected graph are traversed, the fluctuation variance of each node is spliced with the node weight, and is converted into a code string; the code string is taken as an initial input, combined with an environment parameter sequence composed of environmental parameters of the day to form an original seed pool; The environmental parameter sequence is composed of noise levels, illumination intensities, temperature and humidity, and personnel densities in the office area in a fixed order; The original seed pool is iteratively processed, and the number of iterations is set to a preset multiple of the number of current employees in the team; in each iteration process, the fluctuation variance of the node of the fluctuation undirected graph is taken as a fine adjustment amount of a chaos parameter to obtain a chaotic confusion seed; Based on the mean value of the comprehensive emotional index of each employee in the team on the day, the chaotic confusion seed is segmented to obtain a segmented confusion seed; after inserting an expansion parameter in each segmented confusion seed, the segmented confusion seed is hashed respectively and combined into a hash string; the comprehensive emotional index of each employee in the team is mapped to an integer, and the bit combination corresponding to each integer in the hash string is extracted as a key to encrypt and store related data; The security of the encrypted storage of the related data is improved, the risk of being cracked is reduced, and the accuracy of the evaluation of the comprehensive emotional index of the employee is ensured.

Images