Conversion method of five-dimensional radar map for voice emotion analysis and related equipment

Abstract

The invention provides a conversion method of a five-dimensional radar map for voice emotion analysis and related equipment, and relates to the technical field of emotion analysis, and the method comprises the steps: obtaining input voice data, carrying out the preprocessing, carrying out the dual-mode feature extraction of the preprocessed voice data, obtaining a fusion feature vector, inputting a preset VAD emotion recognition model, and obtaining a VAD emotion recognition model; outputting a three-dimensional emotion space vector; and calculating a five-dimensional psychological index based on the three-dimensional emotion space vector and the fusion feature vector, performing normalization processing to generate radar map coordinate data and visual parameters, transmitting the radar map coordinate data and the visual parameters to a terminal, and dynamically rendering and displaying a five-dimensional radar map on a terminal interface of the terminal. Through adaptive normalization processing and radar map visual modeling, five-dimensional psychological indexes can be presented in a unified scale, and structural plotting is realized through radar map coordinate data and visual parameters, so that the problems of rough emotion description, insufficient data dimensions and limited visual ability in the existing scheme are solved.

Description

Technical Field

[0001] This application relates to the technical field of emotion analysis, and in particular to a method and related equipment for converting a five-dimensional radar chart for speech emotion analysis. Background Technology

[0002] With the rapid development of children's smart toys, emotional companionship devices, and family education aids, toys with voice interaction capabilities are gradually becoming common products in parent-child scenarios. These devices typically provide children with companionship, emotional comfort, and behavioral guidance by picking up voice information from their daily conversations. Some products also attempt to integrate cloud services to provide parents with feedback on their children's behavioral characteristics or psychological state.

[0003] In existing solutions, audio is mainly transcribed using a speech recognition model. The semantic emotion category is then identified based on the transcribed text data, or simple emotion labels such as "happy," "angry," and "sad" are inferred through traditional acoustic features (such as pitch, loudness, and speech rate). These labels are then presented on the terminal as icons, color changes, or fixed facial animations. This approach can reflect the child's immediate emotional tendencies to a certain extent and help guardians roughly judge the child's emotional state in a short period of time.

[0004] While single-dimensional emotion recognition based on acoustics or semantics can roughly determine a child's emotions, traditional emotion recognition methods struggle to support multi-dimensional indicator modeling and generate structured and interpretable graphical representations when multi-dimensional analysis, continuous quantification, and periodic presentation of emotions are required. This results in significant limitations in deep interaction scenarios for children's companion toys and mental health support scenarios, with issues such as coarse emotion portrayal, insufficient data dimensions, and limited visualization capabilities. Summary of the Invention

[0005] To overcome the problems of coarse emotion characterization, insufficient data dimensions, and limited visualization capabilities in existing solutions, this application provides a method and related equipment for converting a five-dimensional radar chart for speech emotion analysis.

[0006] On one hand, this invention provides a method for converting a five-dimensional radar chart for voice emotion analysis, comprising: acquiring input voice data and preprocessing the voice data to obtain preprocessed voice data; extracting bimodal features from the preprocessed voice data to obtain a fused feature vector; inputting the fused feature vector into a preset VAD emotion recognition model to output a three-dimensional emotion space vector; calculating a five-dimensional psychological index based on the three-dimensional emotion space vector and the fused feature vector; performing adaptive normalization processing on the five-dimensional psychological index to obtain a normalized five-dimensional psychological index; generating radar chart coordinate data and visualization parameters based on the normalized five-dimensional psychological index; transmitting the radar chart coordinate data and visualization parameters to a terminal, and dynamically rendering and displaying the five-dimensional radar chart on the terminal interface.

[0007] Optionally, the step of acquiring the input speech data and preprocessing the speech data to obtain preprocessed speech data includes: acquiring the input speech data through an audio acquisition module; converting the speech data into a format that converts non-WAV format speech data into WAV format mono audio data to obtain converted speech data; performing noise reduction processing on the converted speech data using spectral subtraction or Wiener filtering algorithms to obtain noise-reduced speech data; performing endpoint detection on the noise-reduced speech data by calculating short-time energy and short-time zero-crossing rate to obtain effective speech segments; performing speech frame processing on the effective speech segments and applying a Hamming window function to window each frame of speech signal to obtain multiple speech frames; and enhancing the high-frequency components of each speech frame using a first-order high-pass filter to obtain preprocessed speech data.

[0008] Optionally, the step of extracting dual-modal features from the preprocessed speech data to obtain a fused feature vector, and inputting the fused feature vector into a preset VAD emotion recognition model to output a three-dimensional emotion space vector includes: performing a fast Fourier transform on each speech frame in the preprocessed speech data to obtain spectral information; calculating the Mel-frequency cepstral coefficients (MFCCs) as timbre feature parameters based on the spectral information; calculating the fundamental frequency F0 of each speech frame in the preprocessed speech data to statistically analyze pitch feature parameters and pitch fluctuation rate parameters; calculating the short-time energy of each speech frame in the preprocessed speech data to statistically analyze energy feature parameters; calculating the duration ratio between adjacent speech frames and the total number of effective speech segments in the preprocessed speech data to obtain speech rate feature parameters; and analyzing the variance of the fundamental frequency F0 and the short-time energy... The fluctuation amplitude is used to obtain emotion stability feature parameters. Laughter feature frequency bands in the preprocessed speech data are detected to obtain laughter detection probability. The timbre feature parameters, pitch feature parameters, pitch fluctuation rate parameters, energy feature parameters, speech rate feature parameters, emotion stability feature parameters, and laughter detection probability are combined to form an acoustic feature vector. Semantic features are extracted from the preprocessed speech data to obtain a semantic vector, a negative word proportion parameter, and a keyword weighting coefficient to construct a semantic feature vector. The acoustic feature vector and the semantic feature vector are concatenated dimensionally using a feature concatenation method to obtain a fused feature vector. The fused feature vector is input into a preset VAD emotion recognition model to output a three-dimensional emotion space vector, wherein the three-dimensional emotion space vector includes pleasure, activation, and dominance.

[0009] Optionally, the step of extracting semantic features from the preprocessed speech data to obtain a semantic vector, a negative word proportion parameter, and keyword weighting coefficients to construct a semantic feature vector includes: converting the preprocessed speech data into text data using an automatic speech recognition model; inputting the text data into a pre-trained Chinese language model for text embedding encoding to obtain a semantic vector; using an emotion dictionary matching algorithm to count the number of negative words appearing in the text data and calculating the ratio of the number of negative words to the total number of words to obtain a negative word proportion parameter; using a keyword recognition algorithm to identify preset psychological cue keywords in the text data to obtain keyword weighting coefficients, wherein the keyword weighting coefficients include stress keyword weighting coefficients, conflict keyword weighting coefficients, and positive keyword weighting coefficients; and combining the semantic vector, the negative word proportion parameter, and the keyword weighting coefficients to form a semantic feature vector.

[0010] Optionally, the step of calculating the five-dimensional psychological index based on the three-dimensional emotion space vector and the fused feature vector includes: extracting values ​​of pleasure, activation, and dominance from the three-dimensional emotion space vector; extracting energy standard deviation, pitch standard deviation, fundamental frequency variance, negative word proportion parameter, laughter detection probability, stress keyword weighting coefficient, conflict keyword weighting coefficient, and positive keyword weighting coefficient from the fused feature vector; calculating a learning stress index using the energy standard deviation, pitch standard deviation, negative word proportion parameter, and stress keyword weighting coefficient; and using the pleasure... The parent-child communication index is calculated using the negative word ratio parameter and the conflict keyword weighting coefficient; the social relationship index is calculated using the laughter detection probability, the pleasure level, and the positive keyword weighting coefficient; the emotional stability index is calculated using the activation level, the fundamental frequency variance, and the preset fundamental frequency fluctuation penalty coefficient; the negative emotion index is calculated using the pleasure level, the negative word ratio parameter, and the activation level; and the learning pressure index, the parent-child communication index, the social relationship index, the emotional stability index, and the negative emotion index are combined to generate a five-dimensional psychological index.

[0011] Optionally, the step of adaptively normalizing the five-dimensional psychological index to obtain a normalized five-dimensional psychological index, and generating radar chart coordinate data and visualization parameters based on the normalized five-dimensional psychological index, includes: checking the numerical range of each dimension index in the five-dimensional psychological index, limiting the value range to between 0 and 1, multiplying each dimension index by 100 to obtain a preliminary normalized index value, performing nonlinear adjustment on the preliminary normalized index value, and truncating outliers on the adjusted index value to obtain the normalized five-dimensional psychological index; calculating the coordinates of each dimension on a preset radar chart based on the values ​​of each dimension in the normalized five-dimensional psychological index. The radial coordinates are used to calculate the Cartesian coordinates of each dimension's vertices using the polar angles of each dimension in a preset polar coordinate system and the radial coordinates. The Cartesian coordinates of adjacent dimension vertices are then connected sequentially to form a closed pentagonal region. The area of ​​the pentagonal region is calculated and used as a comprehensive psychological health index. The dispersion of the five-dimensional psychological index is then calculated. A comprehensive psychological state rating is performed based on the comprehensive psychological health index and the dispersion. A color mapping scheme is assigned to each dimension region of the five-dimensional radar chart. The Cartesian coordinates of each dimension's vertices, the area S of the pentagonal region, the dispersion, the comprehensive psychological state rating, and the color mapping scheme are combined to generate radar chart coordinate data and visualization parameters.

[0012] Optionally, the step of transmitting the radar chart coordinate data and visualization parameters to the terminal and dynamically rendering and displaying the five-dimensional radar chart on the terminal interface includes: transmitting the radar chart coordinate data and visualization parameters to the terminal; creating a canvas object in the graphics rendering engine of the terminal; drawing a pentagonal outline connecting each vertex on the canvas object according to the Cartesian coordinates of each dimension vertex in the radar chart coordinate data; filling the area inside the pentagonal outline according to the color mapping scheme in the visualization parameters; and filling the corresponding sector of each dimension with the corresponding color.

[0013] On the other hand, this application also provides a five-dimensional radar chart conversion system for voice emotion analysis, comprising: a data acquisition module for acquiring input voice data and preprocessing the voice data to obtain preprocessed voice data; a feature extraction module for performing bimodal feature extraction on the preprocessed voice data to obtain a fused feature vector, inputting the fused feature vector into a preset VAD emotion recognition model, and outputting a three-dimensional emotion space vector; a mapping calculation module for calculating a five-dimensional psychological index based on the three-dimensional emotion space vector and the fused feature vector; a data generation module for performing adaptive normalization processing on the five-dimensional psychological index to obtain a normalized five-dimensional psychological index, and generating radar chart coordinate data and visualization parameters based on the normalized five-dimensional psychological index; and a data transmission module for transmitting the radar chart coordinate data and visualization parameters to a terminal, and dynamically rendering and displaying the five-dimensional radar chart on the terminal interface of the terminal.

[0014] On the other hand, this application also provides an electronic device, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the five-dimensional radar map conversion method for voice emotion analysis as described above.

[0015] On the other hand, this application also provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the five-dimensional radar map conversion method for speech emotion analysis as described above.

[0016] This application provides a method and related equipment for converting a five-dimensional radar chart for speech emotion analysis, which has the following technical effects: By performing a complete preprocessing process on the input speech data, the preprocessed speech data maintains stable audio quality and structural consistency; through bimodal feature extraction, acoustic and semantic information can simultaneously participate in emotion recognition and psychological index calculation, enabling the five-dimensional psychological index to reflect the acoustic trend of speech changes; by inputting the fused feature vector into a preset VAD emotion recognition model, the three core emotional factors of pleasure, activation, and dominance can be expressed in a unified emotion space representation; subsequently, using the three-dimensional emotion space vector and various parameters in the fused feature vector, a five-dimensional psychological index system is established, enabling the five psychological dimensions of learning pressure, parent-child communication, social relationships, emotional stability, and negative emotions to be calculated independently but in a related manner, achieving multi-angle quantitative analysis of psychological states; finally, through adaptive normalization processing and radar chart visualization modeling, the five-dimensional psychological index can be presented at a unified scale, and structured plotting is achieved through radar chart coordinate data and visualization parameters, overcoming the problems of coarse emotion characterization, insufficient data dimensions, and limited visualization capabilities in existing solutions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for converting a five-dimensional radar map for voice emotion analysis, provided in an embodiment of the present invention. Figure 2 This is a schematic block diagram of the structure of a five-dimensional radar map conversion system for voice emotion analysis provided in an embodiment of the present invention; Figure 3 This is a schematic block diagram of the structure of the electronic device provided in the embodiment of the present invention.

[0019] Explanation of reference numerals in the attached figures: 10. Five-dimensional radar map conversion system for speech emotion analysis; 11. Data acquisition module; 12. Feature extraction module; 13. Mapping calculation module; 14. Data generation module; 15. Data transmission module; 20. Electronic equipment; 21. Bus; 22. Processor; 23. Transceiver; 24. Memory. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0023] Example 1: like Figure 1 The present application provides a method for converting a five-dimensional radar chart for speech emotion analysis. This method can be used to extract acoustic and semantic features from input speech data, generate a fused feature vector, obtain a three-dimensional emotion space vector, further calculate a five-dimensional psychological index, and generate radar chart coordinate data and visualization parameters, enabling the terminal to display the five-dimensional radar chart intuitively. This conversion method mainly includes the following steps S1 to S5: Step S1: Obtain the input voice data and preprocess the voice data to obtain preprocessed voice data.

[0024] The audio acquisition module collects speech data in real time, and the acquired speech data is input into the preprocessing unit. The preprocessing unit performs necessary steps on the speech data, including format normalization, noise suppression, silence removal, and basic signal enhancement, to obtain preprocessed speech data that represents valid speech content. This preprocessing process ensures that the subsequent feature extraction module obtains a structurally complete, temporally continuous, and low-noise speech signal, improving the reliability of subsequent bimodal feature extraction and emotion recognition.

[0025] The audio acquisition module can be a microphone array or a single digital microphone, with a sampling rate set to 16kHz and a sampling bit depth set to 16bit. It converts the acquired speech data into linear PCM encoding in WAV format, performs energy threshold silence detection on the speech data, removes silent segments with continuous energy below the preset threshold, and removes low-frequency mechanical noise through simple high-pass filtering, thereby obtaining preprocessed speech data.

[0026] Step S2: Extract dual-modal features from the preprocessed speech data to obtain a fused feature vector. Input the fused feature vector into the preset VAD emotion recognition model to output a three-dimensional emotion space vector.

[0027] The preprocessed speech data is processed by both acoustic feature extraction and semantic feature extraction modules. The acoustic feature extraction module analyzes the changes in timbre, pitch, energy, and speech rate in the speech data from both time and frequency domains. The semantic feature extraction module converts the preprocessed speech data into text data using an automatic speech recognition model and then constructs a text embedding vector based on a language model. Subsequently, a feature fusion unit combines the acoustic and semantic feature vectors to form a fused feature vector. This fused feature vector is then input into a pre-defined VAD emotion recognition model, and a nonlinear mapping is performed on the fused feature vector based on this model to obtain a three-dimensional emotion space vector.

[0028] Acoustic features can be obtained by framing speech data and performing a Fast Fourier Transform to obtain the spectrum, and then calculating the basic timbre coefficients and pitch parameters. Semantic features can be obtained by using a deep speech recognition model to obtain text, and then inputting it into a Chinese BERT model to obtain a semantic vector. After fusing the feature vector input into a VAD emotion recognition model containing a multi-layer neural network, a three-dimensional emotion space vector including three dimensions: pleasure, activation, and dominance can be output.

[0029] Step S3: Calculate the five-dimensional psychological index based on the three-dimensional emotion space vector and the fused feature vector.

[0030] The system extracts values ​​of pleasure, activation, and dominance from a three-dimensional emotional space vector, and extracts parameters that reflect emotional fluctuations, semantic tendencies, and acoustic stability from a fused feature vector. Through a pre-defined weight combination logic, it forms a learning pressure index, a parent-child communication index, a social relationship index, an emotional stability index, and a negative emotion index, so that each dimension is jointly determined by acoustic and semantic information, thus forming a five-dimensional psychological index system.

[0031] Pleasure level can be used as the main factor affecting the parent-child communication index and social relationship index, activation level can be used as the main factor affecting the emotional stability index, and speech rate, short-term energy change and negative vocabulary ratio in the integrated feature vector can be used as key reference parameters for the learning pressure index and negative emotion index. Finally, they are combined into a five-dimensional psychological index.

[0032] Step S4: Perform adaptive normalization on the five-dimensional psychological index to obtain the normalized five-dimensional psychological index, and generate radar chart coordinate data and visualization parameters based on the normalized five-dimensional psychological index.

[0033] Range constraints and linear mappings are applied to all values ​​in the five-dimensional psychological index to keep all index values ​​within the standard range of 0 to 1. Non-linear adjustments are then made based on the sensitivity characteristics of the psychological index to enhance the visualization of differences between different dimensions. Subsequently, a coordinate system for the five-dimensional radar chart is established based on the normalized five-dimensional psychological index, generating the polar angle, radial coordinates, vertex coordinates, and color parameters for visual rendering corresponding to each psychological dimension, thus obtaining radar chart coordinate data and visualization parameters.

[0034] The five-dimensional psychological index can be linearly mapped to a display range of 0 to 100, and the numerical differences can be enhanced by a fixed curve transformation. The five psychological dimensions are evenly distributed on a 360-degree circle, and the radial length of each dimension is calculated and converted into Cartesian coordinates. A color mapping scheme is further assigned so that different psychological dimensions have different color representations when visualized, so as to form an intuitive presentation through radar chart coordinate data and visualization parameters.

[0035] Step S5: Transmit the radar map coordinate data and visualization parameters to the terminal, and dynamically render and display the five-dimensional radar map on the terminal interface.

[0036] The radar chart coordinate data and visualization parameters are sent to the terminal device in a structured data format. The terminal parses the structured data through its local graphics rendering engine and draws the wireframe and filled area of ​​the five-dimensional radar chart according to the Cartesian coordinates. Visual rendering is performed through color mapping parameters, so that the user interface can dynamically display the overall distribution of the five-dimensional psychological index.

[0037] On a smartphone, a JavaScript graphics rendering framework can be used to parse the data points of a five-dimensional radar chart, draw a pentagonal outline based on the coordinates of each vertex, and then apply color fills to the internal areas of each dimension so that the terminal can display the changes in the five-dimensional psychological index in real time when the interface is refreshed.

[0038] In this embodiment, input speech data is acquired and preprocessed to obtain preprocessed speech data. Based on this, bimodal feature extraction is performed on the preprocessed speech data to obtain acoustic and semantic features, which are then fused to form a fused feature vector. This fused feature vector is then input into a pre-defined VAD emotion recognition model to obtain a three-dimensional emotion space vector reflecting emotional states. Using the three-dimensional emotion space vector and multiple feature parameters from the fused feature vector, a five-dimensional psychological index describing psychological states is calculated. The five-dimensional psychological index is then adaptively normalized to obtain a normalized five-dimensional psychological index. Radar chart coordinate data and visualization parameters are generated based on the normalized five-dimensional psychological index. Finally, the radar chart coordinate data and visualization parameters are transmitted to the terminal, enabling the terminal interface to dynamically render and display the five-dimensional radar chart, thus realizing a complete processing flow from speech data to the visualization of the five-dimensional psychological index.

[0039] This example demonstrates how to extract multimodal information from speech data, including acoustic and semantic features, without altering its natural expression. This allows for the acquisition of a three-dimensional emotion space vector to characterize immediate emotional states, and the construction of a five-dimensional psychological index based on fused feature vectors. This index reflects more comprehensive, detailed, and long-term psychological characteristics. Through adaptive normalization, the five-dimensional psychological index is unified within a consistent visualization range, enabling the intuitive presentation of different psychological features within the same coordinate system. Furthermore, dynamic rendering based on radar chart coordinate data and visualization parameters allows the terminal to display the overall state and dimensional differences of the five-dimensional psychological index in a structured, quantitative, and easily understandable manner. This significantly improves the interpretability and usability of emotion analysis, helping guardians or related systems more accurately identify emotional trends and psychological feature distributions. It enhances the value and applicability of speech emotion analysis, overcoming the problems of coarse emotion characterization, insufficient data dimensions, and limited visualization capabilities in existing solutions.

[0040] In another example, step S10 may preferably be: The audio acquisition module acquires the input voice data, performs format conversion on the voice data, and converts non-WAV format voice data into WAV format mono audio data to obtain the converted voice data.

[0041] The audio acquisition module acquires input speech data, which is then imported into the format conversion unit. The encoding format of the input speech data is detected; if it is not in WAV format, the format conversion engine is invoked to convert the input speech data into WAV format mono audio data. During the conversion process, the sampling rate is resampled to 16kHz while maintaining a sampling bit depth of 16bit. Simultaneously, multi-channel audio data is converted into mono format using a channel synthesis algorithm. This ensures that subsequent processing modules only need to process the converted speech data in a unified format, thereby improving subsequent processing efficiency and the stability of the processing chain.

[0042] If the input voice data is in MP3, AAC, or AMR format, the FFmpeg format conversion module is called to perform decoding and re-encoding operations, converting it into WAV format linear PCM encoding, and using a weighted average channel mixing method to synthesize the stereo into a mono, thus obtaining the converted voice data.

[0043] The converted speech data is denoised using spectral subtraction or Wiener filtering algorithms to remove background noise and environmental interference signals, resulting in denoised speech data.

[0044] Frequency domain noise reduction is performed on the converted speech data. First, the converted speech data is segmented and windowed. Then, a Fast Fourier Transform is performed on each frame to obtain a noise spectrum estimate. Subsequently, either the spectral subtraction algorithm or the Wiener filtering algorithm is called to estimate and suppress noise for each frequency component. The spectral subtraction algorithm is a commonly used frequency domain noise reduction method in existing technology. It achieves noise suppression by estimating the noise power spectrum and subtracting the noise spectrum point by point from the speech power spectrum. The Wiener filtering algorithm is a linear minimum mean square error filtering method. It calculates the filtering gain by constructing the ratio of the speech power spectrum to the noise power spectrum, thus preserving frequency components with a high signal-to-noise ratio and suppressing frequency components with a low signal-to-noise ratio, thereby obtaining the denoised speech data.

[0045] When using spectral subtraction, the noise estimation update parameter can be set to 0.98, and spectral subtraction can be performed within the frequency range of 50Hz to 8kHz. When using the Wiener filtering algorithm, the initial noise power spectrum can be set to the average power spectrum of the first second speech segment, and the noise estimation can be updated in a fixed 5-millisecond step to finally obtain the denoised speech data.

[0046] Endpoint detection is performed on the denoised speech data by calculating short-time energy and short-time zero-crossing rate to identify the start and end points of the denoised speech data, and silent segments before and after the speech data are removed to obtain effective speech segments.

[0047] The short-time energy and short-time zero-crossing rate are calculated frame by frame for the denoised speech data and compared with preset energy and zero-crossing rate thresholds. If the short-time energy and short-time zero-crossing rate of consecutive frames are both higher than the corresponding thresholds, it is determined as the start point of the speech; if the short-time energy and short-time zero-crossing rate of consecutive frames are both lower than the thresholds, it is determined as the end point of the speech. The effective speech segments are extracted from the denoised speech data using an endpoint detection algorithm, and preceding and following silence segments are removed to obtain the effective speech segments.

[0048] If the short-time energy threshold is set to 20% of the global short-time energy average and the short-time zero-crossing rate threshold is set to 30% of the global zero-crossing rate average, when the short-time energy of consecutive 20ms to 40ms speech frames is higher than the energy threshold and the short-time zero-crossing rate is higher than the zero-crossing rate threshold, the position is marked as the speech start point; when several consecutive frames are lower than the above thresholds, the position is marked as the speech end point, thus obtaining a valid speech segment.

[0049] The effective speech segments are processed by speech framing, with the frame length set to 20 to 30 milliseconds and the frame shift set to 10 to 15 milliseconds. The Hamming window function is used to window each frame of speech signal to obtain multiple speech frames.

[0050] The effective speech segments are divided into equal-length segments according to a set frame length. By setting the frame length to 20 to 30 milliseconds and the frame shift to 10 to 15 milliseconds, the speech signal is ensured to have reasonable temporal resolution. Subsequently, each frame of speech signal is multiplied by a Hamming window function to smooth the frame interface, making the edge transitions of the speech frames more natural, thereby reducing spectral leakage and enhancing the accuracy of subsequent frequency domain analysis, ultimately resulting in multiple speech frames.

[0051] The frame length can be set to 25 milliseconds, the frame shift to 10 milliseconds, and each frame of speech signal can be multiplied by a Hamming window function with a length of 400 points to make the segmented speech frames have smooth characteristics, thus obtaining multiple speech frames that can be used for subsequent spectrum analysis.

[0052] The high-frequency components of each speech frame are enhanced by a first-order high-pass filter, with the pre-emphasis coefficient set to 0.95 to 0.97, resulting in preprocessed speech data.

[0053] A first-order high-pass filter operation is performed on each speech frame to enhance the high-frequency components of the speech signal through pre-emphasis, making the high-frequency details in the spectrum more apparent. The pre-emphasis coefficient is set in the range of 0.95 to 0.97, and the speech frame is processed through the differential structure of the high-pass filter to generate preprocessed speech data, providing a clearer high-frequency profile for subsequent feature extraction.

[0054] By using a pre-emphasis coefficient of 0.97, each frame of speech signal is processed using the formula y(t)=x(t)-0.97·x(t-1) to make the speech signal with enhanced high-frequency components more suitable for understanding timbre changes, thus obtaining preprocessed speech data.

[0055] In another example, step S20 may preferably be: Fast Fourier Transform is performed on each speech frame in the preprocessed speech data to obtain spectral information. Mel-frequency cepstral coefficients (MFCCs) are calculated based on the spectral information, and 13- to 39-dimensional MFCC coefficients are extracted as timbre feature parameters. The fundamental frequency F0 of each speech frame in the preprocessed speech data is calculated using the autocorrelation function method or the cepstral method. The mean, variance, and rate of change of the fundamental frequency F0 in the entire speech segment are statistically analyzed to obtain pitch feature parameters and pitch fluctuation rate parameters.

[0056] A Fast Fourier Transform (FFT) is performed on each speech frame in the preprocessed speech data to obtain spectral information. Then, a Mel filter bank is used to weight the spectral information, mapping the spectrum to the Mel scale, taking the logarithm, and then performing a Discrete Cosine Transform (DCT) to obtain the Mel frequency cepstral coefficients (MFCCs). Depending on the application requirements, 13-dimensional to 39-dimensional MFCC coefficients are extracted as timbre feature parameters. Simultaneously, the fundamental frequency (F0) of each frame is calculated using either the autocorrelation function method or the cepstral method, and the mean, variance, and rate of change of the fundamental frequency (F0) are statistically analyzed to form pitch feature parameters and pitch fluctuation rate parameters.

[0057] For each frame, a 512-point FFT is performed and 40 Mel filters are applied. A 13-dimensional MFCC is extracted by discrete cosine transform, and the first-order and second-order difference coefficients of the MFCC are additionally calculated to form a 39-dimensional MFCC. The fundamental frequency F0 is calculated using the autocorrelation function method, and its mean, variance, and rate of change in the entire speech segment are statistically analyzed to form a stable pitch feature description.

[0058] The short-time energy of each speech frame in the preprocessed speech data is calculated, and the mean, standard deviation and dynamic range of the short-time energy are statistically analyzed to obtain energy characteristic parameters. The duration ratio between adjacent speech frames and the total number of effective speech segments in the preprocessed speech data are calculated to obtain speech rate characteristic parameters.

[0059] The short-time energy of each frame in the preprocessed speech data is calculated, and the mean, standard deviation, and dynamic range of the short-time energy are statistically analyzed to obtain energy feature parameters. Simultaneously, the inter-frame duration ratio of adjacent speech frames and the total number of effective speech segments are statistically analyzed to calculate speech rate feature parameters, ensuring that speech rate reflects the rhythmic changes in the expressed content.

[0060] Among them, short-time energy can be obtained by summing the squares of a frame of speech signal, and the dynamic range of energy can be calculated by the difference between the maximum and minimum short-time energy; speech rate feature parameters can be calculated by counting the total number of effective speech segments and combining the frame shift of each frame. For example, if the effective speech segment contains 300 frames and the frame shift is 10 milliseconds, the time feature corresponding to the speech rate can be calculated as 300 × 10 milliseconds.

[0061] By analyzing the variance of the fundamental frequency F0 and the fluctuation amplitude of short-term energy, the characteristic parameters of emotional stability are obtained. The frequency band of laughter in the preprocessed speech data is detected by the spectrum pattern recognition algorithm, and the probability value of laughter is calculated to obtain the probability of laughter detection.

[0062] Emotional stability characteristic parameters are calculated based on the fundamental frequency F0 variance and short-term energy fluctuation amplitude. The degree of emotional stability is reflected by analyzing the smoothness of fundamental frequency changes and the continuity of energy changes. At the same time, a spectrum pattern recognition algorithm is used to detect the laughter frequency band in the preprocessed speech data. For example, the energy concentration range of laughter patterns in the range of 1kHz to 4kHz is detected, and the probability of laughter detection is obtained by statistically analyzing the frame proportion of energy in this frequency band.

[0063] Emotional stability characteristic parameters can be calculated by combining the standard deviation of the fundamental frequency F0 and the standard deviation of the short-time energy. A spectrum pattern recognition algorithm based on feature templates is used to detect typical laughter frequency bands. When a frequency band energy enhancement that matches the laughter template occurs, the frame is counted as a laughter frame. Finally, the proportion of laughter frames to the total number of frames is used as the laughter detection probability.

[0064] The acoustic feature vector is formed by combining timbre feature parameters, pitch feature parameters, pitch fluctuation rate parameters, energy feature parameters, speech rate feature parameters, emotional stability feature parameters, and laughter detection probability.

[0065] The timbre feature parameters, pitch feature parameters, pitch fluctuation rate parameters, energy feature parameters, speech rate feature parameters, emotional stability feature parameters, and laughter detection probability are combined in a fixed order to form a structured acoustic feature vector. This ensures that each type of acoustic feature has a fixed position in the acoustic feature vector, guaranteeing that the subsequent feature fusion model can correctly parse the acoustic feature vector.

[0066] The 39-dimensional MFCC coefficients can be placed in the first 39 dimensions of the acoustic feature vector, the mean, variance, and rate of change of the fundamental frequency F0 can be placed in the next 3 dimensions, the mean, standard deviation, and dynamic range of the short-time energy can be placed in the 3 dimensions, the speech rate feature parameters can be placed in the 2 dimensions, and the emotion stability feature parameters and the laughter detection probability can be placed in the 1 dimension respectively, thus forming a complete acoustic feature vector.

[0067] Semantic features are extracted from the preprocessed speech data to obtain semantic vectors, negative word proportion parameters, and keyword weighting coefficients, in order to construct semantic feature vectors.

[0068] The semantic feature extraction module performs text conversion, semantic vector construction, negative word proportion parameter calculation, and keyword weighting coefficient calculation on the preprocessed speech data to obtain semantic vector, negative word proportion parameter, and keyword weighting coefficient, which are finally combined to form semantic feature vector.

[0069] The preprocessed speech data is converted into text data by an automatic speech recognition model, and then input into a trained Chinese BERT model to obtain a 768-dimensional semantic vector. The negative word ratio parameter is calculated by matching negative words using a sentiment dictionary. The frequency of occurrence of stress keywords, conflict keywords and positive keywords is statistically analyzed by a keyword recognition algorithm, and their respective weighting coefficients are calculated to obtain the keyword weighting coefficients.

[0070] Acoustic and semantic feature vectors are concatenated dimensionally using a feature concatenation method, or a weighted fusion of acoustic and semantic feature vectors is performed using an attention mechanism to obtain a fused feature vector. This fused feature vector is then input into a pre-defined VAD emotion recognition model, which includes an input layer, multiple hidden layers, and an output layer. The hidden layers employ a bidirectional Long Short-Term Memory (LSTM) network structure. The VAD emotion recognition model performs nonlinear mapping and emotion space modeling on the fused feature vector, outputting a three-dimensional emotion space vector. This three-dimensional emotion space vector includes values ​​for three dimensions: pleasure, activation, and dominance, with each dimension ranging from 0 to 1.

[0071] The acoustic and semantic feature vectors are directly concatenated along their feature dimensions using a feature concatenation method, or a weighted fusion of the two vectors is performed using an attention mechanism to obtain a fused feature vector. This fused feature vector is then input into a pre-defined VAD emotion recognition model, which consists of an input layer, multiple hidden layers, and an output layer. The hidden layers employ a bidirectional Long Short-Term Memory (LSTM) network structure to capture the temporal dependencies of speech features, enabling a non-linear mapping of the fused feature vector and constructing an emotion space model. This results in a three-dimensional emotion space vector representing pleasure, activation, and dominance.

[0072] When using the feature concatenation method, all dimensions of the acoustic feature vector are placed before the semantic feature vector to form a unified vector. When using the attention mechanism, dot product attention is used to calculate the attention weights between the acoustic and semantic feature vectors and generate a fused feature representation. After inputting the fused feature vector into a VAD emotion recognition model with a 3-layer bidirectional LSTM, the output is a three-dimensional emotion space vector with all three dimensions in the range of 0 to 1.

[0073] The step of extracting semantic features from the preprocessed speech data to obtain semantic vectors, negative word proportion parameters, and keyword weighting coefficients to construct semantic feature vectors can be preferably further optimized as follows: The preprocessed speech data is converted into text data using an automatic speech recognition model, which is an end-to-end speech recognition model based on a deep neural network. The text data is then input into a pre-trained Chinese language model for text embedding encoding. The pre-trained Chinese language model is either a BERT model or an ERNIE model, resulting in a 768-dimensional semantic vector.

[0074] The preprocessed speech data is fed frame by frame into an end-to-end speech conversion architecture via an automatic speech recognition model, performing acoustic modeling, decoding modeling, and output modeling to ultimately generate text data. The text data is then input into a pre-trained Chinese language model for embedding encoding, resulting in a 768-dimensional semantic vector.

[0075] Automatic speech recognition models can adopt end-to-end speech recognition models based on the Transformer structure, which perform speech-to-text conversion through feature encoder and decoder modules; the text data is then input into the Chinese BERT model, which generates 768-dimensional semantic vectors through its encoding layer.

[0076] The number of negative words appearing in the text data is counted by the sentiment dictionary matching algorithm, and the ratio of the number of negative words to the total number of words is calculated to obtain the negative word ratio parameter.

[0077] The sentiment dictionary matching algorithm scans each word in the text data, strictly matches the word with the entries in the negative dictionary, counts the number of negative words, and calculates the ratio of the number of negative words to the total number of words to form the negative word ratio parameter.

[0078] If the text data contains 100 words, and the words in the negative dictionary appear 8 times, then the negative word ratio parameter is calculated as 8 / 100, or 0.08.

[0079] The keyword recognition algorithm identifies preset psychological cue keywords in text data. These psychological cue keywords include stress-related words, loneliness-related words, conflict-related words, and positive-related words. The frequency of each type of keyword is counted, and a corresponding weighting coefficient is calculated based on the frequency of each type of keyword to obtain the keyword weighting coefficient. The keyword weighting coefficient includes the stress keyword weighting coefficient, the conflict keyword weighting coefficient, and the positive keyword weighting coefficient.

[0080] Based on a pre-defined list of psychological cue keywords, keyword matching is performed on the text data. By traversing each word and comparing it with lists of stress-related words, conflict-related words, and positive-related words, the frequency of occurrence of each type of keyword is counted. Based on pre-defined weighting rules, the weighting coefficients for stress keywords, conflict keywords, and positive keywords are calculated to form the keyword weighting coefficients.

[0081] If stress-related words are assigned a weight of 1.2, conflict-related words a weight of 1.5, and positive-related words a weight of 0.8, and stress-related words appear 3 times, conflict-related words appear 2 times, and positive-related words appear 5 times in the text data, then the corresponding weighting coefficients are 3×1.2, 2×1.5, and 5×0.8, respectively.

[0082] The semantic vector is formed by combining the semantic vector, the negative word ratio parameter, and the keyword weighting coefficient.

[0083] The semantic vector, negative word ratio parameter, and keyword weighting coefficient are combined in a predefined dimensional order to form a semantic feature vector in a structured manner. This ensures that the semantic vector, negative word ratio parameter, and keyword weighting coefficient all have fixed dimensional positions, providing a consistent input format for subsequent fusion.

[0084] The 768-dimensional semantic vector can be placed in the first 768 dimensions of the semantic feature vector, the negative word ratio parameter can be placed in the 769th dimension, and the weighting coefficients of pressure keywords, conflict keywords, and positive keywords can be placed in the 770th to 772nd dimensions, respectively, thus forming a complete semantic feature vector.

[0085] In another example, step S30 may preferably be: The values ​​of pleasure, activation and dominance are extracted from the three-dimensional emotion space vector. The standard deviation of energy, standard deviation of pitch, fundamental frequency variance, negative word proportion parameter, laughter detection probability, weighting coefficient of stress keyword, weighting coefficient of conflict keyword and weighting coefficient of positive keyword are extracted from the fused feature vector.

[0086] The values ​​of pleasure, activation, and dominance are read one by one from the three-dimensional emotion space vector, ensuring that the values ​​of each dimension are within the range of 0 to 1. Simultaneously, the standard deviation of energy, standard deviation of pitch, fundamental frequency variance, negative word proportion parameter, laughter detection probability, weighting coefficient of stress keywords, weighting coefficient of conflict keywords, and weighting coefficient of positive keywords are read from the fused feature vector. All feature parameters are extracted in a structured index form, ensuring that they participate in the calculation of the five-dimensional psychological index according to a fixed logic.

[0087] A fixed-dimensional interval of the fused feature vector can be set as the storage area for energy standard deviation, pitch standard deviation and fundamental frequency variance, and another fixed-dimensional interval can be used as the storage area for negative word proportion parameter, laughter detection probability, stress keyword weighting coefficient, conflict keyword weighting coefficient and positive keyword weighting coefficient, so that the extraction process can read the data of the corresponding dimension through the index.

[0088] The learning stress index is calculated by multiplying the product of the energy standard deviation and the pitch standard deviation by 0.5 and the sum of the negative word proportion parameter, and then multiplying by the stress keyword weighting coefficient. The smaller value between the calculated result and 1 is taken as the learning stress index.

[0089] The product of the energy standard deviation and the pitch standard deviation is calculated, and then the negative word proportion parameter is multiplied by 0.5. The two results are added together and then multiplied by the stress keyword weighting coefficient to obtain the initial value of the learning stress index. This initial value is then compared to 1, and the smaller value is taken as the learning stress index, ensuring that the value of the learning stress index does not exceed 1.

[0090] If the standard deviation of energy is 0.4, the standard deviation of pitch is 0.3, the negative vocabulary proportion parameter is 0.2, and the weighting coefficient of stress keywords is 1.5, then the initial value of the learning stress index is calculated as follows: (0.4×0.3 + 0.2×0.5) × 1.5 = (0.12 +0.1) × 1.5 = 0.22 × 1.5 = 0.33. Since the calculated result is less than 1, the final value of the learning stress index is 0.33.

[0091] The parent-child communication index is calculated by multiplying the pleasure level by 1 minus the negative word ratio parameter, then multiplying by the difference between 1 and the weighting coefficient of conflicting keywords, and taking the larger value between the calculated result and 0.

[0092] The balance between positive and negative emotions in verbal expression is calculated by multiplying the level of pleasure by (1 - the proportion of negative words). Then, the value of (1 - the weighting coefficient of conflicting keywords) is calculated. Finally, these two parts are multiplied together to obtain the initial value of the parent-child communication index. This initial value is then compared to 0, and the larger value is taken as the parent-child communication index, ensuring that the index does not become negative.

[0093] If the pleasure level is 0.7, the negative word ratio parameter is 0.1, and the conflict keyword weighting coefficient is 0.3, then the initial value of the parent-child communication index is: 0.7 × (1-0.1) × (1-0.3) = 0.7 × 0.9 × 0.7 = 0.441. Since the result is greater than 0, the parent-child communication index is 0.441.

[0094] The social relationship index is calculated by summing the product of the probability of laughter detection, the weighted coefficient of pleasure level and positive keywords, and then dividing by 2.

[0095] The probability of laughter detection is added to the product of the pleasure level and the weighting coefficient of positive keywords, and the sum is divided by 2 so that the social relationship index can simultaneously consider the pleasure level, the occurrence of laughter, and the occurrence of positive keywords in the speech.

[0096] If the laughter detection probability is 0.2, the pleasure level is 0.6, and the positive keyword weighting coefficient is 1.0, then the social relationship index = (0.2 + 0.6 × 1.0) ÷ 2 = (0.2 + 0.6) ÷ 2 = 0.8 ÷ 2 = 0.4.

[0097] The emotional stability index is calculated by subtracting the absolute value of the difference between the activation level and 0.5 from 1, and then subtracting the product of the fundamental frequency variance and the preset fundamental frequency fluctuation penalty coefficient.

[0098] The activation offset is obtained by calculating the absolute value of the difference between the activation level and 0.5. Then, the initial value of stability is calculated by subtracting the offset value from 1. At the same time, the fundamental frequency variance is read from the fused feature vector, multiplied by the preset fundamental frequency fluctuation penalty coefficient, and the result is subtracted from the initial value of stability to obtain the result of the emotion stability index.

[0099] If the activation level is 0.4, the fundamental frequency variance is 0.03, and the preset fundamental frequency fluctuation penalty coefficient is 2, then the formula for calculating the emotional stability index is: 1 - |0.4 - 0.5| - (0.03 × 2) = 1 - 0.1 - 0.06 = 0.84.

[0100] The negative emotion index is calculated by subtracting the pleasure level from 1, multiplying it by the negative word ratio parameter, and then multiplying it by the sum of the products of 1, activation level, and 0.3.

[0101] The negative emotion index is obtained by multiplying (1 - pleasure level) by the negative word ratio parameter, and then multiplying it by (1 + activation level × 0.3). This negative emotion index can comprehensively reflect the interaction between emotional states such as sadness, anger, and excitement.

[0102] If the pleasure level is 0.3, the negative vocabulary ratio parameter is 0.1, and the activation level is 0.6, then the negative emotion index is: (1-0.3) × 0.1 × (1 + 0.6×0.3) = 0.7 × 0.1 × (1 + 0.18) = 0.07 × 1.18 =0.0826.

[0103] The five-dimensional psychological index is generated by combining the learning stress index, parent-child communication index, social relationship index, emotional stability index, and negative emotion index.

[0104] The five-dimensional psychological index is formed by combining the learning stress index, parent-child communication index, social relationship index, emotional stability index, and negative emotion index in a fixed order, so that the data of the five dimensions can participate in the calculation in a unified order in the subsequent visualization construction.

[0105] The five indices can be arranged in the following order: “learning pressure, parent-child communication, social relationships, emotional stability, and negative emotions” to construct a five-dimensional psychological index, such as [0.33, 0.441, 0.4, 0.84, 0.0826].

[0106] In another example, step S40 may preferably be: The numerical range of each dimension index in the five-dimensional psychological index is checked, limiting the value range to between 0 and 1. Each dimension index is multiplied by 100 to obtain a preliminary normalized index value, the value range of which is between 0 and 100. The preliminary normalized index value is then nonlinearly adjusted using a power function transformation, calculated as follows: the adjusted index value equals 1 of the preliminary normalized index value divided by the sensitivity power. Outlier truncation is then performed on the adjusted index value; if the adjusted index value is greater than 100, it is set to 100; if the adjusted index value is less than 0, it is set to 0, thus obtaining the normalized five-dimensional psychological index.

[0107] The value range of the five-dimensional psychological index is checked dimension by dimension, limiting all indices to between 0 and 1. Then, the five-dimensional psychological index is multiplied by 100 to convert it into a preliminary normalized index value within the range of 0 to 100. Next, a power function transformation is used to perform non-linear adjustment, calculated as "adjusted index value = preliminary normalized index value^(1 / sensitivity)," ensuring that different psychological dimensions maintain different sensitivities to change. Finally, outlier truncation is performed, setting values ​​exceeding 100 to 100 and values ​​less than 0 to 0, resulting in the normalized five-dimensional psychological index.

[0108] If the initial normalization index is 64 and the sensitivity is set to 2, then the adjusted index value is calculated as 64^(1 / 2) = 8. If the index value of a certain dimension exceeds 100 due to calculation error, for example, 105, then it is truncated and set to 100.

[0109] A polar coordinate system for a five-dimensional radar chart is established, with five dimensions—learning stress, parent-child communication, social relationships, emotional stability, and negative emotions—evenly distributed clockwise on a 360-degree circle. The angle between each dimension is 72 degrees. The polar angles for the learning stress dimension are set to 0 degrees, parent-child communication dimension to 72 degrees, social relationship dimension to 144 degrees, emotional stability dimension to 216 degrees, and negative emotions dimension to 288 degrees. The radial coordinates of each dimension on the radar chart are calculated based on the normalized values ​​of the five-dimensional psychological indices. The value of the radial coordinate is equal to the normalized index value of that dimension. The Cartesian coordinates of the vertices of each dimension are calculated using the polar angles and radial coordinates in the pre-defined polar coordinate system. The formula is: x = radial coordinate multiplied by the cosine of the polar angle, y = radial coordinate multiplied by the sine of the polar angle. The Cartesian coordinates of adjacent dimension vertices are then connected sequentially to form a closed pentagonal region.

[0110] A five-dimensional polar coordinate system was established, with learning stress, parent-child communication, social relationships, emotional stability, and negative emotions evenly distributed clockwise on a 360-degree circle, and the included angle of each dimension fixed at 72 degrees. The polar angles correspond to 0 degrees, 72 degrees, 144 degrees, 216 degrees, and 288 degrees, respectively. The normalized five-dimensional psychological index was used as the radial coordinate value, combined with the corresponding polar angles to form the polar coordinate points of the five dimensions.

[0111] If the normalized exponent value of a certain dimension is 85, then its radial coordinate is 85; if the polar angle corresponding to this dimension is 144 degrees, then its Cartesian coordinates are x = 85 × cos(144°) and y = 85 × sin(144°).

[0112] The area S of the pentagonal region is calculated using the polygon area calculation formula: S equals 0.5 multiplied by the product of the radial coordinates of the five dimensions and the radial coordinates of the adjacent dimensions, and then multiplied by the sum of the sine values ​​of 72 degrees. The area S is used as a comprehensive psychological health indicator to calculate the dispersion of the five-dimensional psychological index.

[0113] The area of ​​a pentagonal region, S, is calculated using the formula for calculating the area of ​​a regular polygon. This is done by calculating 0.5 × r(i) × r(i+1) × sin(72°) for each pair of adjacent radial coordinates. The sum of these five values ​​yields S, which is then used as a comprehensive psychological health indicator. Simultaneously, the dispersion of the five-dimensional psychological index is calculated. The variance of this index is then used as an indicator of psychological balance, assessing whether there are any significant biases in the five psychological dimensions.

[0114] If the five radial coordinates are 80, 70, 60, 90, and 85 respectively, then the area contribution of the five items can be calculated in sequence and summed to obtain S; the dispersion can be calculated using the variance formula, for example, variance=120.

[0115] A comprehensive psychological state rating is conducted based on the comprehensive psychological health index S and the variance. When S is greater than the preset first threshold T1 and the variance is less than the preset second threshold T2, the rating is good psychological state. When S is less than the preset third threshold T3 or the variance is greater than the preset fourth threshold T4, the rating is that attention is needed. In other cases, the rating is that psychological state is average. Color mapping schemes are assigned to each dimension of the five-dimensional radar chart.

[0116] A comprehensive psychological state rating is determined based on the values ​​of the comprehensive psychological health index and its dispersion. If the area S > T1 and the variance < T2, the rating is "good psychological state"; if the area S < T3 or the variance > T4, the rating is "needs attention"; otherwise, the rating is "average psychological state". A color mapping scheme is then configured for the five dimensions, so that each dimension displays a corresponding color in the radar chart.

[0117] If T1=2000, T2=150, T3=800, T4=300, and the calculated results are S=2200 and variance=120, then the rating is good psychological state.

[0118] The learning stress dimension is mapped to orange-red with RGB values ​​of 255, 140, and 0; the parent-child communication dimension is mapped to warm yellow with RGB values ​​of 255, 215, and 0; the social relationship dimension is mapped to vibrant green with RGB values ​​of 50, 205, and 50; the emotional stability dimension is mapped to tranquil blue with RGB values ​​of 70, 130, and 180; and the negative emotion dimension is mapped to warning purple with RGB values ​​of 147, 112, and 219. The saturation and transparency of the corresponding colors are dynamically adjusted according to the normalized index values ​​of each dimension. The Cartesian coordinates of the vertices of each dimension, the area S of the pentagonal region, the dispersion, the comprehensive psychological state rating, and the color mapping scheme are combined to generate radar chart coordinate data and visualization parameters.

[0119] The five-dimensional Cartesian coordinates, the area S of the pentagon, the variance, the comprehensive psychological state rating, and the five-dimensional color mapping scheme are combined and encapsulated into radar chart coordinate data and visualization parameters for terminal loading and rendering.

[0120] The dimensions of learning stress, parent-child communication, social relationships, emotional stability, and negative emotions are mapped to RGB(255,140,0), respectively. The transparency is dynamically adjusted based on the normalization index, for example, a transparency of 0.6.

[0121] In another example, step S50 may preferably be: The radar chart coordinate data and visualization parameters are transmitted to a terminal, which may be a smartphone, tablet, or personal computer; a canvas object is created in the terminal's graphics rendering engine.

[0122] The radar chart coordinate data and visualization parameters are sent to the terminal, which can be a smartphone, tablet, or personal computer, via a network communication interface. A canvas object is then created in the terminal's graphics rendering engine to provide a drawing container for the radar chart.

[0123] In iOS devices, Core Graphics is used to create a drawing context; in Android devices, a Canvas object is used; and in Windows devices, a canvas object is created using OpenGL or Direct2D.

[0124] Based on the Cartesian coordinates of the vertices in each dimension of the radar chart coordinate data, draw a pentagonal outline connecting the vertices on the canvas object. The outline has a line width of 2 to 4 pixels and a line color of dark gray. According to the color mapping scheme in the visualization parameters, fill the area inside the pentagonal outline. Fill the corresponding sector of each dimension with the corresponding color, and set the transparency of the fill color to 50% to 70%. Label the endpoints of the rays in each dimension of the radar chart with dimension names, including learning pressure, parent-child communication, social relationships, emotional stability, and negative emotions. The font size of the labeled text is 12 to 16 pixels. A data panel is displayed on the right or below the radar chart. The data panel includes the specific values ​​of psychological indices for each dimension, the area value of the comprehensive psychological health index, the dispersion value of the psychological state balance index, and the comprehensive psychological state rating.

[0125] Based on the comprehensive psychological state rating, preset mental health advice text is displayed on the data panel. When the rating is "good" psychological state, the preset text is encouraging. When the rating is "needs attention" psychological state, the preset text is warning and prompts you to seek professional psychological counseling. When the rating is "average" psychological state, the preset text is improvement advice.

[0126] Based on the Cartesian coordinates in the radar chart data, draw the outline of a pentagon on the canvas and fill the five dimensional regions according to the color mapping scheme in the visualization parameters, setting the fill transparency to 50% to 70%. Draw the dimension name text at the ray endpoints of each dimension of the radar chart, and simultaneously display a data panel in the vicinity of the radar chart, including the five-dimensional psychological index, area S, dispersion variance, and rating results. Display different mental health advice texts in the data panel based on the rating results.

[0127] If the rating is "Good mental state", it will display "Current mental state is stable, please continue to maintain positive communication"; if the rating is "Needs attention", it will display "Psychological risk characteristics detected, it is recommended to seek professional psychological counseling support in a timely manner".

[0128] Example 2: like Figure 2 As shown, the present invention also provides a five-dimensional radar map conversion system 10 for voice emotion analysis. The conversion system includes a data acquisition module 11, a feature extraction module 12, a mapping calculation module 13, a data generation module 14, and a data transmission module 15.

[0129] The data acquisition module 11 is mainly used to acquire the input voice data and preprocess the voice data to obtain preprocessed voice data.

[0130] The feature extraction module 12 is mainly used to extract dual-modal features from the preprocessed speech data to obtain a fused feature vector. The fused feature vector is then input into a preset VAD emotion recognition model to output a three-dimensional emotion space vector.

[0131] The mapping calculation module 13 is mainly used to calculate the five-dimensional psychological index based on the three-dimensional emotion space vector and the fused feature vector.

[0132] The data generation module 14 is mainly used to perform adaptive normalization processing on the five-dimensional psychological index to obtain the normalized five-dimensional psychological index, and generate radar chart coordinate data and visualization parameters based on the normalized five-dimensional psychological index.

[0133] The data transmission module 15 is mainly used to transmit radar map coordinate data and visualization parameters to the terminal, and dynamically render and display the five-dimensional radar map on the terminal interface.

[0134] In this embodiment, by setting a clear data flow relationship between the data acquisition module 11, feature extraction module 12, mapping calculation module 13, data generation module 14, and data transmission module 15, the conversion process of the five-dimensional radar chart for the entire voice emotion analysis is effectively improved in terms of the system's accuracy, stability, and interpretability in multi-dimensional emotion modeling, psychological data quantification, and visualization under a modular structure.

[0135] Example 3: The present invention also provides an electronic device 20, which includes a processor 22 and a memory 24. The memory 24 stores at least one instruction or at least one program, which is loaded and executed by the processor 22 to implement a five-dimensional radar map conversion method for voice emotion analysis as described above.

[0136] In one alternative embodiment, an electronic device 20 is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device includes a processor 22 and a memory 24. The processor 22 and the memory 24 are connected, for example, via a bus 21. Optionally, the electronic device 20 may further include a transceiver 23, which can be used for data interaction between the electronic device 20 and other electronic devices 20, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 23 is not limited to one unit, and the structure of the electronic device 20 does not constitute a limitation on the embodiments of this application.

[0137] In this embodiment, by storing a method for converting a five-dimensional radar chart of voice emotion analysis in the memory 24 of the electronic device 20 in the form of at least one instruction or at least one program, the electronic device 20 can complete a series of operations such as voice data processing, emotion recognition, and psychological visualization during the execution of the program by the processor 22. This enables the method for converting a five-dimensional radar chart of voice emotion analysis to achieve a stable and complete execution process in the electronic device 20, realizing the efficient conversion of voice emotion analysis to visual psychological assessment. It has the advantages of clear structure, reliable calculation, and flexible application scenarios.

[0138] Example 4: The present invention also provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement a method for converting a five-dimensional radar map of speech emotion analysis as described above.

[0139] In this embodiment, by storing at least one instruction or at least one program in a computer-readable storage medium, the processor can implement all the processing steps of the five-dimensional radar chart conversion method for speech emotion analysis as described above when reading and executing the instruction or program. This ensures the portability, scalability, and platform independence of the speech data processing process, enabling the five-dimensional radar chart conversion method for speech emotion analysis to have cross-platform execution capability and flexible deployment capability, effectively improving the reusability and maintainability of emotion recognition technology in practical applications.

[0140] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0141] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0142] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing the relevant hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0143] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for converting a five-dimensional radar chart for speech emotion analysis, characterized in that, include: The input voice data is acquired and preprocessed to obtain preprocessed voice data. The preprocessed speech data is subjected to bimodal feature extraction to obtain a fused feature vector. The fused feature vector is then input into a preset VAD emotion recognition model to output a three-dimensional emotion space vector. A five-dimensional psychological index is calculated based on the three-dimensional emotion space vector and the fused feature vector. The five-dimensional psychological index is adaptively normalized to obtain a normalized five-dimensional psychological index, and radar chart coordinate data and visualization parameters are generated based on the normalized five-dimensional psychological index. The radar map coordinate data and visualization parameters are transmitted to the terminal, and the five-dimensional radar map is dynamically rendered and displayed on the terminal interface.

2. The method for converting a five-dimensional radar chart for speech emotion analysis according to claim 1, characterized in that, The step of acquiring input speech data and preprocessing the speech data to obtain preprocessed speech data includes: The audio acquisition module acquires the input voice data, and the voice data is converted into a format, converting non-WAV format voice data into WAV format mono audio data to obtain the converted voice data. The converted speech data is denoised using spectral subtraction or Wiener filtering algorithms to obtain denoised speech data. By calculating the short-time energy and short-time zero-crossing rate, endpoint detection is performed on the denoised speech data to obtain effective speech segments. The effective speech segment is processed by speech framing, and the Hamming window function is used to window each frame of speech signal to obtain multiple speech frames. The high-frequency components of each speech frame are enhanced by a first-order high-pass filter to obtain preprocessed speech data.

3. The method for converting a five-dimensional radar chart for speech emotion analysis according to claim 1, characterized in that, The steps of extracting dual-modal features from the preprocessed speech data to obtain a fused feature vector, inputting the fused feature vector into a preset VAD emotion recognition model, and outputting a three-dimensional emotion space vector include: Fast Fourier Transform is performed on each speech frame in the preprocessed speech data to obtain spectral information. Based on the spectral information, Mel-frequency cepstral coefficients (MFCC) are calculated as timbre feature parameters. The fundamental frequency (F0) of each speech frame in the preprocessed speech data is calculated to statistically analyze pitch feature parameters and pitch fluctuation rate parameters. Calculate the short-time energy of each speech frame in the preprocessed speech data to statistically analyze the energy feature parameters. Calculate the duration ratio between adjacent speech frames and the total number of effective speech segments in the preprocessed speech data to obtain the speech rate feature parameters. By analyzing the variance of the fundamental frequency F0 and the fluctuation amplitude of the short-time energy, the emotional stability characteristic parameters are obtained, and the laughter characteristic frequency band in the preprocessed speech data is detected to obtain the laughter detection probability. The timbre feature parameters, pitch feature parameters, pitch fluctuation rate parameters, energy feature parameters, speech rate feature parameters, emotional stability feature parameters, and laughter detection probability are combined to form an acoustic feature vector; Semantic features are extracted from the preprocessed speech data to obtain semantic vectors, negative word proportion parameters, and keyword weighting coefficients, in order to construct semantic feature vectors. The acoustic feature vector and the semantic feature vector are concatenated according to their dimensions using a feature concatenation method to obtain a fused feature vector. The fused feature vector is then input into a preset VAD emotion recognition model to output a three-dimensional emotion space vector, which includes pleasure, activation, and dominance.

4. The method for converting a five-dimensional radar chart for speech emotion analysis according to claim 3, characterized in that, The step of extracting semantic features from the preprocessed speech data to obtain semantic vectors, negative word proportion parameters, and keyword weighting coefficients to construct semantic feature vectors includes: The preprocessed speech data is converted into text data using an automatic speech recognition model. The text data is then input into a pre-trained Chinese language model for text embedding encoding to obtain a semantic vector. The number of negative words appearing in the text data is counted by the sentiment dictionary matching algorithm, and the ratio of the number of negative words to the total number of words is calculated to obtain the negative word ratio parameter. The keyword recognition algorithm identifies preset psychological cue keywords in the text data and obtains keyword weighting coefficients, wherein the keyword weighting coefficients include stress keyword weighting coefficients, conflict keyword weighting coefficients, and positive keyword weighting coefficients; The semantic vector, the negative word ratio parameter, and the keyword weighting coefficient are combined to form a semantic feature vector.

5. The method for converting a five-dimensional radar chart for speech emotion analysis according to claim 4, characterized in that, The step of calculating the five-dimensional psychological index based on the three-dimensional emotion space vector and the fused feature vector includes: The values ​​of pleasure, activation and dominance are extracted from the three-dimensional emotion space vector. The standard deviation of energy, standard deviation of pitch, fundamental frequency variance, negative word ratio parameter, laughter detection probability, weighting coefficient of stress keyword, weighting coefficient of conflict keyword and weighting coefficient of positive keyword are extracted from the fused feature vector. The learning stress index is calculated using the energy standard deviation, the pitch standard deviation, the negative vocabulary ratio parameter, and the stress keyword weighting coefficient. The parent-child communication index is calculated using the pleasure level, the proportion of negative words, and the weighting coefficient of conflicting keywords. The social relationship index is calculated using the laughter detection probability, the level of pleasure, and the weighting coefficient of positive keywords. The emotional stability index is calculated based on the activation level, the fundamental frequency variance, and the preset fundamental frequency fluctuation penalty coefficient. A negative emotion index is calculated based on the pleasure level, the proportion of negative words, and the activation level. The learning stress index, parent-child communication index, social relationship index, emotional stability index, and negative emotion index are combined to generate a five-dimensional psychological index.

6. The method for converting a five-dimensional radar chart for speech emotion analysis according to claim 1, characterized in that, The steps of adaptively normalizing the five-dimensional psychological index to obtain a normalized five-dimensional psychological index, and generating radar chart coordinate data and visualization parameters based on the normalized five-dimensional psychological index, include: The numerical range of each dimension index in the five-dimensional psychological index is checked, and the value range is limited to between 0 and 1. Each dimension index is multiplied by 100 to obtain the index value after preliminary normalization. The index value after preliminary normalization is adjusted nonlinearly, and the outlier truncation is performed on the adjusted index value to obtain the normalized five-dimensional psychological index. Based on the values ​​of each dimension in the normalized five-dimensional psychological index, calculate the radial coordinates of each dimension on the preset radar chart. Calculate the Cartesian coordinates of the vertices of each dimension using the polar angles of each dimension in the preset polar coordinate system and the radial coordinates. Connect the Cartesian coordinates of adjacent dimension vertices in sequence to form a closed pentagonal region. Calculate the area of ​​the pentagonal region, use the area as a comprehensive psychological health index, and calculate the dispersion of the five-dimensional psychological index; A comprehensive psychological state rating is performed based on the comprehensive psychological health index and the dispersion, and a color mapping scheme is assigned to each dimension region of the five-dimensional radar chart. The Cartesian coordinates of the vertices in each dimension, the area and dispersion of the pentagonal region, the comprehensive psychological state rating, and the color mapping scheme are combined to generate radar chart coordinate data and visualization parameters.

7. The method for converting a five-dimensional radar chart for speech emotion analysis according to claim 6, characterized in that, The step of transmitting the radar map coordinate data and visualization parameters to the terminal, and dynamically rendering and displaying the five-dimensional radar map on the terminal interface, includes: The radar map coordinate data and the visualization parameters are transmitted to the terminal, and a canvas object is created in the terminal's graphics rendering engine. Based on the Cartesian coordinates of the vertices in each dimension of the radar chart coordinate data, a pentagonal outline connecting each vertex is drawn on the canvas object. According to the color mapping scheme in the visualization parameters, the area inside the pentagonal outline is filled, and the sector corresponding to each dimension is filled with the corresponding color.

8. A five-dimensional radar map conversion system for voice emotion analysis, characterized in that, include: The data acquisition module is used to acquire input voice data and preprocess the voice data to obtain preprocessed voice data. The feature extraction module is used to extract dual-modal features from the preprocessed speech data to obtain a fused feature vector, and input the fused feature vector into a preset VAD emotion recognition model to output a three-dimensional emotion space vector. The mapping calculation module is used to calculate a five-dimensional psychological index based on the three-dimensional emotion space vector and the fused feature vector. The data generation module is used to perform adaptive normalization processing on the five-dimensional psychological index to obtain the normalized five-dimensional psychological index, and generate radar chart coordinate data and visualization parameters based on the normalized five-dimensional psychological index. The data transmission module is used to transmit the radar map coordinate data and visualization parameters to the terminal, and dynamically render and display the five-dimensional radar map on the terminal interface.

9. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the method for converting a five-dimensional radar map of speech emotion analysis as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the five-dimensional radar chart conversion method for voice emotion analysis as described in any one of claims 1 to 8.

Images