A method and device for adaptive ambient sound emotion recognition

Abstract

The application discloses a self-adaptive environmental sound emotion recognition method and device, and the method comprises the following steps: collecting environmental sound by using crowd-sourcing audio data and a mobile device, and labeling the environmental sound to construct an environmental sound sample library; performing self-adaptive segmentation on sound data in the sample library through audio signal energy change to obtain sound data of different lengths; extracting sound perception features based on the perception characteristics of human ears for the sound data of different lengths obtained through segmentation; taking the sound perception features as input, performing environmental sound emotion training through a recurrent neural network method of a long short-term memory network, and performing emotion prediction on actually collected environmental sound data by using the trained model.

Description

Technical Field

[0001] This invention relates to the field of sound data processing, and in particular to an adaptive environmental sound emotion recognition method and apparatus. Background Technology

[0002] Sound is a crucial means for humans to perceive their environment and an important component of the human living environment. Sound landscape design has become an emerging research topic in urban planning, architecture, and related fields. Sound emotion, the subjective feeling people have about environmental sounds, is an important indicator for evaluating the sound environment. Currently, there is extensive research on environmental sound type identification, with many methods based on audio signal processing and machine learning proposed. However, compared to sound type identification, research on methods for environmental sound emotion identification is still relatively limited.

[0003] Existing voice emotion recognition methods are mostly designed for human speech or music, and these methods are generally based on sound signal processing or machine learning. Signal processing-based methods identify emotion types by capturing the changing patterns of sound signals, such as zero-crossing rate and signal decomposition features. These methods have successfully achieved emotion recognition in some scenarios; however, achieving good accuracy requires designing complex information extraction algorithms. Machine learning-based methods do not require complex information processing algorithms; methods such as Gaussian Mixture Models (GMM) and Support Vector Machines (SVM) have already demonstrated impressive recognition accuracy. These methods, using the ideas of machine learning, first construct a sound sample database for specific themes and emotion types, then extract sound features from different perspectives, and finally perform automated classification through machine learning training mechanisms. Numerous voice emotion datasets, sound feature extraction methods, and voice emotion recognition methods exist for different languages ​​and music genres. For example, the RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song) public speech and music emotion sample database on Kaggle, the world's largest open machine learning data platform, contains human voice samples, mostly short samples of 3-4 seconds. However, unlike human speech and music, environmental sounds are more complex, containing various sound sources such as traffic, natural sounds, and mechanical sounds, and are also more variable, changing rapidly in a short time as the source of the sound changes or moves. Therefore, methods for emotion recognition based on environmental sounds still need further research. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to adaptively identify the dynamic emotions in variable-length environmental sound data, and to provide a method and device for environmental sound emotion recognition based on neural networks, so as to achieve adaptive dynamic emotion recognition for any variable-length environmental sound data.

[0005] Technical solution: According to a first aspect of the present invention, an adaptive environmental sound emotion recognition method includes the following steps:

[0006] (1) Collect and label environmental sounds using crowdsourced audio data and mobile devices to build an environmental sound sample library;

[0007] (2) For the sound data in the sample library, adaptive segmentation of the sound scene is performed by changing the energy of the audio signal to obtain scene sound data of different lengths;

[0008] (3) Extract sound perception features based on the perceptual characteristics of the human ear from the segmented sound data of different lengths;

[0009] (4) Using sound perception features as input, the environmental sound emotion recognition model is trained by the recurrent neural network method of long short-term memory network;

[0010] (5) Use the trained model to predict the sentiment of the actual collected environmental sound data.

[0011] Further, step (1) includes: downloading publicly available environmental sound datasets from home and abroad using web crawling technology; labeling the collected sounds with emotion types using the Wundt discrete emotion model, specifically including six emotions: pleasure, boredom, excitement, calmness, tension, and relaxation; and supplementing the number of sound samples in the dataset whose proportion of emotion type samples is lower than a specified threshold, so that the number of sound samples of each type is balanced.

[0012] In one implementation, step (2) includes: setting the audio analysis window size, calculating the signal energy of the audio data window by window, and considering the signal energy state to have changed when the audio signal energy of a specified number of windows is less than or greater than a preset audio energy threshold. Then, setting sound scene segmentation points, performing adaptive segmentation of the sound data, and obtaining environmental sound data of different lengths.

[0013] In another embodiment, step (2) includes: setting the audio analysis window size, calculating the signal energy of the audio data window by window and calculating the second gradient of the audio signal energy, searching for the zero-crossing point of the second gradient of the signal energy, and when the second gradient is zero, that is, the zero cross point of the second derivative of the signal, it is considered that the signal energy state has changed abruptly, and the point is used as the adaptive segmentation point of the sound data.

[0014] Furthermore, the formula for calculating the energy of the audio data signal is as follows:

[0015]

[0016] In the formula, N is the number of audio frames within the window, and a iLet be the audio intensity value of the i-th frame.

[0017] Further, in step (3), the sound perception features are extracted based on Mel-frequency cepstral coefficients (MFCC), including: dividing the input audio signal into frames to obtain stable frame signals; converting the audio data from the time domain to the frequency domain using fast Fourier transform to obtain a sound spectrum; performing regional statistics on the sound energy of different frequencies, defining a triangular Mel filter bank with M filters, applying the Mel filter bank to the spectrum, and summing the energy results of each filter; calculating the logarithmic value of the energy of all filter groups; eliminating the correlation between them and reducing the dimension by discrete cosine transform, calculating the discrete cosine transform of the energy values ​​of all logarithmic filter groups, and obtaining a 26-dimensional discrete cosine transform result; retaining the first 20 dimensions of the discrete cosine transform result as the MFCC feature vector of the sound.

[0018] Further, the training of the model in step (4) includes: recording the sequence length of all input sound perception features, setting the length of the longest sequence as a length threshold, and padding the feature sequences smaller than the length threshold with zeros; then training the model by sequentially passing it through LSTM neural units, Dropout neural layers and Softmax fully connected layers, and outputting the trained neural network weights.

[0019] According to a second aspect of the present invention, an adaptive environmental sound emotion recognition device includes:

[0020] The environmental sound sample library construction module uses crowdsourced audio data and mobile devices to collect and label environmental sounds, thereby constructing an environmental sound sample library.

[0021] The adaptive audio data segmentation module performs adaptive segmentation of audio scenes based on changes in audio signal energy in the sample library, resulting in audio scene data of different lengths.

[0022] The environmental sound feature extraction module extracts sound perception features from the segmented sound data of different lengths based on the perceptual characteristics of the human ear.

[0023] The model training module uses sound perception features as input and trains the environmental sound emotion recognition model through a recurrent neural network method based on long short-term memory networks.

[0024] The environmental sound emotion recognition module uses a trained model to predict the emotion of actual collected environmental sound data.

[0025] According to a third aspect of the present invention, a computer device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when executed by the processors, the programs implement the steps of the adaptive environmental sound emotion recognition method as described in the first aspect of the present invention.

[0026] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the adaptive environmental sound emotion recognition method as described in the first aspect of the present invention.

[0027] Beneficial effects: This invention enables adaptive dynamic emotion recognition of environmental sound data of arbitrary length. The sound emotion is represented by a discrete emotion model containing six basic emotion categories, which can be freely expanded and combined. Compared with emotion recognition methods for music and speech, this invention can adapt to environmental sound data of arbitrary length. It first performs adaptive segmentation to obtain the changes in environmental sound, then extracts the perceptual features of the sound, and for variable-length audio data, it uses a recurrent neural network to identify emotions, ultimately obtaining dynamic changes in environmental sound emotion. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention;

[0029] Figure 2 This is a schematic diagram of adaptive segmentation of environmental sound data;

[0030] Figure 3 This is a schematic diagram of environmental sound perception characteristics;

[0031] Figure 4 This is a schematic diagram of a recurrent neural network;

[0032] Figure 5 It is the result of environmental sound characteristics and emotion recognition. Detailed Implementation

[0033] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0034] The overall approach of this invention is as follows: Addressing the diversity of emotional responses to environmental sounds, this invention utilizes crowdsourced audio data and mobile devices to collect environmental sounds, constructing an environmental sound sample library that ensures a uniform distribution of sound samples across different emotional types. To accommodate the dynamic changes in environmental sounds, adaptive audio length segmentation is performed. Based on this, and considering the characteristics of human auditory perception of sound, perceptual features of the segmented sounds are extracted to construct feature vectors. Finally, a recurrent neural network, combined with contextual information, is used to classify the emotional responses of the sounds, outputting dynamic emotional categories. This method is applicable to emotion recognition of variable-length environmental sound data and can automatically identify dynamic changes in emotional responses.

[0035] Reference Figure 1 An adaptive environmental sound emotion recognition method includes the following steps:

[0036] Step 1: Collect and label ambient sounds to build an ambient sound sample library, which contains audio samples of sounds from typical environments.

[0037] We downloaded publicly available environmental sound datasets from both domestic and international sources using web scraping techniques, such as the domestic Ear0.com sound sharing platform (https: / / www.ear0.com / ) and the international Freesound sharing platform (https: / / freesound.org / browse / ). Then, based on the Wundt emotion dimension model, we manually labeled the collected sound data with six basic emotions: pleasure, annoyance, excitement, calmness, tension, and relaxation. These six emotion models can be combined to express more complex emotions and can also be expanded to include more emotion types. For emotion types with a limited number of samples in the dataset, we conducted field sampling as a supplement, ensuring that the ratio of the most frequent to the least frequent emotion type in the sound samples did not exceed 3:1.

[0038] Step 2: Based on the sound data in the sample library, perform adaptive segmentation of the sound scene to obtain environmental sound data of different lengths.

[0039] Adaptive segmentation of sound scenes is performed based on changes in audio signal energy, and the results are as follows: Figure 2 As shown. First, the energy of the sound data is monitored. Then, abrupt changes in the average energy data of the sound are detected, and the energy abrupt change points are used as adaptive segmentation points for the sound data. Specifically, the audio analysis window size is first set. Taking audio data with a sampling rate of 16000 as an example, a length of 10ms is selected as the window size. Then, the signal energy of the audio data is calculated window by window. The formula for calculating the signal energy of a certain audio window is shown below:

[0040]

[0041] Where N is the number of audio frames within the window, a i Let be the audio intensity value of the i-th frame.

[0042] Furthermore, the audio data segmentation points are determined according to the changes in audio energy. In this invention, the abrupt change points of audio data signal energy are used as adaptive segmentation points for audio data. According to one embodiment, an audio energy threshold is set, and energy state is judged based on the audio energy threshold. If the audio energy of a certain window number is greater than the threshold, the state is set to a high energy range; otherwise, it is a low energy range. When the audio energy of a certain window number is less than or greater than the audio energy threshold, it can be considered that a change in energy state has occurred, and a sound scene segmentation point is set. According to another embodiment, the second gradient of the audio signal energy is first calculated. When the second gradient is greater than zero, the energy exhibits a concave change; when the second gradient is less than zero, the energy exhibits a convex change; when the second gradient is equal to zero, the concavity and convexity of the energy change, which can be regarded as abrupt change points of signal energy. Therefore, the zero-crossing points of the second gradient of the signal energy are searched. When the second gradient is zero, that is, the zero crossover point of the second derivative of the signal, it can be considered that the signal energy has undergone abrupt change nearby, which is regarded as a significant change in the ambient sound, and this point is used as the adaptive segmentation point of the sound data. Finally, adaptive segmentation of the sound data is performed based on the sound scene segmentation points to obtain environmental sound data of different lengths.

[0043] Step 3: Extract the perceptual features of ambient sound.

[0044] For the segmented sound data of different lengths, the essence is signal intensity that varies over time. The data is too discrete, and directly performing emotion recognition would cause data redundancy. Therefore, it is necessary to extract sound features first. Here, the Mel-scale Frequency Cepstral Coefficients (MFCCs) of the sound are extracted as the feature vector for environmental sound emotion recognition. The MFCC extraction steps are as follows: Figure 3 As shown, the specific operation is as follows:

[0045] (1) Convert the input audio signal into short frames to obtain a stable frame signal;

[0046] (2) Considering that human hearing perception is based on the vibration of sound at different frequencies, the audio data is converted from the time domain to the frequency domain through fast Fourier transform to obtain the sound spectrum.

[0047] (3) Considering that the human ear cannot distinguish the difference between two sounds with similar frequencies, regional statistics can be performed on the sound energy of different frequencies. A triangular Mel filter bank with M filters is defined. The Mel filter bank is applied to the spectrum and the energy results of each filter are added together.

[0048] (4) Considering that the human ear perceives sound nonlinearly, that is, to double the perceived volume, the sound energy needs to be increased by 8 times, calculate the logarithm of the energy groups of all filters.

[0049] (5) Since the filter banks are overlapping, there is a strong correlation between the filter energies. Therefore, the correlation between them is eliminated by discrete cosine transform and the dimension is reduced. The discrete cosine transform of the energy values ​​of all logarithmic filter banks is calculated to obtain the 26-dimensional discrete cosine transform result.

[0050] (6) Considering that the high-dimensional discrete cosine transform result represents the rapid change of filter bank energy without compensation, the first 20 dimensions of the discrete cosine transform result are retained as the MFCC feature vector of the sound.

[0051] Step 4: Use the perceptual features of sound to train a voice emotion recognition model based on a recurrent neural network.

[0052] Using sound perception features as input, an environmental sound emotion prediction method based on a Long Short-Term Memory (LSTM) recurrent neural network is used for training. The steps are as follows: Figure 4 As shown, the specific operation is as follows:

[0053] (1) Feature vector alignment. Considering that the lengths of the input audio data are different, the lengths of their feature vectors are also different, so it is necessary to align the lengths of the feature vectors. First, record the sequence lengths of all input sound perception features, and set the length of the longest sequence as the length threshold. For feature sequences that are shorter than the length threshold, pad them with zeros to the length threshold.

[0054] (2) LSTM Neural Unit. A recurrent neural network (RNN) is a type of neural network specifically designed for processing time series data. It has a chain structure with a large number of repetitive neural network modules and can adapt well to the analysis of sequence data by using context information. However, during the training of long sequences, general RNNs also suffer from the gradient vanishing problem, which is why LSTM time units are introduced.

[0055] Compared to typical recurrent neural networks, which often contain simple chain structures like tanh layers, LSTM neurons consist of three stages. The first is the forgetting layer, which selectively forgets the data input from the previous node. Information from previous hidden layers and the current input is passed between the two through a sigmoid function, with weights ranging from 0 to 1. Weights closer to 0 indicate forgetting, while weights closer to 1 indicate retention.

[0056] f t =σ(W f ·[h t-1 ,xt ]+b f )

[0057] Next, the input layer updates the information. The hidden layer input and the current input are passed through the sigmoid function, converting the values ​​to a range of 0 to 1 to determine whether to update; 0 indicates no update, and 1 indicates an update is needed. Additionally, the hidden layer input and the current input are processed by the tanh function, yielding values ​​between -1 and 1. The tanh output is then multiplied by the sigmoid output; the sigmoid output determines which important information is retained from the tanh output.

[0058] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0059]

[0060] Then, the neuron state is calculated. The neuron state is multiplied by the forgetting layer vector. Then, the output is obtained from the input gate, and pointwise addition is performed to update the neuron state with the relevant new values, resulting in a new neuron state.

[0061]

[0062] Finally, the output layer determines the state of the next hidden layer. First, the previous hidden layer and the current input are passed through the sigmoid function. Then, the newly modified neuron state is passed to the tanh function, and the tanh output is multiplied by the sigmoid output to determine what information the hidden layer should contain. The new neuron state and the new hidden layer are then carried over to the next time step.

[0063] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0064] h t =o t *tanh(C t )

[0065] (3) Dropout layer. To prevent overfitting during neural network training, a dropout layer is introduced. During training, the dropout layer randomly sets the input units to 0 at a certain frequency p. Inputs that are not set to 0 are amplified by a ratio of 1 / (1-p) to keep the sum of all inputs constant. Here, 0.5 is taken as the probability value.

[0066] (4) Softmax fully connected layer. The Softmax function can transform an n-dimensional vector into a probability distribution of n objects, that is, the final neural network is mapped to the emotion type of the ambient sound.

[0067]

[0068] (5) Model Training. The environmental sound emotion sample set is divided into training and testing samples in a 4:1 ratio. The model is trained using a GPU, and the weights of each layer are updated through backpropagation. As the number of training iterations increases, the loss function continuously decreases. When the loss function reaches a threshold or the maximum number of training iterations is reached, training is stopped and the weights are saved.

[0069] Step 5: Use the trained model to perform sentiment prediction on the actual collected environmental sound data.

[0070] The trained model is applied to emotion recognition on new environmental sound data. After adaptive segmentation and neural network recognition of the actual collected environmental sound data, the dynamic environmental sound emotion categories are output in time sequence. Taking environmental sound data from Nanjing as an example, the results of its dynamic environmental sound emotion recognition are as follows: Figure 5 As shown, the states are, in order: calm, calm, calm, restless, tense, and restless. This demonstrates that the method effectively extracts the dynamic changes in environmental sound over a short period.

[0071] Based on the same technical concept as the method embodiments, the present invention also provides an adaptive environmental sound emotion recognition device, comprising:

[0072] The environmental sound sample library construction module uses crowdsourced audio data and mobile devices to collect and label environmental sounds, thereby constructing an environmental sound sample library.

[0073] The adaptive audio data segmentation module performs adaptive segmentation of audio scenes based on changes in audio signal energy in the sample library, resulting in audio scene data of different lengths.

[0074] The environmental sound feature extraction module extracts sound perception features from the segmented sound data of different lengths based on the perceptual characteristics of the human ear.

[0075] The model training module uses sound perception features as input and trains the environmental sound emotion recognition model through a recurrent neural network method based on long short-term memory networks.

[0076] The environmental sound emotion recognition module uses a trained model to predict the emotion of actual collected environmental sound data.

[0077] According to an embodiment of the present invention, the environmental sound sample library construction module includes:

[0078] The data acquisition unit is used to download publicly available environmental sound datasets from both domestic and international sources using web crawler technology.

[0079] The data labeling unit is used to label the collected sounds with emotion types in conjunction with the Wundt discrete emotion model, specifically including six emotions: pleasure, boredom, excitement, calmness, tension, and relaxation.

[0080] The data preprocessing unit is used to supplement the number of sound samples whose proportion of emotion type samples in the dataset is lower than a specified threshold, so as to balance the number of sound samples of each type.

[0081] According to an embodiment of the present invention, the adaptive audio data segmentation module includes:

[0082] The signal energy calculation unit is used to set the size of the audio analysis window and calculate the signal energy of the audio data window by window. The formula for calculating the signal energy of the audio data is as follows:

[0083]

[0084] In the formula, N is the number of audio frames within the window, and a i Let be the audio intensity value of the i-th frame;

[0085] The signal energy change recognition unit is used to identify changes in audio energy and regards abrupt changes in audio energy as the dividing points where significant changes occur in the ambient sound.

[0086] In one implementation, the signal energy change recognition unit determines the state change based on the audio energy threshold. Specifically, when the audio signal energy of a specified window number is less than or greater than the preset audio energy threshold, it is considered that the signal energy state has changed. Sound scene segmentation points are set, and adaptive segmentation of sound data is performed to obtain environmental sound data of different lengths.

[0087] In another embodiment, the signal energy change recognition unit determines the state change based on the second-order gradient of the signal energy. Specifically, it calculates the second-order gradient of the audio signal energy, searches for the zero-crossing points of the second-order gradient, and considers a sudden change in the signal energy state to occur when the second-order gradient is zero, i.e., the zero-crossing point of the second derivative of the signal. This point is then used as the adaptive segmentation point for the sound data.

[0088] The audio segmentation unit is used to divide the audio data into environmental sound data of different lengths based on the segmentation points determined by the signal energy change identification unit.

[0089] According to an embodiment of the present invention, the environmental sound feature extraction module extracts sound perception features based on Mel-frequency cepstral coefficients (MFCC), including: dividing the input audio signal into frames to obtain stable frame signals; converting the audio data from the time domain to the frequency domain using a fast Fourier transform to obtain a sound spectrum; performing regional statistics on the sound energy at different frequencies, defining a triangular Mel filter bank with M filters, applying the Mel filter bank to the spectrum, and summing the energy results of each filter; calculating the logarithmic values ​​of all filter energy groups; eliminating correlations and reducing dimensions through discrete cosine transform, calculating the discrete cosine transform of the energy values ​​of all logarithmic filter banks to obtain a 26-dimensional discrete cosine transform result; and retaining the first 20 dimensions of the discrete cosine transform result as the MFCC feature vector of the sound.

[0090] Furthermore, the model training module trains the model by: recording the sequence lengths of all input sound perception features, setting the length of the longest sequence as a length threshold, and padding feature sequences smaller than the length threshold with zeros; then training the model by sequentially passing it through LSTM neural units, Dropout neural layers, and Softmax fully connected layers, and outputting the trained neural network weights.

[0091] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the adaptive environmental sound emotion recognition method as described above.

[0092] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the adaptive environmental sound emotion recognition method as described above.

[0093] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] This invention is described with reference to flowchart illustrations of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the processes... Figure 1 A device for a function specified by one or more processes.

[0095] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0096] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 A process or multiple processes specify the steps of a function.

Claims

1. An adaptive environmental sound emotion recognition method, characterized in that, Includes the following steps: (1) Collect and label environmental sounds using crowdsourced audio data and mobile devices to build an environmental sound sample library; (2) For the sound data in the sample library, adaptive segmentation of sound scenes is performed by changing the audio signal energy to obtain sound data of different lengths; specifically, the following method is adopted: set the audio analysis window size, calculate the signal energy of the audio data window by window, and when the audio signal energy of the specified number of windows is less than or greater than the preset audio energy threshold, it is considered that the signal energy state has changed, set the sound scene segmentation point, and perform adaptive segmentation of the sound data to obtain environmental sound data of different lengths; or the following method is adopted: set the audio analysis window size, calculate the signal energy of the audio data window by window and calculate the second gradient of the audio signal energy, search for the zero crossing point of the second gradient of the signal energy, and when the second gradient is zero, that is, the zero crossing point of the second derivative of the signal, it is considered that the signal energy state has changed abruptly, and this point is used as the adaptive segmentation point of the sound data; (3) Extract sound perception features based on the perceptual characteristics of the human ear from the segmented sound data of different lengths; Specifically, sound perception features are extracted based on Mel-frequency cepstral coefficients (MFCC), including: segmenting the input audio signal into frames to obtain stable frame signals; converting the audio data from the time domain to the frequency domain using Fast Fourier Transform (FFT) to obtain a sound spectrogram; performing regional statistical analysis on the sound energy at different frequencies, defining a triangular Mel-frequency filter bank with M filters, applying the Mel-frequency filter bank to the spectrogram, and summing the energy results from each filter; calculating the logarithmic values ​​of all filter energy groups; eliminating correlations and reducing dimensions through Discrete Cosine Transform (DCT), calculating the DCT of the energy values ​​of all logarithmic filter banks to obtain a 26-dimensional DCT result; and retaining the first 20 dimensions of the DCT result as the MFCC feature vector of the sound. (4) Using sound perception features as input, the environmental sound emotion recognition model is trained by the recurrent neural network method of long short-term memory network; (5) Use the trained model to perform sentiment prediction on the actual collected environmental sound data.

2. The adaptive environmental sound emotion recognition method according to claim 1, characterized in that, Step (1) includes: downloading publicly available environmental sound datasets from home and abroad using web crawling technology; labeling the collected sounds with emotion types using the Wundt discrete emotion model, specifically including six emotions: pleasure, boredom, excitement, calmness, tension, and relaxation; and supplementing the number of sound samples in the dataset whose proportion of emotion type samples is lower than a specified threshold, so that the number of sound samples of each type is balanced.

3. The adaptive environmental sound emotion recognition method according to claim 1, characterized in that, The formula for calculating the energy of audio data signals is: ； In the formula, N is the number of audio frames within the window, and a i Let be the audio intensity value of the i-th frame.

4. The adaptive environmental sound emotion recognition method according to claim 1, characterized in that, The training of the model in step (4) includes: recording the sequence length of all input sound perception features, setting the length of the longest sequence as the length threshold, and padding the feature sequences smaller than the length threshold with zeros; then training the model by sequentially passing it through LSTM neural units, Dropout neural layers and Softmax fully connected layers, and outputting the trained neural network weights.

5. An adaptive environmental sound emotion recognition device, characterized in that, For implementing the adaptive environmental sound emotion recognition method as described in any one of claims 1-4, the apparatus comprises: The environmental sound sample library construction module uses crowdsourced audio data and mobile devices to collect and label environmental sounds, thereby constructing an environmental sound sample library. The adaptive audio data segmentation module performs adaptive segmentation of audio scenes based on changes in audio signal energy in the sample library, resulting in audio scene data of different lengths. The environmental sound feature extraction module extracts sound perception features from the segmented sound data of different lengths based on the perceptual characteristics of the human ear. The model training module uses sound perception features as input and trains the environmental sound emotion recognition model through a recurrent neural network method based on long short-term memory networks. The environmental sound emotion recognition module uses a trained model to predict the emotion of actual collected environmental sound data.

6. A computer device, characterized in that, include: One or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the adaptive environmental sound emotion recognition method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the adaptive environmental sound emotion recognition method as described in any one of claims 1-4.

Images