Paralinguistic information recognition method and system based on fine-grained contrastive learning

Abstract

This invention discloses a method and system for identifying paralinguistic information based on fine-grained contrastive learning. The method comprises: generating natural language description texts of different styles based on paralinguistic labels in a paralinguistic dataset, followed by noise reduction processing; performing feature extraction using an audio encoder and a text encoder, then performing fine-grained contrastive learning on audio feature data samples and text feature data samples, using a fine-grained loss as a loss function to train the audio encoder and text encoder; performing feature extraction using the trained audio encoder and text encoder, calculating the similarity between the new audio feature data and the new text feature data, and selecting the descriptive text data with the highest similarity as the paralinguistic information of the test audio data. This invention can capture more comprehensive and open paralinguistic information from speech, effectively improving the accuracy of feature extraction compared to traditional methods.

Description

Technical Field

[0001] The present invention relates to a paralanguage information recognition technology, and in particular to a paralanguage information recognition method and system based on fine-grained contrastive learning. Background Art

[0002] Speech not only conveys explicit semantic information but also contains rich paralinguistic features that provide deep insights into the speaker's intent, emotion, and state. Paralinguistic features are those that go beyond the pure phonetic and linguistic content and can supplement or modify semantic information. Based on their temporal properties, these features can be divided into long-term, medium-term, and short-term features, depending on their duration. For example, long-term features include gender, age, and dialect; medium-term features involve emotional state and stuttering; and short-term features include emotion and changes in speech rate. Extracting this paralinguistic information is crucial to enhancing the naturalness and intelligence of human-computer interaction, as it enables machines to not only understand semantic content but also identify the speaker's underlying emotions and intentions. For example, intelligent systems can dynamically adjust their interaction with users by identifying emotional information in speech, thereby providing a more personalized and context-aware experience.

[0003] Currently, many techniques for extracting paralinguistic information from speech are typically designed for specific tasks, often focusing on identifying only a few predefined attributes. For example, speech emotion recognition is often treated as a classification problem, while the identification of long-term features such as gender and age is also often a single-task problem. Some research has introduced transfer learning, which leverages pretrained models for other paralinguistic recognition tasks, such as using speaker recognition models for emotion detection. However, these methods typically operate within a closed-set framework, meaning each model can only handle a fixed set of labels and tasks, limiting their ability to capture paralinguistic features in a wider range of complex speech scenarios. To address the limitations of closed-set learning, multi-task learning has been investigated as a potential solution. Through multi-task learning, models can simultaneously predict multiple paralinguistic features, such as age, gender, and emotion, further expanding the capabilities of paralinguistic recognition. However, this approach introduces new complexity as the number of tasks increases. Whenever new tasks or labels are added, the model architecture needs to be adapted, resulting in poor scalability. Furthermore, multi-task learning still operates within a predefined set of labels, limiting the model's adaptability to identifying new paralinguistic features. Summary of the Invention

[0004] The technical problem to be solved by the present invention is as follows: In response to the above-mentioned problems in the prior art, a method and system for identifying paralinguistic information based on fine-grained contrastive learning are provided, which can capture more comprehensive and open paralinguistic information from speech, effectively improving the accuracy of feature extraction compared to traditional methods.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for identifying paralinguistic information based on fine-grained contrastive learning includes the following steps:

[0007] Generate natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, and then perform noise reduction processing;

[0008] Using an audio encoder to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, and using a text encoder to extract features from the natural language description text after noise reduction to obtain text feature data samples;

[0009] Performing fine-grained comparative learning on the audio feature data samples and the text feature data samples to obtain a similarity matrix of the audio feature data samples and the text feature data samples, as well as an enhanced true value matrix of the audio feature data samples and the text feature data samples, and then using the fine-grained loss of the similarity matrix and the enhanced true value matrix as a loss function to train the audio encoder and the text encoder;

[0010] Use the trained audio encoder to extract features from the test audio data to obtain new audio feature data, and use the trained text encoder to extract features from the descriptive text data in the test set to obtain new text feature data. Calculate the similarity between the new audio feature data and the new text feature data, and select the descriptive text data with the highest similarity as the paralinguistic information of the test audio data.

[0011] Furthermore, when generating natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, it specifically includes:

[0012] Input the initial prompt word and paralinguistic label into the large language model, obtain the initial description text output by the large language model and record it in the context variable;

[0013] Input the content recorded in the context variable and the new prompt word into the large language model, obtain the new description text output by the large language model and record it in the context variable, and repeat this step until the description text output by the large language model meets the requirements.

[0014] Furthermore, the noise reduction process includes:

[0015] Classify the natural language description text into corresponding paralanguage labels to obtain different paralanguage label groups;

[0016] For each paralinguistic label group, the similarity of all samples in the group is calculated, and samples whose similarity with other samples is lower than the preset lower limit are regarded as noise samples;

[0017] The noise sample is input into the large language model to obtain a regenerated sample. The similarity between the regenerated sample and other samples in the corresponding paralinguistic label group is calculated. If the similarity with other samples is lower than the preset lower limit, the regenerated sample is input into the large language model again, and the similarity between the regenerated sample and other samples in the corresponding paralinguistic label group is calculated again. This step is repeated until the similarity with other samples exceeds the preset lower limit.

[0018] Furthermore, the audio feature data samples and the text feature data samples are high-dimensional feature vectors. Before performing fine-grained comparative learning on the audio feature data samples and the text feature data samples, it also includes: mapping the audio feature data samples and the text feature data samples into a shared embedding space to perform effective semantic alignment and paralinguistic feature learning.

[0019] Furthermore, when performing fine-grained comparative learning on the audio feature data samples and the text feature data samples, the following steps are included:

[0020] Calculate the similarity matrix of the audio feature data sample and the text feature data sample;

[0021] Constructing sub-true value matrices corresponding to each paralinguistic label from the audio feature data samples and the text feature data samples according to the corresponding paralinguistic labels, assigning weights to each sub-true value matrix according to the importance of each paralinguistic label, and then combining the weighted sub-true value matrices with the identity matrix to obtain an enhanced true value matrix;

[0022] The KL divergence between the similarity matrix and the augmented ground truth matrix is ​​calculated as the fine-grained loss.

[0023] Furthermore, the similarity matrix expression is as follows:

[0024] S=A·T

[0025] Wherein, A represents the feature vector of the audio feature data sample, and T represents the feature vector of the text feature data sample.

[0026] Furthermore, the enhanced true value matrix expression is as follows:

[0027]

[0028] Where I represents the identity matrix, G i represents the sub-truth value matrix corresponding to the i-th paralinguistic label, w i represents the importance distribution weight of the i-th paralanguage tag, and K represents the number of paralanguage tag types in the paralanguage dataset.

[0029] The present invention also provides a paralanguage information recognition system based on fine-grained contrastive learning, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any step of the paralanguage information recognition method based on fine-grained contrastive learning.

[0030] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods for identifying paralinguistic information based on fine-grained contrastive learning.

[0031] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods for identifying paralinguistic information based on fine-grained contrastive learning.

[0032] Compared with the prior art, the advantages of the present invention are:

[0033] 1. The present invention generates natural language description texts of different styles based on the paralanguage dataset and then performs noise reduction processing, thereby ensuring the sentence diversity and semantic consistency of the training text data, thereby ensuring the training effect of the model.

[0034] 2. By embedding audio and description text in a shared space and aligning them, the present invention can simultaneously capture paralinguistic information in multiple audios, thereby improving the versatility of paralinguistic feature recognition in speech.

[0035] 3. By introducing a fine-grained contrast loss function, this invention enables the model to better capture the detailed similarity relationship between samples and improve the discrimination ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flowchart of a method according to an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of the principle of fine-grained contrastive learning in an embodiment of the present invention.

[0038] Figure 3 Schematic diagram of the principle of performing paralinguistic information recognition on test audio in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the scope of protection of the present invention is not limited thereby.

[0040] Example 1

[0041] Given the limitations of existing technologies in extracting paralinguistic information from speech, this embodiment proposes a paralinguistic information recognition method based on fine-grained contrastive learning. This method can extract a wider range of paralinguistic information from speech and, compared with traditional methods, effectively improves the accuracy of feature extraction.

[0042] like Figure 1 As shown, the method of this embodiment includes the following steps:

[0043] S1) Prompt Engineering and Data Noise Reduction: Generate natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, and then perform noise reduction processing;

[0044] S2) Feature Extraction: Using an audio encoder to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, and using a text encoder to extract features from the natural language description text after noise reduction to obtain text feature data samples;

[0045] S3) Fine-grained contrastive learning: performing fine-grained contrastive learning on the audio feature data samples and the text feature data samples to obtain a similarity matrix of the audio feature data samples and the text feature data samples, as well as an enhanced true value matrix of the audio feature data samples and the text feature data samples. Then, a fine-grained loss of the similarity matrix and the enhanced true value matrix is ​​used as a loss function to train the audio encoder and the text encoder.

[0046] S4) Identification of paralinguistic information in speech: Use the trained audio encoder to extract features from the test audio data to obtain new audio feature data, and use the trained text encoder to extract features from the descriptive text data in the test set to obtain new text feature data. Calculate the similarity between the new audio feature data and the new text feature data, and select the descriptive text data with the highest similarity as the paralinguistic information of the test audio data.

[0047] Through the above steps, the audio encoder and text encoder can capture more comprehensive and open paralinguistic information from speech through fine-grained comparative learning.

[0048] Each step is described in detail below.

[0049] This embodiment uses natural language descriptions of speaking style and speech to train the audio encoder and text encoder. Natural language descriptions can effectively integrate various paralinguistic information, resulting in stronger expressiveness. This approach also facilitates the expansion of the model to a more open set of labels and tasks. To ensure greater diversity and accuracy in style descriptions, step S1 of this embodiment utilizes thirteen datasets containing multimodal paralinguistic information (IEMOCAP, MELD, MSP-Pocast, RAVDESS, SAVEE, ComparE2016-Deception, Prompt-Speech, SEP-28k, FluecyBank, TIMIT, Google-Accent Archive, and VCTK). These datasets contain paralinguistic information in nine categories: emotion, speaking rate, deception, stuttering, mood, dialect, accent, gender, and age. Based on this paralinguistic information, prompt engineering and data denoising are performed to generate text data describing the speaking style, ensuring the quality of the input data in the text modality.

[0050] Specifically, in step S1, when generating natural language description texts of different styles based on the paralanguage tags of the paralanguage dataset, a large language model (such as GPT-3.5-turbo) is used to generate natural language descriptions based on the paralanguage tags. Simulated memory mechanisms are introduced to track previous prompts and responses through context variables to ensure that the text generated by the large language model has more expression diversity and avoids generating monotonous descriptions. Specifically, the following steps are included:

[0051] S101) inputting the initial prompt word and the paralinguistic tag into the large language model, obtaining the initial description text output by the large language model and recording it into the context variable;

[0052] S102) Input the content recorded in the context variable and the new prompt word into the large language model, obtain the new description text output by the large language model and record it in the context variable, and repeat this step until the description text output by the large language model meets the requirements. For example, in the process of generating natural language descriptions multiple times, the corresponding paralinguistic tag context variable C and the prompt "Please use a different sentence structure and vary the description style from your previous responses when describing the speaker's speaking style" are input into the GPT-3.5-turbo API each time to generate a natural language description.

[0053] In step S1 of this embodiment, when performing noise reduction processing, specifically, after the generated description text is categorized into labels, outliers with low similarity within the same paralinguistic label group are filtered out, including the following steps:

[0054] S201) Classifying the natural language description text into corresponding paralanguage tags to obtain different paralanguage tag groups. In this embodiment, to ensure consistency between the generated text and the paralanguage tags, the Claude2.1 API is called to perform tag classification on the description text generated by GPT-3.5-turbo to detect whether it can be classified into the paralanguage tag corresponding to the generated text;

[0055] S202) For each paralinguistic label group, the similarity of all samples in the group is calculated, and samples whose similarity with other samples is lower than a preset lower bound are regarded as noise samples. In this embodiment, the SentenceBERT model is introduced to calculate the semantic similarity of the text, and the interquartile range (IQR) method is used to filter out outliers with low similarity within the same paralinguistic label group. Noise samples are samples whose similarity with other samples is lower than the lower bound (Lower Bound = Q1 - 1.5 × (Q3 - Q1), where Q1 and Q3 are the 25th and 75th percentiles, respectively.

[0056] S203) To ensure semantic consistency of generated texts of the same paralinguistic label category, the detected noise sample is input into the large language model for regeneration. Specifically, the noise sample is input into the large language model to obtain a regenerated sample, and the similarity between the regenerated sample and other samples in the corresponding paralinguistic label group is calculated. If the similarity with the other samples is lower than a preset lower limit, the regenerated sample is input into the large language model, and the similarity between the regenerated sample and other samples in the corresponding paralinguistic label group is calculated again. This step is repeated until the similarity with the other samples exceeds the preset lower limit.

[0057] In this embodiment, a contrastive learning structure is adopted, and in step S2, the audio encoder f audio and the text encoder f text For audio data X a and text data X t Perform feature extraction separately and map them into a shared embedding space.

[0058] Specifically, in step S2, when the audio encoder is used to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, the audio data is obtained from multiple collected paralanguage datasets. Input audio encoder. The audio encoder in this embodiment adopts HTSAT model. The audio encoder (HTSAT model) receives audio input and converts it into a high-dimensional audio feature vector, for example, This high-dimensional audio feature vector is used to represent the sentence-level features of the input audio.

[0059] Similarly, in step S2, when the text encoder is used to extract the features of the natural language description text after noise reduction to obtain the text feature data sample, the description text obtained in step S1 is specifically extracted. Input text encoder. The text encoder in this embodiment adopts RoBERTa model. The text encoder (RoBERTa model) converts the text input into a corresponding high-dimensional text feature vector, for example, This text feature vector represents the sentence-level features of the input text.

[0060] In step S2 of this embodiment, after obtaining the audio feature data sample and the text feature data sample, the feature vectors of the audio feature data sample and the feature vectors of the text feature data sample are mapped to a shared embedding space. Specifically, both features are processed into 512-dimensional vectors, representing sentence-level features of the two modalities, so as to perform effective semantic alignment and learn paralinguistic features. These features are compared and aligned in the same vector space to effectively capture the correlation between speech and text.

[0061] Step S3 of this embodiment builds on traditional contrastive learning by further considering the similarity between paralinguistic labels. Based on the existing identity matrix as the ground truth matrix, a sub-ground truth matrix representing the relevance of paralinguistic information is added to the original identity matrix to form a new ground truth matrix. This allows for more fine-grained loss calculation and improves the model's ability to recognize various paralinguistic features in speech.

[0062] Specifically, in step S3, when fine-grained comparative learning is performed on the audio feature data sample and the text feature data sample, as shown in FIG. Figure 2 As shown, the following steps are included:

[0063] S301) Calculate the similarity matrix of the audio feature data sample and the text feature data sample. In this embodiment, based on the audio embedding and text embedding in the embedding space, the similarity matrix between the samples is calculated to represent the similarity relationship between the audio and text samples. The expression is as follows:

[0064] S=A·T

[0065] Where A represents the feature vector of the audio feature data sample, and T represents the feature vector of the text feature data sample;

[0066] S302) Constructing a sub-true value matrix corresponding to each paralanguage label from the audio feature data samples and the text feature data samples according to the corresponding paralanguage labels. In this embodiment, when constructing a sub-true value matrix corresponding to each paralanguage label from the audio feature data samples and the text feature data samples according to the corresponding paralanguage labels, the corresponding sub-true value matrix G is constructed based on the same relationship between each paralanguage feature in the paralanguage labels corresponding to the audio feature data samples and the text feature data samples. i , matrix G i The part with a value of 1 in indicates that the two samples at the corresponding position have the same label i, otherwise it is 0, thus reflecting the same relationship between the samples in different paralinguistic features. For example, the paralinguistic label corresponding to an audio feature data sample has the following features: emotion = happy, gender = male, and the paralinguistic label corresponding to a text feature data sample has the following features: emotion = happy, gender = female. Therefore, the corresponding value in the sub-true value matrix of the emotion feature is 1, and the corresponding value in the sub-true value matrix of the gender feature is 0;

[0067] S303) Weights are assigned according to the importance of each paralinguistic tag, and weights are assigned to each sub-truth matrix. In this embodiment, more basic tags such as gender and age are considered secondary tags and are assigned lower weights, while key features such as emotions and accents are considered primary features and are assigned higher weights. The weighted sub-truth matrices are then combined with the identity matrix to obtain an enhanced truth matrix, which is expressed as follows:

[0068]

[0069] Where I represents the identity matrix, G i represents the sub-truth value matrix corresponding to the i-th paralinguistic label, w i represents the importance distribution weight of the i-th paralanguage tag, and K represents the number of paralanguage tag types in the paralanguage dataset;

[0070] S303) Calculate the KL divergence between the similarity matrix and the enhanced true value matrix as the fine-grained loss, which is expressed as follows:

[0071] Loss fg =KL(S||G)

[0072] Among them, KL(S||G) represents the similarity matrix S and the enhanced true value matrix G new The KL divergence between .

[0073] This embodiment is based on the similarity matrix S and the enhanced true value matrix G newPerform loss calculation and use the KL divergence between the two as the loss to minimize the difference between the audio and the corresponding text feature representation, capture the similarity relationship between the samples in multiple paralinguistic features, and ensure high-precision recognition in fine-grained contrastive learning. When the fine-grained loss is subsequently used as the loss function to train the audio encoder and text encoder, the fine-grained loss Loss can be used. fg The gradients of the parameters of the audio and text encoders are calculated using a backpropagation algorithm. Based on the gradients, an optimization algorithm is used to update the parameters of both encoders. The goal of the parameter update is to minimize the loss function, thereby improving the alignment of audio and text features and the ability to recognize paralinguistic features. It should be noted that the specific process of calculating the gradients of the loss function with respect to the parameters of the two encoders and subsequently using these gradients to update the parameters of the two encoders is well known to those skilled in the art. The method of this embodiment does not involve improvements in this process, so the specific calculation process is not detailed here.

[0074] like Figure 3 As shown, in step S4 of this embodiment, a test audio is input into the audio encoder trained in step S3. Then, the style description texts in the test set are sequentially input into the text encoder trained in step S3. Next, the cosine similarity between the audio feature vectors extracted by the audio encoder and the feature vectors of each description text extracted by the text encoder is calculated. The text with the highest similarity is the style description (paralinguistic information) that matches the current test audio.

[0075] Example 2

[0076] This embodiment further provides a paralinguistic information recognition system based on fine-grained contrastive learning, comprising a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the paralinguistic information recognition method based on fine-grained contrastive learning described in the first embodiment.

[0077] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for identifying paralinguistic information based on fine-grained contrastive learning described in the first embodiment are implemented.

[0078] This embodiment further provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method for identifying paralinguistic information based on fine-grained contrastive learning described in the first embodiment are implemented.

[0079] In summary, this invention discloses a method and system for identifying paralinguistic information based on fine-grained contrastive learning. This fine-grained contrastive learning strategy enables the capture of more comprehensive and open paralinguistic information from speech. Furthermore, this invention improves the accuracy of speech style recognition through more detailed loss calculation, providing a more effective solution for speech style recognition.

[0080] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying paralinguistic information based on fine-grained contrastive learning, characterized in that: The following steps are involved: Generate natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, and then perform noise reduction processing; Using an audio encoder to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, and using a text encoder to extract features from the natural language description text after noise reduction to obtain text feature data samples; Performing fine-grained comparative learning on the audio feature data samples and the text feature data samples to obtain a similarity matrix of the audio feature data samples and the text feature data samples, as well as an enhanced true value matrix of the audio feature data samples and the text feature data samples, and then using the fine-grained loss of the similarity matrix and the enhanced true value matrix as a loss function to train the audio encoder and the text encoder; Use the trained audio encoder to extract features from the test audio data to obtain new audio feature data, and use the trained text encoder to extract features from the descriptive text data in the test set to obtain new text feature data. Calculate the similarity between the new audio feature data and the new text feature data, and select the descriptive text data with the highest similarity as the paralinguistic information of the test audio data.

2. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: When generating natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, it specifically includes: Input the initial prompt word and paralinguistic label into the large language model, obtain the initial description text output by the large language model and record it in the context variable; Input the content recorded in the context variable and the new prompt word into the large language model, obtain the new description text output by the large language model and record it in the context variable, and repeat this step until the description text output by the large language model meets the requirements.

3. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: When performing noise reduction processing, it includes: Classify the natural language description text into corresponding paralanguage labels to obtain different paralanguage label groups; For each paralinguistic label group, the similarity of all samples in the group is calculated, and samples whose similarity with other samples is lower than the preset lower limit are regarded as noise samples; The noise sample is input into the large language model to obtain a regenerated sample. The similarity between the regenerated sample and other samples in the corresponding paralinguistic label group is calculated. If the similarity with other samples is lower than the preset lower limit, the regenerated sample is input into the large language model again, and the similarity between the regenerated sample and other samples in the corresponding paralinguistic label group is calculated again. This step is repeated until the similarity with other samples exceeds the preset lower limit.

4. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: The audio feature data samples and the text feature data samples are high-dimensional feature vectors. Before performing fine-grained comparative learning on the audio feature data samples and the text feature data samples, the method further includes mapping the audio feature data samples and the text feature data samples into a shared embedding space to perform effective semantic alignment and paralinguistic feature learning.

5. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: When performing fine-grained comparative learning on audio feature data samples and text feature data samples, the following steps are included: Calculate the similarity matrix of the audio feature data sample and the text feature data sample; Constructing sub-true value matrices corresponding to each paralinguistic label from the audio feature data samples and the text feature data samples according to the corresponding paralinguistic labels, assigning weights to each sub-true value matrix according to the importance of each paralinguistic label, and then combining the weighted sub-true value matrices with the identity matrix to obtain an enhanced true value matrix; The KL divergence between the similarity matrix and the augmented ground truth matrix is ​​calculated as the fine-grained loss.

6. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 5, characterized in that: The similarity matrix expression is as follows: S=A·T Wherein, A represents the feature vector of the audio feature data sample, and T represents the feature vector of the text feature data sample.

7. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 5, characterized in that: The enhanced true value matrix expression is as follows: Where I represents the identity matrix, G i represents the sub-truth value matrix corresponding to the i-th paralinguistic label, w i represents the importance distribution weight of the i-th paralanguage tag, and K represents the number of paralanguage tag types in the paralanguage dataset.

8. A paralinguistic information recognition system based on fine-grained contrastive learning, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method for identifying paralinguistic information based on fine-grained contrastive learning according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying paralinguistic information based on fine-grained contrastive learning according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for identifying paralinguistic information based on fine-grained contrastive learning according to any one of claims 1 to 7 are implemented.

Images