Parabolingual information identification method and system based on fine-grained comparative learning

Abstract

The invention discloses a sublingual information identification method and system based on fine-grained comparative learning, and the method comprises the steps: generating different styles of natural language description texts according to sublingual tags of a sublingual data set, and then carrying out the noise reduction processing; after feature extraction is carried out by using an audio encoder and a text encoder, fine-grained contrast learning is carried out on audio feature data samples and text feature data samples, and fine-grained loss is used as a loss function to train the audio encoder and the text encoder; and performing feature extraction by using the trained audio encoder and text encoder, calculating the similarity between the new audio feature data and the new text feature data, and selecting the description text data with the highest similarity as the sublingual information of the test audio data. According to the method, more comprehensive and open side language information can be captured from voice, and compared with a traditional method, the accuracy of feature extraction is effectively improved.

Description

Technical Field

[0001] The present invention relates to a paralanguage information recognition technology, and in particular to a paralanguage information recognition method and system based on fine-grained contrastive learning. Background Art

[0002] Speech not only conveys explicit semantic information, but also contains rich paralinguistic features that provide deep insights into the speaker's intentions, emotions, and states. Paralinguistic features refer to those features that go beyond the pure speech and language content and can supplement or modify semantic information. According to their temporal properties, these features can be divided into long-term, medium-term, and short-term features according to their duration. For example, long-term features include gender, age, and dialect; medium-term features involve emotional state, stuttering, etc.; and short-term features include emotions and changes in speech rate. Extracting this paralinguistic information is crucial to enhancing the naturalness and intelligence of human-computer interaction because it enables machines to not only understand semantic content but also identify the speaker's underlying emotions and intentions. For example, intelligent systems can dynamically adjust the way they interact with users by identifying emotional information in speech, thereby providing a more personalized and context-aware experience.

[0003] Currently, many techniques for extracting paralinguistic information from speech are usually designed for specific tasks and often focus on identifying only a few predefined attributes. For example, speech emotion recognition is often treated as a classification problem, while the identification of long-term features such as gender and age is also common in single-task scenarios. Some studies have introduced transfer learning by using pre-trained models for other paralinguistic recognition tasks, such as using speaker recognition models for emotion detection. However, these methods usually operate under a closed set framework, that is, each model can only handle a fixed set of labels and tasks, which limits their ability to capture paralinguistic features in a wider range and complex speech scenarios. To address the limitations of the closed set, multi-task learning has been studied as a potential solution. Through multi-task learning, the model can simultaneously predict multiple paralinguistic features, such as attributes such as age, gender, and emotion, further expanding the ability of paralinguistic recognition. However, this approach also introduces new complexities as the number of tasks increases. Whenever new tasks or labels are added, the model architecture needs to be adjusted accordingly, resulting in poor scalability. In addition, multi-task learning still operates within a predefined label set, limiting the model's adaptability to identifying new paralinguistic features. Summary of the invention

[0004] The technical problem to be solved by the present invention is as follows: In view of the above-mentioned problems in the prior art, a method and system for identifying paralinguistic information based on fine-grained contrastive learning are provided, which can capture more comprehensive and open paralinguistic information from speech, and effectively improve the accuracy of feature extraction compared with traditional methods.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for identifying paralinguistic information based on fine-grained contrastive learning comprises the following steps:

[0007] Generate natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, and then perform noise reduction processing;

[0008] Using an audio encoder to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, and using a text encoder to extract features from the natural language description text after noise reduction to obtain text feature data samples;

[0009] Performing fine-grained comparative learning on the audio feature data samples and the text feature data samples to obtain a similarity matrix of the audio feature data samples and the text feature data samples, as well as an enhanced true value matrix of the audio feature data samples and the text feature data samples, and then using the fine-grained loss of the similarity matrix and the enhanced true value matrix as a loss function to train the audio encoder and the text encoder;

[0010] Use the trained audio encoder to extract features from the test audio data to obtain new audio feature data, and use the trained text encoder to extract features from the description text data in the test set to obtain new text feature data, calculate the similarity between the new audio feature data and the new text feature data, and select the description text data with the highest similarity as the paralinguistic information of the test audio data.

[0011] Furthermore, when generating natural language description texts of different styles according to the paralanguage labels of the paralanguage dataset, it specifically includes:

[0012] Input the initial prompt words and paralinguistic labels into the large language model, obtain the initial description text output by the large language model and record it into the context variable;

[0013] Input the content recorded in the context variable and the new prompt word into the large language model, obtain the new description text output by the large language model and record it in the context variable, and repeat this step until the description text output by the large language model meets the requirements.

[0014] Furthermore, when performing noise reduction processing, it includes:

[0015] Classify the natural language description text into corresponding paralanguage labels to obtain different paralanguage label groups;

[0016] For each paralinguistic label group, the similarity of all samples in the group is calculated, and the samples whose similarity with other samples is lower than the preset lower limit are regarded as noise samples;

[0017] The noise sample is input into the large language model to obtain a regenerated sample, and the similarity between the regenerated sample and other samples in the corresponding paralanguage label group is calculated. If the similarity with other samples is lower than the preset lower limit, the regenerated sample is input into the large language model, and the similarity between the regenerated sample and other samples in the corresponding paralanguage label group is calculated again. This step is repeated until the similarity with other samples is higher than the preset lower limit.

[0018] Furthermore, the audio feature data samples and the text feature data samples are high-dimensional feature vectors. Before performing fine-grained comparative learning on the audio feature data samples and the text feature data samples, it also includes: mapping the audio feature data samples and the text feature data samples to a shared embedding space to perform effective semantic alignment and paralinguistic feature learning.

[0019] Furthermore, when fine-grained comparative learning is performed on the audio feature data samples and the text feature data samples, the following steps are included:

[0020] Calculate the similarity matrix of the audio feature data sample and the text feature data sample;

[0021] The audio feature data samples and the text feature data samples are used to construct sub-true value matrices corresponding to each paralanguage label according to the corresponding paralanguage labels, and weights are assigned according to the importance of each paralanguage label to weight each sub-true value matrix, and then the weighted sub-true value matrix is ​​combined with the unit matrix to obtain an enhanced true value matrix;

[0022] The KL divergence between the similarity matrix and the augmented ground-truth matrix is ​​calculated as the fine-grained loss.

[0023] Furthermore, the similarity matrix expression is as follows:

[0024] S=A·T

[0025] Wherein, A represents the feature vector of the audio feature data sample, and T represents the feature vector of the text feature data sample.

[0026] Furthermore, the enhanced true value matrix expression is as follows:

[0027]

[0028] Where I represents the identity matrix, G i represents the sub-truth value matrix corresponding to the i-th paralinguistic label, w i represents the importance distribution weight of the i-th paralanguage tag, and K represents the number of paralanguage tag types in the paralanguage dataset.

[0029] The present invention also proposes a paralanguage information recognition system based on fine-grained contrastive learning, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement any step of the paralanguage information recognition method based on fine-grained contrastive learning.

[0030] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the steps of any one of the methods for identifying paralinguistic information based on fine-grained contrastive learning.

[0031] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods for identifying paralinguistic information based on fine-grained contrastive learning.

[0032] Compared with the prior art, the advantages of the present invention are:

[0033] 1. The present invention generates natural language description texts of different styles based on the paralanguage dataset and then performs noise reduction processing, thereby ensuring the sentence diversity and semantic consistency of the training text data, thereby ensuring the training effect of the model.

[0034] 2. The present invention can simultaneously capture the paralinguistic information in multiple audios by embedding the audio and the description text in a shared space and aligning them, thereby improving the universality of paralinguistic feature recognition in speech.

[0035] 3. By introducing a fine-grained contrast loss function, the present invention enables the model to better capture the detailed similarity relationship between samples and improve the discrimination ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of the principle of fine-grained contrastive learning according to an embodiment of the present invention.

[0038] Figure 3 The figure is a schematic diagram of the principle of performing paralanguage information recognition on test audio in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0040] Embodiment 1

[0041] In view of the limitations of the prior art in extracting paralinguistic information from speech, this embodiment proposes a paralinguistic information recognition method based on fine-grained contrastive learning, which can extract a wider range of paralinguistic information from speech and effectively improves the accuracy of feature extraction compared to traditional methods.

[0042] like Figure 1 As shown, the method of this embodiment includes the following steps:

[0043] S1) Prompt engineering and data denoising: Generate natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, and then perform denoising;

[0044] S2) Feature extraction: using an audio encoder to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, and using a text encoder to extract features from the natural language description text after noise reduction to obtain text feature data samples;

[0045] S3) Fine-grained contrastive learning: performing fine-grained contrastive learning on the audio feature data samples and the text feature data samples to obtain a similarity matrix of the audio feature data samples and the text feature data samples, and an enhanced true value matrix of the audio feature data samples and the text feature data samples, and then using the fine-grained loss of the similarity matrix and the enhanced true value matrix as a loss function to train the audio encoder and the text encoder;

[0046] S4) Identification of paralinguistic information in speech: Use the trained audio encoder to extract features from the test audio data to obtain new audio feature data, and use the trained text encoder to extract features from the description text data in the test set to obtain new text feature data, calculate the similarity between the new audio feature data and the new text feature data, and select the description text data with the highest similarity as the paralinguistic information of the test audio data.

[0047] Through the above steps, the audio encoder and text encoder can capture more comprehensive and open paralinguistic information from speech through fine-grained contrastive learning.

[0048] Each step is described in detail below.

[0049] In this embodiment, natural language description and speech of speaking style are used to train audio encoder and text encoder. Natural language description can effectively integrate multiple paralinguistic information and has stronger expressive ability. This method also promotes the expansion of the model to more open sets of labels and tasks. In order to ensure that the style description has higher diversity and accuracy, step S1 of this embodiment uses thirteen data containing multimodal paralinguistic information (IEMOCAP, MELD, MSP-Pocast, RAVDESS, SAVEE, ComparE2016-Deception, Prompt-Speech, SEP-28k, FluecyBank, TIMIT, Google-Accent, Archive, VCTK). The paralinguistic information contained in these data sets includes: emotion, speaking speed, fraud, stuttering, emotion, dialect, accent, gender and age, a total of nine directions. Based on these paralinguistic information, prompt engineering and data denoising are performed to generate text data of speaking style description to ensure the quality of input data of text modality.

[0050] Specifically, in step S1, when generating natural language description texts of different styles based on the paralanguage tags of the paralanguage dataset, a large language model (such as GPT-3.5-turbo) generates natural language descriptions based on the paralanguage tags, and introduces a simulated memory mechanism to track previous prompts and responses through context variables to ensure that the text generated by the large language model has more expression diversity and avoids generating monotonous descriptions. Specifically, the following steps are included:

[0051] S101) inputting the initial prompt word and the paralanguage tag into the large language model, obtaining the initial description text output by the large language model and recording it into the context variable;

[0052] S102) Input the content recorded in the context variable and the new prompt word into the large language model, obtain the new description text output by the large language model and record it into the context variable, and repeat this step until the description text output by the large language model meets the requirements. For example, in the process of generating natural language descriptions multiple times, each time the corresponding paralanguage tag context variable C and the prompt "Please use a different sentence structure and vary the description style from your previous responses when describing the speaker's speaking style" are input into the GPT-3.5-turbo API to generate a natural language description.

[0053] In step S1 of this embodiment, when performing noise reduction processing, specifically, after the generated description text is label-classified, outliers with low similarity in the same paralanguage label group are filtered out, including the following steps:

[0054] S201) Classifying the natural language description text into corresponding paralanguage tags to obtain different paralanguage tag groups. In this embodiment, in order to ensure the consistency between the generated text and the paralanguage tag, the Claude2.1 API is called to classify the description text generated by GPT-3.5-turbo to detect whether it can be classified into the paralanguage tag corresponding to the generated text;

[0055] S202) For each paralanguage tag group, the similarity of all samples in the group is calculated, and samples whose similarity with other samples is lower than a preset lower bound value are regarded as noise samples. In this embodiment, the SentenceBERT model is introduced to calculate the semantic similarity of the text, and the interquartile range (IQR) method is used to filter out outliers with low similarity in the same paralanguage tag group. The noise samples are samples whose similarity with other samples is lower than the lower bound Lower Bound = Q1-1.5×(Q3-Q1), where Q1 and Q3 are the 25th and 75th percentiles respectively;

[0056] S203) In order to ensure the semantic consistency of the generated texts of the same paralanguage tag category, the detected noise sample is input into the large language model for regeneration. Specifically, the noise sample is input into the large language model to obtain a regenerated sample, and the similarity between the regenerated sample and other samples in the corresponding paralanguage tag group is calculated. If the similarity with the other samples is lower than a preset lower limit, the regenerated sample is input into the large language model, and the similarity between the regenerated sample and other samples in the corresponding paralanguage tag group is calculated. This step is repeated until the similarity with the other samples is higher than the preset lower limit.

[0057] In this embodiment, a contrastive learning structure is adopted, and in step S2, the audio encoder f audio and the text encoder f text For audio data X a and text data X t Feature extraction is performed separately and they are mapped into a shared embedding space.

[0058] Specifically, in step S2, when the audio encoder is used to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, the audio data is obtained from multiple collected paralanguage datasets. Input audio encoder. The audio encoder in this embodiment adopts HTSAT model. The audio encoder (HTSAT model) receives audio input and converts it into a high-dimensional audio feature vector. For example, This high-dimensional audio feature vector is used to represent the sentence-level features of the input audio.

[0059] Similarly, in step S2, when the text encoder is used to extract the features of the natural language description text after the noise reduction process to obtain the text feature data sample, the description text obtained in step S1 is extracted. Input text encoder. The text encoder in this embodiment adopts RoBERTa model. The text encoder (RoBERTa model) converts the text input into a corresponding high-dimensional text feature vector, for example, This text feature vector represents the sentence-level features of the input text.

[0060] In step S2 of this embodiment, after obtaining the audio feature data sample and the text feature data sample, the feature vector of the audio feature data sample and the feature vector of the text feature data sample are mapped to a shared embedding space. In detail, both features are processed into 512-dimensional vectors to represent sentence-level features of two modalities for effective semantic alignment and paralinguistic feature learning. These features are compared and aligned in the same vector space to effectively capture the correlation between speech and text.

[0061] In step S3 of this embodiment, the similarity relationship between paralinguistic labels is further considered on the basis of traditional contrastive learning. On the basis of the original use of the unit matrix as the ground truth matrix, a sub ground truth matrix representing the relevance of paralinguistic information is added to form a new ground truth matrix together with the original unit matrix, so as to achieve more fine-grained loss calculation and improve the model's ability to recognize multiple paralinguistic features in speech.

[0062] Specifically, in step S3, when fine-grained comparative learning is performed on the audio feature data sample and the text feature data sample, Figure 2 As shown, the following steps are included:

[0063] S301) Calculate the similarity matrix of the audio feature data sample and the text feature data sample. In this embodiment, based on the audio embedding and text embedding in the embedding space, the similarity matrix between the samples is calculated to represent the similarity relationship between the audio and text samples. The expression is as follows:

[0064] S=A·T

[0065] Where A represents the feature vector of the audio feature data sample, and T represents the feature vector of the text feature data sample;

[0066] S302) constructing a sub-true value matrix corresponding to each paralanguage label from the audio feature data sample and the text feature data sample according to the corresponding paralanguage label. In this embodiment, when constructing a sub-true value matrix corresponding to each paralanguage label from the audio feature data sample and the text feature data sample according to the corresponding paralanguage label, the corresponding sub-true value matrix G is constructed according to the same relationship of each paralanguage feature in the paralanguage label corresponding to the audio feature data sample and the text feature data sample. i , the matrix G i The part with 1 in indicates that the two samples at the corresponding position are the same in label i, otherwise it is 0, thus reflecting the same relationship of the samples in different paralinguistic features. For example, the paralinguistic label corresponding to an audio feature data sample has the following features: emotion = happy, gender = male, and the paralinguistic label corresponding to a text feature data sample has the following features: emotion = happy, gender = female, so the corresponding value in the sub-true value matrix of the emotion feature is 1, and the corresponding value in the sub-true value matrix of the gender feature is 0;

[0067] S303) assigning weights according to the importance of each paralinguistic tag, and weighting each sub-truth value matrix. In this embodiment, more basic tags such as gender and age are regarded as secondary tags and are assigned lower weights, while key features such as emotions and accents are regarded as primary features and are assigned higher weights first. Then, the weighted sub-truth value matrices are combined with the identity matrix to obtain an enhanced truth value matrix, which is expressed as follows:

[0068]

[0069] Where I represents the identity matrix, G i represents the sub-truth value matrix corresponding to the i-th paralinguistic label, w i represents the importance distribution weight of the i-th paralanguage tag, K represents the number of paralanguage tag types in the paralanguage dataset;

[0070] S303) Calculate the KL divergence between the similarity matrix and the enhanced true value matrix as the fine-grained loss, and the expression is as follows:

[0071] Loss fg =KL(S||G)

[0072] Among them, KL(S||G) represents the similarity matrix S and the enhanced true value matrix G new The KL divergence between .

[0073] This embodiment is based on the similarity matrix S and the enhanced true value matrix G newLoss calculation is performed, and the KL divergence between the two is used as the loss to minimize the difference between the audio and the corresponding text feature representation, capture the similarity relationship between samples in multiple paralinguistic features, and ensure high-precision recognition in fine-grained contrastive learning. When the fine-grained loss is used as the loss function to train the audio encoder and the text encoder, the fine-grained loss Loss fg The gradients of the parameters of the audio encoder and the text encoder are calculated by the back propagation algorithm, and the parameters of the two encoders are updated using the optimization algorithm according to the gradients. The goal of the parameter update is to minimize the loss function, thereby improving the alignment of audio and text features and the recognition ability of paralinguistic features. It should be noted that the specific process of how to calculate the gradient of the loss function with respect to the parameters of the two encoders and how to use these gradients to update the parameters of the two encoders is well known to those skilled in the art. The method of this embodiment does not involve improvements in the relevant process, so the specific calculation process will not be repeated.

[0074] like Figure 3 As shown, in step S4 of this embodiment, a test audio is input into the audio encoder trained in step S3, and then the style description texts in the test set are sequentially input into the text encoder trained in step S3. Then, the cosine similarity between the audio feature vector extracted by the audio encoder and the feature vectors of each description text extracted by the text encoder is calculated. The text with the highest similarity is the style description (paralinguistic information) that matches the current test audio.

[0075] Embodiment 2

[0076] This embodiment further provides a paralinguistic information recognition system based on fine-grained contrastive learning, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the paralinguistic information recognition method based on fine-grained contrastive learning described in the first embodiment.

[0077] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for identifying paralinguistic information based on fine-grained contrastive learning described in the first embodiment are implemented.

[0078] This embodiment further provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method for identifying paralinguistic information based on fine-grained contrastive learning described in the first embodiment are implemented.

[0079] In summary, the present invention discloses a method and system for identifying paralinguistic information based on fine-grained contrastive learning, which uses a fine-grained contrastive learning strategy to capture more comprehensive and open paralinguistic information from speech. In addition, the present invention improves the accuracy of speech style recognition through more detailed loss calculation, providing a more effective solution for speech style recognition.

[0080] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for identifying paralinguistic information based on fine-grained contrastive learning, characterized in that: The following steps are involved: Generate natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, and then perform noise reduction processing; Using an audio encoder to extract features from the audio data of the paralanguage dataset to obtain audio feature data samples, and using a text encoder to extract features from the natural language description text after noise reduction to obtain text feature data samples; Performing fine-grained comparative learning on the audio feature data samples and the text feature data samples to obtain a similarity matrix of the audio feature data samples and the text feature data samples, as well as an enhanced true value matrix of the audio feature data samples and the text feature data samples, and then using the fine-grained loss of the similarity matrix and the enhanced true value matrix as a loss function to train the audio encoder and the text encoder; Use the trained audio encoder to extract features from the test audio data to obtain new audio feature data, and use the trained text encoder to extract features from the description text data in the test set to obtain new text feature data, calculate the similarity between the new audio feature data and the new text feature data, and select the description text data with the highest similarity as the paralinguistic information of the test audio data.

2. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: When generating natural language description texts of different styles based on the paralanguage labels of the paralanguage dataset, it specifically includes: Input the initial prompt words and paralinguistic labels into the large language model, obtain the initial description text output by the large language model and record it into the context variable; Input the content recorded in the context variable and the new prompt word into the large language model, obtain the new description text output by the large language model and record it in the context variable, and repeat this step until the description text output by the large language model meets the requirements.

3. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: When performing noise reduction processing, it includes: Classify the natural language description text into corresponding paralanguage labels to obtain different paralanguage label groups; For each paralinguistic label group, the similarity of all samples in the group is calculated, and the samples whose similarity with other samples is lower than the preset lower limit are regarded as noise samples; The noise sample is input into the large language model to obtain a regenerated sample, and the similarity between the regenerated sample and other samples in the corresponding paralanguage label group is calculated. If the similarity with other samples is lower than the preset lower limit, the regenerated sample is input into the large language model, and the similarity between the regenerated sample and other samples in the corresponding paralanguage label group is calculated again. This step is repeated until the similarity with other samples is higher than the preset lower limit.

4. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: The audio feature data samples and the text feature data samples are high-dimensional feature vectors. Before fine-grained comparative learning is performed on the audio feature data samples and the text feature data samples, the method further includes: mapping both the audio feature data samples and the text feature data samples to a shared embedding space to perform effective semantic alignment and paralinguistic feature learning.

5. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 1, characterized in that: When performing fine-grained comparative learning on audio feature data samples and text feature data samples, the following steps are included: Calculate the similarity matrix of the audio feature data sample and the text feature data sample; The audio feature data samples and the text feature data samples are used to construct sub-true value matrices corresponding to each paralanguage label according to the corresponding paralanguage labels, and weights are assigned according to the importance of each paralanguage label to weight each sub-true value matrix, and then the weighted sub-true value matrix is ​​combined with the unit matrix to obtain an enhanced true value matrix; The KL divergence between the similarity matrix and the augmented ground-truth matrix is ​​calculated as the fine-grained loss.

6. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 5, characterized in that: The similarity matrix expression is as follows: S=A·T Wherein, A represents the feature vector of the audio feature data sample, and T represents the feature vector of the text feature data sample.

7. The method for identifying paralinguistic information based on fine-grained contrastive learning according to claim 5, characterized in that: The enhanced true value matrix expression is as follows: Where I represents the identity matrix, G i represents the sub-truth value matrix corresponding to the i-th paralinguistic label, w i represents the importance distribution weight of the i-th paralanguage tag, and K represents the number of paralanguage tag types in the paralanguage dataset.

8. A paralinguistic information recognition system based on fine-grained contrastive learning, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the paralinguistic information recognition method based on fine-grained contrastive learning according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the paralinguistic information recognition method based on fine-grained contrastive learning according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the paralinguistic information recognition method based on fine-grained contrastive learning according to any one of claims 1 to 7 are implemented.

Images