Artificial intelligence-based pre-training optimization methods, devices, equipment, and media

Abstract

This application relates to the field of artificial intelligence technology, and particularly to a pre-training optimization method, apparatus, device, and medium based on artificial intelligence. The method uses a first sentiment prediction model to extract frame-level features from sentence-level speech. The sentence-level sentiment labels of the sentence-level speech are used as the sentiment categories of the frame-level features. The frame-level features are encoded and predicted, with the goal of minimizing the negative cosine similarity between the encoded and predicted results. A pre-set encoder is trained, and the frame-level features are input into the trained encoder to output updated frame-level features. All updated frame-level features are clustered, and pseudo-labels for the corresponding frame data are determined based on the clustering results. A second sentiment prediction model is trained based on this. By maximizing the similarity between features with the same label through the pre-set encoder and pre-set prediction network, and further strengthening the correlation between low-dimensional features and sentiment information through clustering, the accuracy of the model's sentiment prediction is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a pre-training optimization method, apparatus, device, and medium based on artificial intelligence. Background Technology

[0002] Currently, Speech Emotion Recognition (SER) is an emerging research direction in the field of digital speech signal processing, opening up new avenues for human-computer interaction and playing an important role in many scenarios. Call centers use SER technology to track customer emotions and provide better service; in the medical field, SER-based diagnostic systems can analyze the degree of depression and distress in patients; and many other applications also utilize efficient SER systems to improve their work efficiency.

[0003] Emotions in human voices are influenced by various factors, such as gender, age, speaker, dialect, and culture. Therefore, better modeling emotions has always been a key research focus. Currently, deep learning-based methods have become mainstream. Among them, self-supervised pre-trained models offer a high-performance solution. While pre-training can utilize large-scale heterogeneous datasets to obtain powerful and general-purpose models, the pre-training task is not entirely consistent with the target task—that is, the pre-training domain and the target domain differ—leading to less than ideal performance for specific tasks. In SER tasks, a large amount of unlabeled data is typically used for pre-training, requiring the pre-trained model to accurately extract emotional information from unlabeled data. Therefore, optimizing the pre-trained model to improve its accuracy in extracting emotional information from unlabeled data is a pressing issue. Summary of the Invention

[0004] In view of this, embodiments of this application provide a pre-training optimization method, apparatus, device, and medium based on artificial intelligence to solve the problem of how to pre-train and optimize a pre-trained model to improve the accuracy of the pre-trained model in extracting sentiment information from unlabeled data.

[0005] In a first aspect, embodiments of this application provide a pre-training optimization method based on artificial intelligence, the pre-training optimization method comprising:

[0006] For any sentence-level speech in the training set, the first sentiment prediction model is used to extract the frame-level features corresponding to each frame of data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the frame-level features to obtain the sentiment category corresponding to the frame-level features of all frame data in the training set.

[0007] For any sentiment category, all frame-level features belonging to the sentiment category are input into a preset encoder for encoding, and at least one frame-level feature is randomly selected from all frame-level features of the sentiment category and input into a preset prediction network for prediction.

[0008] With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, the preset encoder is trained to obtain the trained preset encoder. All frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated frame-level features are output to obtain the updated frame-level features of all frame data in the training set.

[0009] Based on the updated frame-level features and corresponding sentiment categories of all frame data, cluster all updated frame-level features, and update the sentiment categories corresponding to all updated frame-level features according to the clustering results, and determine the updated results as pseudo-labels for the corresponding frame data.

[0010] Based on the pseudo-labels of all frame data in the training set, the second sentiment prediction model is trained using the training set to obtain a pre-trained second sentiment prediction model. Both the first sentiment prediction model and the second sentiment prediction model have time-step aligned feature encoders.

[0011] In one embodiment, based on the updated frame-level features and corresponding sentiment categories of all frame data, all updated frame-level features are clustered, and the sentiment categories corresponding to all updated frame-level features are updated according to the clustering results. Determining the updated results as pseudo-labels for the corresponding frame data includes:

[0012] Cluster the frame-level features of all emotion categories to obtain N cluster centers, where N is a positive integer;

[0013] The mean of all frame-level features belonging to the same sentiment category is used as the anchor point to obtain M anchor points. The distance between each cluster center point and each anchor point is calculated, where M is a positive integer.

[0014] For any cluster center point, the anchor point closest to the cluster center point is determined as the target anchor point. When the target anchor point meets the preset conditions, the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo-label of all frame-level features within the cluster center point, thus obtaining the pseudo-labels of all frame data in the training set.

[0015] In one embodiment, after calculating the distances between all cluster centers and each anchor point, the method further includes:

[0016] For any cluster center point, determine all anchor points whose distance to the cluster center point is less than a certain distance threshold;

[0017] Determining the anchor point closest to the cluster center as the target anchor point includes:

[0018] From all anchor points whose distance to the cluster center is less than the first distance threshold, determine the anchor point that is closest to the cluster center as the target anchor point.

[0019] In one embodiment, for any cluster center point, the anchor point closest to the cluster center point is determined as the target anchor point. When the target anchor point meets a preset condition, the sentence-level sentiment tag corresponding to the target anchor point is used as a pseudo-tag for all frame-level features within the cluster center point, including:

[0020] For any cluster center point, determine the anchor point that is closest to the cluster center point as the target anchor point;

[0021] Detect whether the distance between the target anchor point and the cluster center point is less than a second distance threshold;

[0022] If the distance between the target anchor point and the cluster center point is less than the second distance threshold, then the target anchor point is determined to meet the preset conditions, and the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo tag for all frame-level features within the cluster center point.

[0023] In one embodiment, after detecting whether the distance between the target anchor point and the cluster center point is less than a second distance threshold, the method further includes:

[0024] If the distance between the target anchor point and the cluster center point is not less than the second distance threshold, it is determined that the target anchor point does not meet the preset conditions, other types of anchor points are created, and the sentence-level sentiment tags defined by the other types of anchor points are used as pseudo-tags for all frame-level features within the cluster center point.

[0025] In one embodiment, after training the second sentiment prediction model using the training set based on the pseudo-labels of all frame data in the training set to obtain a pre-trained second sentiment prediction model, the method further includes:

[0026] For any sentence-level speech in the training set, the pre-trained second sentiment prediction model is used to extract the pre-trained frame-level features corresponding to each frame of data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the pre-trained frame-level features to obtain the sentiment category corresponding to the pre-trained frame-level features of all frames of data in the training set.

[0027] For any sentiment category, all pre-trained frame-level features belonging to the sentiment category are input into a preset encoder for encoding, and at least one frame-level feature is randomly selected from all pre-trained frame-level features of the sentiment category and input into a preset prediction network for prediction.

[0028] With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, the preset encoder is trained to obtain the trained preset encoder. All pre-trained frame-level features belonging to the emotion category are input into the trained preset encoder, and the corresponding updated pre-trained frame-level features are output to obtain the updated pre-trained frame-level features of all frame data in the training set.

[0029] Based on the updated pre-trained frame-level features and corresponding sentiment categories of all frame data, cluster all updated pre-trained frame-level features, and update the sentiment categories corresponding to all updated pre-trained frame-level features according to the clustering results, and determine the updated results as the pre-trained pseudo-labels of the corresponding frame data.

[0030] Based on the pre-trained pseudo-labels of all frame data in the training set, the second sentiment prediction model is trained using the training set to obtain the trained second sentiment prediction model.

[0031] In one embodiment, the first sentiment prediction model includes a first feature encoder, a bidirectional LSTM layer, and a first fully connected layer. The first feature encoder is composed of CNN layers. The second sentiment prediction model includes a second feature encoder, a Transformer layer, and a second fully connected layer. The second feature encoder is composed of CNN layers. The number of CNN layers in the first feature encoder is the same as the number of CNN layers in the second feature encoder.

[0032] Secondly, embodiments of this application provide a pre-training optimization device based on artificial intelligence, the pre-training optimization device comprising:

[0033] The first training module is used to extract the frame-level features corresponding to each frame of data in any sentence-level speech in the training set using the first sentiment prediction model, and use the sentence-level sentiment label of the sentence-level speech as the sentiment category of the frame-level features to obtain the sentiment category corresponding to the frame-level features of all frame data in the training set.

[0034] The encoding prediction module is used to input all frame-level features belonging to any sentiment category into a preset encoder for encoding, and randomly select at least one frame-level feature from all frame-level features of the sentiment category into a preset prediction network for prediction.

[0035] The feature update module is used to train the preset encoder with the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, and to obtain the trained preset encoder. All frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated frame-level features are output to obtain the updated frame-level features of all frame data in the training set.

[0036] The pseudo-label determination module is used to cluster all updated frame-level features based on the updated frame-level features and corresponding sentiment categories of all frame data, and update the sentiment categories corresponding to all updated frame-level features based on the clustering results, and determine the updated results as pseudo-labels for the corresponding frame data.

[0037] The second training module is used to train the second sentiment prediction model using the pseudo-labels of all frame data in the training set, so as to obtain a pre-trained second sentiment prediction model. Both the first sentiment prediction model and the second sentiment prediction model have time-step aligned feature encoders.

[0038] In one embodiment, the pseudo-tag determination module includes:

[0039] The feature clustering unit is used to cluster the frame-level features of all emotion categories to obtain N cluster centers, where N is an integer greater than zero.

[0040] The distance calculation unit is used to take the mean of all frame-level features belonging to the same sentiment category as anchor points to obtain M anchor points, and calculate the distance between each cluster center point and each anchor point, where M is an integer greater than zero;

[0041] The pseudo-label determination unit is used to determine the anchor point closest to any cluster center point as the target anchor point. When the target anchor point meets the preset conditions, the sentence-level sentiment label corresponding to the target anchor point is used as the pseudo-label of all frame-level features within the cluster center point, thereby obtaining the pseudo-labels of all frame data in the training set.

[0042] In one embodiment, the pseudo-tag determination module further includes:

[0043] Anchor point filtering unit is used to, after calculating the distance between each anchor point and all cluster centers, determine all anchor points whose distance to the cluster center is less than a certain distance threshold for any given cluster center.

[0044] The pseudo-tag determination unit includes:

[0045] The first target anchor point determination subunit is used to determine the anchor point closest to the cluster center point from all anchor points whose distance to the cluster center point is less than the first distance threshold as the target anchor point.

[0046] In one embodiment, the pseudo-tag determination unit includes:

[0047] The second target anchor point determination subunit is used to determine the anchor point closest to any cluster center point as the target anchor point;

[0048] A distance detection subunit is used to detect whether the distance between the target anchor point and the cluster center point is less than a second distance threshold.

[0049] The first pseudo-label determination subunit is used to determine that the target anchor point meets the preset condition if the distance between the target anchor point and the cluster center point is less than the second distance threshold, and to use the sentence-level sentiment label corresponding to the target anchor point as the pseudo-label of all frame-level features within the cluster center point.

[0050] In one embodiment, the pseudo-tag determination unit further includes:

[0051] The second pseudo-label determination subunit is used to determine that the target anchor does not meet the preset conditions after detecting whether the distance between the target anchor and the cluster center is less than the second distance threshold, and if it is detected that the distance between the target anchor and the cluster center is not less than the second distance threshold, then other types of anchors are created, and the sentence-level sentiment labels defined by the other types of anchors are used as pseudo-labels for all frame-level features within the cluster center.

[0052] In one embodiment, the pre-training optimization device further includes:

[0053] The fine-tuning module is specifically used for:

[0054] Based on the pseudo-labels of all frame data in the training set, the second sentiment prediction model is trained using the training set to obtain a pre-trained second sentiment prediction model. Then, for any sentence-level speech in the training set, the pre-trained second sentiment prediction model is used to extract the pre-trained frame-level features corresponding to each frame data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the pre-trained frame-level features to obtain the sentiment category corresponding to the pre-trained frame-level features of all frame data in the training set.

[0055] For any sentiment category, all pre-trained frame-level features belonging to the sentiment category are input into a preset encoder for encoding, and at least one frame-level feature is randomly selected from all pre-trained frame-level features of the sentiment category and input into a preset prediction network for prediction.

[0056] With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, the preset encoder is trained to obtain the trained preset encoder. All pre-trained frame-level features belonging to the emotion category are input into the trained preset encoder, and the corresponding updated pre-trained frame-level features are output to obtain the updated pre-trained frame-level features of all frame data in the training set.

[0057] Based on the updated pre-trained frame-level features and corresponding sentiment categories of all frame data, cluster all updated pre-trained frame-level features, and update the sentiment categories corresponding to all updated pre-trained frame-level features according to the clustering results, and determine the updated results as the pre-trained pseudo-labels of the corresponding frame data.

[0058] Based on the pre-trained pseudo-labels of all frame data in the training set, the second sentiment prediction model is trained using the training set to obtain the trained second sentiment prediction model.

[0059] In one embodiment, the first sentiment prediction model includes a first feature encoder, a bidirectional LSTM layer, and a first fully connected layer. The first feature encoder is composed of CNN layers. The second sentiment prediction model includes a second feature encoder, a Transformer layer, and a second fully connected layer. The second feature encoder is composed of CNN layers. The number of CNN layers in the first feature encoder is the same as the number of CNN layers in the second feature encoder.

[0060] Thirdly, embodiments of this application provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the pre-training optimization method as described in the first aspect.

[0061] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the pre-training optimization method as described in the first aspect.

[0062] The beneficial effects of this application embodiment compared with the prior art are as follows: For any sentence-level speech in the training set, this application uses a first sentiment prediction model to extract the frame-level features corresponding to each frame of data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the frame-level features to obtain the sentiment categories corresponding to the frame-level features of all frame data in the training set. For any sentiment category, all frame-level features belonging to the sentiment category are input into a preset encoder for encoding. At least one frame-level feature is randomly selected from all frame-level features of the sentiment category and input into a preset prediction network for prediction. The preset encoder is trained with the goal of minimizing the negative cosine similarity between the encoded result and the predicted result. The trained preset encoder is then obtained. All frame-level features belonging to the sentiment category are input into the trained preset encoder. The corresponding updated frame-level features are obtained, and the updated frame-level features of all frame data in the training set are obtained. Based on the updated frame-level features of all frame data and the corresponding sentiment categories, all updated frame-level features are clustered, and the sentiment categories corresponding to all updated frame-level features are updated according to the clustering results. The updated results are determined as pseudo-labels of the corresponding frame data. Based on the pseudo-labels of all frame data in the training set, the second sentiment prediction model is trained using the training set to obtain a pre-trained second sentiment prediction model. The similarity between features under the same label is maximized by a preset encoder and a preset prediction network. Clustering is used to further strengthen the correlation between low-dimensional features and sentiment information. The clustering distribution is then used as the sentiment pseudo-label, and the model is trained accordingly to improve the accuracy of the model in predicting sentiment information. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of an application environment for a pre-training optimization method based on artificial intelligence provided in Embodiment 1 of this application;

[0065] Figure 2 This is a flowchart illustrating a pre-training optimization method based on artificial intelligence provided in Embodiment 2 of this application;

[0066] Figure 3 This is a schematic diagram of the structure of a pre-training optimization device based on artificial intelligence provided in Embodiment 3 of this application;

[0067] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of this application. Detailed Implementation

[0068] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0069] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0070] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0071] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0072] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0073] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0074] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0075] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0076] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0077] To illustrate the technical solution of this application, specific embodiments are described below.

[0078] The pre-training optimization method for a sentiment prediction model provided in Embodiment 1 of this application can be applied to, for example, Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0079] See Figure 2 This is a flowchart illustrating a pre-training optimization method for a sentiment prediction model provided in Embodiment 2 of this application. The aforementioned pre-training optimization method for the sentiment prediction model is applied to... Figure 1 The server-side computer connects to a corresponding database to retrieve training data. This computer can also connect to a client-side computer, which is operated by the user. The user can then provide the training set to the server through the client. For example... Figure 2 As shown, the pre-training optimization method for this sentiment prediction model may include the following steps:

[0080] Step S201: For any sentence-level speech in the training set, use the first sentiment prediction model to extract the frame-level features corresponding to each frame of data in the sentence-level speech, and use the sentence-level sentiment label of the sentence-level speech as the sentiment category of the frame-level features to obtain the sentiment category corresponding to the frame-level features of all frame data in the training set.

[0081] In this application, the first sentiment prediction model can be a wav2vec model, which includes a feature encoder, a bidirectional LSTM layer and a fully connected layer, wherein the softmax function can be used for normalization in the fully connected layer.

[0082] The feature encoder is set up to align with the second sentiment prediction model in terms of time steps. If the second sentiment prediction model uses the wav2vec2.0 model, which contains a feature encoder consisting of a multi-layer CNN network, then the feature encoder in the wav2vec model also needs to have a CNN network with the same number of layers.

[0083] Frame-level features are the features of each frame of data. That is, the sentence-level speech is segmented into frames of speech, and features are extracted from each frame of speech. Each sentence-level speech corresponds to a sentiment label, and the sentiment category corresponding to the features of all frames of speech in the sentence-level speech is the sentiment label of the sentence-level speech.

[0084] Step S202: For any sentiment category, input all frame-level features belonging to the sentiment category into a preset encoder for encoding, and randomly select at least one frame-level feature from all frame-level features of the sentiment category into a preset prediction network for prediction.

[0085] This application provides a Simsiamese module, which includes an encoder f() and a prediction network h(). Frame-level features with the same sentiment category are input into the encoder, and at least one frame-level feature is selected from these features and input into the prediction network to obtain the re-encoded result z of the frame-level features. n and prediction results p m .

[0086] Step S203: With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, a preset encoder is trained to obtain a trained preset encoder. All frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated frame-level features are output to obtain the updated frame-level features of all frame data in the training set.

[0087] In this application, the encoder consists of a ResNet and a three-layer fully connected network, while the prediction network is a two-layer fully connected network. The corresponding calculation of negative cosine similarity is as follows:

[0088]

[0089] The preset encoder is trained by minimizing the negative cosine similarity. This trained encoder is then used to re-encode the frame-level features to obtain updated frame-level features. Using the Simsiamese module described above can improve the similarity of frame-level features within the same sentiment category, thereby enhancing the clustering effect and improving the accuracy and efficiency of subsequent clustering.

[0090] Step S204: Based on the updated frame-level features and corresponding sentiment categories of all frame data, cluster all updated frame-level features, and update the sentiment categories corresponding to all updated frame-level features according to the clustering results, and determine the updated results as pseudo-labels for the corresponding frame data.

[0091] Optionally, based on the updated frame-level features and corresponding sentiment categories of all frame data, cluster all updated frame-level features, and update the sentiment categories corresponding to all updated frame-level features according to the clustering results. The pseudo-labels for the corresponding frame data are determined by the updated results, including:

[0092] Cluster the frame-level features of all emotion categories to obtain N cluster centers, where N is a positive integer;

[0093] The mean of all frame-level features belonging to the same sentiment category is used as the anchor point to obtain M anchor points. The distance between each cluster center point and each anchor point is calculated, where M is a positive integer.

[0094] For any cluster center point, the anchor point closest to the cluster center point is determined as the target anchor point. When the target anchor point meets the preset conditions, the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo-label of all frame-level features within the cluster center point, thus obtaining the pseudo-labels of all frame data in the training set.

[0095] In this application, an improved K-means clustering algorithm is used to cluster frame-level features into N clusters. The cluster center point corresponds to the mean of all frame-level features in each cluster. Since the extracted frame-level features can already represent certain sentiment information, the mean of all frame-level features within the same sentiment category is extracted as anchor points, resulting in M ​​anchor points, where M represents the total number of sentiment categories. Using the clustering results described above, the distance between the cluster center point of each cluster and each anchor point is calculated.

[0096] Optionally, after calculating the distances between all cluster centers and each anchor point, the method also includes:

[0097] For any cluster center point, determine all anchor points whose distance to the cluster center point is less than the first distance threshold;

[0098] Determining the target anchor point as the anchor point that is closest to the cluster center includes:

[0099] From all anchor points whose distance to the cluster center is less than the first distance threshold, determine the anchor point that is closest to the cluster center as the target anchor point.

[0100] Optionally, for any cluster center point, the anchor point closest to the cluster center point is determined as the target anchor point. When the target anchor point meets preset conditions, the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo-label for all frame-level features within the cluster center point, including:

[0101] For any cluster center, determine the anchor point that is closest to the cluster center as the target anchor point;

[0102] Detect whether the distance between the target anchor point and the cluster center point is less than the second distance threshold;

[0103] If the distance between the target anchor point and the cluster center point is less than the second distance threshold, the target anchor point is determined to meet the preset conditions, and the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo-tag of all frame-level features within the cluster center point.

[0104] Optionally, after detecting whether the distance between the target anchor point and the cluster center point is less than the second distance threshold, the method further includes:

[0105] If the distance between the target anchor point and the cluster center point is not less than the second distance threshold, it is determined that the target anchor point does not meet the preset conditions. Other anchor points are created, and the sentence-level sentiment tags defined by the other anchor points are used as pseudo-tags for all frame-level features within the cluster center point.

[0106] Where the distance d between the cluster center and the anchor point ij If the distance is less than or equal to γ, then the pseudo-label of this cluster center point is mapped to the sentiment category corresponding to the anchor point, where γ represents the preset threshold, i∈(0,M), j∈(0,N); if the distance is greater than γ, then a new other class anchor point is created as a unified category representation for non-sentiment categories, and finally there are a total of M+1 pseudo-labels.

[0107] Step S205: Based on the pseudo-labels of all frame data in the training set, train the second sentiment prediction model using the training set to obtain the pre-trained second sentiment prediction model.

[0108] In this application, both the first sentiment prediction model and the second sentiment prediction model have time-step aligned feature encoders.

[0109] Optionally, the first sentiment prediction model includes a first feature encoder, a bidirectional LSTM layer, and a first fully connected layer, wherein the first feature encoder is composed of CNN layers; the second sentiment prediction model includes a second feature encoder, a Transformer layer, and a second fully connected layer, wherein the second feature encoder is composed of CNN layers; the number of CNN layers in the first feature encoder is the same as the number of CNN layers in the second feature encoder; before extracting the frame-level features corresponding to each frame of data in any sentence-level speech in the training set using the first sentiment prediction model, the following steps are also included:

[0110] The first sentiment prediction model is trained using a feature set, which includes sentence-level speech samples and their corresponding sentiment labels and sentiment category labels.

[0111] The output of the first fully connected layer is normalized using the softmax function to obtain the predicted value;

[0112] The first sentiment prediction model is trained by repeating the process of using the feature set to measure loss using the first cross-entropy function until the loss measurement result meets the preset conditions, thus obtaining the trained first sentiment prediction model. The first cross-entropy function L... g include:

[0113]

[0114] In the formula, Z represents the total number of samples, C represents the total number of emotion categories, and y i c represents the sentiment category label corresponding to sample i. j Representing sentiment labels, p(c j |X i ) represents the corresponding input feature x i c j The predicted posterior probability of the class.

[0115] The second feature encoder consists of 7 CNN layers, then obtains the context representation through the Transformer layer, and then predicts the pseudo-label category of the occluded part through a linear multi-head prediction consisting of two fully connected layers.

[0116] Optionally, the second fully connected layer consists of two fully connected layers. Based on the pseudo-labels of all frames in the training set, the second sentiment prediction model is trained using the training set, resulting in a pre-trained second sentiment prediction model including:

[0117] The second sentiment prediction model is trained using all frames in the training set and their corresponding pseudo-labels.

[0118] The second sentiment prediction model is trained by repeatedly using all frames in the training set and their corresponding pseudo-labels, with the loss measured by the second cross-entropy function, until the loss measurement result meets the preset conditions, thus obtaining the pre-trained second sentiment prediction model. The second cross-entropy function L... v include:

[0119]

[0120] In the formula, This represents the low-dimensional features encoded and output by the feature encoder on the frame data, t represents the masked portion masked by the second fully connected layer, and z represents the low-dimensional features encoded by the feature encoder. t This represents the contextual representation of the occluded portion extracted using the Transformer layer. This represents the posterior probability prediction of the occluded part of the context.

[0121] Optionally, after training the second sentiment prediction model using the pseudo-labels of all frames in the training set to obtain a pre-trained second sentiment prediction model, the process also includes:

[0122] For any sentence-level speech in the training set, the pre-trained second sentiment prediction model is used to extract the pre-trained frame-level features corresponding to each frame of data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the pre-trained frame-level features to obtain the sentiment category corresponding to the pre-trained frame-level features of all frames of data in the training set.

[0123] For any sentiment category, all pre-trained frame-level features belonging to the sentiment category are input into a preset encoder for encoding, and at least one frame-level feature is randomly selected from all pre-trained frame-level features of the sentiment category and input into a preset prediction network for prediction.

[0124] With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, a preset encoder is trained to obtain a trained preset encoder. All pre-trained frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated pre-trained frame-level features are output to obtain the updated pre-trained frame-level features of all frame data in the training set.

[0125] Based on the updated pre-trained frame-level features and corresponding sentiment categories of all frame data, cluster all updated pre-trained frame-level features, and update the sentiment categories corresponding to all updated pre-trained frame-level features according to the clustering results, and determine the updated results as the pre-trained pseudo-labels of the corresponding frame data.

[0126] Based on the pre-trained pseudo-labels of all frames in the training set, the second sentiment prediction model is trained using the training set to obtain the trained second sentiment prediction model.

[0127] Since the pseudo-label category is related to the sentiment category, this method can focus on predicting the sentiment information of the masked sequence. After pre-training, wav2vec2.0 can be directly used to replace wav2vec, and the pre-trained second sentiment prediction model can be fine-tuned according to steps S201 to S205 to obtain the trained second sentiment prediction model.

[0128] In this embodiment, for any sentence-level speech in the training set, a first sentiment prediction model is used to extract frame-level features corresponding to each frame of data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the frame-level features to obtain the sentiment categories corresponding to the frame-level features of all frames in the training set. For any sentiment category, all frame-level features belonging to the sentiment category are input into a preset encoder for encoding. At least one frame-level feature is randomly selected from all frame-level features of the sentiment category and input into a preset prediction network for prediction. The preset encoder is trained with the goal of minimizing the negative cosine similarity between the encoded result and the predicted result. The trained preset encoder is then input into the trained preset encoder, and the corresponding updated frame-level features are output. The updated frame-level features of all frames in the training set are obtained. Based on the updated frame-level features and their corresponding sentiment categories, all updated frame-level features are clustered, and the sentiment categories corresponding to all updated frame-level features are updated according to the clustering results. The updated results are determined as pseudo-labels for the corresponding frame data. Based on the pseudo-labels of all frames in the training set, the second sentiment prediction model is trained using the training set to obtain a pre-trained second sentiment prediction model. The similarity between features with the same label is maximized by a preset encoder and a preset prediction network. Clustering is used to further strengthen the correlation between low-dimensional features and sentiment information. The clustering distribution is then used as the sentiment pseudo-label and trained accordingly, thereby improving the model's accuracy in predicting sentiment information.

[0129] Corresponding to the pre-training optimization method of the sentiment prediction model in the above embodiment, Figure 3 This diagram illustrates the structural block diagram of the pre-training optimization device for the sentiment prediction model provided in Embodiment 3 of this application. The pre-training optimization device is applied to... Figure 1 The server-side computer device connects to a corresponding database to obtain training data. This computer device can also connect to a client-side computer device, which is operated by the user. The user can provide the training set to the server through the client. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0130] See Figure 3 The pre-trained optimization device includes:

[0131] The first training module 31 is used to extract the frame-level features corresponding to each frame of data in any sentence-level speech in the training set using the first sentiment prediction model, and use the sentence-level sentiment label of the sentence-level speech as the sentiment category of the frame-level features to obtain the sentiment category corresponding to the frame-level features of all frame data in the training set.

[0132] The encoding prediction module 32 is used to input all frame-level features belonging to any sentiment category into a preset encoder for encoding, and randomly select at least one frame-level feature from all frame-level features of the sentiment category into a preset prediction network for prediction.

[0133] The feature update module 33 is used to train a preset encoder with the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, and to obtain the trained preset encoder. All frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated frame-level features are output to obtain the updated frame-level features of all frame data in the training set.

[0134] The pseudo-label determination module 34 is used to cluster all updated frame-level features based on the updated frame-level features and corresponding sentiment categories of all frame data, and update the sentiment categories corresponding to all updated frame-level features based on the clustering results, and determine the updated results as pseudo-labels of the corresponding frame data.

[0135] The second training module 35 is used to train the second sentiment prediction model using the pseudo-labels of all frame data in the training set, so as to obtain a pre-trained second sentiment prediction model. Both the first sentiment prediction model and the second sentiment prediction model have time step aligned feature encoders.

[0136] Optionally, the pseudo-label determination module 34 includes:

[0137] The feature clustering unit is used to cluster the frame-level features of all emotion categories to obtain N cluster centers, where N is an integer greater than zero.

[0138] The distance calculation unit is used to take the mean of all frame-level features belonging to the same sentiment category as anchor points to obtain M anchor points, and calculate the distance between each cluster center point and each anchor point, where M is an integer greater than zero;

[0139] The pseudo-label determination unit is used to determine the anchor point closest to any cluster center point as the target anchor point. When the target anchor point meets the preset conditions, the sentence-level sentiment label corresponding to the target anchor point is used as the pseudo label of all frame-level features within the cluster center point, thus obtaining the pseudo labels of all frame data in the training set.

[0140] Optionally, the pseudo-label determination module 34 also includes:

[0141] Anchor point filtering unit is used to determine all anchor points whose distance to each cluster center point is less than a certain distance threshold after calculating the distance between each cluster center point and each anchor point.

[0142] The pseudo-label determination unit includes:

[0143] The first target anchor point determination subunit is used to determine the anchor point closest to the cluster center point from all anchor points whose distance to the cluster center point is less than a first distance threshold.

[0144] Optionally, the pseudo-label determination unit includes:

[0145] The second target anchor point determination subunit is used to determine the anchor point closest to any cluster center point as the target anchor point;

[0146] The distance detection subunit is used to detect whether the distance between the target anchor point and the cluster center point is less than the second distance threshold.

[0147] The first pseudo-label determination subunit is used to determine that the target anchor point meets the preset conditions if the distance between the target anchor point and the cluster center point is less than the second distance threshold, and to use the sentence-level sentiment label corresponding to the target anchor point as the pseudo-label of all frame-level features within the cluster center point.

[0148] Optionally, the pseudo-label determination unit also includes:

[0149] The second pseudo-label determination subunit is used to determine whether the distance between the target anchor point and the cluster center point is less than the second distance threshold after detecting whether the distance between the target anchor point and the cluster center point is not less than the second distance threshold. If it is detected that the distance between the target anchor point and the cluster center point is not less than the second distance threshold, the target anchor point is determined to not meet the preset condition, other types of anchor points are created, and the sentence-level sentiment labels defined by the other types of anchor points are used as pseudo-labels for all frame-level features within the cluster center point.

[0150] Optionally, the pre-trained optimization device also includes:

[0151] The fine-tuning module is specifically used for:

[0152] Based on the pseudo-labels of all frames in the training set, the second sentiment prediction model is trained using the training set to obtain a pre-trained second sentiment prediction model. Then, for any sentence-level speech in the training set, the pre-trained second sentiment prediction model is used to extract the pre-trained frame-level features corresponding to each frame of speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the pre-trained frame-level features to obtain the sentiment category corresponding to the pre-trained frame-level features of all frames in the training set.

[0153] For any sentiment category, all pre-trained frame-level features belonging to the sentiment category are input into a preset encoder for encoding, and at least one frame-level feature is randomly selected from all pre-trained frame-level features of the sentiment category and input into a preset prediction network for prediction.

[0154] With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, a preset encoder is trained to obtain a trained preset encoder. All pre-trained frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated pre-trained frame-level features are output to obtain the updated pre-trained frame-level features of all frame data in the training set.

[0155] Based on the updated pre-trained frame-level features and corresponding sentiment categories of all frame data, cluster all updated pre-trained frame-level features, and update the sentiment categories corresponding to all updated pre-trained frame-level features according to the clustering results, and determine the updated results as the pre-trained pseudo-labels of the corresponding frame data.

[0156] Based on the pre-trained pseudo-labels of all frames in the training set, the second sentiment prediction model is trained using the training set to obtain the trained second sentiment prediction model.

[0157] Optionally, the first sentiment prediction model includes a first feature encoder, a bidirectional LSTM layer, and a first fully connected layer. The first feature encoder is composed of CNN layers. The second sentiment prediction model includes a second feature encoder, a Transformer layer, and a second fully connected layer. The second feature encoder is composed of CNN layers. The number of CNN layers in the first feature encoder is the same as the number of CNN layers in the second feature encoder.

[0158] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0159] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of this application. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, wherein the processor executes the computer program to implement the steps in the pre-training optimization method embodiments of any of the above sentiment prediction models.

[0160] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0161] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0162] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0164] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a computer device, it enables the computer device to execute the steps in the above method embodiments.

[0165] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0166] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A pre-training optimization method based on artificial intelligence, characterized in that, The pre-training optimization method includes: For any sentence-level speech in the training set, the first sentiment prediction model is used to extract the frame-level features corresponding to each frame of data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the frame-level features to obtain the sentiment category corresponding to the frame-level features of all frame data in the training set. For any sentiment category, all frame-level features belonging to the sentiment category are input into a preset encoder for encoding, and at least one frame-level feature is randomly selected from all frame-level features of the sentiment category and input into a preset prediction network for prediction. With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, the preset encoder is trained to obtain the trained preset encoder. All frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated frame-level features are output to obtain the updated frame-level features of all frame data in the training set. Based on the updated frame-level features and corresponding sentiment categories of all frame data, cluster all updated frame-level features, and update the sentiment categories corresponding to all updated frame-level features according to the clustering results, and determine the updated results as pseudo-labels for the corresponding frame data. Based on the pseudo-labels of all frame data in the training set, the second sentiment prediction model is trained using the training set to obtain a pre-trained second sentiment prediction model. Both the first sentiment prediction model and the second sentiment prediction model have time-step aligned feature encoders.

2. The pre-training optimization method according to claim 1, characterized in that, Based on the updated frame-level features and corresponding sentiment categories of all frame data, cluster all updated frame-level features, and update the sentiment categories corresponding to all updated frame-level features according to the clustering results. The updated results are determined as pseudo-labels for the corresponding frame data, including: Cluster the frame-level features of all emotion categories to obtain N cluster centers, where N is a positive integer; The mean of all frame-level features belonging to the same sentiment category is used as the anchor point to obtain M anchor points. The distance between each cluster center point and each anchor point is calculated, where M is a positive integer. For any cluster center point, the anchor point closest to the cluster center point is determined as the target anchor point. When the target anchor point meets the preset conditions, the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo-label of all frame-level features within the cluster center point, thus obtaining the pseudo-labels of all frame data in the training set.

3. The pre-training optimization method according to claim 2, characterized in that, After calculating the distances between all cluster centers and each anchor point, the process also includes: For any cluster center point, determine all anchor points whose distance from the cluster center point is less than a first distance threshold; Determining the anchor point closest to the cluster center as the target anchor point includes: From all anchor points whose distance to the cluster center is less than the first distance threshold, determine the anchor point that is closest to the cluster center as the target anchor point.

4. The pre-training optimization method according to claim 2, characterized in that, For any cluster center point, the anchor point closest to the cluster center point is determined as the target anchor point. When the target anchor point meets preset conditions, the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo-tag for all frame-level features within the cluster center point, including: For any cluster center point, determine the anchor point that is closest to the cluster center point as the target anchor point; Detect whether the distance between the target anchor point and the cluster center point is less than a second distance threshold; If the distance between the target anchor point and the cluster center point is less than the second distance threshold, then the target anchor point is determined to meet the preset conditions, and the sentence-level sentiment tag corresponding to the target anchor point is used as the pseudo tag for all frame-level features within the cluster center point.

5. The pre-training optimization method according to claim 4, characterized in that, After detecting whether the distance between the target anchor point and the cluster center point is less than a second distance threshold, the method further includes: If the distance between the target anchor point and the cluster center point is not less than the second distance threshold, it is determined that the target anchor point does not meet the preset conditions, other types of anchor points are created, and the sentence-level sentiment tags defined by the other types of anchor points are used as pseudo-tags for all frame-level features within the cluster center point.

6. The pre-training optimization method according to claim 1, characterized in that, After training the second sentiment prediction model using the pseudo-labels of all frames in the training set to obtain a pre-trained second sentiment prediction model, the process further includes: For any sentence-level speech in the training set, the pre-trained second sentiment prediction model is used to extract the pre-trained frame-level features corresponding to each frame of data in the sentence-level speech. The sentence-level sentiment label of the sentence-level speech is used as the sentiment category of the pre-trained frame-level features to obtain the sentiment category corresponding to the pre-trained frame-level features of all frames of data in the training set. For any sentiment category, all pre-trained frame-level features belonging to the sentiment category are input into a preset encoder for encoding, and at least one frame-level feature is randomly selected from all pre-trained frame-level features of the sentiment category and input into a preset prediction network for prediction. With the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, the preset encoder is trained to obtain the trained preset encoder. All pre-trained frame-level features belonging to the emotion category are input into the trained preset encoder, and the corresponding updated pre-trained frame-level features are output to obtain the updated pre-trained frame-level features of all frame data in the training set. Based on the updated pre-trained frame-level features and corresponding sentiment categories of all frame data, cluster all updated pre-trained frame-level features, and update the sentiment categories corresponding to all updated pre-trained frame-level features according to the clustering results, and determine the updated results as the pre-trained pseudo-labels of the corresponding frame data. Based on the pre-trained pseudo-labels of all frames in the training set, the second sentiment prediction model is trained using the training set to obtain the trained second sentiment prediction model.

7. The pre-training optimization method according to any one of claims 1 to 6, characterized in that, The first sentiment prediction model includes a first feature encoder, a bidirectional LSTM layer, and a first fully connected layer. The first feature encoder is composed of CNN layers. The second sentiment prediction model includes a second feature encoder, a Transformer layer, and a second fully connected layer. The second feature encoder is composed of CNN layers. The number of CNN layers in the first feature encoder is the same as the number of CNN layers in the second feature encoder.

8. A pre-training optimization device based on artificial intelligence, characterized in that, The pre-training optimization device includes: The first training module is used to extract the frame-level features corresponding to each frame of data in any sentence-level speech in the training set using the first sentiment prediction model, and use the sentence-level sentiment label of the sentence-level speech as the sentiment category of the frame-level features to obtain the sentiment category corresponding to the frame-level features of all frame data in the training set. The encoding prediction module is used to input all frame-level features belonging to any sentiment category into a preset encoder for encoding, and randomly select at least one frame-level feature from all frame-level features of the sentiment category into a preset prediction network for prediction. The feature update module is used to train the preset encoder with the goal of minimizing the negative cosine similarity between the encoded result and the predicted result, and to obtain the trained preset encoder. All frame-level features belonging to the sentiment category are input into the trained preset encoder, and the corresponding updated frame-level features are output to obtain the updated frame-level features of all frame data in the training set. The pseudo-label determination module is used to cluster all updated frame-level features based on the updated frame-level features and corresponding sentiment categories of all frame data, and update the sentiment categories corresponding to all updated frame-level features based on the clustering results, and determine the updated results as pseudo-labels for the corresponding frame data. The second training module is used to train the second sentiment prediction model using the pseudo-labels of all frame data in the training set, so as to obtain a pre-trained second sentiment prediction model. Both the first sentiment prediction model and the second sentiment prediction model have time-step aligned feature encoders.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the pre-training optimization method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the pre-training optimization method as described in any one of claims 1 to 7.

Images