A sentiment analysis method for customer service platform based on large model

Abstract

The invention discloses a customer service platform sentiment analysis method based on a large model, and relates to the technical field of language intelligent data processing. The key points of the technical solution are: using an SVM model and a Transformer model to form an SVM-transformer model as a basic sentiment analysis model for analysis, and using a labeled sentiment data set to fine-tune the SVM-transformer model to improve the accuracy of the model, and evaluating the performance of the model through methods such as cross-validation to ensure that the model can maintain good generalization ability on new data. The invention can more accurately identify the user's current emotional state, such as satisfaction or dissatisfaction, and avoid the problems of low detection efficiency, high cost, poor accuracy, and difficulty in supervision in traditional methods, thereby providing more accurate content recommendation, user behavior analysis and other functions for the customer service platform.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of language intelligent data processing, and more specifically, to a customer service platform sentiment analysis method based on a large model. Background Art

[0002] With the rapid development of the customer service industry, real-time sentiment analysis of users has become an important means to measure user experience and customer service efficiency. Through real-time sentiment analysis, the customer service platform can not only provide real-time feedback to customer service staff to help them adjust their service strategies; it can also make recommendations to users about other things that may be of interest to improve user experience.

[0003] However, the current traditional sentiment analysis methods have problems such as being unable to accurately identify user emotions and being difficult to adapt to rapidly changing content styles. They are also unable to accurately identify changes in user emotions during customer service. The main shortcomings are as follows:

[0004] Traditional sentiment analysis methods are mainly based on lexicon methods and classic machine learning models. Lexicon methods use predefined vocabulary lists to determine the sentiment tendency of texts. Classic machine learning methods, including common models such as support vector machines (SVMs), Naive Bayes, and decision trees, cannot fully consider contextual information, resulting in inaccurate sentiment analysis results.

[0005] In addition to text, the customer service platform also involves customer service response and user evaluation data. The analysis of customer service responses and user evaluations can correct the judgment of user emotions. Traditional sentiment analysis methods are relatively rarely used in this regard. Simple text matching is usually used for sentiment recognition. However, simple text recognition alone often cannot fully understand the tone and emotion expressed by the user, which can easily cause misunderstandings when customer service responds.

[0006] Therefore, a new solution needs to be proposed to solve this problem. Summary of the invention

[0007] The purpose of the embodiments of the present invention is to provide a customer service platform sentiment analysis method based on a large model in order to solve the above-mentioned problems.

[0008] The above technical objectives of the embodiments of the present invention are achieved through the following technical solutions:

[0009] In a first aspect, a customer service platform sentiment analysis method based on a large model comprises the following steps:

[0010] Obtain user chat record data and user emotion labels to form a data set;

[0011] Extracting text features from the chat record data based on the Bert model and converting them to obtain vector output;

[0012] Convert the vector into sentiment feature output based on a convolutional neural network;

[0013] Inputting the text features and the sentiment features into the Bert model to obtain a fused feature vector, and introducing an attention mechanism to adjust the weight matrix size of the fused feature vector;

[0014] The fused feature vector includes utilizing the feature crossover idea to fuse the text feature and the sentiment feature into a fused feature vector Eme;

[0015]

[0016] in, Indicates emotional characteristics, Represents text features;

[0017] The attention mechanism is introduced to adjust the weight matrix size of the fused feature vector. The update formula after the attention mechanism is introduced is as follows:

[0018]

[0019] Among them, Weight represents the training weight matrix;

[0020] The data set and the fused feature vector are pre-trained based on the svm-transformer model, and the pre-trained model is used for sentiment analysis prediction.

[0021] The present invention adopts technology based on SVM-Transformer model fusion, which can more accurately identify the user's current emotional state, such as satisfaction or dissatisfaction, avoiding the problems of low detection efficiency, high cost, poor accuracy, and difficulty in supervision in traditional methods, thereby providing the customer service platform with more accurate content recommendation, user behavior analysis and other functions.

[0022] Optionally, the present invention is further configured as follows: the svm-transformer model includes an svm model and a transformer model, and the data set and the fused feature vector are pre-trained based on the svm-transformer model, wherein the data set is converted into a feature vector fea_svm and a feature vector fea_trans by the Bert model;

[0023] Input the feature vector fea_svm into the svm model for training to obtain a pre-trained svm model;

[0024] Input the feature vector fea_trans into the pre-trained svm model to obtain a feature vector label_svm;

[0025] Combine the feature vector label_svm and the feature vector fea_trans to obtain a new feature set fea_all;

[0026] The feature vector fea_all and the data set are put into the Transformers model to obtain a pre-training result.

[0027] Optionally, the present invention is further configured as follows: the pre-trained model is used for sentiment analysis prediction, including putting the fused feature vector into the pre-trained Transformer model for prediction to obtain a sentiment analysis prediction result.

[0028] Optionally, the present invention is further configured as follows: extracting text features from chat record data based on the Bert model and converting them to obtain vector output includes:

[0029] Use a word segmenter to segment the chat record data into words;

[0030] Use the WordPiece word segmentation method to split the word into multiple small-grained words;

[0031] The position information of the word is obtained, and the position information is input into the Bert model to obtain a vector for output.

[0032] Optionally, the present invention is further configured as follows: the convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer;

[0033] The convolutional neural network captures the local features of the vector through the convolution layer, and finally aggregates the features to the fully connected layer, and outputs the sentiment features through the softmax() function.

[0034] Optionally, the present invention is further configured as follows: the training weight matrix is ​​calculated by the Bert model through a self-attention mechanism, and the calculation method is as follows:

[0035]

[0036] Q, K, V represent the initialized weight matrix, T represents the matrix transpose, Represents the dimension of K.

[0037] In a second aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer or a processor, any one of the above methods is implemented.

[0038] In a third aspect, a computer program product is provided, wherein the computer program product comprises a computer program, and when the computer program is executed by a computer or a processor, the computer or the processor executes any one of the methods described above.

[0039] In summary, the present invention has the following beneficial effects:

[0040] After obtaining the user's chat records, the Bert model is loaded, and the chat records are input into the Bert model to obtain text features. The text features are converted into vectors of fixed length for output, and the converted vectors are input into the CNN network for emotional feature analysis and prediction, so that features that are more in line with the user's emotions can be obtained. These features are further integrated to improve the representation ability of the original feature information, and the extracted text features and emotional features are made into a cross feature vector. The attention mechanism is introduced to automatically adjust the weight of the fused features. At the same time, in order to improve the prediction structure of the model, the SVM model and the Transformer model are integrated to form an SVM-Transformer model, and the prediction results of the SVM model and the original feature data are combined to form a new feature set, which is put into the Transformer model for training and learning. Finally, the results of the model's emotional prediction are fed back, so that the user's current emotional state can be more accurately identified, such as satisfaction or dissatisfaction, avoiding the problems of low detection efficiency, high cost, poor accuracy, and difficulty in supervision in traditional methods. In addition, the adjustment of the weight size in the fused features can better understand the emotions expressed by customers, and then provide the customer service platform with more accurate content recommendations, user behavior analysis and other functions, so that the customer service platform can provide better services to customers. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flow chart of the customer service platform sentiment analysis method based on a large model in the present invention;

[0042] Figure 2 Schematic diagram of feature advancement and fusion in the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1As shown, in a feasible implementation manner, a customer service platform sentiment analysis method based on a large model specifically includes the following steps:

[0045] Step 101: Acquire user chat record data and user emotion labels to form a data set;

[0046] Step 102: extracting text features from the chat record data based on the Bert model and converting them to obtain vector output;

[0047] Step 103: Convert the vector output by the Bert model into an emotional feature output based on a convolutional neural network;

[0048] Step 104: Input the text features and sentiment features into the Bert model to obtain a fused feature vector, and introduce an attention mechanism to adjust the weight matrix size of the fused feature vector;

[0049] Step 105: Pre-train the data set and fused feature vector based on the SVM-Transformer model, and use the pre-trained model for sentiment analysis prediction;

[0050] Specifically, after obtaining the user's chat records, the Bert model is loaded, the chat records are input into the Bert model to obtain text features, the text features are converted into vector outputs of a fixed length, and the converted vectors are input into the CNN network for sentiment feature analysis and prediction, thereby obtaining features that are more in line with the user's emotions.

[0051] Furthermore, these features are further fused to improve the representation ability of the original feature information. The extracted text features and sentiment features are made into a cross feature vector, and the attention mechanism is introduced to automatically adjust the weight of the fused features.

[0052] Furthermore, in order to improve the prediction structure of the model, the SVM model and the Transformer model are fused to form an SVM-transformer model. The prediction results of the SVM model and the original feature data are combined to form a new feature set, which is put into the Transformer model for training and learning. Finally, the results of the model's emotion prediction are fed back, so that the user's current emotional state, such as satisfaction or dissatisfaction, can be more accurately identified, avoiding the problems of low detection efficiency, high cost, poor accuracy, and difficulty in supervision in traditional methods, thereby providing the customer service platform with more accurate content recommendations, user behavior analysis and other functions.

[0053] Specifically, in step 101, for the customer service platform, the SDK data collection method is used to collect the chat record data of users in the customer service session, and the real-time answers of the customer service and the user evaluation are collected at the same time, and uploaded to the back-end server through a network request for subsequent data analysis. The data is uploaded asynchronously to avoid affecting the performance of the application.

[0054] More specifically, when the user selects the SDK of the customer service platform itself, he / she should integrate it into the customer service application according to the documentation and guidelines provided by the SDK, add the necessary library files to the project, and configure the initialization parameters of the SDK according to the installation instructions.

[0055] SDK is a collection of tools, libraries, sample codes and documents for software development, which can help developers build, integrate and use specific software and services more easily. At the same time, a real-time feedback channel based on Kafka is designed, and Kafka is connected to the SDK. The data information obtained by the SDK is pushed to the Kafka message queue using the RTMP protocol. The Flink application is used to read the user chat record data from Kafka for the next step of data analysis.

[0056] Specifically, Flink is a real-time computing framework that is mainly used to process and analyze streaming data and can handle high-throughput data streams. Kafka is a distributed messaging system that is mainly used for data collection and buffering. It can handle communications between high-throughput data producers and consumers and ensure data persistence and reliability. Therefore, the combination of Flink and Kafka can make full use of the advantages of both to achieve efficient data processing and message delivery, and to deliver chat record data in a timely manner.

[0057] Furthermore, a dataset of user sentiment labels is formed, including a dataset of user comments, including positive and negative comments, and manually annotating the sentiment tendency of each comment (usually positive or negative).

[0058] Furthermore, in step 102, the Bert model is a natural language processing model based on the Transformer model, which is mainly used to process tasks such as text classification, question-answering systems, named entity recognition, and semantic similarity calculation. As one of the most advanced pre-trained language models currently, the Bert model is widely used in various natural language processing scenarios.

[0059] Specifically, the Bert model mainly includes the Transformer model and the bidirectional encoder mechanism. By training the text through the bidirectional encoder mechanism, the contextual information in the text can be captured. A key feature of the Bert model is that it can generate different vector representations for each word in different contexts, and can also generate vector representations for a certain sentence. These vectors can be used as input features for subsequent tasks. When in use, the pre-trained Bert model is loaded to extract text features.

[0060] Preferably, in step 103, the user emotion feature extraction is output through a convolutional neural network, which includes neural network structures such as convolutional layers, pooling layers, and fully connected layers. The convolutional neural network extracts local features and texture features of the emotion information through neural network structures such as convolutional layers, pooling layers, and fully connected layers. It has the characteristic of translation invariance and can automatically learn effective feature representations from original image data.

[0061] Specifically, the convolution layer is used to detect local features, such as edges, textures, etc., and each convolution layer is composed of a group of convolution kernels. Each convolution kernel will slide on the image and perform dot product operations with the local area to obtain a feature map. ReLU is used as the activation function to increase the nonlinear expression ability of the model. The maximum pooling method is used to reduce the spatial dimension, reduce the amount of calculation, and improve the calculation speed of the CNN module.

[0062] More specifically, the convolutional neural network captures the local features of the vector through the convolution layer, and finally aggregates the features to the fully connected layer, and outputs the sentiment features through the softmax() function.

[0063] Furthermore, in step 104, the features are fused to improve the representation capability of the original feature information, and the extracted text and sentiment information are represented by a cross feature vector by using the feature cross idea.

[0064] Specifically, an attention mechanism is introduced to automatically adjust the importance weights of different cross-features, thereby improving the sentiment analysis prediction effect of the model. The purpose of the attention mechanism is to enable the model to focus on the most relevant part of the input. The attention mechanism method of the present invention is to assign different weight matrices to different cross-features in detail. During the model training process, the weight matrix size corresponding to different cross-features will be automatically learned to indicate the importance of the features, which is beneficial for the model to focus on more important features. The invention uses the Soft Attention method, which assigns a weight to each element in the input sequence. These weights determine the composition of the context vector. In this process, the model determines the weight by calculating the similarity between the state of the decoder and the output of the encoder.

[0065] Preferably, in step 105, in order to improve the prediction effect of the model, the SVM model and the Transformer model are fused. The SVM model is good at processing high-dimensional vectors and has the advantages of fast speed and high precision, and usually has better effects on small sample problems; while the Transformer model has higher advantages in processing sequence data.

[0066] Specifically, the prediction results of the SVM and the original feature data are combined into a new feature set, which is put into the transformer model for training and learning. The SVM model and the Transformer model are combined to form an SVM-transformer model as the basic sentiment analysis model for analysis, and the SVM-transformer model is fine-tuned using a labeled sentiment dataset to improve the accuracy of the model.

[0067] Preferably, the performance of the model is evaluated through methods such as cross-validation to ensure that the model can maintain good generalization capabilities on new data, thereby making the sentiment prediction more accurate and Shudie customer service can better serve users based on the feedback results.

[0068] See also Figure 1 and Figure 2 As shown, in another feasible embodiment, extracting text features in chat record data based on the Bert model and converting them to obtain vector output includes: using a word segmenter to segment the chat record data to obtain words;

[0069] Preferably, for the user chat records read by the Kafka consumer, a word segmenter is used to segment the chat record data to obtain words. The word segmentation tool jieba library can be used to segment the chat records, and a predefined stop word list is called to represent stop words and useless symbols. The jieba library is a Python library for Chinese word segmentation. Due to its simplicity, ease of use and high efficiency, the jieba library has quickly become the preferred tool for processing Chinese text tasks and has been widely used in the field of natural language processing. For user comments, the jieba library needs to perform the following preprocessing when used:

[0070] 1. Text cleaning: remove useless characters, such as punctuation marks, tag symbols, etc.;

[0071] 2. Word segmentation: Use jieba to split the comment text into meaningful words, such as jieba.cut(clean_text);

[0072] 3. Stop word filtering: load the stop word list and use it to filter the segmented text;

[0073] After the above processing, the text information in the user's chat record data can be processed to obtain multiple words, which is convenient for the subsequent analysis of the information.

[0074] Further, the word is split into multiple small-grained words using the WordPiece word segmentation method;

[0075] Specifically, first use the Tokenizer to further split the input text into Tokens. Use the WordPiece word segmentation method to further split the words into smaller sub-words to optimize the vocabulary after word segmentation and improve the generalization ability of the model; secondly, use a pre-trained embedding matrix to map the tokens after word segmentation into a high-dimensional space. Is the high-dimensional space after mapping:

[0076]

[0077] Since the Bert model can take two sentences as input, an additional embedding is added to each token to distinguish between two different sentences:

[0078] .

[0079] Further, the position information of the word is obtained, and the position information is input into the Bert model to obtain a vector for output;

[0080] More specifically, since the Bert model itself cannot obtain the position information of the current word in the sentence, it is necessary to obtain position embedding to supplement the position information. A unique position vector is calculated for each position. The even position and odd position vectors are obtained as follows. These vectors are learned during the training process:

[0081]

[0082]

[0083] Among them, d_model refers to the embedding vector after word segmentation. represents the embedding vector for even positions, +1 indicates the embedding vector at odd position.

[0084] Furthermore, after the above processing, the input text features will be input into multiple Transformer encoder layers for processing. Each encoder layer includes a self-attention mechanism and a feedforward neural network. The self-attention mechanism allows the model to focus on different positions in the input sequence to better understand the relationship between words. The self-attention mechanism is calculated as follows. Q, K, and V are the initialized weight matrices, and the matrix values ​​Weight will be learned during the training process:

[0085] Weight

[0086] Where T represents the matrix transpose, represents the dimension of K,

[0087] Specifically, the weight values ​​in the weight matrix are the parameters that the neural network needs to learn. The purpose of training is to learn the appropriate parameter size. The self-attention mechanism is to learn a better parameter size, thereby improving the network's ability to analyze and predict.

[0088] Finally, a feedforward fully connected layer module will be designed to output the text features that need to be extracted. , used to further enhance the ability of feature extraction:

[0089]

[0090] in, Represents the output of the pooling layer represents the Relu activation function, represents the training weight matrix, is the bias value for training.

[0091] See also Figure 1 and Figure 2 As shown, in another feasible embodiment, the fusion feature vector includes utilizing the feature crossover idea to fuse the text feature and the sentiment feature into a fusion feature vector Eme:

[0092]

[0093] in, represents the extracted emotional features, represents the extracted text features,

[0094] The cross-fused features can be directly Put it into the next Transformer model for emotion prediction, the present invention obtains Based on the formula, the attention mechanism is introduced to automatically adjust the importance weights of different cross-features. The sentiment feature and text feature will each train a weight matrix to represent the weight corresponding to each position. Then the two feature vectors are weighted and fused to improve the sentiment analysis prediction effect of the model. After adding the attention mechanism, the updated formula is as follows:

[0095]

[0096] Weight represents the weight matrix.

[0097] See also Figure 1 and Figure 2 As shown, in another feasible embodiment, the manually labeled sentiment analysis data is first put into the SVM for training, the result predicted by the SVM is used as a column feature of the data set, the data set is updated, and then the updated data set is put into the Transformer model for training, the data sequence features are learned in depth, and the residual brought by the SVM model is fitted at the same time, and the Transformer model is fine-tuned based on the manually labeled user emotions. The specific steps are as follows:

[0098] Collect a data set of user comments, including positive and negative comments, and manually annotate the sentiment tendency of each comment (usually positive or negative). Then introduce the SVM model, train the SVM model, convert the labeled data into a feature vector through the BERT layer, and divide the feature data into two parts, fea_svm and fea_trans, in a 1:1 ratio. The fea_svm is input into the SVM model for training. The goal of SVM is to find an optimal hyperplane that can maximize the interval from the sample point to the hyperplane and accurately separate the sample points of different categories. Then, fea_trans is input into the trained SVM model for prediction to obtain the label column label_svm. The features label_svm and fea_trans are combined into a new feature set, fea_all=concat[fea_trans,label_svm]. The feature data feal_all and the corresponding sentiment label are put into the Transformer model for training. The point-by-point multiplication and accumulation results are as follows:

[0099]

[0100] in, Represent the new feature features of the corresponding positive and negative comments, and get the final feature data Used to input into the transformer model to improve the effect of user comment sentiment prediction.

[0101] Furthermore, fine-tuning training includes obtaining text messages sent by users in customer service conversations and corresponding emotional labels, labeling these data to form a data set, and dividing the data set into a training set and a test set in a ratio of 80% and 20%.

[0102] Specifically, use the tokenizer of the pre-trained model Transformer to tokenize the text, then add [CLS] and [SEP] to the text for sequence truncation, encode it into an encoding format that the model can recognize, and use labelEnconder to map the sentiment labels to integer labels (such as 0 and 1). Use PyTorch's data loader to load the processed data into the model in batches, select Hugging Face's Transformers library as the pre-trained model, add 4 classification heads, add an additional fully connected layer as a classifier, and output dimension 2, corresponding to the sentiment analysis label category. Use Adam as the optimizer and set the learning rate to 5e-5.

[0103] Furthermore, the batch size is determined according to the available memory, and the pre-trained model is saved. When predicting user emotions, it is only necessary to put the features fused in the previous step into the trained transformer model for prediction. It should be noted that the feature dimensions during prediction and pre-training should be kept consistent.

[0104] Preferably, the results of the model's sentiment prediction are fed back to the customer service, and the trained Transformer model is deployed in the production environment for real-time sentiment analysis. User comments on the customer service platform are processed in real time, and the results of the sentiment analysis are returned so that the customer service can better serve users based on the feedback results.

[0105] Through the above steps, the user's current emotional state, such as satisfaction or dissatisfaction, can be identified more accurately, avoiding the problems of low detection efficiency, high cost, poor accuracy, and difficulty in supervision in traditional methods, thereby providing the customer service platform with more accurate content recommendations, user behavior analysis and other functions.

[0106] An embodiment of the present application also provides a computer-readable storage medium, which can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk), etc. The computer-readable storage medium contains instructions that instruct the computing device to execute the aforementioned time synchronization method.

[0107] An embodiment of the present application also provides a computer program product comprising instructions. The computer program product may be a software or program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computer device, the computing device executes the aforementioned time synchronization method.

[0108] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A customer service platform sentiment analysis method based on a large model, characterized in that: The method comprises the following steps: Obtain user chat record data and user emotion labels to form a data set; Extracting text features from the chat record data based on the Bert model and converting them to obtain vector output; Convert the vector into sentiment feature output based on a convolutional neural network; Inputting the text features and the sentiment features into the Bert model to obtain a fused feature vector, and introducing an attention mechanism to adjust the weight matrix size of the fused feature vector; The fused feature vector includes utilizing the feature crossover idea to fuse the text feature and the sentiment feature into a fused feature vector Eme; in, Indicates emotional characteristics, Represents text features; The attention mechanism is introduced to adjust the weight matrix size of the fused feature vector. The update formula after the attention mechanism is introduced is as follows: Among them, Weight represents the training weight matrix; The data set and the fused feature vector are pre-trained based on the svm-transformer model, and the pre-trained model is used for sentiment analysis prediction.

2. The method for sentiment analysis of a customer service platform based on a large model according to claim 1, characterized in that: The svm-transformer model includes an svm model and a transformer model, and the data set and the fused feature vector are pre-trained based on the svm-transformer model, wherein the data set is converted into a feature vector fea_svm and a feature vector fea_trans by the Bert model; Input the feature vector fea_svm into the svm model for training to obtain a pre-trained svm model; Input the feature vector fea_trans into the pre-trained svm model to obtain a feature vector label_svm; Combine the feature vector label_svm and the feature vector fea_trans to obtain a new feature set fea_all; The feature vector fea_all and the data set are put into the Transformers model to obtain a pre-training result.

3. The method for sentiment analysis of a customer service platform based on a large model according to claim 2, characterized in that: The pre-trained model is used for sentiment analysis prediction, including putting the fused feature vector into the pre-trained Transformer model for prediction to obtain a sentiment analysis prediction result.

4. The method for sentiment analysis of a customer service platform based on a large model according to claim 1, characterized in that: The text features in the chat record data are extracted based on the Bert model and converted to obtain vector outputs including: Use a word segmenter to segment the chat record data into words; Use the WordPiece word segmentation method to split the word into multiple small-grained words; The position information of the word is obtained, and the position information is input into the Bert model to obtain a vector for output.

5. The method for sentiment analysis of a customer service platform based on a large model according to claim 4, characterized in that: The convolutional neural network includes a convolutional layer, a pooling layer, and a fully connected layer; The convolutional neural network captures the local features of the vector through the convolution layer, and finally aggregates the features to the fully connected layer, and outputs the sentiment features through the softmax() function.

6. The method for sentiment analysis of a customer service platform based on a large model according to claim 1, characterized in that: The training weight matrix is ​​calculated by the Bert model through the self-attention mechanism, and the calculation method is as follows: Q, K, V represent the initialized weight matrix, T represents the matrix transpose, Represents the dimension of K.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer or a processor, the method according to any one of claims 1 to 6 is implemented.

8. A computer program product, characterized in that: The computer program product comprises a computer program, and when the computer program is executed by a computer or a processor, the computer or the processor is caused to execute the method according to any one of claims 1 to 6.

Images