Semantic vector representation method, device, computer equipment and storage medium

Abstract

The present application relates to the field of artificial intelligence, and provides a semantic vector representation method, apparatus, computer equipment, and storage medium, which obtains text data, classifies the text data, and determines the category corresponding to the text data; performs word segmentation processing on the text data to obtain multiple feature words and corresponding text sequences; calculates the weight value of each feature word in the corresponding category; constructs a weight matrix corresponding to the text data according to the weight value; constructs an embedding matrix of the text sequence, adds the weight matrix to the self-attention mechanism of a preset text representation model, processes the embedding matrix through the preset text representation model, and obtains the semantic vector corresponding to the text data. The semantic vector representation method, apparatus, computer equipment, and storage medium provided by the present application can accurately obtain the semantic vector of the text data.

Description

Technical Field

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a semantic vector representation method, apparatus, computer equipment, and storage medium. Background Art

[0002] In intelligent question-answering or information retrieval scenarios, it's often necessary to represent sentences and derive their semantic vectors. Currently, deep neural networks with a self-attention mechanism are typically used to represent sentences. This mechanism weights the word vectors within a sentence by the vectors of the contextual words. However, the self-attention mechanism has a drawback: if a sentence contains a large proportion of unimportant words, this can distort the semantic representation and, consequently, affect the subsequent similarity calculation. For example, in a question-and-answer system for travel guides to cities in the Guangdong-Hong Kong-Macao Greater Bay Area, the term "Guangdong-Hong Kong-Macao Greater Bay Area cities" is not particularly relevant because the questions in the question-and-answer system revolve around "Guangdong-Hong Kong-Macao Greater Bay Area cities." For example, among the three sentences: "Buy a ticket to a key city in the Guangdong-Hong Kong-Macao Greater Bay Area; Find a hotel in a key city in the Guangdong-Hong Kong-Macao Greater Bay Area; Buy a train ticket," the first and third sentences are more semantically similar. However, using a deep network semantic representation model with a self-attention mechanism, the vectors for the first and second sentences are closer. As a result, the accuracy of the semantic vectors for the sentences is lower. Summary of the Invention

[0003] The main purpose of this application is to provide a semantic vector representation method, apparatus, computer equipment and storage medium, aiming to solve the technical problem of low accuracy of semantic vectors obtained when semantic representation is performed through the self-attention mechanism.

[0004] To achieve the above objectives, this application provides a semantic vector representation method, comprising the following steps:

[0005] Acquiring text data, and classifying the text data to determine a category corresponding to the text data;

[0006] Performing word segmentation processing on the text data to obtain multiple feature words and corresponding text sequences;

[0007] Calculating the weight value of each of the feature words in the corresponding category;

[0008] Constructing a weight matrix corresponding to the text data according to the weight values;

[0009] An embedding matrix of the text sequence is constructed, the weight matrix is ​​added to the self-attention mechanism of a preset text representation model, and the embedding matrix is ​​processed by the preset text representation model to obtain a semantic vector corresponding to the text data.

[0010] Furthermore, the step of calculating the weight value of each of the feature words in the corresponding category includes:

[0011] Obtaining sample data of the category corresponding to the text data;

[0012] The inverse text frequency value of each of the feature words is calculated in the sample data, and the corresponding inverse text frequency value is used as the weight value corresponding to each of the feature words.

[0013] Furthermore, the step of constructing a weight matrix corresponding to the text data according to the weight values ​​includes:

[0014] The weight values ​​corresponding to the characters in the text data are used to construct a corresponding weight matrix according to the order of the characters in the text data.

[0015] Furthermore, the step of constructing an embedding matrix for the text sequence, adding the weight matrix to the self-attention mechanism of a preset text representation model, processing the embedding matrix by the preset text representation model, and obtaining a semantic vector corresponding to the text data includes:

[0016] Constructing the embedding matrix according to the text sequence;

[0017] Input the embedding matrix into the self-attention mechanism to obtain the corresponding Q matrix, K matrix and V matrix;

[0018] Multiply the result of multiplying the Q matrix and the K matrix by the weight matrix to obtain the target matrix;

[0019] The target matrix is ​​calculated by a normalized exponential function, multiplied by the V matrix, and then input into the feedforward neural network part of the text representation model for processing to obtain the semantic vector corresponding to the text data.

[0020] Furthermore, the step of calculating the weight value of each of the feature words includes:

[0021] Get the weight prediction model of the corresponding category;

[0022] The text sequence is input into a weight prediction model of a corresponding category to calculate the weight value of each feature word; wherein the weight prediction model is trained based on an extreme gradient boosting model.

[0023] Furthermore, after the step of obtaining the semantic vector corresponding to the text data, the following steps are included:

[0024] The text data is added to the corresponding category as sample data of other text data.

[0025] The present application also provides a semantic vector representation device, comprising:

[0026] an acquiring unit, configured to acquire text data, classify the text data, and determine a category corresponding to the text data;

[0027] A word segmentation processing unit, configured to perform word segmentation processing on the text data to obtain a plurality of feature words and corresponding text sequences;

[0028] A calculation unit, used to calculate the weight value of each of the feature words in the corresponding category;

[0029] A first construction unit is configured to construct a weight matrix corresponding to the text data according to the weight value;

[0030] The second candidate construction unit is used to construct an embedding matrix of the text sequence, add the weight matrix to the self-attention mechanism of a preset text representation model, process the embedding matrix through the preset text representation model, and obtain a semantic vector corresponding to the text data.

[0031] Furthermore, the computing unit includes:

[0032] The first acquisition subunit is used to obtain sample data of the category corresponding to the text data; the first calculation subunit is used to calculate the inverse text frequency value of each of the feature words in the sample data, and use the corresponding inverse text frequency value as the weight value corresponding to each of the feature words.

[0033] The present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above-mentioned semantic vector representation methods are implemented.

[0034] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the above-mentioned semantic vector representation methods are implemented.

[0035] The semantic vector representation method, apparatus, computer equipment, and storage medium provided in this application introduce weight information of feature words statistically analyzed through specific categories in the semantic representation process, and add the weight matrix of text data to the self-attention mechanism, which can make the semantic representation of sentences pay more attention to important words and less attention to unimportant words, thereby improving the effect of similarity calculation of text data and improving the accuracy of the obtained semantic vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the steps of the semantic vector representation method in one embodiment of the present application;

[0037] Figure 2 This is a schematic diagram of the network structure of the self-attention mechanism in one embodiment of the present application;

[0038] Figure 3 This is a structural block diagram of a semantic vector representation device in one embodiment of the present application;

[0039] Figure 4 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0040] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0042] Reference Figure 1 , an embodiment of the present application provides a semantic vector representation method, comprising the following steps:

[0043] Step S1, obtaining text data, and classifying the text data to determine the category corresponding to the text data;

[0044] Step S2, performing word segmentation processing on the text data to obtain multiple feature words and corresponding text sequences;

[0045] Step S3, calculating the weight value of each of the feature words in the corresponding category;

[0046] Step S4, constructing a weight matrix corresponding to the text data according to the weight values;

[0047] Step S5: construct an embedding matrix for the text sequence, add the weight matrix to the self-attention mechanism of a preset text representation model, process the embedding matrix through the preset text representation model, and obtain a semantic vector corresponding to the text data.

[0048] In this embodiment, as described in step S1 above, text data is obtained. The text data may be user input or converted from voice data. The text data is classified using a preset classification method to determine the category to which the text data corresponds. Specifically, multiple categories are pre-set, each category having multiple sample data. Keywords in the text data can be extracted to determine which category the data belongs to.

[0049] As described in step S2 above, in Chinese text, words are continuous. The text data can be segmented using Jieba word segmentation, HanLP, etc. to obtain multiple feature words. For example, the text data "how to buy a product" can be segmented to obtain three feature words: "how", "buy", and "product".

[0050] As described in step S3 above, in the corresponding category, a weight value is calculated for each feature word. In different categories, important words and unimportant words are different. Therefore, the weight values ​​calculated for the feature words after word segmentation are also different in different categories. After obtaining the weight words of the feature words, a weight matrix of the text data is constructed.

[0051] As described in step S4 above, the text representation model is trained based on the BERT (Bidirectional Encoder Representations from Transformers) model. The network structure of the text representation model mainly consists of four parts: input, self-attention mechanism, feedforward neural network (feedforward) part and vector output.

[0052] In the input, if the text data contains a total of L characters, these L characters are passed to the input, and each character is converted into a one-dimensional character vector with a dimension of dk. During training, the model automatically learns from the character vectors to derive a text vector that captures the global semantic information of the text and combines it with the semantic information of individual characters / words. Because characters / words appearing at different positions in the text carry different semantic information (for example, "I love you" and "You love me"), different position vectors are assigned to characters / words at different positions to distinguish them. Finally, the sum of the character vector, text vector, and position vector serves as the model input. The vectors of all characters are combined into an embedding matrix X (of shape: L*dk), where each row of this matrix is ​​a vector representing each character in the text data.

[0053] The weight matrix of the text data is incorporated into the self-attention mechanism, allowing it to focus more on important words and less on unimportant ones during sentence representation. After processing the embedding matrix through the self-attention mechanism, it is input into the feedforward neural network for forward propagation. The feedforward neural network consists of two layers: the first layer uses a linear activation function, and the second layer uses a Reluctant Unit (ReLU) activation function. After processing by the feedforward neural network, the output is a semantic vector for each character / word in the text data, which incorporates the semantic information of the entire text.

[0054] In this embodiment, the weight information of feature words statistically obtained by specific categories is introduced in the semantic representation process, which can make the semantic representation of sentences pay more attention to important words, reduce the attention to unimportant words, improve the effect of calculating the similarity of text data, and improve the accuracy of the obtained semantic vectors. At the same time, by this method, there is no need to retrain the text representation model. Only the statistically obtained weight matrix needs to be introduced into the model inference process, without increasing the data annotation cost and the model retraining cost.

[0055] In one embodiment, step S3 of calculating the weight value of each feature word in the corresponding category includes:

[0056] Step S31: Obtain the sample data of the category corresponding to the text data;

[0057] Step S32: Calculate the inverse document frequency value of each feature word in the sample data, and use the corresponding inverse document frequency value as the weight value corresponding to each feature word.

[0058] In this embodiment, first, the sample data within a certain category is statistically analyzed, then all the sample data is segmented to count all the words, and then the inverse document frequency (IDF) value of each sample feature word is calculated. The IDF value represents the importance of the corresponding sample feature word. By statistically analyzing the IDF value of the sample feature word, the importance of the sample feature word in the sample data is represented based on the IDF value. If some sample feature words appear very frequently in the sample data, these sample feature words are often considered unimportant. For example, words such as "de", "a", and "zenme" often appear in the sample data, indicating that the importance of these sample feature words is relatively low. In different categories, the unimportant sample feature words are different, and the unimportant sample feature words can also be set manually.

[0059] In this embodiment, the inverse document frequency is used to represent the weight of the feature word. The sample data of the corresponding category is obtained, and the IDF value of each feature word is calculated in the sample data. The IDF value calculation formula for a feature word w is:

[0060] where α is a adjustment factor, N is the number of sample data in this category, and N w is the number of sample data containing the feature word W.

[0061] Specifically, for example, the sample data of the categories corresponding to the text data "how to buy products" are the following four: "how to buy equipment", "how to buy train tickets", "how to repair", and "how to declare products". After word segmentation processing, the first sample data obtains three sample feature words: "how", "buy", and "equipment". After word segmentation processing, the second sample data obtains three sample feature words: "how", "buy", and "train tickets". After word segmentation processing, the third sample data obtains two sample feature words: "how" and "repair". After word segmentation processing, the fourth sample data obtains "how", "declare", and "product". Among them, there are 3 sample data containing the word "how" among the above 4 sample data, so the importance weight of the feature word "how" is IDF('how') = log(4 / 3), and there is 1 sample data containing the word "product", so the IDF('how') of the feature word "product" = log(4 / 1). In the above category, the importance of "product" is greater than that of the word "how", so the IDF values ​​of the three feature words of "how to buy products" are log(4 / 3), log(4 / 2), and log(4 / 1) respectively.

[0062] In one embodiment, the step S4 of constructing a weight matrix corresponding to the text data according to the weight values ​​includes:

[0063] Step S41 , constructing a corresponding weight matrix based on the weight values ​​corresponding to the characters in the text data according to the order of the characters in the text data.

[0064] In this embodiment, the weight value of each feature word is obtained by calculation, and then a weight matrix is ​​constructed according to the weight value. For example, the text data of "how to buy a product" constructs a weight matrix [how, buy, product]-[log(4 / 3),log(4 / 2),log(4 / 1)], and each word constructs a weight matrix according to its order in the text data, that is, w=[log(4 / 3),log(4 / 3),log(4 / 2),log(4 / 2),log(4 / 1),log(4 / 1)]; the specific value calculated is w=[0.125,0.125,0.3,0.3,0.6,0.6]; finally, it is converted into a weight matrix W:

[0065] Where l represents the number of characters in the text data.

[0066] In one embodiment, step S5 of constructing an embedding matrix for the text sequence, adding the weight matrix to a self-attention mechanism of a preset text representation model, and processing the embedding matrix using the preset text representation model to obtain a semantic vector corresponding to the text data includes:

[0067] Step S51, constructing the embedding matrix according to the text sequence;

[0068] Step S52: input the embedding matrix into the self-attention mechanism to obtain the corresponding Q matrix, K matrix and V matrix;

[0069] Step S53, performing bitwise multiplication of the result of multiplying the Q matrix and the K matrix with the weight matrix to obtain a target matrix;

[0070] In step S54, the target matrix is ​​calculated by a normalized exponential function, multiplied by the V matrix, and then input into the feedforward neural network part of the text representation model for processing to obtain the semantic vector corresponding to the text data.

[0071] In this embodiment, an embedding matrix X is constructed and input into the self-attention mechanism to obtain three matrices Q (Query), K (Key), and V (Value). In the existing self-attention matrix, let the matrix X = Q = K = V, which can be obtained by the formula The character representations in the text data are modified so that the character vectors become contextually relevant. This formula indicates that the vector of a character at a certain position in the text data is the weighted sum of the context vectors. This weight is calculated by calculating the dot product of the current character vector with the vectors of characters at other positions. X′ is the adjusted character vector that captures the context. All the character vectors in the text data are then fed into a fully connected layer, the feedforward neural network, to obtain the semantic vector of the text data.

[0072] In real-world applications, the words in a sentence's context have varying degrees of importance. Some words don't require significant attention because they aren't crucial to the semantic representation of the corresponding category. The self-attention mechanism in the above process weights the word vector at a given position by the word vectors in the context, ensuring that words that are not crucial to the final semantic understanding are not ignored.

[0073] See also Figure 2 In this embodiment, the weight matrix of text data is added to the self-attention mechanism. In the process of sentence representation, more attention is paid to important words, less attention is paid to important words, and the influence of unimportant words on the final semantics is reduced. The result of multiplying the Q matrix and the K matrix is ​​multiplied by the weight matrix W to obtain the target matrix. The bitwise multiplication is to multiply the vectors or matrices of the same dimension bit by bit. The target matrix is ​​multiplied by the V matrix after the softmax calculation and then input into the feedforward neural network part for processing, and finally the semantic vector is obtained. The processing in the self-attention mechanism can be expressed as:

[0074] Here, ⊙ represents Hadamard product, which is positional multiplication.

[0075] With the above modifications, when obtaining a word's context-based vector, the vector for a word at a specific position is calculated by weighted summing the context vectors. The weights corresponding to unimportant words are reduced, allowing the final sentence's semantic vector to retain more important semantic information. This modification is equivalent to introducing prior knowledge based on unsupervised sample data of the corresponding category. This prior knowledge is used to determine which unimportant semantics should be ignored during sentence representation, reducing the impact of unimportant words on the overall semantics of the sentence.

[0076] In one embodiment, the step of calculating the weight value of each of the feature words includes:

[0077] Get the weight prediction model of the corresponding category;

[0078] The text sequence is input into a weight prediction model of a corresponding category to calculate the weight value of each feature word; wherein the weight prediction model is trained based on an extreme gradient boosting model.

[0079] In this embodiment, a weight prediction matrix is ​​pre-trained for each category. Training samples can be manually labeled using effective features to manually mark the weights of important words in a sentence. Then, an Xgboost (eXtreme Gradient Boosting) model is used to train these labeled training samples, so that after training, the weights of keywords can be accurately predicted. Effective features may include: IDF value, TF (Term Frequency) value, term length, term part of speech, word position, whether it is a stop word, whether it is a modal particle, whether it is a proper noun (person / place name), etc.

[0080] In another embodiment, the feature words in the sample data can be extracted by the YAKE (Yet Another Keyword Extractor) algorithm to calculate the weight of each feature word. The YAKE algorithm can extract the feature words from the uppercase term (T case ), word position (T Position ), word frequency (TF), contextual relationship (T Rel ), the frequency of the word in the sentence (T sentence ) Comprehensively extract keywords from five aspects, and then use the formula Calculate the score of each feature word. The lower the score, the higher the importance. Then input the group into the logarithmic function with a base of 0.5 to calculate the weight of each feature word.

[0081] In one embodiment, after the step of obtaining the semantic vector corresponding to the text data, the following steps are included:

[0082] The text data is added to the corresponding category as sample data of other text data.

[0083] In this embodiment, after the text data obtains the corresponding semantic vector, the text data is added to the corresponding category, and then the weight of each feature word is adjusted to a certain extent, so that more important words can be learned when the text vector is subsequently represented.

[0084] See also Figure 3 , an embodiment of the present application provides a semantic vector representation device, comprising:

[0085] an acquiring unit, configured to acquire text data, classify the text data, and determine a category corresponding to the text data;

[0086] A word segmentation processing unit, configured to perform word segmentation processing on the text data to obtain a plurality of feature words and corresponding text sequences;

[0087] A calculation unit, used to calculate the weight value of each of the feature words in the corresponding category;

[0088] A first construction unit is configured to construct a weight matrix corresponding to the text data according to the weight value;

[0089] The second candidate construction unit is used to construct an embedding matrix of the text sequence, add the weight matrix to the self-attention mechanism of a preset text representation model, process the embedding matrix through the preset text representation model, and obtain a semantic vector corresponding to the text data.

[0090] In one embodiment, the calculation unit 30 includes:

[0091] A first acquisition subunit, configured to acquire sample data of a category corresponding to the text data;

[0092] The first calculation subunit is configured to calculate the inverse text frequency value of each of the feature words in the sample data, and use the corresponding inverse text frequency value as the weight value corresponding to each of the feature words.

[0093] In one embodiment, the first building unit 40 includes:

[0094] The first constructing subunit is configured to construct a corresponding weight matrix based on the weight values ​​corresponding to the characters in the text data according to the order of the characters in the text data.

[0095] In one embodiment, the second building unit 50 includes:

[0096] A second construction subunit is used to construct the embedding matrix according to the text sequence;

[0097] A first input subunit is used to input the embedding matrix into the self-attention mechanism to obtain the corresponding Q matrix, K matrix and V matrix;

[0098] The bit multiplier unit is used to perform bit multiplication on the result of multiplying the Q matrix and the K matrix with the weight matrix to obtain the target matrix;

[0099] The second input subunit is used to calculate the target matrix through a normalized exponential function, multiply it with the V matrix, and then input it into the feedforward neural network part of the text representation model for processing to obtain the semantic vector corresponding to the text data.

[0100] In one embodiment, the calculation unit 30 includes:

[0101] The second acquisition subunit is used to obtain a weight prediction model of the corresponding category;

[0102] The second calculation subunit is used to input the text sequence into the weight prediction model of the corresponding category to calculate the weight value of each feature word; wherein, the weight prediction model is trained based on the extreme gradient boosting model.

[0103] In one embodiment, the semantic vector representation device further includes:

[0104] The adding unit is used to add the text data to the corresponding category as sample data of other text data.

[0105] In this embodiment, the specific implementation of each of the above units and sub-units can be referred to the description in the above method embodiment, which will not be repeated here.

[0106] Reference Figure 4 In the embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data, etc. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a semantic vector representation method is implemented.

[0107] Those skilled in the art will understand that Figure 4 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0108] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements a semantic vector representation method when the computer program is executed by a processor.

[0109] In summary, the semantic vector representation method, device, computer equipment and storage medium provided in the embodiment of the present application obtain text data, classify the text data, determine the category corresponding to the text data; perform word segmentation on the text data to obtain multiple feature words and corresponding text sequences; calculate the weight value of each feature word in the corresponding category; construct a weight matrix corresponding to the text data according to the weight value; construct an embedding matrix of the text sequence, add the weight matrix to the self-attention mechanism of the preset text representation model, and process the embedding matrix through the preset text representation model to obtain the semantic vector corresponding to the text data. Through the semantic vector representation method, device, computer equipment and storage medium provided by the present application, the weight information of the feature words counted by specific categories is introduced in the semantic representation process, which can make the semantic representation of the sentence pay more attention to important words and reduce the attention to unimportant words, improve the effect of the similarity calculation of text data, and improve the accuracy of the obtained semantic vector. At the same time, through this method, there is no need to retrain the text representation model, only the statistical weight matrix needs to be introduced into the model inference process, without increasing the data annotation cost and model retraining cost.

[0110] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM).

[0111] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0112] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A semantic vector representation method, characterized in that: The following steps are involved: Acquiring text data, and classifying the text data to determine a category corresponding to the text data; Performing word segmentation processing on the text data to obtain multiple feature words and corresponding text sequences; Calculating the weight value of each of the feature words in the corresponding category; Constructing a weight matrix corresponding to the text data according to the weight values; Constructing an embedding matrix for the text sequence, adding the weight matrix to the self-attention mechanism of a preset text representation model, and processing the embedding matrix through the preset text representation model to obtain a semantic vector corresponding to the text data; The steps of constructing an embedding matrix for the text sequence, adding the weight matrix to a self-attention mechanism of a preset text representation model, and processing the embedding matrix by the preset text representation model to obtain a semantic vector corresponding to the text data include: Constructing the embedding matrix according to the text sequence; Input the embedding matrix into the self-attention mechanism to obtain the corresponding Q matrix, K matrix and V matrix; Multiply the result of multiplying the Q matrix and the K matrix by the weight matrix to obtain the target matrix; The target matrix is ​​calculated by a normalized exponential function, multiplied by the V matrix, and then input into the feedforward neural network part of the text representation model for processing to obtain the semantic vector corresponding to the text data.

2. The semantic vector representation method according to claim 1, characterized in that: The step of calculating the weight value of each of the feature words in the corresponding category includes: Obtaining sample data of the category corresponding to the text data; The inverse text frequency value of each of the feature words in the sample data is calculated, and the corresponding inverse text frequency value is used as the weight value corresponding to each of the feature words.

3. The semantic vector representation method according to claim 1, characterized in that The step of constructing a weight matrix corresponding to the text data according to the weight values ​​comprises: The weight values ​​corresponding to the characters in the text data are used to construct a corresponding weight matrix according to the order of the characters in the text data.

4. The semantic vector representation method according to claim 1, characterized in that: The step of calculating the weight value of each of the feature words includes: Get the weight prediction model of the corresponding category; The text sequence is input into a weight prediction model of a corresponding category to calculate the weight value of each feature word; wherein the weight prediction model is trained based on an extreme gradient boosting model.

5. The semantic vector representation method according to claim 2, characterized in that: After the step of obtaining the semantic vector corresponding to the text data, the method further includes: The text data is added to the corresponding category as sample data of other text data.

6. A semantic vector representation device, characterized in that: include: an acquiring unit, configured to acquire text data, classify the text data, and determine a category corresponding to the text data; A word segmentation processing unit, configured to perform word segmentation processing on the text data to obtain a plurality of feature words and corresponding text sequences; A calculation unit, used to calculate the weight value of each of the feature words in the corresponding category; A first construction unit is configured to construct a weight matrix corresponding to the text data according to the weight value; A second construction unit is configured to construct an embedding matrix for the text sequence, add the weight matrix to a self-attention mechanism of a preset text representation model, and process the embedding matrix using the preset text representation model to obtain a semantic vector corresponding to the text data; The second building block comprises: A second construction subunit is used to construct the embedding matrix according to the text sequence; A first input subunit is used to input the embedding matrix into the self-attention mechanism to obtain the corresponding Q matrix, K matrix and V matrix; The bit multiplier unit is used to perform bit multiplication on the result of multiplying the Q matrix and the K matrix with the weight matrix to obtain the target matrix; The second input subunit is used to calculate the target matrix through a normalized exponential function, multiply it with the V matrix, and then input it into the feedforward neural network part of the text representation model for processing to obtain the semantic vector corresponding to the text data.

7. The semantic vector representation device according to claim 6, characterized in that The computing unit comprises: A first acquisition subunit, configured to acquire sample data of a category corresponding to the text data; The first calculation subunit is configured to calculate the inverse text frequency value of each of the feature words in the sample data, and use the corresponding inverse text frequency value as the weight value corresponding to each of the feature words.

8. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the semantic vector representation method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the semantic vector representation method according to any one of claims 1 to 5 are implemented.

Images