A semantic representation method, device, storage medium and equipment

Abstract

The application discloses a semantic representation method and device, a storage medium and equipment. The method comprises the following steps: firstly, obtaining a target text to be processed; then, extracting a language identification code vector, an entity word information code vector, a position code vector and a word vector of the target text; then, inputting the language identification code vector, the entity word information code vector, the position code vector and the word vector of the target text into a pre-constructed semantic representation model to predict a semantic representation vector corresponding to the target text; wherein, the semantic representation model is obtained by comparative training of positive sample text pairs and negative sample text pairs of the same language and positive sample text pairs and negative sample text pairs of different languages by using language identification and an entity library. Therefore, it is no longer necessary to customize a model for a single language, and the semantic representation of cross languages can be realized only by using the pre-constructed semantic representation model, the representation effect is effectively improved, and it is beneficial to subsequent related tasks of multiple languages.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a semantic representation method, apparatus, storage medium and device. Background Technology

[0002] With the continuous breakthroughs in artificial intelligence technology and the increasing popularity of various smart terminal devices, human-computer interaction is occurring more and more frequently in people's daily work and life, bringing great convenience to people. The core technologies in human-computer interaction are semantic understanding and semantic representation.

[0003] Currently, semantic understanding and representation technologies for Chinese and English are relatively mature. However, methods that can simultaneously support semantic representation for multiple languages ​​have yet to emerge. Existing multilingual tasks require customized models for each language and scenario, meaning a separate model needs to be deployed for semantic representation in each language and scenario. This is cumbersome and complex, both in the training phase and in the real-world deployment environment. Therefore, how to achieve cross-language semantic representation and improve its performance is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] The main objective of this application is to provide a semantic representation method, apparatus, storage medium, and device that can achieve cross-language semantic representation and effectively improve the representation effect.

[0005] This application provides a semantic representation method, including:

[0006] Obtain the target text to be processed;

[0007] Extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text;

[0008] The language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text are input into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text.

[0009] The semantic representation model is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases.

[0010] In one possible implementation, the semantic representation model is constructed as follows:

[0011] Obtain training sample text pairs, which include positive and negative sample text pairs in the same language, as well as positive and negative sample text pairs in different languages.

[0012] Extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and the negative sample text pair;

[0013] The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and negative sample text pair are input into the initial semantic representation model to train the semantic representation model.

[0014] The initial multimodal feature extraction model includes a first initial network model, a second initial network model, a concatenation layer, and an output layer; both the first and second initial network models include an input layer, a transformer encoding layer, a keyword mask attention KMA encoding layer, and a connection layer; the parameters of the first and second initial network models are shared.

[0015] In one possible implementation, the step of inputting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and the negative sample text pair into the initial semantic representation model, and training the semantic representation model, includes:

[0016] The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the first sample text in the positive sample text pair are input into the input layer of the first initial network model to obtain the first sample input vector; and the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the second sample text in the positive sample text pair are input into the input layer of the second initial network model to obtain the second sample input vector.

[0017] The first sample input vector is input into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the first sample and the sample encoding vector of each word in the first sample text; and the second sample input vector is input into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the second sample and the sample encoding vector of each word in the second sample text.

[0018] The sample encoding vector of each word in the first sample text is input into the KMA encoding layer of the first initial network model to obtain the first sample keyword encoding vector of the first sample text; and the sample encoding vector of each word in the second sample text is input into the KMA encoding layer of the second initial network model to obtain the second sample keyword encoding vector of the second sample text.

[0019] The first sample overall encoding vector and the first sample keyword encoding vector are input into the connection layer of the first initial network model and connected to obtain the first sample vector of the first sample text; the second sample overall encoding vector and the second sample keyword encoding vector are input into the connection layer of the second initial network model and connected to obtain the second sample vector of the second sample text.

[0020] The first sample vector and the second sample vector are input into the concatenation layer of the initial multimodal feature extraction model and concatenated to obtain the first concatenated vector;

[0021] The first concatenated vector is used for classification, and the first classification result is output through the output layer of the initial multimodal feature extraction model;

[0022] The initial multimodal feature extraction model is trained using the first classification result and the corresponding classification and recognition labels of the positive sample text pairs, and the first initial network model obtained after training is used as the semantic representation model.

[0023] In one possible implementation, the step of inputting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and the negative sample text pair into the initial semantic representation model, and training the semantic representation model, includes:

[0024] The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the third sample text in the negative sample text pair are input into the input layer of the first initial network model to obtain the third sample input vector; and the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the fourth sample text in the negative sample text pair are input into the input layer of the second initial network model to obtain the fourth sample input vector.

[0025] The third sample input vector is input into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the third sample and the sample encoding vector of each word in the third sample text; and the fourth sample input vector is input into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the fourth sample and the sample encoding vector of each word in the fourth sample text.

[0026] The sample encoding vector of each word in the third sample text is input into the KMA encoding layer of the first initial network model to obtain the third sample keyword encoding vector of the third sample text; and the sample encoding vector of each word in the fourth sample text is input into the KMA encoding layer of the second initial network model to obtain the fourth sample keyword encoding vector of the fourth sample text.

[0027] The overall encoding vector of the third sample and the keyword encoding vector of the third sample are input into the connection layer of the first initial network model and connected to obtain the third sample vector of the third sample text; the overall encoding vector of the fourth sample and the keyword encoding vector of the fourth sample are input into the connection layer of the second initial network model and connected to obtain the fourth sample vector of the fourth sample text.

[0028] The third sample vector and the fourth sample vector are input into the concatenation layer of the initial multimodal feature extraction model and concatenated to obtain the second concatenated vector;

[0029] The second concatenated vector is used for classification, and the second classification result is output through the output layer of the initial multimodal feature extraction model;

[0030] The initial multimodal feature extraction model is trained using the second classification result and the corresponding classification and recognition labels of the negative sample text pairs, and the first initial network model obtained after training is used as the semantic representation model.

[0031] In one possible implementation, the method further includes:

[0032] Obtain the verification text pair; and extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the verification text pair;

[0033] The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the verification text pair are input into the initial semantic representation model to predict the classification result corresponding to the verification text pair;

[0034] When the classification result corresponding to the verification text pair is inconsistent with the classification label result corresponding to the verification text pair, the verification text pair is used again as the training sample text pair to update the initial semantic representation model, so as to obtain the updated semantic representation model.

[0035] In one possible implementation, the step of inputting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text includes:

[0036] The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text are input into the input layer of the pre-constructed semantic representation model to obtain the input vector;

[0037] The input vector is fed into the transformer encoding layer of the semantic representation model for encoding to obtain the overall encoding vector of the target text and the encoding vector of each word;

[0038] The encoding vector of each word in the target text is input into the KMA encoding layer of the semantic representation model to perform keyword encoding and obtain the keyword encoding vector.

[0039] The overall encoding vector of the target text and the keyword encoding vector are input into the connection layer of the semantic representation model and connected, and the resulting connection vector is used as the semantic representation vector corresponding to the target text.

[0040] In one possible implementation, the step of inputting the encoding vector of each word in the target text into the KMA encoding layer of the semantic representation model to perform keyword encoding and obtain keyword encoding vectors includes:

[0041] The encoding vector of each word in the target text is input into the KMA encoding layer of the semantic representation model. The encoding vector of each word is multiplied by the encoding vector of each keyword in the target text to obtain the keyword encoding vector corresponding to each word in the target text.

[0042] The keyword encoding vectors corresponding to each word in the target text are concatenated, and the resulting concatenated vector is used as the keyword encoding vector corresponding to the target text.

[0043] In one possible implementation, the entity word information encoding vector includes a core verb encoding vector, a general entity word encoding vector, a person name encoding vector, an organization name encoding vector, and a place name encoding vector.

[0044] This application also provides a semantic representation device, including:

[0045] The first acquisition unit is used to acquire the target text to be processed;

[0046] The first extraction unit is used to extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text;

[0047] The first prediction unit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text.

[0048] The semantic representation model is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases.

[0049] In one possible implementation, the device further includes:

[0050] The second acquisition unit is used to acquire training sample text pairs, which include positive and negative sample text pairs in the same language, as well as positive and negative sample text pairs in different languages.

[0051] The second extraction unit is used to extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the positive sample text pair and the negative sample text pair;

[0052] The training unit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the positive sample text pair and negative sample text pair into the initial semantic representation model, and train the semantic representation model to obtain the semantic representation model;

[0053] The initial multimodal feature extraction model includes a first initial network model, a second initial network model, a concatenation layer, and an output layer; both the first and second initial network models include an input layer, a transformer encoding layer, a keyword mask attention KMA encoding layer, and a connection layer; the parameters of the first and second initial network models are shared.

[0054] In one possible implementation, the training unit includes:

[0055] The first input subunit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the first sample text in the positive sample text pair into the input layer of the first initial network model to obtain the first sample input vector; and to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the second sample text in the positive sample text pair into the input layer of the second initial network model to obtain the second sample input vector;

[0056] The second input subunit is used to input the first sample input vector into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the first sample and the sample encoding vector of each word in the first sample text; and to input the second sample input vector into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the second sample and the sample encoding vector of each word in the second sample text.

[0057] The third input subunit is used to input the sample encoding vector of each word in the first sample text into the KMA encoding layer of the first initial network model to obtain the first sample keyword encoding vector of the first sample text; and to input the sample encoding vector of each word in the second sample text into the KMA encoding layer of the second initial network model to obtain the second sample keyword encoding vector of the second sample text.

[0058] The first connection subunit is used to input the first sample overall encoding vector and the first sample keyword encoding vector into the connection layer of the first initial network model for connection to obtain the first sample vector of the first sample text; and to input the second sample overall encoding vector and the second sample keyword encoding vector into the connection layer of the second initial network model for connection to obtain the second sample vector of the second sample text.

[0059] The first splicing subunit is used to input the first sample vector and the second sample vector into the splicing layer of the initial multimodal feature extraction model for splicing to obtain the first splicing vector;

[0060] The first classification subunit is used to classify using the first concatenated vector and output the first classification result through the output layer of the initial multimodal feature extraction model;

[0061] The first training subunit is used to train the initial multimodal feature extraction model using the first classification result and the corresponding classification and recognition labels of the positive sample text, and to use the first initial network model obtained after training as the semantic representation model.

[0062] In one possible implementation, the training unit includes:

[0063] The fourth input subunit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the third sample text in the negative sample text pair into the input layer of the first initial network model to obtain the third sample input vector; and to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the fourth sample text in the negative sample text pair into the input layer of the second initial network model to obtain the fourth sample input vector;

[0064] The fifth input subunit is used to input the third sample input vector into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the third sample and the sample encoding vector of each word in the third sample text; and to input the fourth sample input vector into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the fourth sample and the sample encoding vector of each word in the fourth sample text.

[0065] The sixth input subunit is used to input the sample encoding vector of each word in the third sample text into the KMA encoding layer of the first initial network model to obtain the third sample keyword encoding vector of the third sample text; and to input the sample encoding vector of each word in the fourth sample text into the KMA encoding layer of the second initial network model to obtain the fourth sample keyword encoding vector of the fourth sample text.

[0066] The second connection subunit is used to input the overall encoding vector of the third sample and the keyword encoding vector of the third sample into the connection layer of the first initial network model for connection to obtain the third sample vector of the third sample text; and to input the overall encoding vector of the fourth sample and the keyword encoding vector of the fourth sample into the connection layer of the second initial network model for connection to obtain the fourth sample vector of the fourth sample text.

[0067] The second splicing subunit is used to input the third sample vector and the fourth sample vector into the splicing layer of the initial multimodal feature extraction model for splicing to obtain the second splicing vector;

[0068] The second classification subunit is used to classify using the second concatenated vector and output the second classification result through the output layer of the initial multimodal feature extraction model;

[0069] The second training subunit is used to train the initial multimodal feature extraction model using the second classification result and the corresponding classification and recognition labels of the negative sample text, and to use the first initial network model obtained after training as the semantic representation model.

[0070] In one possible implementation, the device further includes:

[0071] The third acquisition unit is used to acquire verification text pairs; and extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pairs;

[0072] The second prediction unit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pair into the initial semantic representation model to predict the classification result corresponding to the verification text pair;

[0073] The update unit is used to update the initial semantic representation model by reusing the verification text pair as the training sample text pair when the classification result corresponding to the verification text pair is inconsistent with the classification label result corresponding to the verification text pair, so as to obtain the updated semantic representation model.

[0074] In one possible implementation, the first prediction unit includes:

[0075] The seventh input subunit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text into the input layer of the pre-constructed semantic representation model to obtain the input vector;

[0076] The first encoding subunit is used to input the input vector into the transformer encoding layer of the semantic representation model for encoding, so as to obtain the overall encoding vector of the target text and the encoding vector of each word;

[0077] The second encoding subunit is used to input the encoding vector of each word in the target text into the KMA encoding layer of the semantic representation model to encode keywords and obtain keyword encoding vectors.

[0078] The third connection subunit is used to input the overall encoding vector of the target text and the keyword encoding vector into the connection layer of the semantic representation model for connection, and use the resulting connection vector as the semantic representation vector corresponding to the target text.

[0079] In one possible implementation, the second coding subunit includes:

[0080] The computational subunit is used to input the encoding vector of each word in the target text into the KMA encoding layer of the semantic representation model, and to perform dot product calculation on the encoding vector of each word and the encoding vector of each keyword in the target text to obtain the keyword encoding vector corresponding to each word in the target text.

[0081] The third splicing subunit is used to splice the keyword encoding vectors corresponding to each word in the target text, and use the spliced ​​vector as the keyword encoding vector corresponding to the target text.

[0082] In one possible implementation, the entity word information encoding vector includes a core verb encoding vector, a general entity word encoding vector, a person name encoding vector, an organization name encoding vector, and a place name encoding vector.

[0083] This application also provides a semantic representation device, including: a processor, a memory, and a system bus;

[0084] The processor and the memory are connected via the system bus;

[0085] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described semantic representation methods.

[0086] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described semantic representation methods.

[0087] This application also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described semantic representation methods.

[0088] This application provides a semantic representation method, apparatus, storage medium, and device. First, the target text to be processed is acquired. Then, the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text are extracted. Next, the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text are input into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text. The semantic representation model is obtained by comparative training using language identifiers and an entity database based on positive and negative sample text pairs of the same language and positive and negative sample text pairs of different languages.

[0089] As can be seen, since the semantic representation model pre-built in this application is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases, it is possible to achieve cross-language semantic representation without the need for single-language model customization when performing semantic representation. This effectively improves the representation effect and is beneficial for subsequent multilingual related tasks. Attached Figure Description

[0090] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0091] Figure 1 A flowchart illustrating a semantic representation method provided in an embodiment of this application;

[0092] Figure 2 An example diagram of the embedding matrix provided in the embodiments of this application;

[0093] Figure 3An example diagram provided for embodiments of this application shows how to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text.

[0094] Figure 4 Example diagrams of two text sentences with different meanings in the same language provided in this application embodiment;

[0095] Figure 5 This is a schematic diagram of the network structure of the semantic representation model provided in the embodiments of this application;

[0096] Figure 6 This is a schematic diagram of the encoding mechanism of the KMA encoding layer in the semantic representation model provided in the embodiments of this application;

[0097] Figure 7 This is a schematic diagram illustrating the composition of a semantic representation device provided in an embodiment of this application. Detailed Implementation

[0098] With globalization and increasingly close international exchanges, various international businesses are emerging one after another. In order to better facilitate international communication, multilingual human-computer interaction has attracted more attention. The core technologies of human-computer interaction are semantic understanding and semantic representation. Semantic representation refers to extracting a vector representation of a sentence from a sentence, which is a deeper level of representation of semantic understanding.

[0099] Currently, semantic understanding and representation technologies for Chinese and English are relatively mature. However, a semantic representation method that can support multiple languages ​​simultaneously has yet to emerge. Existing semantic representation methods are basically customized for a single language, with models trained separately for each language. This approach has two typical drawbacks: First, when integrating and deploying semantic representation models into human-computer interaction systems, a large number of models need to be deployed. For example, if there are N languages, at least N models need to be deployed, resulting in a significant consumption of computing resources. Second, this approach does not fully utilize the semantic commonalities of multiple languages. Although the languages ​​are different, some semantically similar data can actually be fully utilized through model sharing, leading to poor representation results.

[0100] To address the aforementioned shortcomings, this application provides a semantic representation method. First, the target text to be processed is acquired. Then, the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text are extracted. Next, the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text are input into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text. The semantic representation model is trained by comparing positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and an entity database.

[0101] As can be seen, since the semantic representation model pre-built in this application is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases, it is possible to achieve cross-language semantic representation without the need for single-language model customization when performing semantic representation. This effectively improves the representation effect and is beneficial for subsequent multilingual related tasks.

[0102] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0103] First Embodiment

[0104] See Figure 1 This is a flowchart illustrating a semantic representation method provided in this embodiment. The method includes the following steps:

[0105] S101: Obtain the target text to be processed.

[0106] In this embodiment, any text that is semantically represented using this embodiment is defined as the target text to be processed. Furthermore, this embodiment does not limit the language type of the target text. For example, the target text can be Chinese text, English text, or Japanese text, etc. This embodiment also does not limit the source and acquisition method of the target text. For example, the target text can be the result of speech recognition or news text data obtained from an online platform, etc. This embodiment also does not limit the type of target text. For example, the target text can be question text in people's daily conversations or dialogue text in scenarios such as banks and insurance companies, etc.

[0107] It is understandable that the target text can be a sentence text, which is a collection of words. After obtaining the sentence text as the target text to be processed, it can be processed according to the subsequent steps S102-S103 to obtain its corresponding more accurate semantic representation.

[0108] S102: Extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text.

[0109] In this embodiment, after obtaining the target text to be processed through step S101, in order to improve the semantic representation effect of the target text, the entity word information encoding vector of the target text can be extracted according to the pre-built entity library corresponding to each language, and the language identifier encoding vector, position encoding vector and word vector of the target text can be extracted using existing or emerging vector extraction methods, so as to execute the subsequent step S103.

[0110] The language identifier encoding vector of the target text is determined according to the existing standard of general language encoding vector identifier. For example, when the target text is Chinese text, its corresponding language identifier encoding vector is "zh", or when the target text is English text, its corresponding language identifier encoding vector is "en", or when the target text is Japanese text, its corresponding language identifier encoding vector is "ja", etc.

[0111] Entity databases for various languages ​​can typically be obtained by aligning and translating using Chinese entity databases. One possible implementation is that the Chinese entity database can contain a vast number of entity words, including but not limited to "core verbs," "general entity words," "personal names," "organization names," and "place names." Correspondingly, the entity word information encoding vectors for the target text can include, but are not limited to, core verb encoding vectors, general entity word encoding vectors, person name encoding vectors, organization name encoding vectors, and place name encoding vectors. These can be represented using symbols such as "1," "2," "3," "4," and "5."

[0112] The positional encoding vector of the target text can be represented by "0", "1", "2", "3", "4", etc., from front to back, according to the position of each word in the target text.

[0113] The word vectors of the target text can be obtained using methods such as Word2vec, and this application does not limit the specific extraction method.

[0114] It should be noted that all the encoded information of the extracted target text can be implemented by introducing an embedding matrix, and the numerical representations of the various encoded vectors of the target text, such as "0", "1", "2", "3", "4", etc., correspond to indices on the embedding matrix. For example, ... Figure 2 As shown, for the language embedding of the target text, the numerical representations "zh", "en", and "ru" are all indices on the language embedding matrix, with "0", "1", and "2" representing the indices. For the entity type embedding of the target text, the numerical representations "1", "2", "3", and "4" representing the indices on the entity type embedding matrix are all indices.

[0115] S103: Input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text into the pre-built semantic representation model to predict the semantic representation vector corresponding to the target text.

[0116] In this embodiment, after extracting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text in step S102, in order to effectively improve the semantic representation effect of the target text, the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text can be further input into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text, which can then be used for subsequent multilingual related tasks.

[0117] Since the semantic representation model is trained by comparing positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases, when performing semantic representation on target text, it is no longer necessary to customize a single-language model according to the language type of the target text. Instead, cross-language semantic representation can be achieved using this semantic representation model, which can effectively improve the semantic representation effect of the target text.

[0118] It should be noted that the present application does not limit the specific structure of the semantic representation model. According to a preferred implementation, the semantic representation model may include an input layer, a transformer encode layer, a key-word mask attention (KMA) encoding layer (KMA layer), and a connection layer.

[0119] Based on the above, according to an optional implementation, the specific implementation process of the above step S103 may include the following steps S1031 to S1034:

[0120] Step S1031: inputting the language identification coding vector, entity word information coding vector, position coding vector and word vector of the target text into the input layer of a pre-constructed semantic representation model to obtain an input vector.

[0121] Specifically, in this implementation, after obtaining the language identification coding vector, entity word information coding vector, position coding vector and word vector of the target text, these vectors may be further input together into the input layer of the pre-constructed semantic representation model to obtain the input vector for performing the subsequent step S1032.

[0122] For example, as shown in Figure 3 , assuming that the target text is the Japanese sentence "我想去看樱花", the language of the target text is Japanese, and the corresponding language identification coding vector (that is, the language embedding in Figure 3 ) is "ja". In the target text, "桜" represents a general entity word, and "見" represents a core verb, the entity word information coding vectors corresponding to the two (that is, the entityType embedding in Figure 3 ) can be marked as 2 and 1 respectively, and other words are marked as 0. The position coding vector of the target text is the identifiers 0 to 11 corresponding to the position embedding in Figure 3 , and the word vector of the target text is the token embedding in Figure 3 . All these vectors can be input into the input layer of the pre-constructed semantic representation model together to obtain the input vector.

[0123] Step S1032: inputting the input vector into the transformer encoding layer of the semantic representation model for encoding, to obtain the overall encoding vector of the target text and the encoding vector of each word.

[0124] In this implementation, the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text are input to the input layer of the pre-constructed semantic representation model through step S1031. After the input vector is obtained, the input vector can be further input to the trasnformer encoding layer of the semantic representation model for encoding, as Figure 3 shown, the overall encoding vector of the target text (which can be defined herein as h(CLS)) and the encoding vector of each word are obtained, wherein, Figure 3 the two light gray boxes output by the trasnformer encoding layer correspond to the encoding vectors of the entity words "桜" and "見", and the other 10 dark gray boxes correspond to the encoding vectors of other non-entity words. The encoding vector of each word output by the trasnformer encoding layer can be defined as h1, h2...h respectively 12 .

[0125] Step S1033: input the encoding vector of each word in the target text to the KMA encoding layer of the semantic representation model, perform keyword encoding, and obtain a keyword encoding vector.

[0126] In this implementation, after predicting and obtaining the encoding vector of each word in the target text (h1, h2...h 12 ) through step S1032, the encoding vector of each word in the target text (h1, h2...h 12 ) can be further input to the KMA encoding layer of the semantic representation model for keyword encoding, that is, paying more attention to keyword differences in the target text, to obtain a keyword encoding vector, as Figure 3 shown.

[0127] Specifically, in an optional implementation, after inputting the encoding vector of each word in the target text into the KMA encoding layer of the semantic representation model, the encoding vector of each word (i.e., h1, h2...h 12 ) and the encoding vector of each keyword in the target text (i.e., as shown in Figure 3 , the encoding vectors h3 and h5 corresponding to "桜" and "見") are subjected to dot product calculation to obtain the keyword encoding vector corresponding to each word in the target text, then the keyword encoding vectors corresponding to each word in the target text are spliced, and the spliced vector obtained is used as the keyword encoding vector corresponding to the target text, which can be defined as h(KMA) herein to represent the essential core point of the target text.

[0128] Step S1034: input the overall encoding vector of the target text and the keyword encoding vector into the connection layer of the semantic representation model for connection, and use the obtained connection vector as the semantic representation vector corresponding to the target text.

[0129] In this implementation, after the overall coding vector of the target text (h(CLS)) is obtained through prediction in step S1032, and the keyword coding vector representing the essential core point of the target text (h(KMA)) is obtained through step S1033, both can be further input into the connection layer of the semantic representation model, such as Figure 3 the connection layer, for connection to obtain a connection vector, which is defined as S1, as shown in Figure 3 , the obtained S1 is the semantic representation of the target text "私は桜を見に行きたいです".

[0130] Wherein, the specific connection mode of h(CLS) and h(KMA) is not limited, which may be direct concatenation, or weighted processing, etc.

[0131] In this way, through the introduction of language identification coding vectors and entity word information coding vectors, and the coding of keywords by the KMA coding layer in the semantic representation model, the representation of the meaning of the target text by core words can be enhanced, thereby increasing the accuracy of semantic representation and improving the effect of semantic representation.

[0132] Next, the construction process of the semantic representation model will be introduced in this embodiment, wherein, in an optional implementation, the construction process of the semantic representation may specifically include the following steps A-C:

[0133] Step A: Obtain training sample text pairs, wherein the training sample text pairs include positive sample text pairs and negative sample text pairs of the same language, as well as positive sample text pairs and negative sample text pairs of different languages.

[0134] It should be noted that, in order to improve the semantic representation effect of the semantic representation model on cross-lingual texts, during model training, it is necessary to construct positive sample text pairs and negative sample text pairs of the same language, as well as positive sample text pairs and negative sample text pairs of different languages, which are collectively used as training sample text pairs to perform subsequent step B, so as to effectively improve the semantic representation ability of the model.

[0135] Specifically, in this implementation, in order to construct the semantic representation model, a large amount of preparation work needs to be done in advance. First, a large number of aligned text pairs in different languages, pairs with the same meaning in the same language, and two text sentences with different meanings in the same language (such as Figure 4The text includes phrases like "How to scan the code to join the group buying group" and "How to scan the code to join a group buying group," as well as two text sentences with different meanings in different languages. These include voice texts from people in daily conversations in various languages ​​such as Chinese, English, and Japanese, representing the same and different semantic understandings. These are used as positive and negative sample text pairs to form the training data for the model.

[0136] Furthermore, after obtaining training sample text pairs containing positive and negative sample text pairs, it is necessary to manually label the similarity classification labels of the training sample text pairs. For example, the similarity classification label of positive sample text pairs can be marked as 1, and the similarity classification label of negative sample text pairs can be marked as 0.

[0137] Step B: Extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector from the positive sample text pairs and negative sample text pairs.

[0138] After obtaining the training sample text pairs containing positive and negative sample text pairs through step A, the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive and negative sample text pairs can be further extracted. For the specific extraction method, please refer to step S102 above. The only difference is that the target text is replaced with each sample text in the positive and negative sample text pairs, which will not be elaborated here.

[0139] Step C: Input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and the negative sample text pair into the initial semantic representation model to train the semantic representation model.

[0140] One optional implementation is that the initial multimodal feature extraction model may include, but is not limited to, a first initial network model, a second initial network model, a concatenation layer, and an output layer. Both the first and second initial network models include an input layer, a transform encoder layer, a keyword mask attention KMA encoder layer, and a connection layer; the parameters of the first and second initial network models are shared. Figure 5 As shown, the first initial network model and the second initial network model are respectively Figure 5 The networks on the left and right sides both contain an input layer, a Transformer encoding layer, a keyword mask attention KMA encoding layer, and a connection layer, and their network parameters are shared.

[0141] After extracting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive and negative sample text pairs in step B, these vectors can be further input into the initial semantic representation model to predict the similarity between the two texts in the input training sample text pairs (i.e., positive or negative sample text pairs). This yields the similarity classification result for the training sample text pairs, which is then compared with manually labeled similarity classification tags. The model parameters are adjusted based on the comparison results (i.e., the differences between the two). This process can be repeated, employing a contrastive learning approach. Using the training sample text pairs (i.e., positive and negative sample text pairs), the model is trained through multiple rounds until the training termination condition is met (e.g., reaching a preset number of training iterations). At this point, the trained model is considered complete. Figure 5 The first network model that generates semantic representation vectors on the left can be used as a trained semantic representation model.

[0142] Specifically, one optional implementation is that when the training sample text pairs are positive sample text pairs, the specific implementation process of step C above may include the following steps C11-C17:

[0143] Step C11: Input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the first sample text in the positive sample text pair into the input layer of the first initial network model to obtain the first sample input vector; and input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the second sample text in the positive sample text pair into the input layer of the second initial network model to obtain the second sample input vector.

[0144] For example: Figure 5As shown in the example, assuming that in the positive sample text pair, the first sample text is "how to scan the code to enter the resident group", and the second sample text is "how to scan QR code into resident Group", when the language identification coding vector, entity word information coding vector, position coding vector and word vector of the two are input into the first initial network model and the second initial network model respectively, in addition to adding position embedding to each character, language embedding is additionally added. For example, the embedding "zh" corresponding to Chinese is added to each character, and the embedding "en" corresponding to English is added. Additional entityType embedding is also added. During training, all characters are added with entity type embedding. For example, each character in the original string is added with embedding whose index is "0", while "scan code (扫码)" is added with "1", corresponding to Figure 2 the first row of the entity type embedding matrix, which represents a core verb, and "resident group (居民群)" is added with "2", corresponding to Figure 2 the second row of the entity type embedding matrix, which represents a general entity word.

[0145] Step C12: input the first sample input vector into the transformer encoding layer of the first initial network model to obtain the first overall sample encoding vector and the sample encoding vector of each word in the first sample text; and input the second sample input vector into the transformer encoding layer of the second initial network model to obtain the second overall sample encoding vector and the sample encoding vector of each word in the second sample text.

[0146] After obtaining the first sample input vector and the second sample input vector through step C11, the first sample input vector integrated with entity information and language information can be further input into Figure 5 the transformer encoding layer shown in the left figure to obtain an intermediate representation vector of the whole sentence of the first sample text (e.g., "how to scan the code to enter the resident group") (this is defined as the first overall sample encoding vector, e.g., Figure 5 h<[BOS_never_used_51bce0c785ca2f68081bfa7d91973934]> on the left, which represents all information of the whole sentence, and learns which characters in the sentence play a role in the semantic representation of the whole sentence by virtue of the self-attention自带 in transformer) and the sample encoding vector of each word in the first sample text (which are defined as h1, h2…h n ), and input the second sample input vector integrated with entity information and language information into Figure 5The transformer encoding layer in the right-hand diagram obtains an intermediate representation vector of the entire sentence of the second sample text (e.g., "how to scan QR code into resident Group") (here, it is defined as the overall encoding vector of the second sample, e.g., ...). Figure 5 The right-hand side h(CLS) represents all the information of the entire sentence and, through the transformer's built-in attention, learns which words in the sentence contribute to the semantic representation of the entire sentence. It is also the sample encoding vector of each word in the second sample text.

[0147] Step C13: Input the sample encoding vector of each word in the first sample text into the KMA encoding layer of the first initial network model to obtain the first sample keyword encoding vector of the first sample text; and input the sample encoding vector of each word in the second sample text into the KMA encoding layer of the second initial network model to obtain the second sample keyword encoding vector of the second sample text.

[0148] It should be noted that even though the transformer encoding layer has the ability to learn which words contribute to the semantic representation of the entire sentence from the training text, deep models often cannot accurately capture key information in sentence pairs due to the lack of additional supervision information, thus failing to accurately distinguish the similarity of training sample text pairs. If the key entity words could be explicitly revealed during model training, it would be like drawing a small circle around the key points for the model, making it easier to learn accurate sentence representations. Therefore, this embodiment introduces a Keyword Mask Attention (KMA) encoding layer into the model, such as... Figure 5 As shown, this is very important for training semantic representation models.

[0149] Based on this, after obtaining the sample encoding vector of each word in the first sample text and the sample encoding vector of each word in the second sample text through step C12, the sample encoding vector of each word in the first sample text can be further input into the KMA encoding layer of the first initial network model to obtain the first sample keyword encoding vector of the first sample text; and the sample encoding vector of each word in the second sample text can be input into the KMA encoding layer of the second initial network model to obtain the second sample keyword encoding vector of the second sample text.

[0150] For example: Figure 6 As shown, for the training text "How to scan the code to join the resident group", after obtaining the sample encoding vector of each word through the transformer encoding layer, the encoding vector of each character and the encoding vector of each keyword can be calculated separately (i.e., ...). Figure 6 The dot product of the encoded vectors corresponding to each character in the "QR code" and "resident group" data is performed, such as... Figure 6 The arrows indicate how to obtain the keyword encoding vector for each word in the training text "How to scan the code to join the resident group". After concatenating these vectors, the keyword encoding vector for "How to scan the code to join the resident group" is obtained, as shown below. Figure 6 h(KMA) in the context.

[0151] In this way, the KMA mechanism can force the model to focus on the keyword differences in the training text, thereby learning the essential core of the training text and finally outputting the keyword encoding vector h(KMA). Unlike the vector h(CLS), both reflect the semantic representation of the whole sentence, but with different focuses.

[0152] Step C14: Input the first sample overall encoding vector and the first sample keyword encoding vector into the connection layer of the first initial network model to connect them, and obtain the first sample vector of the first sample text; input the second sample overall encoding vector and the second sample keyword encoding vector into the connection layer of the second initial network model to connect them, and obtain the second sample vector of the second sample text.

[0153] The first sample's overall encoding vector is obtained through steps C12 and C13 (e.g., ...). Figure 5 The left side h(CLS)) and the first sample keyword encoding vector (e.g. Figure 5 h(KMA) on the left, and the overall encoding vector of the second sample (e.g.) Figure 5 The right side of h(CLS)) and the second sample keyword encoding vector (e.g. Figure 5 After h(KMA) on the right, the overall encoding vector of the first sample (such as...) can be further encoded. Figure 5 The left side h(CLS)) and the first sample keyword encoding vector (e.g. Figure 5 The input on the left, h(KMA), is as follows: Figure 5 The first initial network model on the left connects the layers to obtain the first sample vector of the first sample text, which is defined as s1. Simultaneously, the overall encoding vector of the second sample (such as...) can be... Figure 5 The right side of h(CLS)) and the second sample keyword encoding vector (e.g. Figure 5 The input for h(KMA) on the right is as follows: Figure 5 The connection layers of the second initial network model on the right are connected to obtain the second sample vector of the second sample text, and it is defined as s2.

[0154] Step C15: Input the first sample vector and the second sample vector into the concatenation layer of the initial multimodal feature extraction model and concatenate them to obtain the first concatenated vector.

[0155] The first sample vector is obtained through step C14 (e.g.) Figure 5 The left side s1) and the second sample vector (e.g. Figure 5 After s2 on the right, the two can be further input into the splicing layer of the initial multimodal feature extraction model for splicing. The specific splicing method is not limited and can be set according to the actual situation. For example, s1 and s2 can be spliced ​​to obtain the first splicing vector as [s1, s2, s1*s2, s1-s2], so that the information of the first initial network model and the second initial network model on the left and right can be fully integrated.

[0156] Step C16: Classify using the first concatenated vector and output the first classification result through the output layer of the initial multimodal feature extraction model.

[0157] The first concatenation vector is obtained through step C15 (e.g.) Figure 5 After [s1, s2, s1*s2, s1-s2] shown, it can be further input into a fully connected layer so that the output layer outputs a classification result (defined here as the first classification result) that represents whether the first sample text and the second sample text in the current input positive sample text pair express the same meaning. That is, 0 or 1, where 0 means that the first sample text and the second sample text do not express the same meaning, and 1 means that the first sample text and the second sample text express the same meaning.

[0158] Step C17: Use the first classification result and the corresponding classification and recognition labels of the positive sample text to train the initial multimodal feature extraction model, and use the first initial network model obtained after training as the semantic representation model.

[0159] After obtaining the first classification result (0 or 1) corresponding to the positive sample text pair in step C16, it can be compared with the similarity classification label (1) corresponding to the manually labeled positive sample text pair. The model parameters are adjusted according to the comparison result (whether the two are consistent) until the training termination condition is met (such as the first classification result being consistent with the manually labeled classification label). At this point, the trained model is considered complete. Figure 5 The first network model that generates semantic representation vectors on the left can be used as a trained semantic representation model.

[0160] Similarly, another alternative implementation is that when the training sample text pairs are negative sample text pairs, the specific implementation process of step C above may include the following steps C21-C27:

[0161] Step C21: Input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the third sample text in the negative sample text pair into the input layer of the first initial network model to obtain the third sample input vector; and input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the fourth sample text in the negative sample text pair into the input layer of the second initial network model to obtain the fourth sample input vector.

[0162] Step C22: Input the third sample input vector into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the third sample and the sample encoding vector of each word in the third sample text; and input the fourth sample input vector into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the fourth sample and the sample encoding vector of each word in the fourth sample text.

[0163] Step C23: Input the sample encoding vector of each word in the third sample text into the KMA encoding layer of the first initial network model to obtain the third sample keyword encoding vector of the third sample text; and input the sample encoding vector of each word in the fourth sample text into the KMA encoding layer of the second initial network model to obtain the fourth sample keyword encoding vector of the fourth sample text.

[0164] Step C24: Input the overall encoding vector of the third sample and the keyword encoding vector of the third sample into the connection layer of the first initial network model to connect them, so as to obtain the third sample vector of the third sample text; and input the overall encoding vector of the fourth sample and the keyword encoding vector of the fourth sample into the connection layer of the second initial network model to connect them, so as to obtain the fourth sample vector of the fourth sample text.

[0165] Step C25: Input the third sample vector and the fourth sample vector into the concatenation layer of the initial multimodal feature extraction model and concatenate them to obtain the second concatenated vector.

[0166] Step C26: Classify using the second concatenated vector, and output the second classification result through the output layer of the initial multimodal feature extraction model.

[0167] Step C27: Use the second classification result and the corresponding classification and recognition labels of the negative sample text to train the initial multimodal feature extraction model, and use the first initial network model obtained after training as the semantic representation model.

[0168] It should be noted that the implementation principle of steps C21-C27 above is the same as that of steps C11-17 above. The only difference is that the positive sample text pairs are replaced with negative sample text pairs for model training. The specific training process can be referred to the implementation process of steps C11-17 above, and will not be repeated here.

[0169] Based on this, after training and generating a semantic representation model from the training sample text, the generated semantic representation model can be further validated using validation text. The specific validation process may include the following steps (1)-(3):

[0170] Step (1): Obtain the verification text pair; and extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pair.

[0171] In this embodiment, in order to verify the semantic representation model, it is first necessary to obtain verification text pairs. For example, 1,000 text pairs with the same and different meanings in different languages ​​can be downloaded from the web page as verification text pairs. Here, verification text pairs refer to text information that can be used to verify the semantic representation model. After obtaining these verification text pairs, the extraction method described in step S102 above can be used to replace the target text with the verification text in the verification text pair, and extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pair for subsequent steps (2).

[0172] Step (2): Input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pair into the initial semantic representation model to predict the classification result corresponding to the verification text pair.

[0173] After obtaining the verification text pair in step (1) and extracting the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pair, the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pair can be further input into the initial semantic representation model to predict the classification result corresponding to the verification text pair, so as to execute the subsequent step (3).

[0174] Step (3): When the classification result corresponding to the verification text pair is inconsistent with the classification label result corresponding to the verification text pair, the verification text pair is used again as the training sample text pair to update the initial semantic representation model and obtain the updated semantic representation model.

[0175] After predicting the classification result (i.e. 0 or 1) corresponding to the verification text pair through step (2), if the predicted classification result is inconsistent with the actual classification result (i.e., the manually labeled similarity classification label 0 or 1) corresponding to the verification text pair, the verification text pair can be used as a training sample text pair again to update the parameters of the initial semantic representation model, and the first initial network model in the updated initial semantic representation model can be used as the updated semantic representation model.

[0176] Through the above embodiments, the semantic representation model can be effectively validated using verification text pairs. When the classification result predicted by the model is inconsistent with the actual classification result corresponding to the verification text pair (i.e., the manually labeled similarity classification label 0 or 1), the semantic representation model can be adjusted and updated in a timely manner, thereby helping to improve the prediction accuracy and precision of the model.

[0177] In summary, the semantic representation method provided in this embodiment first obtains the target text to be processed, and then extracts the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text. Next, the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text are input into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text. The semantic representation model is obtained by comparing and training positive and negative sample text pairs of the same language and positive and negative sample text pairs of different languages ​​using language identifiers and entity databases.

[0178] As can be seen, since the semantic representation model pre-built in this application is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases, it is possible to achieve cross-language semantic representation without the need for single-language model customization when performing semantic representation. This effectively improves the representation effect and is beneficial for subsequent multilingual related tasks.

[0179] Second Embodiment

[0180] This embodiment will introduce a semantic representation device; please refer to the above method embodiment for related content.

[0181] See Figure 7 This is a schematic diagram of the composition of a semantic representation device provided in this embodiment. The device 700 includes:

[0182] The first acquisition unit 701 is used to acquire the target text to be processed;

[0183] The first extraction unit 702 is used to extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text;

[0184] The first prediction unit 703 is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text.

[0185] The semantic representation model is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases.

[0186] In one implementation of this embodiment, the apparatus further includes:

[0187] The second acquisition unit is used to acquire training sample text pairs, which include positive and negative sample text pairs in the same language, as well as positive and negative sample text pairs in different languages.

[0188] The second extraction unit is used to extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the positive sample text pair and the negative sample text pair;

[0189] The training unit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the positive sample text pair and negative sample text pair into the initial semantic representation model, and train the semantic representation model to obtain the semantic representation model;

[0190] The initial multimodal feature extraction model includes a first initial network model, a second initial network model, a concatenation layer, and an output layer; both the first and second initial network models include an input layer, a transformer encoding layer, a keyword mask attention KMA encoding layer, and a connection layer; the parameters of the first and second initial network models are shared.

[0191] In one implementation of this embodiment, the training unit includes:

[0192] The first input subunit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the first sample text in the positive sample text pair into the input layer of the first initial network model to obtain the first sample input vector; and to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the second sample text in the positive sample text pair into the input layer of the second initial network model to obtain the second sample input vector;

[0193] The second input subunit is used to input the first sample input vector into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the first sample and the sample encoding vector of each word in the first sample text; and to input the second sample input vector into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the second sample and the sample encoding vector of each word in the second sample text.

[0194] The third input subunit is used to input the sample encoding vector of each word in the first sample text into the KMA encoding layer of the first initial network model to obtain the first sample keyword encoding vector of the first sample text; and to input the sample encoding vector of each word in the second sample text into the KMA encoding layer of the second initial network model to obtain the second sample keyword encoding vector of the second sample text.

[0195] The first connection subunit is used to input the first sample overall encoding vector and the first sample keyword encoding vector into the connection layer of the first initial network model for connection to obtain the first sample vector of the first sample text; and to input the second sample overall encoding vector and the second sample keyword encoding vector into the connection layer of the second initial network model for connection to obtain the second sample vector of the second sample text.

[0196] The first splicing subunit is used to input the first sample vector and the second sample vector into the splicing layer of the initial multimodal feature extraction model for splicing to obtain the first splicing vector;

[0197] The first classification subunit is used to classify using the first concatenated vector and output the first classification result through the output layer of the initial multimodal feature extraction model;

[0198] The first training subunit is used to train the initial multimodal feature extraction model using the first classification result and the corresponding classification and recognition labels of the positive sample text, and to use the first initial network model obtained after training as the semantic representation model.

[0199] In one implementation of this embodiment, the training unit includes:

[0200] The fourth input subunit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the third sample text in the negative sample text pair into the input layer of the first initial network model to obtain the third sample input vector; and to input the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the fourth sample text in the negative sample text pair into the input layer of the second initial network model to obtain the fourth sample input vector;

[0201] The fifth input subunit is used to input the third sample input vector into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the third sample and the sample encoding vector of each word in the third sample text; and to input the fourth sample input vector into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the fourth sample and the sample encoding vector of each word in the fourth sample text.

[0202] The sixth input subunit is used to input the sample encoding vector of each word in the third sample text into the KMA encoding layer of the first initial network model to obtain the third sample keyword encoding vector of the third sample text; and to input the sample encoding vector of each word in the fourth sample text into the KMA encoding layer of the second initial network model to obtain the fourth sample keyword encoding vector of the fourth sample text.

[0203] The second connection subunit is used to input the overall encoding vector of the third sample and the keyword encoding vector of the third sample into the connection layer of the first initial network model for connection to obtain the third sample vector of the third sample text; and to input the overall encoding vector of the fourth sample and the keyword encoding vector of the fourth sample into the connection layer of the second initial network model for connection to obtain the fourth sample vector of the fourth sample text.

[0204] The second splicing subunit is used to input the third sample vector and the fourth sample vector into the splicing layer of the initial multimodal feature extraction model for splicing to obtain the second splicing vector;

[0205] The second classification subunit is used to classify using the second concatenated vector and output the second classification result through the output layer of the initial multimodal feature extraction model;

[0206] The second training subunit is used to train the initial multimodal feature extraction model using the second classification result and the corresponding classification and recognition labels of the negative sample text, and to use the first initial network model obtained after training as the semantic representation model.

[0207] In one implementation of this embodiment, the apparatus further includes:

[0208] The third acquisition unit is used to acquire verification text pairs; and extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pairs;

[0209] The second prediction unit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the verification text pair into the initial semantic representation model to predict the classification result corresponding to the verification text pair;

[0210] The update unit is used to update the initial semantic representation model by reusing the verification text pair as the training sample text pair when the classification result corresponding to the verification text pair is inconsistent with the classification label result corresponding to the verification text pair, so as to obtain the updated semantic representation model.

[0211] In one implementation of this embodiment, the first prediction unit 703 includes:

[0212] The seventh input subunit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text into the input layer of the pre-constructed semantic representation model to obtain the input vector;

[0213] The first encoding subunit is used to input the input vector into the transformer encoding layer of the semantic representation model for encoding, so as to obtain the overall encoding vector of the target text and the encoding vector of each word;

[0214] The second encoding subunit is used to input the encoding vector of each word in the target text into the KMA encoding layer of the semantic representation model to encode keywords and obtain keyword encoding vectors.

[0215] The third connection subunit is used to input the overall encoding vector of the target text and the keyword encoding vector into the connection layer of the semantic representation model for connection, and use the resulting connection vector as the semantic representation vector corresponding to the target text.

[0216] In one implementation of this embodiment, the second encoding subunit includes:

[0217] The computational subunit is used to input the encoding vector of each word in the target text into the KMA encoding layer of the semantic representation model, and to perform dot product calculation on the encoding vector of each word and the encoding vector of each keyword in the target text to obtain the keyword encoding vector corresponding to each word in the target text.

[0218] The third splicing subunit is used to splice the keyword encoding vectors corresponding to each word in the target text, and use the spliced ​​vector as the keyword encoding vector corresponding to the target text.

[0219] In one implementation of this embodiment, the entity word information encoding vector includes a core verb encoding vector, a general entity word encoding vector, a person name encoding vector, an organization name encoding vector, and a place name encoding vector.

[0220] Furthermore, embodiments of this application also provide a semantic representation device, including: a processor, a memory, and a system bus;

[0221] The processor and the memory are connected via the system bus;

[0222] The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform any of the above-described implementations of the semantic representation method.

[0223] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the semantic representation method.

[0224] Furthermore, this application embodiment also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described semantic representation methods.

[0225] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0226] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0227] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0228] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A semantic representation method, characterized in that, include: Obtain the target text to be processed; Extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text; The language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text are input into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text. The semantic representation model is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases. The entity database is obtained by aligning and translating a Chinese entity database. The semantic representation model includes an input layer, a transformer encoding layer, a keyword mask attention KMA encoding layer, and a connection layer. The step of inputting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text includes: The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the target text are input into the input layer of the pre-constructed semantic representation model to obtain the input vector; The input vector is fed into the transformer encoding layer of the semantic representation model for encoding to obtain the overall encoding vector of the target text and the encoding vector of each word; The encoding vector of each word in the target text is input into the KMA encoding layer of the semantic representation model. The encoding vector of each word is multiplied by the encoding vector of each keyword in the target text to obtain the keyword encoding vector corresponding to each word in the target text. The keyword encoding vectors corresponding to each word in the target text are concatenated, and the resulting concatenated vector is used as the keyword encoding vector corresponding to the target text. The overall encoding vector of the target text and the keyword encoding vector are input into the connection layer of the semantic representation model and connected, and the resulting connection vector is used as the semantic representation vector corresponding to the target text.

2. The method according to claim 1, characterized in that, The semantic representation model is constructed as follows: Obtain training sample text pairs, which include positive and negative sample text pairs in the same language, as well as positive and negative sample text pairs in different languages. Extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and the negative sample text pair; The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and negative sample text pair are input into the initial semantic representation model to train the semantic representation model. The initial semantic representation model includes a first initial network model, a second initial network model, a splicing layer, and an output layer; both the first and second initial network models include an input layer, a transformer encoding layer, a keyword mask attention KMA encoding layer, and a connection layer; the parameters of the first and second initial network models are shared.

3. The method according to claim 2, characterized in that, The step of inputting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and the negative sample text pair into the initial semantic representation model, and training the semantic representation model, includes: The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the first sample text in the positive sample text pair are input into the input layer of the first initial network model to obtain the first sample input vector; and the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the second sample text in the positive sample text pair are input into the input layer of the second initial network model to obtain the second sample input vector. The first sample input vector is input into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the first sample and the sample encoding vector of each word in the first sample text; and the second sample input vector is input into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the second sample and the sample encoding vector of each word in the second sample text. The sample encoding vector of each word in the first sample text is input into the KMA encoding layer of the first initial network model to obtain the first sample keyword encoding vector of the first sample text; and the sample encoding vector of each word in the second sample text is input into the KMA encoding layer of the second initial network model to obtain the second sample keyword encoding vector of the second sample text. The first sample overall encoding vector and the first sample keyword encoding vector are input into the connection layer of the first initial network model and connected to obtain the first sample vector of the first sample text; the second sample overall encoding vector and the second sample keyword encoding vector are input into the connection layer of the second initial network model and connected to obtain the second sample vector of the second sample text. The first sample vector and the second sample vector are input into the concatenation layer of the initial semantic representation model and concatenated to obtain the first concatenated vector; The first concatenated vector is used for classification, and the first classification result is output through the output layer of the initial semantic representation model. The initial semantic representation model is trained using the first classification result and the corresponding classification and recognition labels of the positive sample text, and the first initial network model obtained after training is used as the semantic representation model.

4. The method according to claim 2, characterized in that, The step of inputting the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the positive sample text pair and the negative sample text pair into the initial semantic representation model, and training the semantic representation model, includes: The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the third sample text in the negative sample text pair are input into the input layer of the first initial network model to obtain the third sample input vector; and the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the fourth sample text in the negative sample text pair are input into the input layer of the second initial network model to obtain the fourth sample input vector. The third sample input vector is input into the trasnformer encoding layer of the first initial network model to obtain the overall encoding vector of the third sample and the sample encoding vector of each word in the third sample text; and the fourth sample input vector is input into the trasnformer encoding layer of the second initial network model to obtain the overall encoding vector of the fourth sample and the sample encoding vector of each word in the fourth sample text. The sample encoding vector of each word in the third sample text is input into the KMA encoding layer of the first initial network model to obtain the third sample keyword encoding vector of the third sample text; and the sample encoding vector of each word in the fourth sample text is input into the KMA encoding layer of the second initial network model to obtain the fourth sample keyword encoding vector of the fourth sample text. The overall encoding vector of the third sample and the keyword encoding vector of the third sample are input into the connection layer of the first initial network model and connected to obtain the third sample vector of the third sample text; the overall encoding vector of the fourth sample and the keyword encoding vector of the fourth sample are input into the connection layer of the second initial network model and connected to obtain the fourth sample vector of the fourth sample text. The third sample vector and the fourth sample vector are input into the concatenation layer of the initial semantic representation model and concatenated to obtain the second concatenated vector; The second concatenated vector is used for classification, and the second classification result is output through the output layer of the initial semantic representation model. The initial semantic representation model is trained using the second classification result and the corresponding classification and recognition labels of the negative sample text, and the first initial network model obtained after training is used as the semantic representation model.

5. The method according to claim 2, characterized in that, The method further includes: Obtain the verification text pair; and extract the language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the verification text pair; The language identifier encoding vector, entity word information encoding vector, position encoding vector, and word vector of the verification text pair are input into the initial semantic representation model to predict the classification result corresponding to the verification text pair; When the classification result corresponding to the verification text pair is inconsistent with the classification label result corresponding to the verification text pair, the verification text pair is used again as the training sample text pair to update the initial semantic representation model, so as to obtain the updated semantic representation model.

6. The method according to any one of claims 1-5, characterized in that, The entity word information encoding vector includes core verb encoding vector, general entity word encoding vector, personal name encoding vector, organization name encoding vector, and place name encoding vector.

7. A semantic representation device, characterized in that, include: The first acquisition unit is used to acquire the target text to be processed; The first extraction unit is used to extract the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text; The first prediction unit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text into a pre-constructed semantic representation model to predict the semantic representation vector corresponding to the target text. The semantic representation model is obtained by comparing and training positive and negative sample text pairs in the same language and positive and negative sample text pairs in different languages ​​using language identifiers and entity databases. The entity database is obtained by aligning and translating a Chinese entity database. The semantic representation model includes an input layer, a transformer encoding layer, a keyword mask attention KMA encoding layer, and a connection layer. The first prediction unit includes: The seventh input subunit is used to input the language identifier encoding vector, entity word information encoding vector, position encoding vector and word vector of the target text into the input layer of the pre-constructed semantic representation model to obtain the input vector; The first encoding subunit is used to input the input vector into the transformer encoding layer of the semantic representation model for encoding, so as to obtain the overall encoding vector of the target text and the encoding vector of each word; The computational subunit is used to input the encoding vector of each word in the target text into the KMA encoding layer of the semantic representation model, and to perform dot product calculation on the encoding vector of each word and the encoding vector of each keyword in the target text to obtain the keyword encoding vector corresponding to each word in the target text. The third splicing subunit is used to splice the keyword encoding vectors corresponding to each word in the target text, and use the resulting spliced ​​vector as the keyword encoding vector corresponding to the target text. The third connection subunit is used to input the overall encoding vector of the target text and the keyword encoding vector into the connection layer of the semantic representation model for connection, and use the resulting connection vector as the semantic representation vector corresponding to the target text.

8. A semantic representation device, characterized in that, include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method described in any one of claims 1-6.

Images