Methods, devices, apparatuses, and computer-readable storage media for constructing a contextually aggregated encoding representation of a text element

Abstract

A method of constructing a context aggregated encoding representation of a text element is disclosed, comprising: obtaining at least two context texts in which a target text element is located; determining an implicit sequence pattern and a pattern support degree of the implicit sequence pattern; obtaining a local encoding representation of at least part of text elements in the at least two context texts; and aggregating the local encoding representation of the at least part of text elements using the pattern support degrees of different implicit sequence patterns to obtain the context aggregated encoding representation of the target text element. Corresponding apparatuses, devices, computer readable storage media, etc. are also disclosed.

Description

TECHNICAL FIELD

[0001] The present application relates to the construction of an encoded representation of a text element, and more particularly, to a method, an apparatus, a device, a computer readable storage medium and the like for constructing a context-aggregated encoded representation of a text element. BACKGROUND

[0002] Encoded representations of words and similarity computations based thereon are common steps in natural language processing and are difficult problems to be solved in many application scenarios, including machine reading, synonym recognition, natural language understanding, question answering systems, term construction, sentiment analysis, text classification, and the like. Measuring the similarity of the semantics between two words often plays a crucial role in downstream tasks. For example, in machine reading, the semantics of the article to be read must first be learned, and the words mentioned in the context of the article are not the same, but are often strongly related in semantics. Identifying the relationship between words in the context is undoubtedly important for downstream understanding tasks. For another example, in a question answering system, providing an accurate answer to a given question relies on accurate matching and understanding of the question, which also involves determining the relationship between words in the context. For yet another example, in sentiment analysis applications, accurate synonym aggregation of the evaluation object must be performed to achieve better classification.

[0003] There are some technical solutions for word similarity computation. For example, word similarity computation based on a word forest, which is based on a synonym forest. The synonym forest classification adopts a hierarchical system, and all included words are organized together according to a tree-like hierarchical structure, divided into three categories: large, medium, and small. These words are further divided into several word groups according to the proximity and relevance of the word meaning, with a five-layer structure. As the level increases, the word meaning is described in more detail. When computing the similarity of two words, the result is obtained by analyzing their relative positions in the word forest. For another example, word similarity computation based on HowNet, which describes the semantic features of a single concept through basic meaning primitives, the grammatical features of a word through grammatical meaning primitives, and the relationships between concepts through relationship meaning primitives. For the similarity computation of two words, it is reduced to the similarity problem of two meaning items (concepts), i.e., if the words have meaning items: , have meaning items: , then the similarity of the words and is the maximum value of the similarity of each meaning item, i.e., .

[0004] However, the methods based on the HowNet and the CNKI both rely on expert knowledge and artificial rules, on the one hand, there are time-consuming and laborious disadvantages, on the other hand, the method based on the knowledge base has poor generalization ability, and the user who is not a language expert encounters an unregistered word and has an unfriendly problem. SUMMARY

[0005] Embodiments of the present application provide a method, apparatus, device and computer readable storage medium for constructing a context-aggregated encoding representation of a text element.

[0006] According to a first aspect of the present application, there is provided a method for constructing a context-aggregated encoding representation of a text element, comprising the steps of:

[0007] obtaining at least two context texts in which a target text element is located from a text to be processed;

[0008] determining an implied sequence pattern and a pattern support degree of the implied sequence pattern for each of the at least two context texts, the implied sequence pattern being a frequent sub-sequence of the at least two context texts, the frequent sub-sequence being a sub-sequence whose ratio of the number of context texts including the frequent sub-sequence to the number of the at least two context texts is greater than or equal to a predetermined pattern support degree threshold, and the pattern support degree of the implied sequence pattern being the ratio of the number of context texts including the corresponding implied sequence pattern to the number of the at least two context texts;

[0009] obtaining a local encoding representation of at least part of the text elements in the at least two context texts, the local encoding representation being an encoding representation generated based on a single context text; and

[0010] aggregating the local encoding representations of the at least part of the text elements using the pattern support degrees of the respective different implied sequence patterns to obtain the context-aggregated encoding representation of the target text element.

[0011] In one embodiment, the determining of the implied sequence pattern comprises applying frequent sequence pattern mining to the at least two context texts, and selecting one frequent sub-sequence corresponding to each of the at least two context texts as the implied sequence pattern of the corresponding context text, and removing the duplicates to obtain the respective different implied sequence patterns of the at least two context texts.

[0012] In one embodiment, the implied sequence pattern of the context text is the longest frequent sub-sequence corresponding to the context text.

[0013] In one embodiment, the determining of the implied sequence pattern for each of the at least two context texts comprises:

[0014] processing each of the at least two context texts to generate at least two sequences corresponding to the at least two context texts respectively, each sequence being formed by text elements of the respective context text in the order of appearance in the respective context text;

[0015] selecting, in the at least two sequences, one or more subsequences of text elements as the frequent subsequences, wherein a ratio of a number of sequences containing the subsequence to a number of the at least two sequences is greater than or equal to the predetermined pattern support threshold; and

[0016] for each sequence of the at least two sequences, determining, from the frequent subsequences, frequent subsequences belonging to the sequence, and selecting, from the frequent subsequences belonging to the sequence, a longest frequent subsequence as the implicit sequential pattern of the respective sequence.

[0017] In an embodiment, the selecting, in the at least two sequences, one or more subsequences of text elements as the frequent subsequences comprises:

[0018] counting, for each text element in the at least two sequences, a frequency ratio of the text element, the frequency ratio of the text element being a ratio of a number of sequences of the at least two sequences containing the text element to a number of sequences of the at least two sequences, determining, as a prefix of unit length 1, a respective text element whose frequency ratio is greater than or equal to the predetermined pattern support threshold, and deleting, from the at least two sequences, a respective text element whose frequency ratio is less than the predetermined pattern support threshold;

[0019] for each i value in ascending order, where i is an integer, 1≤i≤K, K being a maximum value of a number of text elements in each sequence after the deleting, performing the following steps:

[0020] performing a determining step of determining a prefix of unit length i as a frequent subsequence, and for each prefix of unit length i, determining, from each sequence after the deleting, a set of projected sequences of the prefix, and counting, for each text element in the set of projected sequences, a projected frequency ratio of the text element, the projected frequency ratio of the text element being a ratio of a number of projected sequences of the set of projected sequences containing the text element to a number of sequences of the at least two sequences,

[0021] for each prefix of unit length i, when the projected frequency ratio is greater than or equal to the predetermined pattern support threshold, combining a respective text element with the prefix to generate a prefix of unit length i=i+1, and going to the determining step.

[0022] In one embodiment, when the at least two sequences both include the target text element, the at least partial text elements are all text elements in the longest frequent subsequence of the respective sequence with the target text element removed.

[0023] In one embodiment, when the at least two sequences both do not include the target text element, the at least partial text elements are all text elements in the longest frequent subsequence of the respective sequence.

[0024] In one embodiment, the text element is a character, a word, or a phrase, and the at least two context texts from which the target text element is obtained from the text to be processed include:

[0025] At least two sentences including the target text element are obtained, divided by punctuation marks, and the punctuation marks, letters, and numbers in the at least two sentences are removed to generate the at least two context texts.

[0026] In one embodiment, the text element is a character, a word, or a phrase, and the at least two context texts from which the target text element is obtained from the text to be processed include:

[0027] At least two sentences including the target text element are obtained, divided by punctuation marks, and the punctuation marks, letters, and numbers in the at least two sentences are removed to generate the at least two context texts.

[0028] In one embodiment, the obtaining the local encoding representation of the at least partial text elements in the at least two context texts includes: generating the local encoding representation of the at least partial text elements in the at least two context texts by using a Word2Vec model.

[0029] In one embodiment, the target text element and the category of the text element both include any one of the following categories: character, word, phrase, sentence, paragraph, and article, and the category of the target text element and the text element is the same.

[0030] In one embodiment, when the target text element and the category of the text element are both characters, determining the implicit sequence pattern for each of the at least two context texts includes:

[0031] performing character segmentation on each of the at least two context texts and generating a character sequence in the order of the appearance of each character in the respective context text, thereby obtaining at least two character sequences; and

[0032] performing frequent sequence pattern mining on the at least two character sequences to obtain the longest frequent subsequence of each of the at least two character sequences as the implicit sequence pattern of the respective character sequence.

[0033] In one embodiment, the determining the implicit sequential pattern for each of the at least two contextual texts, when the target text element and the category of the text element are both words, comprises:

[0034] performing tokenization on each of the at least two contextual texts and generating a word sequence according to the order of the words appearing in the corresponding contextual text, thereby obtaining at least two word sequences; and

[0035] performing frequent sequential pattern mining on the at least two word sequences to obtain a longest frequent sub-sequence of each of the at least two word sequences as the implicit sequential pattern of the corresponding word sequence.

[0036] In one embodiment, the aggregating the local encoding representations of the at least partial text elements using the pattern support of each of the different implicit sequential patterns to obtain the contextual aggregated encoding representation of the target text element comprises:

[0037] determining the text elements included in the different implicit sequential patterns as the at least partial text elements;

[0038] weighting and summing the local encoding representations of the text elements in each of the different implicit sequential patterns using the pattern support of the corresponding implicit sequential pattern, and taking the result of the weighted sum for each of the different implicit sequential patterns as the contextual aggregated encoding representation of the target text element.

[0039] In one embodiment, the determining the implicit sequential pattern further comprises: deleting the implicit sequential pattern with a unit length of 1 from the de-duplicated implicit sequential patterns, and performing the aggregating using the deleted implicit sequential patterns.

[0040] According to a second aspect of the present disclosure, there is provided a method for determining the similarity between text elements by constructing the contextual aggregated encoding representation of the text elements using the method according to the first aspect of the present disclosure, comprising: obtaining the contextual aggregated encoding representation of the text elements; and calculating the cosine distance or Euclidean distance between the contextual aggregated encoding representations of the text elements as the similarity between the text elements, wherein the contextual aggregated encoding representation is a vector representation.

[0041] According to a third aspect of the present disclosure, there is provided an apparatus for constructing the contextual aggregated encoding representation of text elements, comprising:

[0042] a contextual text obtaining module configured to obtain at least two contextual texts in which a target text element is located from a text to be processed;

[0043] A determination module is configured to determine a hidden sequence pattern and a pattern support for each of the at least two context texts. The determination of the hidden sequence pattern includes: applying frequent sequence pattern mining to the at least two context texts, selecting a frequent subsequence corresponding to each of the at least two context texts as the hidden sequence pattern of the corresponding context text, and obtaining each different hidden sequence pattern of the at least two context texts after deduplication. The pattern support is the ratio of the number of context texts including the corresponding pattern to the number of the at least two context texts. The frequent subsequence is a subsequence whose ratio of the number of context texts including the frequent subsequence to the number of the at least two context texts is greater than or equal to a predetermined pattern support threshold.

[0044] A local encoding representation acquisition module, configured to acquire local encoding representations of at least a portion of text elements in the at least two context texts, the local encoding representations being generated based on a single context text; and

[0045] An aggregation module is configured to aggregate the local encoded representations of at least a portion of the text element using the pattern support of various hidden sequence patterns to obtain a context-aggregated encoded representation of the target text element.

[0046] According to a fourth aspect of this disclosure, an apparatus for determining the similarity of text elements is provided, comprising: a context aggregation encoding representation acquisition module configured to acquire a context aggregation encoding representation of a text element generated using the apparatus for constructing a context aggregation encoding representation of a text element as described in claim 16; and a similarity determination module configured to calculate a cosine distance or Euclidean distance between the context aggregation encoding representations of the text elements as a similarity between the text elements, wherein the context aggregation encoding representation is a vector representation.

[0047] According to a fifth aspect of this disclosure, an apparatus for constructing a context-aggregated encoded representation of text elements is provided, comprising: a processor; and

[0048] A memory configured to store computer-executable instructions thereon, which, when executed in the processor, cause the implementation of the method according to the first aspect of this disclosure.

[0049] According to a sixth aspect of this disclosure, an apparatus for determining the similarity of text elements is provided, comprising: a processor; and a memory configured to store computer-executable instructions thereon, which, when executed in the processor, cause the implementation of the method of the second aspect.

[0050] According to a seventh aspect of the present disclosure, there is provided a computer readable storage medium having instructions stored therein, which when executed on a computer, cause the computer to implement the method according to the first aspect and the second aspect of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative effort based on these drawings.

[0052] Figure 1 FIG. 1 illustrates an exemplary environment architecture diagram to which the method of constructing a context-aggregated encoding representation of a text element and determining text similarity according to embodiments of the present application can be applied.

[0053] Figure 2a FIG. 2 illustrates a flowchart of the method of constructing a context-aggregated encoding representation of a text element according to embodiments of the present application.

[0054] Figure 2b FIG. 3 illustrates a further flowchart of the method of constructing a context-aggregated encoding representation of a text element according to embodiments of the present application.

[0055] Figure 2c FIG. 4 illustrates a still further flowchart of the method of constructing a context-aggregated encoding representation of a text element according to embodiments of the present application.

[0056] Figure 2d FIG. 5 illustrates a flowchart of the method of determining text similarity according to embodiments of the present application.

[0057] Figure 3 FIG. 6 illustrates a specific application embodiment of the present application in a question-answering scenario.

[0058] Figure 4a FIG. 7 illustrates a block diagram of an apparatus for constructing a context-aggregated encoding representation of a text element according to embodiments of the present application.

[0059] Figure 4b FIG. 8 illustrates a block diagram of an apparatus for determining text similarity according to embodiments of the present application.

[0060] Figure 5 FIG. 9 illustrates a hardware environment diagram related to constructing a context-aggregated encoding representation of a text element according to embodiments of the present application. DETAILED DESCRIPTION

[0061] For the purposes of the present application, the technical solutions and advantages will be more apparent, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0062] As referred to herein, a text includes one or more paragraphs, a collection of several sentences, one or more articles, etc., and is generally composed of a plurality of sentences. As referred to herein, a text element includes, for example, a word or a character in the text.

[0063] A sequence, as generally understood in the art, is composed of items, and the items have a temporal order. In this context, an item refers to a text element, such as a word or a character.

[0064] A sequence A is a sub-sequence of sequence B if all the item sets of A can be found in the item sets of B. According to this definition, for sequences and sequence if there exists a numerical sequence satisfying then A is called a sub-sequence of B. In the reverse, B is called a super-sequence of A.

[0065] A frequent sub-sequence is a sub-sequence that frequently occurs. A threshold is generally set, and a word sequence greater than or equal to the threshold is called a frequent sub-sequence. Since it is frequently occurring, it can be considered as a pattern, and thus the frequent sub-sequence is also called a frequent sequence pattern or a sequence pattern.

[0066] The purpose of frequent sequence pattern mining is to mine frequent sub-sequences in each sequence.

[0067] In the process of frequent sequence pattern mining of sequences, the concepts of prefix, projected sequence set, and implied sequence pattern are involved. According to mathematical description, a prefix is: for sequences A={a1,a2,...an} and B={b1,b2,...bm}, n≤m, satisfying a1=b1, a2=b2...an−1=bn−1, and an⊆bn, then A is called a prefix of B. For example, for sequence data B= <aabcacdcf>and A = 0 <aabca>If A is a prefix of B, then A is a prefix of B. Of course, B has more than one prefix, such as , <aa> , <aab>All are prefixes of B. The projection sequence is relative to a specific prefix, and the projection sequence of the prefix is the suffix, and the prefix plus the suffix can constitute a complete sequence. For a certain prefix, the subsequence left after the prefix in the sequence is the suffix. If multiple sequences have the same prefix, the suffixes of the prefix in the multiple sequences together constitute the projection sequence set of the sequence, wherein the suffix of the prefix in each sequence is an element in the projection sequence set concept.

[0068] The implicit sequence mode referred to in the present disclosure refers to the frequent subsequence selected after frequent sequence mining and considered to reflect the context characteristics of the sequence.

[0069] The mode support degree of the frequent subsequence refers to the frequency ratio of the frequent subsequence, that is, the ratio of the number of sequences containing the frequent subsequence to the total number of sequences.

[0070] Text elements include words, phrases, sentences, paragraphs, articles, and various different granularity texts.

[0071] Context, i.e. context, is a concept of linguistics (linguistics, sociolinguistics, discourse analysis, pragmatics, semiotics, etc.), and the context text referred to in the present disclosure is relative to the target text element, which is the text in the text window where the target text element is located. The granularity is larger than the target text element, for example, when the target text element is a word, a phrase, or a sentence, the context can be a sentence or a paragraph where the target text element is located, when the target text element is a paragraph, the context can be several paragraphs or articles where the target text element is located, and so on.

[0072] The Word2Vec model mentioned in the present disclosure is a word vector generation model proposed by Tomas Mikolov et al. in 2013. Word2Vec gives two training ideas, namely CBOW model (Continuous Bag-of-Words Model) and Skip-gram model (Continuous Skip-gram Model), which respectively describe the relationship between the current word and the context from different angles.

[0073] Referring to< / aab> < / aa> Figure 1 , Figure 1 Fig. 1 illustrates an exemplary environment architecture diagram to which the method of constructing a context-aggregated encoding representation of a text element and determining text similarity provided by an embodiment of the present application can be applied. The environment architecture can include at least one server 101, which can be connected to at least one terminal 103 through a network 102. Among them, the at least one terminal 103 can be any terminal capable of running an application, including but not limited to a smart phone, a desktop computer, a laptop computer, a tablet computer, a game console, a vehicle-mounted computer, a smart speaker, and the like. The application at least includes an application based on word similarity, including but not limited to an application associated with machine reading, synonym recognition, natural language understanding, question and answer system, and term construction.

[0074] The server 101 can be any computer device capable of providing machine response services, which can be a server configured to support predetermined services, a server cluster, a cloud server, or the like. The server 101 can support the application based on word similarity on the terminal 103. The server 101 can support different terminals based on cloud cooperation, or can independently support different terminals, for example, the terminals can be divided according to the IP address or application type of the terminal login, and then assigned to different servers. Of course, the terminals can also be assigned to different servers in response to real-time centralized scheduling.

[0075] In some examples, the terminal 103 itself can independently support the application based on word similarity without relying on the server 101. Generally, the user triggers a request for server support functions by operating the application on the terminal. It should be noted that "user" herein should be understood broadly, which can be a person or any functional entity capable of operating the application, for example, a functional entity that triggers the operation of the application based on the analysis of the operation behavior of the person.

[0076] The network 102 can include but is not limited to a wide area network, a local area network, a wired network, a wireless network, or any combination thereof.

[0077] The existing encoding representation method of text elements based on single context text, such as Word2Vec model, determines the vector of a text element based on the context within a window in which the text element is located, so that other context information in the entire text to be processed cannot be utilized, and the importance of the text element cannot be reflected. In the case of single context with low quality, too much noise will inevitably be introduced, for example, occasionally appearing context is mistakenly considered to have an important influence on the semantics of the text element, resulting in an insufficient prepared encoding representation, which will also lead to inaccurate judgment of word semantics and similarity between words. Embodiments of the present disclosure hope to at least partially solve this problem.

[0078] The server 101 or the terminal 103 can determine (for example, by calculating) the similarity of words in response to the operation of the user, and in the determination process, the weighted calculation of two or more context sequences is considered, which can better combine and utilize multiple context information to measure and calculate the semantics of words, and can more comprehensively reflect the semantic representation and context of the context in which the words are located, thereby improving the accuracy of word semantics and similarity between words. In some examples, by further mining the frequent patterns of context sequences, the noise introduced by using a certain low-quality context is reduced, thereby enhancing the robustness of synonyms.

[0079] Figure 2a A flowchart of a method for constructing a context aggregated encoding representation of a text element according to an embodiment of the present application is shown. The method can be performed by the terminal 103, or by one or more servers 101. The following does not elaborate on the allocation and addressing of the server 101, but does not exclude the existence of such steps, and the skilled person can utilize centralized scheduling or distributed management to achieve it.

[0080] Referring to Figure 2a First, a target text element is given, that is, a text element to be processed according to the method of an embodiment of the present application, and at least two context texts in which the target text element is located are obtained from the text to be processed in step 201. In actual application, the same text element (for example, a word or a phrase) usually appears in many different contexts (for example, sentences), and the single context information of the text element may not be sufficient to accurately identify the semantics of the text element. Therefore, the multiple context texts (for example, sentences) in which the target text element is located are first retrieved from the text to be processed (for example, a paragraph, an article or even multiple articles).

[0081] In one example, the context text is obtained by identifying the punctuation marks, and the sentence divided by the punctuation marks is taken as the context text. The punctuation mark is a punctuation mark having the meaning of punctuation, such as a period, a question mark, an exclamation mark, etc. For example, for the target text element "apple", in the large corpus "Autumn is coming, and the apple harvest is plentiful. The yield of apples in Shandong is high and the moisture is sufficient.", there are two punctuation marks, both of which are periods. According to the two periods, two context texts "Autumn is coming, and the apple harvest is plentiful." and "The yield of apples in Shandong is high and the moisture is sufficient." are obtained. In other examples, there can be no punctuation marks in the large corpus, and then relevant punctuation analysis means can be used to divide the large corpus into different sentences, and then the context text is obtained.

[0082] In one example, the context text obtained above can be further processed. The punctuation marks in the context text usually have no effect on the semantics of the target word, and thus the punctuation marks in the context text can be removed. For Chinese text, the letters and numbers that have less effect on the semantics can also be removed. The above removal can be performed by filtering using a regular expression, for example.

[0083] In step 202, an implicit sequence pattern and a pattern support degree of the implicit sequence pattern are determined for each of the at least two context texts. The determination includes applying frequent sequence pattern mining to the at least two context texts, and selecting a frequent sub-sequence corresponding to each of the at least two context texts as the implicit sequence pattern of the corresponding context text, and then removing the duplicates to obtain different implicit sequence patterns of the at least two context texts. The pattern support degree is the ratio of the number of context texts including the corresponding pattern to the number of the at least two context texts. For example, "Apple Inc." is a sequence formed by "Apple" and "Company" in the order, and the pattern support degree of "Apple Inc." is the ratio of the number of context texts including both "Apple" and "Company" to the number of all context texts. The frequent sub-sequence is a sub-sequence whose frequency in the context texts is greater than or equal to a predetermined pattern support degree threshold, that is, the frequency ratio is greater than or equal to the predetermined pattern support degree threshold.

[0084] In one example, the longest frequent sub-sequence of the context text is selected as the implicit sequence pattern of the context text, and the pattern support degree of the longest frequent sub-sequence is the pattern support degree of the implicit sequence pattern.

[0085] In one example, if there is an implied sequence pattern with a unit length of 1 in each of the different implied sequence patterns obtained after deduplication, the implied sequence pattern with a unit length of 1 is deleted. Considering that the implied sequence pattern with a unit length of 1 cannot form a frequent sequence with other words in the window, deletion can further reduce noise.

[0086] Alternatively or optionally, the step of determining the implied sequence pattern for each of the at least two contextual texts comprises, referring to Figure 2b

[0087] In step 2021, each of the at least two contextual texts is processed to generate a sequence formed by text elements of the respective contextual text in the order of appearance in the respective contextual text, thereby obtaining at least two sequences corresponding to the at least two contextual texts. For example, the contextual text "Apple Inc. releases the latest version of mobile phone" is segmented, and a sequence < (Apple) (company) (release) (latest) (version) (of) (mobile phone) > is generated in the order of sequence. The text elements in the sequence are arranged in the order of sequence.

[0088] In step 2022, one or more sub-sequences formed by text elements in the at least two sequences are selected as the frequent sub-sequences, where the ratio of the number of sequences containing the sub-sequences to the number of the at least two sequences is greater than or equal to the predetermined pattern support threshold. For example, if the ratio of the number of times the sequence < (Apple) (latest) > appears in the contextual texts to the total number of contextual texts is greater than or equal to the predetermined pattern support threshold, < (Apple) (latest) > is determined as a frequent sub-sequence. Note that when counting the number of times a sequence appears in a contextual text, the items in the sequence are not required to be adjacent in the contextual text. As long as "Apple" and "latest" exist simultaneously in the contextual text and have such an order, it is considered that the sequence is contained in the contextual text.

[0089] In step 2023, for each sequence of the at least two sequences, the frequent sub-sequences belonging to the sequence are determined from the frequent sub-sequences obtained in step 2022, and the longest frequent sub-sequence is selected as the implied sequence pattern of the corresponding sequence. Since in step 2022, the frequent sequence pattern mining is performed on the whole sequence, and the frequent sub-sequences mined are not distinguished as belonging to which sequence, in step 2023, it is necessary to determine which frequent sub-sequences correspond to each sequence. Obviously, this correspondence requires finding those frequent sub-sequences as a subset of the sequence.

[0090] Referring to Figure 2a ​In one example, we set the text element to a word, and of course, the target text element is also a word. Figure 2a In step 202, determining a hidden sequence pattern for each of the at least two context texts specifically includes: segmenting each of the at least two context texts into characters and generating a character sequence according to the order in which each character appears in the corresponding context text, thereby obtaining at least two character sequences; and performing frequent sequence pattern mining on the at least two character sequences to obtain the longest frequent subsequence of each character sequence in the at least two character sequences as the hidden sequence pattern of the corresponding character sequence.

[0091] Or refer to Figure 2a In one example, we set the text element to a word, and of course, the target text element is also a word. Figure 2a In step 202, determining a hidden sequence pattern for each of the at least two context texts specifically includes: segmenting each of the at least two context texts into words, generating word sequences according to the order in which each word appears in the corresponding context text, thereby obtaining at least two word sequences; and performing frequent sequence pattern mining on the at least two word sequences to obtain the longest frequent subsequence of each word sequence in the at least two word sequences as the hidden sequence pattern of the corresponding word sequence.

[0092] Alternatively or optionally, refer to Figure 2c Step 2022 further includes, for each of the at least two sequences:

[0093] In step 20221, for each text element in the at least two sequences, the frequency percentage of the text element is calculated. The frequency percentage of the text element is the ratio of the number of sequences containing the text element in the at least two sequences to the total number of sequences in the at least two sequences. The corresponding text elements with a frequency percentage greater than or equal to the predetermined pattern support threshold are identified as prefixes with a unit length of 1. The corresponding text elements with a frequency percentage less than the predetermined pattern support threshold are deleted from the at least two sequences.

[0094] For each value of i from smallest to largest, execute steps 20222-20223, where i is an integer, 1≤i≤K, and K is the maximum number of text elements in each sequence after deletion.

[0095] At step 20222, a determination step is performed: determine a prefix of unit length i as a frequent subsequence, and for each prefix of unit length i, determine a set of projected sequences of the prefix from each of the sequences after the deletion, and count a projected frequency proportion of each text element in the set of projected sequences, which is a ratio of a number of projected sequences containing the text element to a number of sequences of the at least two sequences in the set of projected sequences of the prefix.

[0096] At step 20223, for each prefix of unit length i, if there is any set of projected sequences (i.e. the set is not empty), and the counted projected frequency proportion is greater than or equal to the predetermined pattern support threshold, combine the corresponding text element with the prefix to generate a prefix of unit length i = i + 1, and go to the determination step, i.e. step 20222, for further execution.

[0097] In this way, frequent subsequences of different unit lengths are generated. Of course, the way of obtaining frequent subsequences is not limited to the above specific way, and those skilled in the art can also use other ways, for example, starting from the end of the sequence and combining step by step forward according to a similar recursive way to search for frequent subsequences.

[0098] At step 203, a local encoding representation of at least part of the text elements in the at least two context texts is obtained, which is an encoding representation generated based on a single context text, for example, a word or word encoding representation generated by a Word2Vec model, which is a vector of a text element determined based on a single context within a window in which the word or word is located, so that other context information in the entire text to be processed is not utilized. Of course, the local encoding representation is not limited to the Word2Vec encoding representation, but any encoding representation generated based on a single context text, including, for example, Onehot encoding representation and the like.

[0099] At step 204, the local encoding representations of the at least part of the text elements are aggregated using the pattern supports of the different implicit sequence patterns to obtain a context aggregated encoding representation of the target text element.

[0100] In one example, the step includes determining the text elements included in the respective different implicit sequential patterns as the at least partial text elements, i.e. filtering out the part of the text elements with pattern support less than the predetermined pattern support threshold, and then summing up the local vector representations of the respective text elements in each of the respective different implicit sequential patterns after weighting by the pattern support of the corresponding implicit sequential pattern, and taking the result of the weighted summing up for each of the implicit sequential patterns as the context aggregated encoding representation of the target text element. Such context aggregated encoding representation can comprehensively embody multiple different contexts, and by weighting by the pattern support, the influence degree of each context is considered, and the encoding representation constructed will be closer to its actual semantics.

[0101] As described in the foregoing step 2021, when performing frequent sequence mining on the context text, the context text is first converted into a corresponding text element sequence, such as a word sequence or a phrase sequence, and then the frequent sequence mining is performed. At this time, there can be two ways. In a first example, the corresponding text element sequence retains the target text element, and in a second example, the corresponding text element sequence does not contain the target text element. In the first example, the final obtained implicit sequential pattern must contain the target text element, because the frequency proportion of the target text element in the context text is 100%, and therefore the pattern support in the corresponding text sequence is 1. Then in the foregoing example of the aggregation process of step 204, the at least partial text elements that are weighted are all the text elements in the corresponding implicit sequential pattern after removing the target text element, i.e. all the text elements in the longest frequent subsequence of the corresponding sequence after removing the target text element. In the second example, because the text element sequence does not contain the target text element, the final obtained implicit sequential pattern must not contain the target text element. Then in the foregoing example of the aggregation process of step 204, the at least partial text elements that are weighted are all the text elements in the corresponding implicit sequential pattern, i.e. all the text elements in the longest frequent subsequence of the corresponding sequence.

[0102] Figure 2d A flow chart of a method for determining text similarity according to an embodiment of the present application is shown. In step 205, the context aggregated encoding representation of the text elements is obtained, which can be completed by the foregoing steps 201-204, or can be a separate step, such as obtaining the context aggregated encoding representation of the text elements generated by other devices according to steps 201-204 through communication connection.

[0103] After obtaining the context aggregation encoding representation of the text element, the cosine distance or Euclidean distance between the context aggregation vector representations of the text elements can be calculated in step 206 to determine the similarity between the text elements, where the context aggregation encoding representation is a vector representation. For example, if the encoding representations of two text elements are A and B respectively, the cosine similarity between the encoding representations of these two text elements is calculated as follows:

[0104]

[0105] where Ai and Bi represent the components of the encoding representations A and B respectively.

[0106] Figure 3 FIG. illustrates a specific application embodiment of the embodiment of the present invention in a question-and-answer scenario. In this example, the text element is a word. Of course, the present disclosure is not limited to the case where the text element is a word. When the text element is a character, phrase, sentence, paragraph, article, etc., a similar application can be performed. In this scenario, the user communicates with the server 101 through the terminal 103. Specifically, the user communicates with the server 101 through a question-and-answer application on the terminal 103.

[0107] In one example, the user asks the server 101 through the terminal 103: "Do you like the apples in Shandong? ”Before answering, the server needs to understand the semantics of "apples" in the user's question in order to provide a more accurate answer and improve the user experience. According to step 201, the server collects the context text sent by the user within a time window and obtains 4 context texts containing apples, as shown in Table 1:

[0108] Apple has released the latest version of the iPhone The latest version of the iPhone released by Apple is not bad Recently, the output of apples is abundant in moisture The output of apples in Shandong is also abundant in moisture

[0109] Table 1.

[0110] Next, according to step 2021, these 4 context texts are tokenized, and a word sequence is generated in the order in which each word appears in the corresponding context text, thereby obtaining 4 word sequences, as shown in Table 2, where "<>" indicates that the content therein is a sequence, and the words obtained after tokenization are separated by spaces. It should be noted that the words obtained after tokenization may contain characters, such as "many", "of", etc.

[0111] Apple has released the latest version of its iPhone <Apple's just-released new version of the iPhone is pretty good, too> <Recent apple production with sufficient moisture> <The yield of Shandong apples is also sufficient in moisture>

[0112] Table 2. <!

[0113] For the sake of simplicity, the symbols "<>" and spaces indicating the sequence and the words in the sequence are omitted below. Although omitted, the division of the items in the sequence is always consistent with that in Table 2.

[0114] Next, frequent subsequences are determined according to step 2022. Specifically, according to step 20221, it is assumed that the predetermined pattern support threshold is 1 / 3, that is, the frequency proportion of a word meets the condition of being greater than or equal to the predetermined pattern support threshold only when the word appears at least 2 times in 4 context texts. The frequency proportion of a word is as explained above, that is, the ratio of the number of word sequences containing the word to the total number of word sequences, for example, for the word "apple", the number of sequences containing the word is 4, and the total number of sequences is also 4, so the frequency proportion of apple is 1, and the frequency proportions of other words can be obtained in the same way. First, determine the prefix with a unit length of 1: after counting, delete the text elements that appear only once: "le", "just", "this", "not bad", "oh", "recently", "Shandong", "also", and determine the words in Table 3 below as the prefix with a unit length of 1:

[0115] Word Occurrence sequence number Frequency proportion Apple 4 1 's 3 3 / 4 Version 2 1 / 2 Output 2 1 / 2 Abundant 2 1 / 2 Many 2 1 / 2 Release 2 1 / 2 Company 2 1 / 2 Mobile phone 2 1 / 2 Moisture 2 1 / 2 Latest 2 1 / 2

[0116] Table 3.

[0117] After deleting the words with a frequency proportion less than the predetermined pattern support threshold from the sequence, the sequence obtained is shown in Table 4.

[0118] Apple has released the latest version of the iPhone The latest version of the iPhone released by Apple The output of apples is abundant in moisture The output of apples is abundant in moisture

[0119] Table 4.

[0120] According to step 20222, the prefix with a unit length of i = 1 is determined as a frequent subsequence, as shown above, when i = 1, the prefixes "apple", "of", "version", "yield", "sufficient", "more", "release", "company", "mobile phone", "moisture", and "latest" are determined as frequent subsequences. And for each prefix with a unit length of i = 1, determine the projection sequence set from the sequence of Table 4 after the above deletion, the projection sequence set is as explained above, that is, the set of corresponding suffixes of apple in 4 sequences, as shown in Table 5.

[0121]

[0122] Table 5.

[0123] According to step 20223, taking the prefix "apple" with unit length i = 1 as an example, the next round of iteration is to mine the prefix and the corresponding suffix with unit length i = 2, and at this time, the projection frequency proportion of each text element in the projection sequence set needs to be considered. As described earlier, the projection frequency proportion of the text element in the projection sequence set of the prefix is the ratio of the number of projection sequences containing the text element in the projection sequence set of the prefix to the sequence number of the at least two sequences. For example, the projection frequency proportion of "company" in the projection sequence set of "apple" is 2 / 4, that is, 1 / 2. The complete results are shown in Table 6. The mining method of other prefixes with unit length i = 2 is consistent with this, and will not be described here. Obviously, the prefixes "mobile phone" and "sufficient" cannot be further mined.

[0124]

[0125] Table 6.

[0126] Next, taking the prefix "apple company" with unit length i = 2 as an example to describe the next round of iteration, mining the prefix and the corresponding suffix with unit length i = 3, after counting the projection frequency proportion of each text element in the projection sequence set, the results are shown in Table 7. The mining method of other prefixes with unit length i = 3 is consistent with this, and will not be described here. Obviously, the prefixes "apple mobile phone" and "apple sufficient" cannot be further mined.

[0127]

[0128] Table 7.

[0129] Next, taking the prefix "apple company release" with unit length i = 3 as an example to describe the next round of iteration, mining the prefix and the corresponding suffix with unit length i = 4, after counting the projection frequency proportion of each text element in the projection sequence set, the results are shown in Table 8. The mining method of other prefixes with unit length i = 4 is consistent with this, and will not be described here. Obviously, the prefix "apple company mobile phone" cannot be further mined.

[0130]

[0131] Table 8.

[0132] Next, taking the prefix "apple company release latest" with unit length i = 4 as an example to describe the next round of iteration, mining the prefix and the corresponding suffix with unit length i = 5, after counting the projection frequency proportion of each text element in the projection sequence set, the results are shown in Table 9. The mining method of other prefixes with unit length i = 5 is consistent with this, and will not be described here. Obviously, the prefix "apple company release mobile phone" cannot be further mined.

[0133]

[0134] Table 9.

[0135] Next, taking the prefix "Apple company releases the latest version" with unit length i = 5 as an example, the next round of iteration is described, the prefix with unit length i = 6 and the corresponding suffix are mined, and after the projection frequency ratio of each text element in the projection sequence set is counted, the result is shown in Table 10. The mining method of other prefixes with unit length i = 6 is consistent with this, and will not be described here. Obviously, the prefix "Apple company releases the latest mobile phone" cannot be further mined.

[0136]

[0137] Table 10.

[0138] Next, the prefix "Apple company releases the latest version of" with unit length i = 6 is mined, the prefix with unit length i = 7 and the corresponding suffix are mined, and after the projection frequency ratio of each text element in the projection sequence set is counted, only the projection frequency ratio of "mobile phone" is greater than the predetermined mode support threshold, the frequency ratio of "Apple" in the suffix is 1 / 4, which is less than the predetermined mode support threshold, and the prefix "Apple company releases the latest version of mobile phone" cannot be further mined, so finally only one prefix with unit length i = 7 is obtained, and the result is shown in Table 11.

[0139] Prefix (i=7) Corresponding suffix Apple has released the latest version of the iPhone

[0140] Table 11.

[0141] After all the prefixes in Table 5 are mined in this way, the iteration ends. All prefixes of different unit lengths are listed as frequent subsequences.

[0142] At this time, according to step 2023, from these frequent subsequences, the frequent subsequences belonging to each sequence of Table 2 are determined. That is, in these frequent subsequences, the subsequences belonging to each sequence of Table 2 are searched, and then in each subsequence of each sequence, the longest frequent subsequence is selected as the implicit sequence pattern of the corresponding sequence.

[0143] In an example, for each frequent subsequence of each sequence of Table 2 mined by the above steps, the supersequence itself contains more reference information, that is, the supersequence includes the context auxiliary word that the subsequence does not include, the supersequence is retained, and the subsequence is deleted, for example, taking the supersequence "Apple company releases the latest version of mobile phone" and the subsequence "Apple company releases the latest version" as an example, the supersequence further includes the context auxiliary word information "mobile phone" on the basis of the subsequence, so the supersequence "Apple company releases the latest version of mobile phone" is retained, and the subsequence "Apple company releases the latest version" is deleted. Finally, the longest frequent subsequence of each sequence is obtained as the implicit sequence pattern of the corresponding sequence.

[0144] After obtaining the implied sequence patterns of each sequence in this way, further de-duplication is performed to obtain different implied sequence patterns.

[0145] In one example, if there is an implied sequence pattern with a unit length of 1 in the different implied sequence patterns obtained after de-duplication, it is deleted. Considering that the implied sequence pattern with a unit length of 1 cannot form a frequent sequence with other words in the window, deletion can further reduce noise.

[0146] Next, at least part of the words in the sequence of Table 2 are locally encoded according to step 203, for example, the local word vectors of each word in the sequence of Table 2 are trained by using the Word2vec word vector model. According to Table 2, the words contained in each context text include:

[0147] 's Version Output Abundant Many Release Mobile phone Moisture Latest This Shandong Recently Not bad Just Company Also The Oh Apple Also Contextual text

[0148] Table 12.

[0149] The word vectors of each word in each context text are trained by using the Word2vec word vector model, for example:

[0150] "the": [0.1, 0.05, 0.8...0.52];

[0151] "version": [0.06, -0.2, 0.37...0.2];

[0152] "yield": [-0.01, 0.7, 0.99...0.003];

[0153] "sufficient": [-0.51, 0.23, 0.159...0.055] ...

[0154] Then, according to step 204, the word vectors of at least part of the words are aggregated by using the pattern support of each different implied sequence pattern. In one example, the target word "apple" in each different implied sequence pattern is removed first to obtain the frequent sub-sequences corresponding to each context text, as shown in Table 13.

[0155] Corresponding sequence pattern Pattern support Apple has released the latest version of the iPhone The company has released the latest version of the phone The latest version of the iPhone released by Apple is not bad 1 / 2 The company has released the latest version of the phone Recently, the output of apples is abundant in moisture 1 / 2 The output of apples is abundant in moisture The output of apples in Shandong is also abundant in moisture 1 / 2 The output of apples is abundant in moisture Figure 3 1 / 2

[0156] Table 13.

[0157] The context aggregation encoding representation of "apple" is calculated as:

[0158] "company" word vector *1 / 2 + "release" word vector *1 / 2 + "latest" word vector *1 / 2 + "version" word vector *1 / 2 + "of" word vector *1 / 2 + "mobile" word vector *1 / 2 + "production" word vector *1 / 2 + "more" word vector *1 / 2 + "moisture" word vector *1 / 2 + "sufficient" word vector *1 / 2.

[0159] Accordingly, the context aggregated encoding representation of each word is obtained, the cosine distance or Euclidean distance between "apple" and other words is calculated based on the context aggregated encoding representation between each pair of words, so as to obtain the similarity between "apple" and other words, based on the similarity, the semantics of "apple" is further understood, and then the corresponding answer is returned to the user.

[0160] In the example of Figure 4a In the example of

[0161] According to the method of the embodiments of the present disclosure, by mining the implicit sequence patterns in the at least two context texts, aggregating the local encoding representation of at least part of the text elements in the context texts by using the pattern support of each different implicit sequence pattern, the context aggregated encoding representation reflecting the at least two context texts is obtained, the semantics of the words can be better combined and utilized to measure and calculate the semantics of the words, the contribution of the context to the word representation can be better quantified, the semantic representation and context of the context where the words are located can be more comprehensively reflected, the representation ability of the word representation is enhanced, and thus the accuracy of the word semantics and the similarity calculation between words is improved. In some examples, by further mining the frequent patterns of the context sequences, the noise introduced due to the use of a certain low-quality context is reduced, and thus the robustness of the word semantics and the similarity calculation between words is enhanced. On this basis, the question and answer system can identify the word meaning according to the context of the user question, so as to better and more accurately complete the multi-round dialogue and improve the user experience.

[0162] Figure 2a-2c Fig. 1 illustrates a block diagram of an apparatus for constructing a context aggregated encoding representation of a text element according to an embodiment of the present disclosure. The apparatus 400a for constructing a context aggregated encoding representation of a text element comprises a context text acquisition module 401, a determination module 402, a local encoding representation acquisition module 403, and an aggregation module 404.

[0163] The context text acquisition module 401 is configured to acquire at least two context texts in which the target text element is located from the to-be-processed text, i.e., step 201 of Fig. 2.

[0164] The determining module 402 is configured to determine an implied sequence pattern and a pattern support degree of the implied sequence pattern for each of the at least two context texts, i.e. step 202 of FIG. 2. The determining of the implied sequence pattern comprises: applying frequent sequence pattern mining to the at least two context texts, and selecting a frequent sub-sequence corresponding to each of the at least two context texts as the implied sequence pattern of the corresponding context text, after de-duplication, obtaining respective different implied sequence patterns of the at least two context texts, and the pattern support degree is a ratio of a number of context texts including the corresponding pattern to a number of the at least two context texts, and the frequent sub-sequence is a sub-sequence whose ratio of a number of context texts including the frequent sub-sequence to the number of the at least two context texts is greater than or equal to a predetermined pattern support degree threshold.

[0165] The local coding representation obtaining module 403 is configured to obtain a local coding representation of at least part of text elements in the at least two context texts, i.e. step 203 of FIG. 2, and the local coding representation is an encoding representation generated based on a single context text.

[0166] The aggregating module 404 is configured to aggregate the local coding representation of the at least part of text elements by using the pattern support degrees of the respective different implied sequence patterns, to obtain a context aggregated encoding representation of the target text element, i.e. step 204 of FIG. 2.

[0167] More details about the modules can be found in the description of steps 201-204 in Figure 4b and will not be repeated here.

[0168] Figure 2a-2c FIG. 2 illustrates a block diagram of an apparatus for determining text element similarity according to an embodiment of the present application. The apparatus 400b for determining text element similarity comprises a context aggregated encoding representation obtaining module 405, and a similarity determining module 406. The context aggregated encoding representation obtaining module 405 is configured to obtain a context aggregated encoding representation of a text element generated by the apparatus 400a for constructing a context aggregated encoding representation of a text element described above, and in one example, the context aggregated encoding representation obtaining module 405 comprises all the modules of the apparatus 400a for constructing a context aggregated encoding representation of a text element described above, and in another example, the context aggregated encoding representation obtaining module 405 is connected to the apparatus 400a for constructing a context aggregated encoding representation of a text element described above. The similarity determining module 406 is configured to calculate a cosine distance or a Euclidean distance between the context aggregated encoding representations of the text elements as a similarity between the text elements, i.e. step 205 of FIG. 2. More details about the module can be found in the description of step 205 in Figure 5 and will not be repeated here.

[0169] See Figure 1 In embodiments of the present invention, the device 500 for constructing a context-aggregated encoding representation of text elements or determining text similarity includes a processor 504, which includes hardware components 510. The processor 504 may include, for example, one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. As used herein, the term "processor" may refer to any of the above-described structures or any other structures suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within, or incorporated into, dedicated hardware and / or software modules configured for constructing context-aggregated encoding representations of text elements. Furthermore, the techniques may be fully implemented in one or more circuit or logic elements. The methods in this disclosure may be implemented in various components, modules, or units, but do not necessarily require implementation by different hardware units. Rather, as described above, various components, modules, or units may be combined or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with suitable software and / or firmware.

[0170] In one or more examples, the above combination ​ - The content depicted in Figure 4 can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored or transmitted as one or more instructions or code on or via computer-readable medium 506 and executed by a hardware-based processor. Computer-readable medium 506 may comprise a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium comprising any medium that facilitates the transmission of a computer program (including one or more of the aforementioned instructions or code) from one place to another, for example, according to a communication protocol. In this way, computer-readable medium 506 may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. Data storage medium may be any available medium that can be read by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may comprise computer-readable medium 506 and one or more instructions or code stored thereon.

[0171] By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other storage medium 512 that can be used to store desired program codes in the form of instructions or data structures and that can be accessed by a computer. Also, any

[0172] The device for constructing a contextually aggregated encoding representation of a text element or the device for determining a text similarity 500 can also include I / O interface(s) for transmitting data, and other functionality 514. The device for constructing a contextually aggregated encoding representation of a text element or the device for determining a text similarity 500 can be included in different terminals such as a computer 516, a mobile device 518, and other terminals 520, among others. Each of these configurations includes devices that can have generally different constructs and capabilities, and thus the device for constructing a contextually aggregated encoding representation of a text element or the device for determining a text similarity 500 can be constructed in accordance with one or more of different device classes. Moreover, the techniques of the present application can also be implemented in whole or in part by using distributed systems, such as over the "cloud" 522, through a platform 524 as described below.

[0173] The cloud 522 includes and / or is representative of the platform 524 for resources 526. The platform 524 abstracts underlying functionality of hardware (e.g., servers) and software resources of the cloud 522. The resources 526 can include applications and / or data that can be utilized while computer processing is executed on servers that are remote from the computing device. Resources 526 can also include services provided over the Internet and / or through a subscriber network, such as a cellular or Wi-Fi network.

[0174] The platform 524 can abstract resources and functions to connect the computing device with other computing devices. The platform 524 can also serve to abstract scaling of resources to provide a corresponding level of scale to encountered demand for the resources 526 that are implemented via the platform 524. Accordingly, in an interconnected device embodiment, implementation of functionality described herein can be distributed throughout the system 500. For example, the functionality can be implemented in part on the computing device as well as via the platform 524 that abstracts the functionality of the cloud 522.

[0175] According to the method of the embodiments of the present disclosure, by mining the implicit sequence patterns in the at least two context texts, aggregating the local coding representations of at least part of the text elements in the context texts by using the pattern support of each different implicit sequence pattern, a context aggregated coding representation reflecting the at least two context texts is obtained, the semantic of the words can be better combined and utilized to measure and calculate, the contribution of the context to the word representation can be better quantified, the semantic representation and context of the context where the words are located can be more comprehensively reflected, the representation ability of the word representation is enhanced, the similarity between the words is more finely described, and thus the accuracy of the word semantic and the similarity calculation between the words is improved. In some examples, by further mining the frequent patterns of the context sequences, the noise introduced by using a certain low-quality context is reduced, and thus the robustness of the word semantic and the similarity calculation between the words is enhanced. The method of the embodiments of the present disclosure is suitable for related application scenarios of natural language processing technology, such as machine reading, synonym recognition, natural language understanding, question and answer system, word construction, sentiment analysis, text classification, and the like, and has great practical significance.

[0176] It should be noted that the "first", "second", and the like expressions appearing in the present disclosure do not represent the importance or the order of steps, but are only used for distinction. The description order of the method steps does not represent their execution order in the absence of special description or premise constraints (i.e., the execution of one step needs to be based on the execution result of another step). The described method steps can be executed in a possible and reasonable order.

[0177] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0178] It is to be understood that the application is not limited to the precise construction already described above and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should only be limited by the claims appended hereto.< / aabca> < / aabcacdcf>

Claims

1. A method for constructing a context-aggregated encoding representation of text elements, characterized in that, Including the following steps: Obtain at least two context texts containing the target text element from the text to be processed; Each of the at least two context texts is processed to generate a sequence formed by the text elements of each context text in the order in which they appear in each context text, so as to obtain at least two sequences corresponding to the at least two context texts; From the at least two sequences, text elements whose frequency proportion is less than a predetermined pattern support threshold are deleted to obtain at least two updated sequences. Subsequent processing steps are then performed on the at least two updated sequences. The frequency proportion of text elements refers to the ratio of the number of sequences containing text elements to the total number of the at least two sequences. In at least two sequences, a subsequence formed by selecting one or more text elements is designated as a frequent subsequence; wherein the pattern support of the frequent subsequence is greater than or equal to a predetermined pattern support threshold, and the pattern support refers to the ratio of the number of context texts including the frequent subsequence to the number of the at least two context texts. For each of the at least two sequences, from the obtained frequent subsequences, determine the frequent subsequences belonging to each of the at least two sequences, and select the longest frequent subsequence from the frequent subsequences belonging to each of the at least two sequences as the hidden sequence pattern of each of the at least two sequences, and use the pattern support of the longest frequent subsequence as the pattern support of the hidden sequence pattern. If the target text element is retained in both of the at least two sequences, then all text elements in the implicit sequence patterns of the at least two sequences after removing the target text element are determined as at least some text elements; if the target text element is not contained in either of the at least two sequences, then all text elements in the implicit sequence patterns of the at least two sequences are determined as at least some text elements. Obtain local encoded representations of the at least some text elements, wherein the local encoded representations are encoded representations generated based on a single context text; By utilizing the pattern support of each hidden sequence pattern in various different hidden sequence patterns, the local encoding representations of each text element belonging to the at least part of the text elements in the corresponding hidden sequence pattern are weighted and summed, and the result of the weighted sum is used as the context aggregation encoding representation of the target text element. The similarity between two text elements is determined by comparing the distance between their context aggregation encoded representations.

2. The method as described in claim 1, characterized in that, The step of determining the hidden sequence pattern further includes: selecting a frequent subsequence corresponding to each of the at least two context texts as the hidden sequence pattern of the corresponding context text, and then removing duplicates to obtain the different hidden sequence patterns of the at least two context texts.

3. The method as described in claim 1, characterized in that, The step of selecting one or more text elements from at least two sequences to form a subsequence as a frequent subsequence includes: The corresponding text elements whose frequency proportion is greater than or equal to the predetermined pattern support threshold are identified as prefixes with a unit length of 1. For each value of i, from smallest to largest, perform the following steps, where i is an integer, 1 ≤ i ≤ K, and K is the maximum number of text elements in each deleted sequence: The determination steps are as follows: Prefixes of unit length i are identified as frequent subsequences. For each prefix of unit length i, a set of projection sequences for the prefix is ​​determined from the deleted sequences. The projection frequency percentage of each text element in the projection sequence set is calculated. The projection frequency percentage of a text element refers to the ratio of the number of projection sequences containing the text element in the prefix's projection sequence set to the number of sequences in the at least two sequences. For each prefix of unit length i, when the projection frequency ratio is greater than or equal to the predetermined pattern support threshold, the corresponding text element is combined with the prefix to generate a prefix of unit length i = i + 1, and then the process proceeds to the determination step.

4. The method as described in claim 1, characterized in that, The text element is a character, word, or phrase, and the step of obtaining at least two context texts containing the target text element from the text to be processed includes: Obtain at least two sentences, divided by punctuation marks, that include the target text element, as the at least two context texts; or Obtain at least two sentences containing the target text elements, divided by punctuation marks, and remove punctuation marks, letters, and numbers from the at least two sentences to generate the at least two context texts.

5. The method of claim 1, wherein obtaining the local encoded representation of the at least some text elements comprises: The Word2Vec model is used to generate local encoded representations of at least some text elements in the at least two context texts.

6. The method as described in claim 1, characterized in that, The target text element and the text element's category both include any of the following categories: character, word, phrase, sentence, paragraph, and article, and the target text element and the text element have the same category.

7. The method as described in claim 1, characterized in that, When both the target text element and the text element's category are characters, a hidden sequence pattern is determined for each of the at least two context texts, including: Each of the at least two context texts is segmented into characters and a character sequence is generated according to the order in which each character appears in the corresponding context text, so as to obtain at least two character sequences; and, frequent sequence pattern mining is performed on the at least two character sequences to obtain the longest frequent subsequence of each character sequence in the at least two character sequences as the implicit sequence pattern of the corresponding character sequence. When both the target text element and the category of the text element are words, a hidden sequence pattern is determined for each of the at least two context texts, including: Each of the at least two context texts is segmented into words and word sequences are generated according to the order in which each word appears in the corresponding context text to obtain at least two word sequences; and frequent sequence pattern mining is performed on the at least two word sequences to obtain the longest frequent subsequence of each word sequence in the at least two word sequences as the implicit sequence pattern of the corresponding word sequence.

8. The method as described in claim 1, characterized in that, The steps for determining hidden sequence patterns also include: deleting hidden sequence patterns with a unit length of 1 from the deduplicated hidden sequence patterns, and using the deleted hidden sequence patterns for aggregation.

9. A method for determining the similarity of text elements by constructing a context aggregation encoding representation of text elements using the method described in claim 1, characterized in that, Including the following steps: Get the context-aggregated encoding representation of the text element; as well as The cosine distance or Euclidean distance between the context aggregation encoding representations of the text elements is calculated as the similarity between the text elements, wherein the context aggregation encoding representation is a vector representation.

10. An apparatus for constructing a context-aggregated encoded representation of text elements, characterized in that, include: The context text acquisition module is configured to acquire at least two context texts containing the target text element from the text to be processed; The determination module is configured to: process each of the at least two context texts to generate a sequence formed by the text elements of each context text in the order in which they appear in each context text, to obtain at least two sequences corresponding to the at least two context texts; delete text elements whose frequency proportion is less than a predetermined pattern support threshold from the at least two sequences to obtain updated at least two sequences; perform subsequent processing steps on the updated at least two sequences, where the frequency proportion of text elements refers to the ratio of the number of sequences containing text elements to the total number of the at least two sequences; and select one or more text elements from the at least two sequences to form a subsequence. The frequent subsequences are defined as follows: the pattern support of the frequent subsequences is greater than or equal to a predetermined pattern support threshold, wherein the pattern support refers to the ratio of the number of context texts including the frequent subsequences to the number of the at least two context texts; for each of the at least two sequences, the frequent subsequences belonging to each of the at least two sequences are determined from the obtained frequent subsequences, and the longest frequent subsequence is selected from the frequent subsequences belonging to each of the at least two sequences as the latent sequence pattern of each of the at least two sequences, and the pattern support of the longest frequent subsequence is used as the pattern support of the latent sequence pattern; The local encoding representation acquisition module is configured to: if the target text element is retained in both of the at least two sequences, then determine all text elements in the implicit sequence patterns of the at least two sequences after removing the target text element as at least some text elements; if the target text element is not contained in either of the at least two sequences, then determine all text elements in the implicit sequence patterns of the at least two sequences as at least some text elements; and acquire the local encoding representation of the at least some text elements, wherein the local encoding representation is an encoding representation generated based on a single context text. The aggregation module is configured to use the pattern support of each hidden sequence pattern in each different hidden sequence pattern to perform a weighted summation on the local encoding representations of each text element belonging to the at least part of the text elements in the corresponding hidden sequence pattern, and use the weighted summation result as the context aggregation encoding representation of the target text element. A module configured to perform the following steps: determine the similarity between two text elements by comparing the distance between their context aggregation encoded representations.

11. An apparatus for constructing a context-aggregated encoded representation of text elements, characterized in that, include: processor; as well as The memory is configured to store computer-executable instructions that, when executed in the processor, cause the method as described in any one of claims 1-9 to be implemented.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when the computer-readable storage medium is run on a computer, cause the computer to perform the method as described in any one of claims 1-9.

Images