Text processing method, device, electronic device and storage medium

Abstract

The embodiments of the present disclosure provide a text processing method, device, electronic device and storage medium. The text processing method includes: for each character in the text to be processed, selecting a text sequence corresponding to the character from the text to be processed; inputting the text sequence corresponding to the character into a pre-trained knowledge dictionary part, performing knowledge information fusion on the subword sequence corresponding to the text sequence through the knowledge dictionary part to obtain a knowledge fusion vector representation of the subword sequence, and obtaining a knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through a post-processing part; wherein the knowledge dictionary part is pre-trained using a sample text sequence and a knowledge information label corresponding to the sample text sequence. The pre-training process of the knowledge dictionary part in the embodiment of the present disclosure is simpler than the construction process of the entity dictionary and the knowledge graph, and the knowledge dictionary part can be flexibly combined with the natural language processing process, and has good adaptability and flexibility.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and in particular to a text processing method, device, electronic device, and storage medium. Background Art

[0002] Natural Language Processing (NLP) is a discipline that studies language issues in human-computer interaction. It investigates the theories and methods that enable effective communication between humans and computers using natural language. It is both a branch of language information processing and a core area of artificial intelligence.

[0003] In NLP tasks, models are often used to analyze text to obtain vector representations of the characters within the text, thereby specifically characterizing the characteristics of the characters within the text. However, for NLP tasks, simply relying on labeled data to improve model performance is limited. To achieve a deeper semantic understanding of natural language, it is increasingly necessary to incorporate the rich knowledge and information generated by the human world into the model.

[0004] There are two ways of knowledge fusion in the existing technology. One is knowledge fusion based on entity dictionaries, which is performed by matching whether the entities in the entity dictionary appear in the text. However, knowledge fusion based on entity dictionaries requires the construction of entity dictionaries for data in specific task fields, and the process of constructing entity dictionaries is too cumbersome. Another way is knowledge fusion based on knowledge graphs, which usually represents text in the form of graphs, and fuses the network constructed by entities and relationships in the knowledge triples <entity, relationship, entity> with the graph represented in the text to be fused, and adds knowledge as supplementary information for nodes and relationships. However, knowledge fusion based on knowledge graphs also requires the construction of knowledge graphs for data in specific task fields, and the process of constructing knowledge graphs is too cumbersome. Therefore, the processing process of knowledge fusion methods in the existing technology is relatively complicated. Summary of the Invention

[0005] In view of the above problems, the embodiments of the present disclosure propose a text processing method, device, electronic device and storage medium, which can more easily fuse knowledge information on characters in a text.

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided a text processing method, comprising:

[0007] For each character in the text to be processed, selecting a text sequence corresponding to the character from the text to be processed;

[0008] Inputting the text sequence corresponding to the character into a pre-trained knowledge dictionary part, performing knowledge information fusion on the subword sequence corresponding to the text sequence through the knowledge dictionary part to obtain a knowledge fusion vector representation of the subword sequence, and obtaining a knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through a post-processing part;

[0009] The knowledge dictionary part is obtained by pre-training using sample text sequences and knowledge information labels corresponding to the sample text sequences.

[0010] Optionally, the knowledge dictionary part includes a word segmentation part and at least one embedding part, one embedding part is obtained by pre-training based on a type of sample text sequence; the knowledge dictionary part performs knowledge information fusion on the sub-word sequence corresponding to the text sequence to obtain a knowledge fusion vector representation of the sub-word sequence, including: performing sub-word granularity segmentation on the text sequence by the word segmentation part to obtain the sub-word sequence corresponding to the text sequence; and performing knowledge information fusion on each sub-word in the sub-word sequence by each embedding part to obtain a knowledge fusion vector representation of the sub-word sequence corresponding to each embedding part.

[0011] Optionally, the post-processing part includes a conversion part and a pooling part; the knowledge fusion vector representation of the character is obtained based on the knowledge fusion vector representation of the sub-word sequence through the post-processing part, including: performing inter-sub-word fusion processing on the knowledge fusion vector representation of the sub-word sequence corresponding to each embedded part through the conversion part to obtain the knowledge fusion vector representation of the text sequence corresponding to each embedded part; and performing aggregation processing on the knowledge fusion vector representation of the text sequence corresponding to all embedded parts through the pooling part to obtain the knowledge fusion vector representation of the character.

[0012] Optionally, the knowledge dictionary part includes a word segmentation part and at least one embedding part, and one embedding part is pre-trained based on a type of sample text sequence; for any embedding part, the knowledge dictionary part is pre-trained in the following manner: a sample text sequence of the type corresponding to the embedding part is input into the word segmentation part in the knowledge dictionary part, and the sample text sequence is segmented at subword granularity by the word segmentation part to obtain a sample subword sequence corresponding to the sample text sequence; knowledge information is fused on each sample subword in the sample subword sequence by the embedding part to obtain a knowledge fusion vector representation of the sample subword sequence; features are fused and extracted on the knowledge fusion vector representation of the sample subword sequence by a text convolutional neural network to obtain a knowledge fusion vector representation of the sample text sequence; after the training is completed based on the knowledge fusion vector representation of the sample text sequence and the knowledge information label of the sample text sequence, the knowledge dictionary part is obtained.

[0013] Optionally, selecting the text sequence corresponding to the character from the text to be processed includes: selecting a text sequence containing the character and at least one character at the front and / or back end of the character from the text to be processed as the text sequence corresponding to the character.

[0014] According to a second aspect of an embodiment of the present disclosure, there is provided a text processing apparatus, comprising:

[0015] A selection module, configured to select, for each character in a text to be processed, a text sequence corresponding to the character from the text to be processed;

[0016] a processing module configured to input a text sequence corresponding to the character into a pre-trained knowledge dictionary, perform knowledge information fusion on a subword sequence corresponding to the text sequence through the knowledge dictionary to obtain a knowledge fusion vector representation of the subword sequence, and obtain a knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through a post-processing module;

[0017] The knowledge dictionary part is obtained by pre-training using sample text sequences and knowledge information labels corresponding to the sample text sequences.

[0018] Optionally, the knowledge dictionary part includes a word segmentation part and at least one embedding part, and one embedding part is obtained by pre-training based on a type of sample text sequence; the processing module includes: a word segmentation unit, which is used to perform sub-word granularity segmentation on the text sequence through the word segmentation part to obtain a sub-word sequence corresponding to the text sequence; an embedding unit, which is used to perform knowledge information fusion on each sub-word in the sub-word sequence through each embedding part, and obtain a knowledge fusion vector representation of the sub-word sequence corresponding to each embedding part.

[0019] Optionally, the post-processing part includes a conversion part and a pooling part; the processing module includes: a conversion unit, which is used to perform inter-subword fusion processing on the knowledge fusion vector representation of the sub-word sequence corresponding to each embedded part through the conversion part, to obtain the knowledge fusion vector representation of the text sequence corresponding to each embedded part; a pooling unit, which is used to perform aggregation processing on the knowledge fusion vector representation of the text sequence corresponding to all embedded parts through the pooling part, to obtain the knowledge fusion vector representation of the character.

[0020] Optionally, the knowledge dictionary part includes a word segmentation part and at least one embedding part, and one embedding part is pre-trained based on a type of sample text sequence; for any embedding part, the knowledge dictionary part is pre-trained through the following modules: a sample word segmentation module, which is used to input the sample text sequence of the type corresponding to the embedding part into the word segmentation part in the knowledge dictionary part, and perform sub-word segmentation on the sample text sequence through the word segmentation part to obtain a sample sub-word sequence corresponding to the sample text sequence; a sample embedding module, which is used to perform knowledge information fusion on each sample sub-word in the sample sub-word sequence through the embedding part to obtain a knowledge fusion vector representation of the sample sub-word sequence; a convolution module, which is used to fuse and extract features from the knowledge fusion vector representation of the sample sub-word sequence through a text convolutional neural network to obtain a knowledge fusion vector representation of the sample text sequence; a determination module, which is used to determine the knowledge dictionary part after the training is completed based on the knowledge fusion vector representation of the sample text sequence and the knowledge information label of the sample text sequence.

[0021] Optionally, the selection module is specifically configured to select a text sequence including the character and at least one character at the front and / or back end of the character from the text to be processed as the text sequence corresponding to the character.

[0022] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: one or more processors; and one or more computer-readable storage media on which instructions are stored; when the instructions are executed by the one or more processors, the processors execute the text processing method as described in any one of the above items.

[0023] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the text processing method as described in any one of the above items.

[0024] The embodiments of the present disclosure provide a text processing method, device, electronic device and storage medium. A knowledge dictionary part is obtained by pre-training a sample text sequence and a knowledge information label corresponding to the sample text sequence; for each character in the text to be processed, a text sequence corresponding to the character is selected from the text to be processed; the text sequence corresponding to the character is input into the pre-trained knowledge dictionary part, and the subword sequence corresponding to the text sequence is fused with knowledge information by the knowledge dictionary part to obtain a knowledge fusion vector representation of the subword sequence, and the knowledge fusion vector representation of the character is obtained based on the knowledge fusion vector representation of the subword sequence by the post-processing part. It can be seen that in the embodiments of the present disclosure, there is no need to construct an entity dictionary and a knowledge graph for data in different task fields. A general knowledge dictionary part is pre-trained by sample data, and knowledge information can be integrated into the text by using the knowledge dictionary part. The pre-training process of the knowledge dictionary part is simpler than the construction process of the entity dictionary and the knowledge graph, and the knowledge dictionary part can be flexibly combined with the natural language processing process, and has good adaptability and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments of the present disclosure. Obviously, the drawings described below are only some drawings of the embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 This is a flowchart of the steps of a text processing method according to an embodiment of the present disclosure.

[0027] Figure 2 2 is a schematic diagram of partial pre-training of a knowledge dictionary according to an embodiment of the present disclosure.

[0028] Figure 3 Schematic diagram of a text classification task according to an embodiment of the present disclosure.

[0029] Figure 4 It is a schematic diagram of a knowledge fusion model according to an embodiment of the present disclosure.

[0030] Figure 5 It is a schematic diagram of a review content interface according to an embodiment of the present disclosure.

[0031] Figure 6 It is a schematic diagram of a scenic spot and amusement park information interface according to an embodiment of the present disclosure.

[0032] Figure 7 It is a structural block diagram of a text processing device according to an embodiment of the present disclosure.

[0033] Figure 8 It is a structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0035] The text processing methods described in the embodiments of this disclosure can be applied to various NLP task scenarios that require the integration of knowledge and information. For example, these NLP task scenarios include, but are not limited to, syntactic and semantic analysis, information extraction, text clustering, text classification, text summarization, text sentiment analysis, text translation, information retrieval, information recommendation, automated question answering, and dialogue systems.

[0036] Reference Figure 1 , shows a step flow chart of a text processing method according to an embodiment of the present disclosure.

[0037] like Figure 1 As shown, the text processing method may include the following steps:

[0038] Step 101: For each character in a text to be processed, select a text sequence corresponding to the character from the text to be processed.

[0039] In the embodiment of the present disclosure, the text to be processed is subjected to knowledge information fusion processing through the pre-trained knowledge dictionary part. Since the knowledge dictionary part is pre-trained using sample text sequences and knowledge information labels corresponding to the sample text sequences, for each character in the text to be processed, the text sequence corresponding to the character is selected from the text to be processed so that the knowledge information fusion processing can be performed on the text sequence corresponding to the character using the knowledge dictionary part.

[0040] In an optional implementation, a text sequence including the character and at least one character at the front and / or back of the character may be selected from the text to be processed as the text sequence corresponding to the character.

[0041] Illustratively, to improve the relevance of a text sequence to a character, a text sequence centered on the character and containing k characters (k is a positive integer greater than or equal to 1) at each end of the character can be selected as the text sequence corresponding to the character. It is understood that if the number of characters at the front or back end is less than the aforementioned k, the characters currently contained at the front or back end can be used.

[0042] For example, for the first character in the text to be processed, the first character and the two characters after the first character can be selected as the text sequence corresponding to the first character; for the second character in the text to be processed, the second character, the one character before the second character and the two characters after the second character can be selected as the text sequence corresponding to the second character; for the third character in the text to be processed, the third character, the two characters before the third character and the two characters after the third character can be selected as the text sequence corresponding to the third character; for the fourth character in the text to be processed, the fourth character, the two characters before the fourth character and the two characters after the fourth character can be selected as the text sequence corresponding to the fourth character, and so on.

[0043] Step 102: input the text sequence corresponding to the character into the pre-trained knowledge dictionary part, perform knowledge information fusion on the subword sequence corresponding to the text sequence through the knowledge dictionary part to obtain the knowledge fusion vector representation of the subword sequence, and obtain the knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through the post-processing part.

[0044] In the disclosed embodiments, there is no need to construct entity dictionaries and knowledge graphs for data in different task areas. The general knowledge dictionary part is pre-trained with sample data, and the knowledge dictionary part can be used to integrate knowledge information into the text. The pre-training process of the knowledge dictionary part is simpler than the construction process of the entity dictionary and knowledge graph, and the knowledge dictionary part can be flexibly combined with the natural language processing process, and has good adaptability and flexibility.

[0045] The text processing method in the embodiment of the present disclosure is introduced in detail below.

[0046] This embodiment is mainly divided into two stages: (1) knowledge dictionary pre-training; (2) multi-dictionary fusion. Each stage is described in detail below.

[0047] (1) Knowledge dictionary pre-training

[0048] In the knowledge dictionary pre-training stage, the main purpose is to obtain the knowledge dictionary part through pre-training, embed the task domain related knowledge into vectors, and obtain the vector representation of knowledge for subsequent knowledge fusion.

[0049] In practical applications, for any task domain, we can select sample text sequences of each type of data from that task domain and train the corresponding embedding component, thereby obtaining the embedding component corresponding to each type of data in that task domain. Accordingly, the knowledge dictionary component can include a word segmentation component and at least one embedding component, where one embedding component is pre-trained based on a sample text sequence of a certain type.

[0050] Illustratively, task domains (i.e., the knowledge information to be integrated) can include, but are not limited to, cities, cuisines, and so on. Different types of data in any task domain can come from different data sources, with each data source corresponding to a specific data type. For example, for the city task domain, data from business names can be considered a specific data type, data from business addresses can be considered a specific data type, data from user search results can be considered a specific data type, and so on.

[0051] The following is an example of pre-training based on any type of sample text sequence.

[0052] Reference Figure 2 , showing a schematic diagram of partial pre-training of a knowledge dictionary in an embodiment of the present disclosure. Figure 2 The word segmentation part and the embedding part constitute the knowledge dictionary part, and the knowledge dictionary part is connected to the text convolutional neural network.

[0053] For any embedding part, the knowledge dictionary part is pre-trained through the following steps a1 to a4:

[0054] Step a1: input the sample text sequence of the type corresponding to the embedding part into the word segmentation part of the knowledge dictionary part, and perform subword segmentation on the sample text sequence through the word segmentation part to obtain a sample subword sequence corresponding to the sample text sequence.

[0055] A sample text sequence is selected from the data of the corresponding type in the currently embedded part, and the knowledge information of the sample text sequence is extracted as the knowledge information label of the sample text sequence. The sample text sequence can be selected according to different task scenarios. The construction of the sample text sequence does not require excessive processing, and it can be selected from the original domain-related text data. For example, a merchant name can be selected from a large amount of merchant information as a sample text sequence, and the city information corresponding to the merchant name can be extracted as the knowledge information label of the sample text sequence.

[0056] Input the sample text sequence into the word segmentation part. Figure 2 In the example, the leftmost input “aa East Road bb Hotel” is the sample text sequence.

[0057] The word segmentation part performs word segmentation on the input sample text sequence at the subword granularity to obtain the sample subword sequence output by the word segmentation part. The sample subword sequence can include multiple sample subwords. Figure 2 In the example text sequence "aa East Road bb Hotel" is segmented at the word segmentation part to obtain the sample subword sequence "[' ', 'aa', 'East Road', 'bb', 'Hotel']".

[0058] Illustratively, the word segmentation part in this embodiment can use the SentencePiece word segmentation module, etc. SentencePiece is an open source natural language processing toolkit that can be used for word segmentation. It integrates different word segmentation methods such as unigram (unary word segmentation), BPE (Byte Pair Encoding, byte pair encoding word segmentation), char (character word segmentation), word (Chinese word segmentation), etc., and can segment a sentence into subword granularity. Subword is a word segmentation granularity between character granularity and word granularity, which can effectively retain the semantic information in the text to the greatest extent while using a relatively small vocabulary size. In practical applications, SentencePiece can use any word segmentation method, as long as it can segment a text into a subword sequence of subword granularity.

[0059] Step a2: performing knowledge information fusion on each sample sub-word in the sample sub-word sequence through the embedding part to obtain a knowledge fusion vector representation of the sample sub-word sequence.

[0060] The sample subword sequence output by the word segmentation part is input into the embedding part. The embedding part performs knowledge information fusion on each sample subword in the sample subword sequence to obtain the knowledge fusion vector representation of the sample subword. Then, the knowledge fusion vector representations of multiple sample subwords constitute the knowledge fusion vector representation of the sample subword sequence.

[0061] Illustratively, the embedding part in this embodiment may use an embedding table, etc., where the embedding table is a variable that can be optimized through learning.

[0062] Step a3: fusing and extracting features from the knowledge fusion vector representation of the sample subword sequence through a text convolutional neural network to obtain a knowledge fusion vector representation of the sample text sequence.

[0063] The knowledge fusion vector representation of the sample subword sequence output by the embedding part is input into a text convolutional neural network (TextCNN). The Text Convolutional Neural Network extracts features from the knowledge fusion vector representation of the sample subword sequence to obtain the knowledge fusion vector representation of the entire sample text sequence. TextCNN is a text processing network structure based on convolutional neural networks. It can extract the semantic features of the sentence from the input word sequence through convolution.

[0064] Step a4, determining that the knowledge dictionary part is obtained after the training is completed based on the knowledge fusion vector representation of the sample text sequence and the knowledge information label of the sample text sequence.

[0065] After obtaining the knowledge fusion vector representation of the sample text sequence, the embedding part can be optimized through predefined tasks (such as classification tasks, etc.).

[0066] A loss function can be calculated based on the knowledge fusion vector representation of the sample text sequence and the knowledge information label of the sample text sequence. When the loss function reaches an optimization goal (e.g., the loss function is less than a certain threshold), training can be determined to be complete. The loss function may include, but is not limited to, at least one of the following: a cross-entropy loss function, an exponential loss function, a hinge loss function, and the like.

[0067] In response to determining that the training is not complete, the parameters of the embedding part can be updated and the optimization training can be continued until the training is completed.

[0068] For example, in the review business, there's a lot of unprocessed information from merchants and users, such as merchant names, addresses, and user search results. We can pre-train the knowledge dictionary using merchant names and addresses as the original lexicon (i.e., sample text sequences) and the corresponding city IDs (representations) as the supervisory signals for classification (i.e., the knowledge information labels for the sample text sequences). The resulting knowledge fusion vector representation of the sample subwords, after pre-training, will contain the corresponding city information. For example, "bb restaurant" contains the location information of "city B."

[0069] (2) Multi-dictionary fusion

[0070] After pre-training the knowledge dictionary, we can obtain a knowledge dictionary containing embeddings corresponding to different types of data within the same task domain. During the multi-dictionary fusion process, we can simply modify the original task in any domain to incorporate the knowledge dictionary and use it to perform the corresponding text processing task.

[0071] This embodiment will take the text classification task as an example for introduction, but in actual application it is not limited to the text classification task.

[0072] Reference Figure 3 , which shows a schematic diagram of a text classification task according to an embodiment of the present disclosure.

[0073] like Figure 3 As shown in FIG, in the text classification task, a knowledge fusion model is added, and the knowledge fusion model includes the above-mentioned knowledge dictionary part.

[0074] First, for each character in the text to be processed, the text sequence corresponding to the character is selected from the text to be processed; then, the text sequence corresponding to each character is input into the knowledge fusion model, and the knowledge fusion model is used to fuse the text sequence corresponding to the character to obtain the knowledge fusion vector representation after the knowledge information of each character in the text to be processed is fused; then, the knowledge fusion vector representation after the knowledge information of each character is fused is input into the BERT (Bidirectional Encoder Representations from Transformer) model for processing; then, the output of the BERT model is input into the MLP (Multilayer Perceptron) for processing; finally, the downstream task is entered.

[0075] Next, the knowledge fusion model is introduced in detail.

[0076] Reference Figure 4 , which shows a schematic diagram of a knowledge fusion model of an embodiment of the present disclosure.

[0077] like Figure 4 As shown in Figure 1, the knowledge fusion model consists of a knowledge dictionary and a post-processing part. The knowledge dictionary contains a segmentation part (SentencePiece) and N (N is a positive integer greater than or equal to 1) embedding parts (embedding tables), and the post-processing part includes a transformation part (Transformer) and a pooling part (Pooling).

[0078] First, for each character in the text to be processed, a text sequence corresponding to the character is selected from the text to be processed as input of the knowledge fusion model.

[0079] Schematically, for each character w of the input text i , with w i Take a text sequence s with a window size of 2k+1 as the center i = <w i-k ,w i-k+1 ,…,w i ,…,wi+k >, as the input of the knowledge fusion model.

[0080] Then, the process of text processing using the knowledge fusion model may include the following steps b1 to b4:

[0081] Step b1: segmenting the text sequence at a subword granularity by the word segmentation part to obtain a subword sequence corresponding to the text sequence.

[0082] In the knowledge fusion model, the input text sequence s i The text sequence s is obtained by performing word segmentation processing on the same word segmentation part (SentencePiece) as in the knowledge dictionary pre-training. i The corresponding subword sequence sw i = <sw i,1 ,sw i,2 ,…,sw i,L >, where L is the length of the subword sequence after segmentation.

[0083] Step b2: performing knowledge information fusion on each subword in the subword sequence through each embedding part to obtain a knowledge fusion vector representation of the subword sequence corresponding to each embedding part.

[0084] The subword sequence sw i Input N embedding parts (Embedding Table) respectively for knowledge information fusion processing, and obtain N subword sequences sw i Each embedding table is initialized by the parameters of the embedding table obtained by the above pre-training.

[0085] Among them, the subword sequence sw obtained by the jth embedding part (Embedding Table) i The knowledge fusion vector is represented as e i,j = <e i,j,1 ,e i,j,2 ,…,e i,j,L >.

[0086] Step b3, performing inter-subword fusion processing on the knowledge fusion vector representation of the subword sequence corresponding to each embedded part through the conversion part, to obtain the knowledge fusion vector representation of the text sequence corresponding to each embedded part.

[0087] Get the subword sequence sw output by each embedding table i The knowledge fusion vector representation {e i,j}After that, use the transformation part (Transformer) to transform each e i,j Perform fusion processing between subwords to obtain each e i,j The corresponding knowledge fusion vector representation of the text sequence is t i,j :

[0088] t i,j =Transformer([e i,j,1 ,e i,j,2 ,…,e i,j,L ])

[0089] Step b4: aggregating the knowledge fusion vector representations of the text sequence corresponding to all embedded parts through the pooling part to obtain the knowledge fusion vector representation of the character.

[0090] The conversion part outputs the knowledge fusion vector representation t of the text sequence corresponding to each embedding part i,j After that, the pooling part (Pooling) is used to i,j Fusion is performed to obtain the final character w i The knowledge fusion vector representation f i .

[0091] Illustratively, the pooling method of the pooling part (Pooling) in this embodiment can adopt any applicable pooling method such as mean-pooling, max-pooling, min-pooling, stochastic-pooling, global average pooling, etc.

[0092] At this point, the knowledge information in multiple dictionaries can be integrated into a BERT text classification task.

[0093] Next, we will take the review recommendation scenario as an example to introduce how the above multi-dictionary fusion method is applied in the recommendation scenario.

[0094] In review recommendation scenarios, identifying the cities to which the content belongs is crucial for distributing high-quality, non-regional content nationwide. However, identifying the cities to which some content belongs can be challenging due to irregular user input, entity aliases, abbreviations, or the absence of associated POIs (Points of Interest).

[0095] Figure 5 Schematic diagram of a review content interface of an embodiment of the present disclosure. Figure 5As shown in the example, user 1 posted a review, "CC Lake, although it's a temple, the scenery is beautiful, but not many people go there." User 2 posted a review, "The EE zone in DD Park is too reminiscent of the game." The cities to which the entities in the reviews posted by user 1 and user 2 belong cannot be identified.

[0096] Among the scenic spots and amusement parks information included in the review platform, there are the places mentioned above.

[0097] Figure 6 Schematic diagram of a scenic spot and amusement park information interface according to an embodiment of the present disclosure. Figure 6 As shown, the scenic area information screen includes information such as "CC Lake Scenic Area, 50 meters southeast of the intersection of XX Road and XX Avenue, City B, Province A." The amusement park information screen includes information such as "Super DD World, No. XX, XX Street, District XX, City C." The information on the scenic area information screen indicates the city to which CC Lake Scenic Area belongs, and the information on the amusement park information screen indicates the city to which Super DD World belongs.

[0098] Therefore, we can use the POI dictionary (including scenic spots) and POI sub-item dictionary (including amusement park projects) included in the reviews to pre-train the city classification task. We can obtain the subword sequence "[' ', 'c', 'c', 'lake', 'scenic spot']" corresponding to "cc Lake Scenic Spot" and the location information contained in its subwords, as well as the subword sequence "[' ', 'super', 'dd', 'world']" corresponding to "Super dd World" and the location information contained in its subwords. Afterwards, we perform the same subword segmentation on the review content to obtain the subword sequence within the window range corresponding to each character. By fusing the dictionary information corresponding to the subword sequence as described above, we can encode city information such as "cc Lake → City B" and "dd World → City C" into character-level vector representations.

[0099] In the embodiments of the present disclosure, without specifically constructing an entity dictionary or knowledge graph, different types of relevant data in the task field are utilized and integrated into the natural language processing task, which can better solve the problem of knowledge fusion of tail data in the corresponding task and has good adaptability and flexibility.

[0100] Reference Figure 7 , shows a structural block diagram of a text processing device according to an embodiment of the present disclosure.

[0101] like Figure 7 As shown, the text processing device may include the following modules:

[0102] A selection module 701 is configured to select, for each character in a text to be processed, a text sequence corresponding to the character from the text to be processed;

[0103] Processing module 702 is configured to input a text sequence corresponding to the character into a pre-trained knowledge dictionary, perform knowledge information fusion on a subword sequence corresponding to the text sequence through the knowledge dictionary to obtain a knowledge fusion vector representation of the subword sequence, and obtain a knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through a post-processing part;

[0104] The knowledge dictionary part is obtained by pre-training using sample text sequences and knowledge information labels corresponding to the sample text sequences.

[0105] Optionally, the knowledge dictionary part includes a word segmentation part and at least one embedding part, and one embedding part is obtained by pre-training based on a type of sample text sequence; the processing module 702 includes: a word segmentation unit, which is used to perform sub-word granularity segmentation on the text sequence through the word segmentation part to obtain a sub-word sequence corresponding to the text sequence; an embedding unit, which is used to perform knowledge information fusion on each sub-word in the sub-word sequence through each embedding part, and obtain a knowledge fusion vector representation of the sub-word sequence corresponding to each embedding part.

[0106] Optionally, the post-processing part includes a conversion part and a pooling part; the processing module 702 includes: a conversion unit, which is used to perform inter-subword fusion processing on the knowledge fusion vector representation of the sub-word sequence corresponding to each embedded part through the conversion part, to obtain the knowledge fusion vector representation of the text sequence corresponding to each embedded part; a pooling unit, which is used to perform aggregation processing on the knowledge fusion vector representation of the text sequence corresponding to all embedded parts through the pooling part, to obtain the knowledge fusion vector representation of the character.

[0107] Optionally, the knowledge dictionary part includes a word segmentation part and at least one embedding part, and one embedding part is pre-trained based on a type of sample text sequence; for any embedding part, the knowledge dictionary part is pre-trained through the following modules: a sample word segmentation module, which is used to input the sample text sequence of the type corresponding to the embedding part into the word segmentation part in the knowledge dictionary part, and perform sub-word segmentation on the sample text sequence through the word segmentation part to obtain a sample sub-word sequence corresponding to the sample text sequence; a sample embedding module, which is used to perform knowledge information fusion on each sample sub-word in the sample sub-word sequence through the embedding part to obtain a knowledge fusion vector representation of the sample sub-word sequence; a convolution module, which is used to fuse and extract features from the knowledge fusion vector representation of the sample sub-word sequence through a text convolutional neural network to obtain a knowledge fusion vector representation of the sample text sequence; a determination module, which is used to determine the knowledge dictionary part after the training is completed based on the knowledge fusion vector representation of the sample text sequence and the knowledge information label of the sample text sequence.

[0108] Optionally, the selection module 701 is specifically configured to select a text sequence including the character and at least one character at the front and / or back end of the character from the text to be processed as the text sequence corresponding to the character.

[0109] In the disclosed embodiments, there is no need to construct entity dictionaries and knowledge graphs for data in different task areas. The general knowledge dictionary part is pre-trained with sample data, and the knowledge dictionary part can be used to integrate knowledge information into the text. The pre-training process of the knowledge dictionary part is simpler than the construction process of the entity dictionary and knowledge graph, and the knowledge dictionary part can be flexibly combined with the natural language processing process, and has good adaptability and flexibility.

[0110] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0111] In an embodiment of the present disclosure, an electronic device is further provided. The electronic device may include one or more processors and one or more computer-readable storage media storing instructions, such as application programs. When the instructions are executed by the one or more processors, the processors perform the text processing method described in any of the above embodiments.

[0112] Reference Figure 8 , which shows a schematic diagram of an electronic device structure according to an embodiment of the present disclosure. Figure 8As shown, the electronic device includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804. The processor 801, the communication interface 802 and the memory 803 communicate with each other via the communication bus 804.

[0113] The memory 803 is used to store computer programs.

[0114] The processor 801 is configured to implement the text processing method of any of the above embodiments when executing the program stored in the memory 803 .

[0115] The communication interface 802 is used for communication between the electronic device and other devices.

[0116] The communication bus 804 mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0117] The processor 801 mentioned above may include but is not limited to: a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0118] The memory 803 mentioned above may include but is not limited to: Read Only Memory (ROM), Random Access Memory (RAM), Compact Disc Read Only Memory (CD-ROM), Electronic Erasable Programmable Read Only Memory (EEPROM), hard disk, floppy disk, flash memory, etc.

[0119] In an embodiment of the present disclosure, a non-temporary computer-readable storage medium is also provided, on which a computer program is stored. The computer program can be executed by a processor of an electronic device. When the computer program is executed by the processor, the processor executes the text processing method described in any of the above embodiments.

[0120] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used together with the teachings herein. Based on the above description, it is apparent that the structure required for constructing such systems is suitable. In addition, the embodiments of the present disclosure are not directed to any specific programming language. It should be understood that various programming languages may be utilized to implement the contents of the embodiments of the present disclosure described herein, and the above description of specific languages is intended to disclose the best mode of implementation of the embodiments of the present disclosure.

[0121] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0122] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present disclosure, various features of the embodiments of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed approach should not be interpreted as reflecting an intention that the claimed embodiments of the present disclosure require more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all of the features of the individual embodiments disclosed above. Accordingly, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the embodiments of the present disclosure.

[0123] Those skilled in the art will appreciate that the modules in the devices of the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device so disclosed may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0124] The various component embodiments of the embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the device for generating dynamic images according to the embodiments of the present disclosure. The embodiments of the present disclosure may also be implemented as a device or apparatus program for executing part or all of the methods described herein. Such a program implementing the embodiments of the present disclosure may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0125] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present disclosure, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The embodiments of the present disclosure may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0126] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] The above description is only a specific implementation of the embodiments of the present disclosure, but the protection scope of the embodiments of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present disclosure, and they should all be covered by the protection scope of the embodiments of the present disclosure.

Claims

1. A text processing method, characterized in that: include: For each character in the text to be processed, selecting a text sequence corresponding to the character from the text to be processed; Inputting the text sequence corresponding to the character into a pre-trained knowledge dictionary part, performing knowledge information fusion on the subword sequence corresponding to the text sequence through the knowledge dictionary part to obtain a knowledge fusion vector representation of the subword sequence, and obtaining a knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through a post-processing part; The knowledge dictionary is pre-trained using sample text sequences and knowledge information labels corresponding to the sample text sequences; The knowledge dictionary part includes a word segmentation part and at least one embedding part, wherein the embedding part is pre-trained based on a type of sample text sequence; the knowledge information of the subword sequence corresponding to the text sequence is fused by the knowledge dictionary part to obtain a knowledge fusion vector representation of the subword sequence, including: Performing word segmentation on the text sequence at a subword granularity by the word segmentation part to obtain a subword sequence corresponding to the text sequence; Performing knowledge information fusion on each subword in the subword sequence through each embedding part, thereby obtaining a knowledge fusion vector representation of the subword sequence corresponding to each embedding part; The post-processing part includes a conversion part and a pooling part; obtaining the knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through the post-processing part includes: The conversion part performs inter-subword fusion processing on the knowledge fusion vector representation of the subword sequence corresponding to each embedded part, thereby obtaining the knowledge fusion vector representation of the text sequence corresponding to each embedded part; The pooling part aggregates the knowledge fusion vector representations of the text sequence corresponding to all the embedded parts to obtain the knowledge fusion vector representation of the character.

2. The method according to claim 1, characterized in that The knowledge dictionary part includes a word segmentation part and at least one embedding part, and each embedding part is pre-trained based on a type of sample text sequence; for any embedding part, the knowledge dictionary part is pre-trained in the following manner: Inputting a sample text sequence of a type corresponding to the embedding part into the word segmentation part of the knowledge dictionary part, and performing subword segmentation on the sample text sequence by the word segmentation part to obtain a sample subword sequence corresponding to the sample text sequence; Performing knowledge information fusion on each sample subword in the sample subword sequence through the embedding part to obtain a knowledge fusion vector representation of the sample subword sequence; fusing and extracting features from the knowledge fusion vector representation of the sample subword sequence through a text convolutional neural network to obtain a knowledge fusion vector representation of the sample text sequence; After the training is completed based on the knowledge fusion vector representation of the sample text sequence and the knowledge information label of the sample text sequence, the knowledge dictionary part is obtained.

3. The method according to claim 1, characterized in that The step of selecting a text sequence corresponding to the character from the text to be processed includes: A text sequence including the character and at least one character at the front and / or back of the character is selected from the text to be processed as the text sequence corresponding to the character.

4. A text processing device, characterized in that: include: A selection module, configured to select, for each character in a text to be processed, a text sequence corresponding to the character from the text to be processed; a processing module configured to input a text sequence corresponding to the character into a pre-trained knowledge dictionary, perform knowledge information fusion on a subword sequence corresponding to the text sequence through the knowledge dictionary to obtain a knowledge fusion vector representation of the subword sequence, and obtain a knowledge fusion vector representation of the character based on the knowledge fusion vector representation of the subword sequence through a post-processing module; The knowledge dictionary is pre-trained using sample text sequences and knowledge information labels corresponding to the sample text sequences; The knowledge dictionary part includes a word segmentation part and at least one embedding part, and each embedding part is pre-trained based on a type of sample text sequence; the processing module includes: A word segmentation unit, configured to segment the text sequence at a subword granularity using the word segmentation part to obtain a subword sequence corresponding to the text sequence; an embedding unit, configured to perform knowledge information fusion on each subword in the subword sequence through each embedding part, and obtain a knowledge fusion vector representation of the subword sequence corresponding to each embedding part; The post-processing part includes a conversion part and a pooling part; the processing module includes: The conversion unit is used to perform inter-subword fusion processing on the knowledge fusion vector representation of the subword sequence corresponding to each embedded part through the conversion part, so as to obtain the knowledge fusion vector representation of the text sequence corresponding to each embedded part; the pooling unit is used to perform aggregation processing on the knowledge fusion vector representation of the text sequence corresponding to all embedded parts through the pooling part, so as to obtain the knowledge fusion vector representation of the character.

5. The device according to claim 4, characterized in that The knowledge dictionary part includes a word segmentation part and at least one embedding part. Each embedding part is pre-trained based on a type of sample text sequence. For any embedding part, the knowledge dictionary part is pre-trained through the following modules: a sample word segmentation module, configured to input a sample text sequence of a type corresponding to the embedding part into the word segmentation part of the knowledge dictionary part, and perform word segmentation on the sample text sequence at a subword granularity by the word segmentation part to obtain a sample subword sequence corresponding to the sample text sequence; a sample embedding module, configured to perform knowledge information fusion on each sample subword in the sample subword sequence through the embedding part to obtain a knowledge fusion vector representation of the sample subword sequence; A convolution module is used to fuse and extract features from the knowledge fusion vector representation of the sample subword sequence through a text convolutional neural network to obtain a knowledge fusion vector representation of the sample text sequence; The determination module is used to determine the knowledge dictionary part obtained after the training is completed based on the knowledge fusion vector representation of the sample text sequence and the knowledge information label of the sample text sequence.

6. The device according to claim 4, characterized in that The selection module is specifically configured to select a text sequence including the character and at least one character at the front and / or back end of the character from the text to be processed as the text sequence corresponding to the character.

7. An electronic device, characterized in that: include: one or more processors; and one or more computer-readable storage media having instructions stored thereon; When the instructions are executed by the one or more processors, the processors are enabled to perform the text processing method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the processor is caused to execute the text processing method according to any one of claims 1 to 3.

Images