Text processing method, device, electronic device and computer-readable storage medium

Abstract

The embodiment of the present invention discloses a text processing method, device, electronic device and computer-readable storage medium. After obtaining text word samples and text sample pairs, the embodiment of the present invention uses a preset text processing model to segment the text sample pairs, and extracts features of target text words and text word samples to obtain text word features and text word sample features. Based on the text word features and text word sample features, keyword categories are identified for the target text words and text word samples. Then, according to the identified first keyword category, the text word features are weighted to obtain text features of the text sample, and the feature distance between the text features is calculated. Then, based on the identified second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship, the preset text processing model is converged, and the trained text processing model is used to retrieve the target text. This scheme can improve the accuracy of text processing.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a text processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] In recent years, with the rapid development of Internet technology, a huge amount of text has appeared on the Internet, and it is necessary to retrieve the required target text from these huge amounts of text. In the process of retrieving these texts, it is often necessary to process the texts to achieve online retrieval of texts. Existing text processing methods often extract text features of texts through the Bert model (twin tower model), and retrieve the target text from the huge amount of texts based on the text features.

[0003] In the process of studying and practicing the prior art, the inventors of the present invention found that the text features extracted by the Bert model often contain some information noise, which makes the extracted text features inaccurate, thus resulting in insufficient accuracy of text processing. Summary of the invention

[0004] Embodiments of the present invention provide a text processing method, device, electronic device and computer-readable storage medium, which can improve the accuracy of text processing.

[0005] A text processing method, comprising:

[0006] Acquire text word samples and text sample pairs, wherein the text word samples include text words marked with keyword categories, and the text sample pairs include text pairs marked with semantic matching relationships;

[0007] Using a preset text processing model to segment the text samples in the text sample pair, and extracting features from the segmented target text words and text word samples to obtain text word features of the target text words and text word sample features of the text word samples;

[0008] Based on the text word features and the text word sample features, performing keyword category recognition on the target text word and the text word sample to obtain a first keyword category of the target text word and a second keyword category of the text word sample;

[0009] According to the first keyword category, weighting the text word features to obtain text features of each text sample in the text sample pair, and calculating feature distances between the text features;

[0010] Based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship, the preset text processing model is converged to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text.

[0011] Accordingly, an embodiment of the present invention provides a text processing device, including:

[0012] An acquisition unit, configured to acquire text word samples and text sample pairs, wherein the text word samples include text words annotated with keyword categories, and the text sample pairs include text pairs annotated with semantic matching relationships;

[0013] A word segmentation unit is used to segment the text samples in the text sample pair using a preset text processing model, and perform feature extraction on the target text words and text word samples after segmentation to obtain text word features of the target text words and text word sample features of the text word samples;

[0014] an identification unit, configured to perform keyword category identification on the target text word and the text word sample based on the text word feature and the text word sample feature, to obtain a first keyword category of the target text word and a second keyword category of the text word sample;

[0015] A weighting unit, configured to weight the text word features according to the first keyword category to obtain text features of each text sample in the text sample pair, and calculate a feature distance between the text features;

[0016] The retrieval unit is used to converge the preset text processing model based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship to obtain a trained text processing model, and use the trained text processing model to retrieve the target text.

[0017] Optionally, in some embodiments, the weighting unit can be specifically used to determine the text weight of the text word feature based on the first keyword category; weight the text word feature based on the text weight, and fuse the weighted text word features to obtain the text feature of each text sample in the text sample pair.

[0018] Optionally, in some embodiments, the weighting unit can be specifically used to identify the category probability of each keyword category in the first keyword category to obtain a first category probability; screen out the category probability of at least one preset key keyword category in the first category probability to obtain a basic category probability; and fuse the basic category probabilities to obtain the text weight of the text word feature.

[0019] Optionally, in some embodiments, the weighting unit can be specifically used to fuse the weighted text word features to obtain fused text features; extract the query text features corresponding to the query text sample and at least one field text feature corresponding to the target text sample from the fused text features; fuse the field text features to obtain target field text features, and use the target field text features and query text features as text features of each text sample in the text sample pair.

[0020] Optionally, in some embodiments, the weighting unit can be specifically used to extract associated features of the field text features to obtain associated features of the field text features; based on the associated features, determine the associated weights of the field text features, and the associated weights are used to indicate the association relationship between field text features; according to the associated weights, weight the field text features, and fuse the weighted field text features to obtain target field text features.

[0021] Optionally, in some embodiments, the retrieval unit can be specifically used to determine the keyword loss information of the text word sample based on the second keyword category and the annotated keyword category; determine the text loss information of the text sample pair according to the annotated semantic matching relationship and feature distance; and converge the preset text processing model based on the keyword loss information and text loss information to obtain a trained text processing model.

[0022] Optionally, in some embodiments, the retrieval unit can be specifically used to identify the category probability of each keyword category in the second keyword category to obtain the second category probability; filter out the category probability corresponding to the annotated keyword category in the second category probability to obtain the target category probability; fuse the target category probability with the annotated keyword category, and calculate the mean of the fused keyword category to obtain the keyword loss information of the text word sample.

[0023] Optionally, in some embodiments, the retrieval unit can be specifically used to determine the matching parameters of the text sample pair based on the annotated semantic matching relationship; when the matching parameters are preset matching parameters and the feature distance is less than a preset distance threshold, the matching parameters are fused with the feature distance to obtain the text loss information of the text sample pair.

[0024] Optionally, in some embodiments, the retrieval unit can be specifically used to calculate the distance difference between the feature distance and the preset distance threshold; calculate the parameter difference between the matching parameter and the preset parameter threshold, and fuse the distance difference with the parameter difference; fuse the fused difference, matching parameter and feature distance to obtain text loss information of the text sample.

[0025] Optionally, in some embodiments, the retrieval unit can be specifically used to obtain loss weights, and based on the loss weights, weight the keyword loss information and the text loss information respectively; fuse the weighted keyword loss information and the weighted text loss information to obtain target loss information; use the weighted keyword loss information to converge the keyword recognition network to obtain a trained keyword recognition network; use the target loss information to converge the feature extraction network to obtain a trained feature extraction network, and use the trained keyword recognition network and the trained feature extraction network as trained text processing models.

[0026] Optionally, in some embodiments, the recognition unit can be specifically used to use the keyword recognition network to normalize the text word features and the text word sample features respectively; based on the normalized text word features, map the category probability of the target text word belonging to each keyword category to obtain the first keyword category of the target text word; based on the normalized text word sample features, map the category probability of the text word sample belonging to each keyword category to obtain the second keyword category of the text word sample.

[0027] Optionally, in some embodiments, the retrieval unit can be specifically used to obtain a candidate text set, and use the trained text processing model to perform feature extraction on each candidate text in the candidate text set to obtain a candidate text feature set; based on the candidate text features in the candidate text feature set, construct index information corresponding to the candidate text feature set; when a query text is received, based on the index information and the query text, screen out at least one candidate text in the candidate text set as a target text.

[0028] Optionally, in some embodiments, the retrieval unit can be specifically used to use the trained text processing model to perform feature extraction on the query text to obtain query text features of the query text; based on the index information, retrieve at least one candidate text feature corresponding to the query text feature in the candidate text feature set to obtain a target candidate text feature; and screen out candidate texts corresponding to the target candidate text feature in the candidate text set to obtain a target text corresponding to the query text.

[0029] Optionally, in some embodiments, the acquisition unit can be specifically used to acquire a text sample set, and screen out at least one text sample and a semantic text sample corresponding to the text sample from the text sample set, wherein the semantic text sample is a text sample that has a semantic relationship with the text sample; the text sample is segmented using the preset text processing model, and keyword categories are marked in the segmented text words to obtain text word samples; and according to the semantic relationship between the text sample and the semantic text sample, a semantic matching relationship is marked in a text pair consisting of the text sample and the semantic text sample to obtain a text sample pair.

[0030] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the text processing method provided by the embodiment of the present invention.

[0031] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps in any text processing method provided by the embodiment of the present invention.

[0032] After obtaining the text word sample and the text sample pair, the embodiment of the present invention uses a preset text processing model to segment the text sample in the text sample pair, and extracts features of the target text word and the text word sample after segmentation to obtain the text word features of the target text word and the text word sample features of the text word sample. Then, based on the text word features and the text word sample features, the target text word and the text word sample are identified by keyword categories to obtain the first keyword category of the target text word and the second keyword category of the text word sample. Then, according to the first keyword category, the text word features are weighted to obtain the text word category of each text sample in the text sample pair. features, and calculate the feature distance between text features. Then, based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship, the preset text processing model is converged to obtain the trained text processing model, and the trained text processing model is used to retrieve the target text. Since the scheme uses a multi-task framework to train the keyword category recognition task and the semantic matching task at the same time, and by identifying the first keyword category, the text word features are weighted, which explicitly enhances the text processing model's ability to recognize word weights in the semantic matching task, thereby effectively reducing information noise. Therefore, the accuracy of text processing can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 is a scenario schematic diagram of a text processing method provided by an embodiment of the present invention;

[0035] Figure 2 is a flowchart of a text processing method provided by an embodiment of the present invention;

[0036] Figure 3 is a retrieval schematic diagram of text retrieval provided by an embodiment of the present invention;

[0037] Figure 4 It is a schematic diagram of the core framework of the text processing flow provided by an embodiment of the present invention;

[0038] Figure 5 It is a multi-task learning framework diagram in the text processing flow provided by an embodiment of the present invention;

[0039] Figure 6 is a flowchart of a keyword recognition task provided by an embodiment of the present invention;

[0040] Figure 7 is a flowchart of a semantic matching task provided by an embodiment of the present invention;

[0041] Figure 8 is another flowchart of the text processing method provided by an embodiment of the present invention;

[0042] Fig. 9 is a structural schematic diagram of a text processing device provided by an embodiment of the present invention;

[0043] Fig.10 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0045] The embodiments of the present invention provide a text processing method, device, electronic device and computer-readable storage medium. The text processing device can be integrated in an electronic device, which can be a server or a terminal.

[0046] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0047] For example, see Figure 1 Taking the text processing device integrated in an electronic device as an example, after obtaining a text word sample and a text sample pair, the electronic device uses a preset text processing model to segment the text samples in the text sample pair, and extracts features of the target text words and text word samples after segmentation to obtain text word features of the target text words and text word sample features of the text word samples. Then, based on the text word features and text word sample features, keyword category recognition is performed on the target text words and text word samples to obtain a first keyword category of the target text words and a second keyword category of the text word samples. Then, according to the first keyword category, the text word features are weighted to obtain text features of each text sample in the text sample pair, and the feature distance between the text features is calculated. Then, based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship, the preset text processing model is converged to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text, thereby improving the accuracy of text processing.

[0048] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0049] This embodiment will be described from the perspective of a text processing device, which can be specifically integrated into an electronic device, which can be a server or a terminal. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device, or other smart devices that can perform text processing.

[0050] A text processing method, comprising:

[0051] A text word sample and a text sample pair are obtained, wherein the text word sample includes a text word with annotated keyword category, and the text sample pair includes a text pair with annotated semantic matching relationship, a preset text processing model is used to segment the text samples in the text sample pair, and feature extraction is performed on the segmented target text words and text word samples to obtain text word features of the target text words and text word sample features of the text word samples, keyword category recognition is performed on the target text words and text word sample based on the text word features and text word sample features to obtain a first keyword category of the target text word and a second keyword category of the text word sample, the text word features are weighted according to the first keyword category to obtain text features of each text sample in the text sample pair, and feature distances between text features are calculated, the preset text processing model is converged based on the second keyword category, the annotated keyword category, the feature distances and the annotated semantic matching relationship to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text.

[0052] like Figure 2 As shown, the specific process of the text processing method is as follows:

[0053] 101. Obtain text word samples and text sample pairs.

[0054] Among them, the text word samples include text words with annotated keyword categories. The so-called keyword category is used to indicate which category of keywords the text word belongs to. There can be multiple keyword categories, for example, they can be divided into three categories, non-keywords, general keywords and important keywords. These three categories of keywords can be represented by different identifiers, for example, non-keywords can be represented by 0, general keywords can be represented by 1, important keywords can be represented by 2, or they can also be represented by other identifiers, but the identifiers of different keyword categories are often different. For example, taking the text sequence of "nanny and confinement nanny appointment, customized screening, professional and reliable" as an example, the corresponding annotated keyword category result can be "nanny / 2, nanny / 2, confinement / 2, nanny / 2, reservation / 1, appointment / 1, order / 0, make / 0, screen / 0, select / 0, professional / 0, industry / 0, reliable / 0, spectrum / 0".

[0055] The text sample pairs include text pairs with annotated semantic matching relationships. The so-called semantic matching relationship can be the semantic correlation between two texts in the text pair. If the correlation is high, the two texts are considered to match. Many natural language processing tasks can be converted into semantic matching problems. For example, web search can be abstracted into the correlation matching problem between the user's query text (query) and the web page content, automatic question answering can be abstracted into the satisfaction matching problem between questions and candidate answers, and text deduplication can be abstracted into the similarity matching problem between texts.

[0056] There are many ways to obtain text word samples and text sample pairs, which can be as follows:

[0057] For example, a text sample set can be obtained, and at least one text sample and a semantic text sample corresponding to the text sample can be screened out from the text sample set, the text sample can be segmented using a preset text processing model, and keyword categories can be marked in the segmented text words to obtain text word samples, and based on the semantic relationship between the text sample and the semantic text sample, a semantic matching relationship can be marked in a text pair consisting of the text sample and the semantic text sample to obtain a text sample pair.

[0058] Among them, the semantic text sample is a text sample that has a semantic relationship with the text sample, and the semantic relationship is semantic match and semantic mismatch. There are many ways to screen out the semantic text samples corresponding to the text samples. For example, the retrieval system can be used to screen out text samples that semantically match the text samples in the text sample set as semantic text samples. At this time, the semantic relationship between the selected semantic text sample and the text sample can be a semantic match, and the text sample pair at this time can be a positive text sample pair. Text samples in the offline library can also be randomly extracted as the semantic text samples corresponding to the text samples. At this time, the semantic relationship between the extracted semantic text sample and the text sample can be a semantic mismatch, and the text sample pair at this time can be a negative text sample pair.

[0059] There are many ways to segment text samples. For example, the tokenization segmentation method in the Bert network in the preset text processing model can be used to segment each text sample into a single token, where token refers to a Chinese character, an English word or a root, etc. There are also many ways to mark the keyword category of the text words after segmentation. For example, the keyword category of the text word can be directly identified, and then, based on the identification result, the keyword category is marked on the text word. Alternatively, the segmented text words can be sent to a marking server, the keyword category of the text word returned by the marking server is received, and the corresponding keyword category is marked on the text word, thereby obtaining a text word sample.

[0060] There are many ways to mark the semantic matching relationship in the text pair composed of the text sample and the semantic text sample. For example, the semantic matching relationship between the text sample and the semantic text sample can be determined according to the semantic relationship between the text sample and the semantic text sample, and the text sample and the corresponding semantic text sample are combined into a text pair, and the semantic matching relationship is marked in the text pair, so as to obtain the text sample pair. The text sample pair can include a positive text sample pair and a negative text sample pair. The text sample in the positive text sample pair matches the semantic text sample, and the text sample in the negative text sample pair does not match the semantic text sample. In addition, when the text sample is a query text sample, the corresponding semantic text sample can be a target text sample.

[0061] 102. Use a preset text processing model to segment the text samples in the text sample pair, and perform feature extraction on the target text words and text word samples after segmentation to obtain text features of the target text words and text word sample features of the text word samples.

[0062] There are many ways to segment text samples in a text sample pair using a preset text processing model, which can be as follows:

[0063] For example, the tokenization method in the Bert network in the preset text processing model can be used to segment each text sample into a single token, which can be a Chinese character, an English word or a root, etc., so as to obtain the target text word after segmentation of each text sample in the text sample pair. Alternatively, each text sample in the text sample pair can be directly segmented by characters and divided into Chinese characters, English words or roots, so as to obtain the target text word after segmentation of each text sample in the text sample pair.

[0064] Among them, in the text sample pair, the text sample is used as the query text sample, and the corresponding semantic text sample can be the target text sample. In the target text sample, it can be first split into multiple fields, and then each field is segmented to obtain the target text words after segmentation of each field.

[0065] After the text samples in the text sample pair are segmented, features can be extracted from the segmented target text words and text word samples. There are many ways to extract features. For example, the Bert network in the preset text processing model or models such as XLNet and ELECTRA can be used to extract features from the target text words and text word samples respectively, so as to obtain the vector representation (q1, q2, q3) of each target text word and the vector representation (t1, t2, t3) of each text word sample, and use the vector representation of the target text word as the text word feature of the target text word, and use the vector representation of the text word sample as the text word sample feature of the text word sample.

[0066] When extracting features from target text words and text word samples, the network parameters of the Bert network can be shared.

[0067] 103. Based on the text word features and the text word sample features, keyword category recognition is performed on the target text word and the text word sample to obtain a first keyword category of the target text word and a second keyword category of the text word sample.

[0068] For example, the keyword recognition network in the preset text processing model can be used to normalize the text word features and the text word sample features respectively. According to the normalized text word features, the category probability of the target text word belonging to each keyword category is mapped to obtain the first keyword category of the target text word. Based on the normalized text word sample features, the category probability of the text word sample belonging to each keyword category is mapped to obtain the second keyword category of the text word sample.

[0069] There are many ways to normalize the text word features and the text word sample features. For example, a fully-connected neural network (Fully-Connected layer, FC) can be used to normalize the text word features and the text word sample features respectively.

[0070] After normalizing the text word features, the first keyword category of the target text word can be calculated based on the normalized text word features. There are many ways to calculate the first keyword category. For example, the category probability of each target text word belonging to each keyword category is calculated by the Softmax function. For example, taking the keyword category as three categories, the calculated category probabilities can be divided into in, is the probability that the i-th target text word belongs to the 0th category keyword, is the probability that the i-th target text word belongs to the first category of keywords, is the probability that the i-th target text word belongs to the second category keyword, As the first keyword category of the target text word.

[0071] After normalizing the features of the text word samples, the second keyword category of the text word samples can be calculated based on the normalized features of the text word samples. There are many ways to calculate the second keyword category. For example, the category probability of each text word sample belonging to each keyword category can be calculated by the Softmax function. For example, taking the keyword category as three categories, the calculated category probability can be: The probability that the i-th text word sample belongs to the 0th category is The probability of belonging to class 1 is The probability of belonging to class 2 is Will and The second keyword category as a sample of text words.

[0072] Among them, it can be found that the keyword recognition network in the preset text processing model is used to identify the keyword category of the target text words and text word samples. The network structure of the keyword recognition network can be an FC-Softmax network, and in the process of identifying the keyword category of text words and text word samples, the FC-Softmax network and the network parameters of the network are shared.

[0073] 104. According to the first keyword category, weight the text word features to obtain the text features of each text sample in the text sample pair, and calculate the feature distance between the text features.

[0074] For example, the text weight of the text word feature can be determined according to the first keyword category, and the text word feature is weighted based on the text weight, and the weighted text word features are fused to obtain the text feature of each text sample in the text sample pair, and the feature distance between the text features is calculated, which can be specifically as follows:

[0075] S1. Determine a text weight of a text word feature according to a first keyword category.

[0076] The text weight is used to indicate the importance of the target text word corresponding to the text word feature in each target text word in the text sample pair. Based on the text weight, the text feature of each text sample in the text sample pair can be more accurately characterized.

[0077] There are multiple ways to determine the text weight of the text word feature according to the first keyword category, which can be specifically as follows:

[0078] For example, the category probability of each keyword category is identified in the first keyword category to obtain the first category probability, the category probability of at least one preset keyword category is screened out in the first category probability to obtain the basic category probability, and the basic category probabilities are fused to obtain the text weight of the text word feature.

[0079] Among them, the probability of the first category can be There are many ways to filter out the category probability of at least one preset keyword category in the first category probability. For example, the category probabilities of the keyword categories of general keywords and important keywords can be filtered out in the first category probability to obtain the basic category probability. Taking the 0th category as non-keywords, the 1st category as general keywords and the 2nd category as important keywords as an example, the basic category probability can be obtained. Filter out as the base class probability.

[0080] After the basic category probabilities are screened out, the basic category probabilities can be fused to obtain the text weight of the text word feature. There are many ways to fuse them. For example, the fusion parameters of each basic category probability are obtained, and the fusion parameters are respectively fused with the corresponding basic category probabilities. Then, the fused basic category probabilities are added to obtain the target basic category probability. Then, the mean of the target basic category probabilities is calculated to obtain the text weight of the text word feature. Specifically, it can be shown as formula (1):

[0081]

[0082] Among them, w qi is the text weight of the i-th text word feature, and are the basic category probabilities respectively.

[0083] S2. Based on the text weight, the text word features are weighted, and the weighted text word features are fused to obtain the text features of each text sample in the text sample pair.

[0084] The text sample pair includes a query text sample and a target text sample. The query text sample may be a text sample, and the target text sample may be a semantic text sample corresponding to the text sample.

[0085] Among them, based on the text weight, the text word features are weighted to obtain the weighted text word features, and the weighted text word features are fused. There are many ways to fuse, which can be as follows:

[0086] For example, based on the text weight, the text word features are weighted to obtain the weighted text word features, and the weighted text word features are fused to obtain the fused text features (vec q =∑ i w qi *q i ). A query text feature corresponding to the query text sample and at least one field text feature corresponding to the target text sample are extracted from the fused text features, the field text features are fused to obtain a target field text feature, and the target field text feature and the query text feature are used as text features of each text sample in the text sample pair.

[0087] There are many ways to extract the query text features corresponding to the query text sample and at least one field text feature corresponding to the target text sample from the fused text features. For example, the text features belonging to the query text sample can be screened out from the fused text features to obtain the query text features corresponding to the query text sample. The text features of each field of the target text sample can be screened out from the fused text features to obtain at least one field text feature.

[0088] After the field text features are screened out, the field text features can be fused to obtain the target field text features. There are many ways to fuse the field text features, for example, extracting the associated features of the field text features to obtain the associated features of the field text features, and determining the associated weights of the field text features based on the associated features. The associated weights are used to indicate the association relationship between the field text features. According to the associated weights, the field text features are weighted, and the weighted field text features are fused to obtain the target field text features. The target field text features here can be understood as the text features corresponding to the target text samples.

[0089] Among them, it can be found that in the process of fusing field text features, the correlation weights between field text features are focused on. There are many ways to determine the correlation weights. For example, the attention network can be used to extract the correlation features of the field text features, and then, based on the correlation features, the correlation weights of each field text feature are calculated.

[0090] S3. Calculate the feature distance between text features.

[0091] The feature distance is used to indicate the semantic matching relationship between text samples corresponding to the text features. There may be multiple types of feature distances, for example, multiple distance forms such as Euclidean distance or cosine distance.

[0092] There are many ways to calculate the feature distance between text features, which can be as follows:

[0093] For example, the cosine distance between text features may be directly calculated to obtain the feature distance between the text features, or the Euclidean distance between text features may be calculated to obtain the feature distance between the text features.

[0094] 105. Based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship, the preset text processing model is converged to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text.

[0095] For example, the keyword loss information of the text word sample can be determined based on the second keyword category and the annotated keyword category, and the text loss information of the text sample pair can be determined according to the annotated semantic matching relationship and the feature distance. Based on the keyword loss information and the text loss information, the preset text processing model is converged to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text, which can be specifically as follows:

[0096] C1. Determine keyword loss information of the text word sample based on the second keyword category and the annotated keyword category.

[0097] The keyword loss information may be loss information generated by a preset text processing model in a keyword category recognition task.

[0098] There are multiple ways to determine the keyword loss information of the text word sample based on the second keyword category and the annotated keyword category, which can be specifically as follows:

[0099] For example, the category probability of each keyword category is identified in the second keyword category to obtain the second category probability, the category probability corresponding to the annotated keyword category is screened out in the second category probability to obtain the target category probability, the target category probability is fused with the annotated keyword category, and the mean of the fused keyword category is calculated to obtain the keyword loss information of the text word sample.

[0100] Among them, taking the keyword category as three categories as an example, the probability of identifying the second category in the second keyword category can be and There are many ways to filter out the target category probability from the second category probability. For example, when the annotated keyword category of the i-th text word sample is category 1, and Filter out is the target category probability.

[0101] After the target category probability is screened out, the target category probability can be fused with the annotated keyword category. There are many ways to fuse. For example, the keyword category parameter of the text word sample can be determined based on the annotated keyword category. When the i-th text word sample belongs to the c-th keyword, the keyword parameter can be 1, otherwise the keyword parameter can be 0. After preprocessing the target category probability, the preprocessed target category probability is multiplied by the keyword parameter to obtain the basic keyword loss information of the i-th text word sample. Then, the basic keyword loss information of the text word sample is accumulated, and the mean of the accumulated keyword loss information is calculated to obtain the keyword loss information of the text word sample. Specifically, it can be shown as formula (2):

[0102]

[0103] Among them, Loss keyword is the keyword loss information of the text word sample, Mark the keyword category (category C), is the target category probability (the category probability belonging to category C).

[0104] C2. Determine the text loss information of the text sample pair based on the annotated semantic matching relationship and feature distance.

[0105] The text loss information may be loss information generated by a preset text processing model in a semantic matching task. The so-called semantic matching task may be understood as calculating the semantic matching relationship between text samples in a text sample pair.

[0106] There are multiple ways to determine the text loss information of the text sample pair according to the annotation semantic matching relationship and feature distance, which can be as follows:

[0107] For example, according to the annotated semantic matching relationship, the matching parameters of the text sample pair are determined. When the matching parameters are preset matching parameters and the feature distance is less than the preset distance threshold, the matching parameters are fused with the feature distance to obtain the text loss information of the text sample pair.

[0108] Among them, the matching parameter is used to indicate the parameter of the semantic matching relationship between the text samples in the text sample pair. For example, when the semantic matching relationship of the text samples in the text sample pair is a match, the corresponding matching parameter can be 1. When the semantic matching relationship of the text samples in the text sample pair is a mismatch, the corresponding matching parameter can be 0. Of course, the matching parameter can also be other parameter values. It should be noted that the semantic matching relationship is different, and the corresponding matching parameters are also different. Taking the matching parameter as 0 or 1 as an example, when the matching parameter is 0 and the feature distance is greater than the preset distance threshold, the text loss information of the text sample pair can be 0. When the matching parameter is 1 and the feature distance is less than the preset distance threshold, the text sample pair has text loss information. Therefore, the condition for the text sample pair to have text loss information is that the matching parameter is the preset matching parameter and the feature distance is less than the preset distance threshold.

[0109] Under the condition that the matching parameter is a preset matching parameter and the feature distance is less than a preset distance threshold, there are multiple ways to fuse the matching parameter with the feature distance. For example, the distance difference between the feature distance and the preset distance threshold is calculated, the parameter difference between the matching parameter and the preset parameter threshold is calculated, and the distance difference is fused with the parameter difference. The fused difference, matching parameter and feature distance are fused to obtain the text loss information of the text sample pair. Specifically, it can be shown as formula (3):

[0110]

[0111] Among them, Loss match is the text loss information of the text sample pair, N is the number of text samples in the text sample pair, y is the matching parameter, d is the feature distance, and margin is a hyperparameter used to indicate the preset distance threshold. The loss function for calculating the text loss information is the contrast loss function, which focuses on learning the parameters of relevant samples and ignores irrelevant samples larger than margin. It has a good effect on the under-recall problem and facilitates the online retrieval module to calculate similarity using cosine distance.

[0112] C3. Based on the keyword loss information and the text loss information, the preset text processing model is converged to obtain the trained text processing model.

[0113] Among them, the preset text processing model includes a feature extraction network and a keyword recognition network.

[0114] There are many ways to converge the preset text processing model, which can be as follows:

[0115] For example, obtain the loss weight, and based on the loss weight, weight the keyword loss information and the text loss information respectively, fuse the weighted keyword loss information and the weighted text loss information to obtain the target loss information, use the weighted keyword loss information to converge the keyword recognition network to obtain the trained keyword recognition network, use the target loss information to converge the feature extraction network to obtain the trained feature extraction network, and use the trained keyword recognition network and the trained feature extraction network as the trained text processing models.

[0116] There are many ways to fuse the weighted keyword loss information and the weighted text loss information. For example, the weighted keyword loss information and the weighted text loss information can be directly added together to obtain the target loss information corresponding to the preset text processing model. Specifically, it can be shown in formula (4):

[0117] Loss total =α*Loss match +β*Loss keyword (4)

[0118] Among them, Loss total is the target loss information, α and β are the loss weights of text loss information and keyword loss information respectively, Loss match is the text loss information, Loss keyword Loss information for keywords.

[0119] After obtaining the target loss information, the network parameters of the preset text processing model can be updated through back propagation, and then iterated multiple times until convergence. During the convergence process, it should be noted that for the keyword recognition network in the preset text processing model, only the weighted keyword loss information is needed to update the network parameters of the keyword recognition network, and the weighted text loss information is not needed to update the network parameters of the keyword recognition network. For the feature extraction network other than the keyword recognition network in the preset text processing model, the target loss information can be used to update the network parameters of the feature extraction network. The keyword recognition network and the feature extraction network are trained iteratively multiple times until convergence, thereby obtaining the trained text processing model. It should be noted that the keyword recognition task is an auxiliary task in order to enhance the noise reduction capability of the semantic matching model, and the weight of the loss value is relatively low.

[0120] C4. Use the trained text processing model to retrieve the target text.

[0121] The target text is the text found through the query text and has a semantic matching relationship with the query text.

[0122] There are many ways to use the trained text processing model to retrieve the target text, which can be as follows:

[0123] For example, a candidate text set can be obtained, and a trained text processing model can be used to extract features from each candidate text in the candidate text set to obtain a candidate text feature set. Index information corresponding to the candidate text feature set can be constructed based on the candidate text features in the candidate text feature set. When a query text is received, at least one candidate text can be screened out from the candidate text set based on the index information and the query text.

[0124] The candidate text set can be processed mainly by offline processing, by calculating the text features of all candidate texts in advance offline through the trained text processing model, thereby obtaining a candidate text feature set corresponding to the candidate text set, and using an index building tool to build an index library of the candidate text feature set, which is provided to an online retrieval system for retrieval. There can be many types of index building tools, for example, index tools such as Faiss or nmslib can be included.

[0125] After constructing the index information corresponding to the candidate text feature set, online retrieval can be performed. The process of online retrieval mainly includes deploying the trained text processing model to the online module. When the user enters the query text, at least one candidate text can be screened out as the target text in the candidate text set based on the index information and the query text. There are many ways to screen. For example, the trained text processing model can be used to extract features of the query text to obtain the query text features of the query text. Based on the index information, at least one candidate text feature corresponding to the query text feature is retrieved from the candidate text feature set to obtain the target candidate text feature. The candidate text corresponding to the target candidate text feature is screened out from the candidate text set to obtain the target text corresponding to the query text.

[0126] Among them, there can be multiple ways to retrieve at least one candidate text feature corresponding to the query text feature in the candidate text feature set based on the index information. For example, the feature similarity between the query text feature and the candidate text feature can be calculated through the index information, and then, based on the feature similarity, the Top K candidate text features with the highest similarity can be retrieved from the candidate text feature set to obtain the target candidate text feature.

[0127] It should be noted that after obtaining the trained text processing model, it is necessary to predict the candidate text features of the candidate text offline and construct the index information corresponding to the candidate text features. Then, the candidate text corresponding to the query text is retrieved online to obtain at least one target text corresponding to the query text. The retrieved target text and its related features are returned to the client through the downstream module for display. The process from the user performing text retrieval on the client to returning the retrieved target text can be as follows: Figure 3 As shown. The user can input the text information of the business or service to be searched through the search control of the application platform, and the client sends the query text (query) input by the user to the server. The server retrieves at least one target text (doc) related to the query from the candidate text set and returns the retrieved doc information to the client, which displays the returned doc information.

[0128] Among them, the entire core framework of text processing can be as follows Figure 4 As shown in the figure, it is mainly divided into three stages: multi-task learning stage, offline vector library generation stage and online vector retrieval stage.

[0129] The multi-task learning stage is mainly used to train the preset text processing model. In the process of training the preset text processing model, the keyword recognition task and the semantic matching task are used to train the preset text processing model. The two tasks will share important modules for parallel training, forming a multi-task learning framework, such as Figure 5 shown.

[0130] In the process of keyword recognition task, the text sample is divided into tokens, and the text features of each token after segmentation are input through the BERT model. The text features here can be text vectors (t1, t2, t3). Then, the category probability of each token belonging to each keyword category is calculated through FC-Softmax (keyword recognition network), and then the keyword loss information is calculated through CE loss (keyword loss function). The specific example is as follows: Figure 6 shown.

[0131] Among them, text vector refers to a piece of text of indefinite length converted into a fixed-length numerical vector in some way. Vectors can be divided into two forms: one is a high-dimensional sparse vector, which usually takes the length of the vocabulary as the length of the vector, and each dimension represents a word. Only the dimension corresponding to the text word has a non-zero value, and most dimensions are zero; the other is a low-dimensional dense vector, which can input the text into models such as neural networks, and output vector representation through training. Each dimension of the vector is basically a non-zero value, and there is no clear physical meaning, but the effect is usually better than a high-dimensional sparse vector.

[0132] In the process of semantic matching task, several queries are randomly selected from the text sample set, and semantically similar docs are retrieved as positive examples (labeled as 1) through the retrieval system. At the same time, docs in the offline library are randomly selected as negative examples (labeled as 0), and the query, doc and label are composed of bigram data as text sample pairs. The so-called bigram data is a bigram data including two texts and a label (represented by 0 or 1) in the text matching scenario. Assume that the two texts are A and B. If the two match, the bigram data is (A, B, 1); if they do not match, the bigram data is (A, B, 0). The dual-tower model is trained by text sample pairs. The dual-tower model here can be understood as using a keyword recognition network and a feature extraction network to generate text vectors for query and doc respectively, so as to obtain the text features of each text sample in the text sample pair. Then, the text loss information of the text vector of the query and the text vector of the doc is calculated by contrasting the loss function. Specifically, it can be as follows: Figure 7 shown.

[0133] In the multi-task learning training process, two tasks are forward propagated simultaneously (shared BERT model and FC-Softmax network), keyword loss information and text loss information are calculated, and then the weighted sum is used to obtain the overall loss information. The BERT model is converged based on the overall loss information, and the FC-Softmax network is converged based on the weighted keyword loss information to obtain the trained text processing model. In addition, other auxiliary tasks can be added during the training of the preset text processing model to improve the accuracy and generalization ability of the preset text processing model in the semantic matching scenario.

[0134] Among them, it should be noted that this solution uses the BERT model to segment text samples, and the segmentation granularity is token, so there is no need to perform Chinese word segmentation, avoiding the accuracy error caused by the word segmentation tool. In addition, the BERT model can obtain insufficient text order information, the feature extraction energy is better, and the accuracy of the extracted text features is more accurate. In addition, a three-category keyword recognition task is designed in the keyword recognition task. By integrating the category probabilities of the predicted keyword categories, the text weight of each token can be effectively obtained, which significantly improves the keyword recognition ability of the preset text processing model, thereby improving the accuracy of the semantic relevance calculation of query and doc, and can effectively alleviate the problems of under-recall and reverse sorting.

[0135] From the above, it can be seen that after obtaining the text word samples and the text sample pairs, the embodiment of the present application uses a preset text processing model to segment the text samples in the text sample pairs, and extracts features of the target text words and text word samples after segmentation to obtain text word features of the target text words and text word sample features of the text word samples. Then, based on the text word features and the text word sample features, keyword category recognition is performed on the target text words and the text word sample to obtain the first keyword category of the target text words and the second keyword category of the text word samples. Then, according to the first keyword category, the text word features are weighted to obtain each text sample in the text sample pairs. The text features are obtained by extracting the text features of the text and calculating the feature distance between the text features. Then, the preset text processing model is converged based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship to obtain the trained text processing model, and the trained text processing model is used to retrieve the target text. Since the scheme uses a multi-task framework to train the keyword category recognition task and the semantic matching task at the same time, and by identifying the first keyword category, the text word features are weighted, which explicitly enhances the text processing model's ability to recognize word weights in the semantic matching task, thereby effectively reducing information noise. Therefore, the accuracy of text processing can be improved.

[0136] According to the method described in the above embodiment, the following is further described in detail with examples.

[0137] In this embodiment, the text processing device is specifically integrated in an electronic device, the electronic device is a server, and the keyword categories include three categories: non-keywords (represented by 0), general keywords (represented by 1) and important keywords (represented by 2).

[0138] like Figure 8 As shown, a text processing method, the specific process is as follows:

[0139] 201. The server obtains text word samples and text sample pairs.

[0140] For example, the server can obtain a text sample set and filter out at least one text sample from the text sample set. The retrieval system can filter out text samples that semantically match the text sample from the text sample set as semantic text samples. At this time, the semantic relationship between the filtered semantic text sample and the text sample can be a semantic match, and the text sample pair at this time can be a positive text sample pair. Text samples in the offline library can also be randomly extracted as semantic text samples corresponding to the text samples. At this time, the semantic relationship between the extracted semantic text sample and the text sample can be a semantic mismatch, and the text sample pair at this time can be a negative text sample pair.

[0141] The server can use the tokenization word segmentation method in the Bert network in the preset text processing model to divide each text sample into a single token, directly identify the keyword category of the token, and then mark the keyword category on the token according to the recognition result. Alternatively, the segmented token can be sent to the marking server, the keyword category of the token returned by the marking server can be received, and the corresponding keyword category can be marked on the token to obtain a text word sample.

[0142] The server can determine the semantic matching relationship between the text sample and the semantic text sample according to the semantic relationship between the text sample and the semantic text sample, form a text pair with the text sample and the corresponding semantic text sample, and mark the semantic matching relationship in the text pair, thereby obtaining a text sample pair. The text sample pair may include a positive text sample pair and a negative text sample pair. The text sample in the positive text sample pair matches the semantic text sample, and the text sample in the negative text sample pair does not match the semantic text sample. In addition, when the text sample is a query text sample, the corresponding semantic text sample can be a target text sample.

[0143] 202. The server uses a preset text processing model to segment the text samples in the text sample pair, and extracts features from the segmented target text words and text word samples to obtain text features of the target text words and text word sample features of the text word samples.

[0144] For example, the server can use the tokenization method in the Bert network in the preset text processing model to segment each text sample into a single token, thereby obtaining the target text word after each text sample in the text sample pair is segmented, or directly perform character segmentation on each text sample in the text sample pair, and segment it into Chinese characters, English words or roots, thereby obtaining the target text word after each text sample in the text sample pair is segmented.

[0145] The server can use the Bert network in the preset text processing model or the XLNet, ELECTRA and other models to extract features of the target text words and text word samples respectively, so as to obtain the vector representation (q1, q2, q3) of each target text word and the vector representation (t1, t2, t3) of each text word sample, and use the vector representation of the target text word as the text word feature of the target text word, and use the vector representation of the text word sample as the text word sample feature of the text word sample.

[0146] 203. The server performs keyword category recognition on the target text word and the text word sample based on the text word feature and the text word sample feature to obtain a first keyword category of the target text word and a second keyword category of the text word sample.

[0147] For example, the server can use a fully connected neural network to normalize the text word features and text word sample features respectively. The Softmax function is used to calculate the category probability of each target text word belonging to each keyword category. Will As the first keyword category of the target text word. The category probability of each text word sample belonging to each keyword category is calculated by the Softmax function ( and ),Will and The second keyword category as a sample of text words.

[0148] Among them, it can be found that the keyword recognition network in the preset text processing model is used to identify the keyword category of the target text words and text word samples. The network structure of the keyword recognition network can be an FC-Softmax network, and in the process of identifying the keyword category of text words and text word samples, the FC-Softmax network and the network parameters of the network are shared.

[0149] 204. The server determines the text weight of the text word feature according to the first keyword category.

[0150] For example, the server identifies the category probability of each keyword category in the first keyword category In the first category probability, the category probabilities of general keywords and important keywords are screened out to obtain the basic category probability. Obtain the fusion parameters of each basic category probability, fuse the fusion parameters with the corresponding basic category probability, then add the fused basic category probabilities to obtain the target basic category probability, then calculate the mean of the target basic category probabilities, and thus the text weight of the text word feature can be specifically shown as formula (1).

[0151] 205. The server weights the text word features based on the text weights, and fuses the weighted text word features to obtain the text features of each text sample in the text sample pair.

[0152] For example, the server weights the text word features based on the text weights to obtain weighted text word features, and fuses the weighted text word features to obtain fused text features (vec q =∑ i w qi *q i ). Filter out the text features belonging to the query text sample from the fused text features, so as to obtain the query text features corresponding to the query text sample. Filter out the text features of each field belonging to the target text sample from the fused text features, so as to obtain at least one field text feature. Perform correlation feature extraction on the field text features to obtain the correlation features of the field text features. Based on the correlation features, determine the correlation weights of the field text features. The correlation weights are used to indicate the correlation relationship between the field text features. According to the correlation weights, the field text features are weighted, and the weighted field text features are fused to obtain the target field text features. The target field text features here can be understood as the text features corresponding to the target text sample. The target field text features and the query text features are used as the text features of each text sample in the text sample pair.

[0153] 206. The server calculates feature distances between text features.

[0154] For example, the server may directly calculate the cosine distance between the text features to obtain the feature distance between the text features, or may also calculate the Euclidean distance between the text features to obtain the feature distance between the text features.

[0155] 207. The server converges the preset text processing model based on the second keyword category, the annotated keyword category, the feature distance, and the annotated semantic matching relationship to obtain a trained text processing model.

[0156] For example, the server may determine the keyword loss information of the text word sample based on the second keyword category and the annotated keyword category, determine the text loss information of the text sample pair according to the annotated semantic matching relationship and the feature distance, and converge the preset text processing model based on the keyword loss information and the text loss information to obtain the trained text processing model, which may be specifically as follows:

[0157] (1) The server determines keyword loss information of the text word sample based on the second keyword category and the annotated keyword category.

[0158] For example, the server identifies the category probability of each keyword category in the second keyword category and obtains the second category probability ( and ), when the annotated keyword category of the i-th text word sample is category 1, then and Filter out is the target category probability. According to the annotated keyword category, the keyword category parameter of the text word sample is determined. When the i-th text word sample belongs to the c-th keyword, the keyword parameter can be 1, otherwise the keyword parameter can be 0. After preprocessing the target category probability, the preprocessed target category probability is multiplied by the keyword parameter to obtain the basic keyword loss information of the i-th text word sample. Then, the basic keyword loss information of the text word sample is accumulated, and the mean of the accumulated keyword loss information is calculated to obtain the keyword loss information of the text word sample, which can be specifically shown in formula (2).

[0159] (2) The server determines the text loss information of the text sample pair based on the annotated semantic matching relationship and feature distance.

[0160] For example, when the semantic matching relationship of the text samples in the text sample pair is a match, the server can determine that the corresponding matching parameter can be 1, and when the semantic matching relationship of the text samples in the text sample pair is a mismatch, the server can determine that the corresponding matching parameter can be 0. When the matching parameter is 1 and the feature distance is less than the preset distance threshold, the distance difference between the feature distance and the preset distance threshold is calculated, the parameter difference between the matching parameter and the preset parameter threshold is calculated, and the distance difference and the parameter difference are fused. The fused difference, matching parameter and feature distance are fused to obtain the text loss information of the text sample pair, which can be specifically shown in formula (3).

[0161] (3) The server converges the preset text processing model based on the keyword loss information and the text loss information to obtain a trained text processing model.

[0162] For example, the server obtains the loss weight, and based on the loss weight, weights the keyword loss information and the text loss information respectively, and directly adds the weighted keyword loss information and the weighted text loss information, so as to obtain the target loss information corresponding to the preset text processing model, which can be specifically shown in formula (4). The weighted keyword loss information is used to converge the keyword recognition network to obtain the trained keyword recognition network, and the target loss information is used to converge the feature extraction network to obtain the trained feature extraction network, and the trained keyword recognition network and the trained feature extraction network are used as the trained text processing model.

[0163] 208. The server uses the trained text processing model to retrieve the target text.

[0164] For example, the server can obtain a candidate text set, calculate the text features of all candidate texts offline in advance through the trained text processing model, so as to obtain a candidate text feature set corresponding to the candidate text set, and use indexing tools such as Faiss or nmslib to build an index library of the candidate text feature set, which is provided to the online retrieval system for retrieval. The trained text processing model is deployed to the online module. When the user enters a query text, the trained text processing model is used to extract features of the query text to obtain the query text features of the query text. The feature similarity between the query text features and the candidate text features is calculated through the index information. Then, based on the feature similarity, the top K candidate text features with the highest similarity are retrieved from the candidate text feature set to obtain the target candidate text features. The candidate text corresponding to the target candidate text features is screened out from the candidate text set to obtain the target text corresponding to the query text. The retrieved target text and its related features are returned to the client through the downstream module for display.

[0165] As can be seen from the above, after obtaining the text word samples and the text sample pairs, the server of this embodiment uses the preset text processing model to segment the text samples in the text sample pairs, and extracts features of the target text words and text word samples after segmentation to obtain text word features of the target text words and text word sample features of the text word samples. Then, based on the text word features and text word sample features, the target text words and text word samples are identified by keyword categories to obtain the first keyword category of the target text words and the second keyword category of the text word samples. Then, according to the first keyword category, the text word features are weighted to obtain the keyword category of each text word in the text sample pairs. The text features of the text are identified and the feature distance between the text features is calculated. Then, based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship, the preset text processing model is converged to obtain the trained text processing model, and the trained text processing model is used to retrieve the target text. Since the scheme uses a multi-task framework to train the keyword category recognition task and the semantic matching task at the same time, and by identifying the first keyword category, the text word features are weighted, which explicitly enhances the text processing model's ability to recognize word weights in the semantic matching task, thereby effectively reducing information noise. Therefore, the accuracy of text processing can be improved.

[0166] In order to better implement the above method, an embodiment of the present invention further provides a text processing device, which can be integrated into an electronic device, such as a server or a terminal. The terminal may include a tablet computer, a laptop computer and / or a personal computer.

[0167] For example, Fig. 9 As shown, the text processing device may include an acquisition unit 301, a word segmentation unit 302, a recognition unit 303, a weighting unit 304 and a retrieval unit 305, as follows:

[0168] (1) Acquisition unit 301;

[0169] The acquisition unit 301 is used to acquire text word samples and text sample pairs. The text word samples include text words marked with keyword categories, and the text sample pairs include text pairs marked with semantic matching relationships.

[0170] For example, the acquisition unit 301 can be specifically used to acquire a text sample set, and screen out at least one text sample and a semantic text sample corresponding to the text sample from the text sample set, use a preset text processing model to segment the text sample, and mark keyword categories in the segmented text words to obtain text word samples, and based on the semantic relationship between the text sample and the semantic text sample, mark the semantic matching relationship in the text pair consisting of the text sample and the semantic text sample to obtain a text sample pair.

[0171] (2) word segmentation unit 302;

[0172] The word segmentation unit 302 is used to segment the text samples in the text sample pair using a preset text processing model, and perform feature extraction on the target text words and text word samples after segmentation to obtain text word features of the target text words and text word sample features of the text word samples.

[0173] For example, the word segmentation unit 302 can be used to segment each text sample into Chinese characters, English words or roots, etc., so as to obtain the target text word after the word segmentation of each text sample in the text sample pair. The Bert network or XLNet, ELECTRA and other models in the preset text processing model are used to extract features of the target text words and text word samples, so as to obtain the vector representation (q1, q2, q3) of each target text word and the vector representation (t1, t2, t3) of each text word sample, and the vector representation of the target text word is used as the text word feature of the target text word, and the vector representation of the text word sample is used as the text word sample feature of the text word sample.

[0174] (3) identification unit 303;

[0175] The recognition unit 303 is used to perform keyword category recognition on the target text word and the text word sample based on the text word feature and the text word sample feature to obtain a first keyword category of the target text word and a second keyword category of the text word sample.

[0176] For example, the recognition unit 303 can be specifically used to use the keyword recognition network in the preset text processing model to normalize the text word features and the text word sample features respectively, and map the category probability of the target text word belonging to each keyword category based on the normalized text word features to obtain the first keyword category of the target text word, and map the category probability of the text word sample belonging to each keyword category based on the normalized text word sample features to obtain the second keyword category of the text word sample.

[0177] (4) weighting unit 304;

[0178] The weighting unit 304 is used to weight the text word features according to the first keyword category to obtain the text features of each text sample in the text sample pair and calculate the feature distance between the text features.

[0179] For example, the weighting unit 304 can be specifically used to determine the text weight of the text word feature according to the first keyword category, weight the text word feature based on the text weight, and fuse the weighted text word features to obtain the text feature of each text sample in the text sample pair, and calculate the feature distance between the text features.

[0180] (5) Retrieval unit 305;

[0181] The retrieval unit 305 is used to converge the preset text processing model based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship to obtain a trained text processing model, and use the trained text processing model to retrieve the target text.

[0182] For example, the retrieval unit 305 can be specifically used to determine the keyword loss information of the text word sample based on the second keyword category and the annotated keyword category, determine the text loss information of the text sample pair based on the annotated semantic matching relationship and the feature distance, and converge the preset text processing model based on the keyword loss information and the text loss information to obtain the trained text processing model. A candidate text set is obtained, and the trained text processing model is used to extract features from each candidate text in the candidate text set to a candidate text feature set, and index information corresponding to the candidate text feature set is constructed based on the candidate text features in the candidate text feature set. When a query text is received, at least one candidate text is screened out from the candidate text set based on the index information and the query text.

[0183] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0184] As can be seen from the above, in this embodiment, after the acquisition unit 301 acquires the text word sample and the text sample pair, the word segmentation unit 302 uses a preset text processing model to segment the text sample in the text sample pair, and extracts features of the target text word and the text word sample after segmentation to obtain the text word features of the target text word and the text word sample features of the text word sample. Then, the recognition unit 303 performs keyword category recognition on the target text word and the text word sample based on the text word features and the text word sample features to obtain the first keyword category of the target text word and the second keyword category of the text word sample. Then, the weighting unit 304 weights the text word features according to the first keyword category to obtain the text word sample. The text features of each text sample in the sample pair are obtained, and the feature distance between the text features is calculated. Then, the retrieval unit 305 converges the preset text processing model based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship to obtain a trained text processing model, and uses the trained text processing model to retrieve the target text; since the scheme uses a multi-task framework to simultaneously train the keyword category recognition task and the semantic matching task, and by identifying the first keyword category, the text word features are weighted, which explicitly enhances the text processing model's ability to recognize word weights in the semantic matching task, thereby effectively reducing information noise, and therefore, can improve the accuracy of text processing.

[0185] The embodiment of the present invention further provides an electronic device, such as Fig.10 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0186] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Fig.10 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0187] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0188] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0189] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0190] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0191] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0192] A text word sample and a text sample pair are obtained, wherein the text word sample includes a text word with annotated keyword category, and the text sample pair includes a text pair with annotated semantic matching relationship, a preset text processing model is used to segment the text samples in the text sample pair, and feature extraction is performed on the segmented target text words and text word samples to obtain text word features of the target text words and text word sample features of the text word samples, keyword category recognition is performed on the target text words and text word sample based on the text word features and text word sample features to obtain a first keyword category of the target text word and a second keyword category of the text word sample, the text word features are weighted according to the first keyword category to obtain text features of each text sample in the text sample pair, and feature distances between text features are calculated, the preset text processing model is converged based on the second keyword category, the annotated keyword category, the feature distances and the annotated semantic matching relationship to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text.

[0193] For example, a text sample set is obtained, and at least one text sample and a semantic text sample corresponding to the text sample are screened out from the text sample set, a preset text processing model is used to segment the text sample, and a keyword category is marked in the segmented text word to obtain a text word sample, and a semantic matching relationship is marked in a text pair composed of the text sample and the semantic text sample according to the semantic relationship between the text sample and the semantic text sample to obtain a text sample pair. Each text sample is segmented into Chinese characters, English words or roots, etc., so as to obtain a target text word after segmentation of each text sample in the text sample pair. The Bert network or XLNet, ELECTRA and other models in the preset text processing model are used to extract features from the target text word and the text word sample, respectively, so as to obtain a vector representation (q1, q2, q3) of each target text word and a vector representation (t1, t2, t3) of each text word sample, and the vector representation of the target text word is used as the text word feature of the target text word, and the vector representation of the text word sample is used as the text word sample feature of the text word sample. The keyword recognition network in the preset text processing model is used to normalize the text word features and the text word sample features respectively. According to the normalized text word features, the category probability of the target text word belonging to each keyword category is mapped to obtain the first keyword category of the target text word. Based on the normalized text word sample features, the category probability of the text word sample belonging to each keyword category is mapped to obtain the second keyword category of the text word sample. According to the first keyword category, the text weight of the text word feature is determined, and based on the text weight, the text word features are weighted, and the weighted text word features are fused to obtain the text features of each text sample in the text sample pair, and the feature distance between the text features is calculated. Based on the second keyword category and the annotated keyword category, the keyword loss information of the text word sample is determined, and the text loss information of the text sample pair is determined according to the annotated semantic matching relationship and the feature distance. Based on the keyword loss information and the text loss information, the preset text processing model is converged to obtain the trained text processing model. A candidate text set is obtained, and a trained text processing model is used to extract features from each candidate text in the candidate text set to obtain a candidate text feature set. According to the candidate text features in the candidate text feature set, index information corresponding to the candidate text feature set is constructed. When a query text is received, at least one candidate text is screened out from the candidate text set according to the index information and the query text.

[0194] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0195] As can be seen from the above, after obtaining the text word samples and the text sample pairs, the embodiments of the present invention use a preset text processing model to segment the text samples in the text sample pairs, and perform feature extraction on the target text words and text word samples after segmentation to obtain text word features of the target text words and text word sample features of the text word samples. Then, based on the text word features and the text word sample features, keyword category recognition is performed on the target text words and the text word sample to obtain the first keyword category of the target text words and the second keyword category of the text word samples. Then, according to the first keyword category, the text word features are weighted to obtain each text sample in the text sample pairs. The text features are obtained by extracting the text features of the text and calculating the feature distance between the text features. Then, the preset text processing model is converged based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship to obtain the trained text processing model, and the trained text processing model is used to retrieve the target text. Since the scheme uses a multi-task framework to train the keyword category recognition task and the semantic matching task at the same time, and by identifying the first keyword category, the text word features are weighted, which explicitly enhances the text processing model's ability to recognize word weights in the semantic matching task, thereby effectively reducing information noise. Therefore, the accuracy of text processing can be improved.

[0196] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0197] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any text processing method provided in the embodiment of the present invention. For example, the instructions can execute the following steps:

[0198] A text word sample and a text sample pair are obtained, wherein the text word sample includes a text word with annotated keyword category, and the text sample pair includes a text pair with annotated semantic matching relationship, a preset text processing model is used to segment the text samples in the text sample pair, and feature extraction is performed on the segmented target text words and text word samples to obtain text word features of the target text words and text word sample features of the text word samples, keyword category recognition is performed on the target text words and text word sample based on the text word features and text word sample features to obtain a first keyword category of the target text word and a second keyword category of the text word sample, the text word features are weighted according to the first keyword category to obtain text features of each text sample in the text sample pair, and feature distances between text features are calculated, the preset text processing model is converged based on the second keyword category, the annotated keyword category, the feature distances and the annotated semantic matching relationship to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text.

[0199] For example, a text sample set is obtained, and at least one text sample and a semantic text sample corresponding to the text sample are screened out from the text sample set, a preset text processing model is used to segment the text sample, and a keyword category is marked in the segmented text word to obtain a text word sample, and a semantic matching relationship is marked in a text pair composed of the text sample and the semantic text sample according to the semantic relationship between the text sample and the semantic text sample to obtain a text sample pair. Each text sample is segmented into Chinese characters, English words or roots, etc., so as to obtain a target text word after segmentation of each text sample in the text sample pair. The Bert network or XLNet, ELECTRA and other models in the preset text processing model are used to extract features from the target text word and the text word sample, respectively, so as to obtain a vector representation (q1, q2, q3) of each target text word and a vector representation (t1, t2, t3) of each text word sample, and the vector representation of the target text word is used as the text word feature of the target text word, and the vector representation of the text word sample is used as the text word sample feature of the text word sample. The keyword recognition network in the preset text processing model is used to normalize the text word features and the text word sample features respectively. According to the normalized text word features, the category probability of the target text word belonging to each keyword category is mapped to obtain the first keyword category of the target text word. Based on the normalized text word sample features, the category probability of the text word sample belonging to each keyword category is mapped to obtain the second keyword category of the text word sample. According to the first keyword category, the text weight of the text word feature is determined, and based on the text weight, the text word features are weighted, and the weighted text word features are fused to obtain the text features of each text sample in the text sample pair, and the feature distance between the text features is calculated. Based on the second keyword category and the annotated keyword category, the keyword loss information of the text word sample is determined, and the text loss information of the text sample pair is determined according to the annotated semantic matching relationship and the feature distance. Based on the keyword loss information and the text loss information, the preset text processing model is converged to obtain the trained text processing model. A candidate text set is obtained, and a trained text processing model is used to extract features from each candidate text in the candidate text set to obtain a candidate text feature set. According to the candidate text features in the candidate text feature set, index information corresponding to the candidate text feature set is constructed. When a query text is received, at least one candidate text is screened out from the candidate text set according to the index information and the query text.

[0200] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0201] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0202] Since the instructions stored in the computer-readable storage medium can execute the steps in any text processing method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any text processing method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0203] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations of the above-mentioned text processing or text retrieval aspects.

[0204] The above is a detailed introduction to a text processing method, device, electronic device and computer-readable storage medium provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A text processing method, characterized in that: include: Acquire text word samples and text sample pairs, wherein the text word samples include text words marked with keyword categories, and the text sample pairs include text pairs marked with semantic matching relationships; Using a preset text processing model to segment the text samples in the text sample pair, and extracting features from the segmented target text words and text word samples to obtain text word features of the target text words and text word sample features of the text word samples; Based on the text word features and the text word sample features, performing keyword category recognition on the target text word and the text word sample to obtain a first keyword category of the target text word and a second keyword category of the text word sample; According to the first keyword category, weighting the text word features to obtain text features of each text sample in the text sample pair, and calculating feature distances between the text features; Based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship, the preset text processing model is converged to obtain a trained text processing model, and the trained text processing model is used to retrieve the target text.

2. The text processing method according to claim 1, characterized in that: The step of weighting the text word features according to the first keyword category to obtain the text features of each text sample in the text sample pair includes: Determining a text weight of the text word feature according to the first keyword category; Based on the text weights, the text word features are weighted, and the weighted text word features are fused to obtain the text features of each text sample in the text sample pair.

3. The text processing method according to claim 2, characterized in that: Determining the text weight of the text word feature according to the first keyword category includes: Identifying the category probability of each keyword category in the first keyword category to obtain a first category probability; Filtering out at least one category probability of a preset key keyword category from the first category probabilities to obtain a basic category probability; The basic category probabilities are fused to obtain the text weights of the text word features.

4. The text processing method according to claim 2, characterized in that: The text sample pair includes a query text sample and a target text sample. The weighted text word features are fused to obtain text features of each text sample in the text sample pair, including: Fusing the weighted text word features to obtain fused text features; Extracting the query text feature corresponding to the query text sample and at least one field text feature corresponding to the target text sample from the fused text features; The field text features are fused to obtain target field text features, and the target field text features and the query text features are used as text features of each text sample in the text sample pair.

5. The text processing method according to claim 4, characterized in that: The step of fusing the field text features to obtain target field text features includes: Extracting associated features from the field text features to obtain associated features of the field text features; Based on the association features, determining association weights of the field text features, wherein the association weights are used to indicate association relationships between field text features; The field text features are weighted according to the association weights, and the weighted field text features are fused to obtain target field text features.

6. The text processing method according to any one of claims 1 to 5, characterized in that: The method of converging the preset text processing model based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship to obtain a trained text processing model includes: determining keyword loss information of the text word sample based on the second keyword category and the annotated keyword category; Determining text loss information of the text sample pair according to the annotated semantic matching relationship and feature distance; Based on the keyword loss information and the text loss information, the preset text processing model is converged to obtain a trained text processing model.

7. The text processing method according to claim 6, characterized in that: The determining the keyword loss information of the text word sample based on the second keyword category and the marked keyword category includes: Identifying the category probability of each keyword category in the second keyword category to obtain a second category probability; Filtering the category probability corresponding to the annotated keyword category from the second category probabilities to obtain the target category probability; The target category probability is fused with the annotated keyword category, and the mean of the fused keyword category is calculated to obtain the keyword loss information of the text word sample.

8. The text processing method according to claim 6, characterized in that: The determining of the text loss information of the text sample pair according to the annotated semantic matching relationship and the feature distance includes: Determining matching parameters of the text sample pair according to the annotated semantic matching relationship; When the matching parameter is a preset matching parameter and the feature distance is less than a preset distance threshold, the matching parameter is fused with the feature distance to obtain text loss information of the text sample pair.

9. The text processing method according to claim 8, characterized in that: The step of fusing the matching parameter with the feature distance to obtain text loss information of the text sample pair includes: Calculating the distance difference between the characteristic distance and the preset distance threshold; Calculating a parameter difference between the matching parameter and a preset parameter threshold, and fusing the distance difference with the parameter difference; The fused difference, matching parameters and feature distance are fused to obtain text loss information of the text sample.

10. The text processing method according to claim 6, characterized in that: The preset text processing network includes a feature extraction network and a keyword recognition network. Based on the keyword loss information and the text loss information, the preset text processing model is converged to obtain a trained text processing model, including: Obtaining loss weights, and weighting the keyword loss information and the text loss information respectively based on the loss weights; The weighted keyword loss information and the weighted text loss information are merged to obtain the target loss information; Using the weighted keyword loss information to converge the keyword recognition network to obtain a trained keyword recognition network; The target loss information is used to converge the feature extraction network to obtain a trained feature extraction network, and the trained keyword recognition network and the trained feature extraction network are used as a trained text processing model.

11. The text processing method according to claim 10, characterized in that: The step of performing keyword category recognition on the target text word and the text word sample based on the text word feature and the text word sample feature to obtain a first keyword category of the target text word and a second keyword category of the text word sample includes: The keyword recognition network is used to normalize the text word features and the text word sample features respectively; According to the normalized text word features, mapping the category probability of the target text word belonging to each keyword category, and obtaining the first keyword category of the target text word; Based on the normalized text word sample features, the category probability of the text word sample belonging to each keyword category is mapped out to obtain the second keyword category of the text word sample.

12. The text processing method according to any one of claims 1 to 5, characterized in that: The step of using the trained text processing model to retrieve the target text includes: Acquire a candidate text set, and use the trained text processing model to perform feature extraction on each candidate text in the candidate text set to obtain a candidate text feature set; Constructing index information corresponding to the candidate text feature set according to the candidate text features in the candidate text feature set; When a query text is received, at least one candidate text is selected from the candidate text set as a target text according to the index information and the query text.

13. The text processing method according to claim 12, characterized in that: The step of selecting at least one candidate text from the candidate text set as the target text according to the index information and the query text includes: Using the trained text processing model to perform feature extraction on the query text to obtain query text features of the query text; Based on the index information, at least one candidate text feature corresponding to the query text feature is retrieved from the candidate text feature set to obtain a target candidate text feature; The candidate texts corresponding to the target candidate text features are screened out from the candidate text set to obtain the target text corresponding to the query text.

14. The text processing method according to any one of claims 1 to 5, characterized in that: The obtaining of text word samples and text sample pairs includes: Acquire a text sample set, and select at least one text sample and a semantic text sample corresponding to the text sample from the text sample set, wherein the semantic text sample is a text sample having a semantic relationship with the text sample; Using the preset text processing model to segment the text sample, and marking the keyword category in the segmented text words to obtain a text word sample; According to the semantic relationship between the text sample and the semantic text sample, a semantic matching relationship is marked in a text pair consisting of the text sample and the semantic text sample to obtain a text sample pair.

15. A text processing device, characterized in that: include: An acquisition unit, configured to acquire text word samples and text sample pairs, wherein the text word samples include text words annotated with keyword categories, and the text sample pairs include text pairs annotated with semantic matching relationships; A word segmentation unit is used to segment the text samples in the text sample pair using a preset text processing model, and perform feature extraction on the target text words and text word samples after segmentation to obtain text word features of the target text words and text word sample features of the text word samples; an identification unit, configured to perform keyword category identification on the target text word and the text word sample based on the text word feature and the text word sample feature, to obtain a first keyword category of the target text word and a second keyword category of the text word sample; A weighting unit, configured to weight the text word features according to the first keyword category to obtain text features of each text sample in the text sample pair, and calculate a feature distance between the text features; The retrieval unit is used to converge the preset text processing model based on the second keyword category, the annotated keyword category, the feature distance and the annotated semantic matching relationship to obtain a trained text processing model, and use the trained text processing model to retrieve the target text.

16. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to execute the steps in the text processing method according to any one of claims 1 to 14.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the text processing method according to any one of claims 1 to 14 are implemented.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the text processing method according to any one of claims 1 to 14.

Images