Text processing method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses a text processing method and device, computer equipment and a computer readable storage medium. The method comprises the steps of obtaining to-be-processed text information; performing example search on the to-be-processed text information to obtain a target example sample set; and performing entity relationship extraction on the to-be-processed text information based on the target example sample set to obtain entity relationship information of the to-be-processed text information. According to the method, combined extraction of entity relations can be achieved, and the problem that error propagation and accumulation are prone to occurring in a pipeline method is solved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text processing method, apparatus, computer device and computer-readable storage medium. Background Art

[0002] The relation extraction task is to extract entities and entity relationships from unstructured text. The relation extraction task is an important task in natural language processing and knowledge graph construction. The traditional relation extraction task uses a pipeline method to extract entities and their relationships. The pipeline method regards entity recognition and relationship classification as two independent subtasks, ignoring the interaction between the two subtasks, which is prone to error propagation and accumulation problems. Summary of the invention

[0003] The embodiments of the present application provide a text processing method, apparatus, computer device and computer-readable storage medium, which can realize the joint extraction of entity relationships and solve the problem of error propagation and accumulation that is prone to occur in the pipeline method.

[0004] The technical solution adopted by the present invention to solve the problem is as follows:

[0005] On the one hand, the present application provides a text processing method, comprising:

[0006] Get the text information to be processed;

[0007] Perform example search on the text information to be processed to obtain a target example sample set;

[0008] Entity relationship extraction is performed on the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

[0009] In some embodiments of the present application, entity relationship extraction is performed on the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed, including:

[0010] Performing segmentation processing on the target example sample set to obtain a processed target example sample set;

[0011] Fusing the processed target example sample set with the text information to be processed to obtain fused text information;

[0012] Obtain preset task definition information, process the task definition information and fused text information, and obtain entity relationship information of the text information to be processed.

[0013] In some implementation schemes of the present application, the task definition information and the fused text information are processed to obtain entity relationship information of the text information to be processed, including:

[0014] Perform multi-head attention processing on the task definition information and the fused text information to obtain first processed information;

[0015] performing summing and normalization processing on the first processed information to obtain second processed information;

[0016] Performing feature extraction on the second processed information to obtain a first feature;

[0017] Performing summing and normalization processing on the first feature to obtain third processing information;

[0018] Performing linear processing on the third processed information to obtain a second feature;

[0019] Perform feature mapping on the second feature to obtain entity relationship information of the text information to be processed.

[0020] In some embodiments of the present application, an example search is performed on the text information to be processed to obtain a target example sample set, including:

[0021] Obtaining a first example sample set;

[0022] Performing a first example search on the first example sample set based on the text information to be processed to obtain a second example sample set;

[0023] Performing a second example search on the first example sample set based on the text information to be processed to obtain a third example sample set;

[0024] Based on the second example sample set and the third example sample set, a target example sample set is constructed.

[0025] In some embodiments of the present application, the first example sample set includes multiple first example samples, and the first example sample set is searched for the first example based on the text information to be processed to obtain the second example sample set, including:

[0026] Extracting and fusing features of each first example sample to obtain a first fused feature, and extracting and fusing features of the text information to be processed to obtain a second fused feature;

[0027] Performing feature segmentation on the first fusion feature to obtain a plurality of first segmentation features;

[0028] Clustering the multiple first segmentation features to obtain cluster center information of each first example sample;

[0029] Performing feature segmentation on the second fused feature to obtain a plurality of second segmentation features;

[0030] Based on the plurality of second segmentation features and cluster center information, determining distance information between the text information to be processed and each first example sample;

[0031] An example search is performed on the first example sample set based on the distance information to obtain a second example sample set.

[0032] In some embodiments of the present application, a second example search is performed on the first example sample set based on the text information to be processed to obtain a third example sample set, including:

[0033] Encoding the text information to be processed to obtain first encoded information;

[0034] Performing encoding processing on each first example sample to obtain second encoding information;

[0035] Based on the first encoding information and the second encoding information, determining first similarity information between the text information to be processed and each first example sample;

[0036] An example search is performed on the first example sample set based on the first similarity information to obtain a third example sample set.

[0037] In some embodiments of the present application, constructing a target example sample set based on the second example sample set and the third example sample set includes:

[0038] Obtaining a fourth example sample set;

[0039] Based on the second example sample set, the third example sample set and the fourth example sample set, construct a fifth example sample set;

[0040] Performing example sorting on the fifth example sample set to obtain a plurality of sorted example sample sets; each sorted example sample set includes a plurality of sorted example samples;

[0041] Determine a first prediction probability of each sorted example sample under a plurality of preset entity relationship labels;

[0042] Based on the first prediction probability, determining a second prediction probability of each sorted example sample set under each entity relationship label;

[0043] determining a first indicator value based on the first predicted probability, and determining a second indicator value based on the second predicted probability;

[0044] Determining a target ranking result of the fifth example sample set based on the first index value and the second index value;

[0045] Based on the target sorting result, the fifth example sample set is concatenated to obtain a target example sample set.

[0046] In a second aspect, the present application provides a text processing device, comprising:

[0047] An information acquisition unit, used for acquiring text information to be processed;

[0048] An example search unit, used for performing example search on the text information to be processed to obtain a target example sample set;

[0049] The text processing unit is used to extract entity relationships from the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

[0050] In a third aspect, the present application further provides a computer device, the computer device comprising:

[0051] one or more processors;

[0052] Memory; and

[0053] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement any one of the text processing methods in the first aspect.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the text processing methods in the first aspect.

[0055] The beneficial effects of the present invention are as follows: by performing example search on the text information to be processed to obtain a target example sample set, entity relationship extraction is performed on the text information to be processed based on the target example sample set, and the target example sample set is used as a context prompt for entity relationship extraction, thereby realizing joint extraction of entity relationships and solving the problem of error propagation and accumulation that is prone to occur in pipeline methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a flowchart of a text processing method provided by an embodiment of the present invention;

[0058] Figure 2 is a flowchart of a specific embodiment of the text processing method provided by an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of the structure of a pre-training model provided in an embodiment of the present invention;

[0060] Figure 4 is a principle block diagram of a text processing device provided by an embodiment of the present invention;

[0061] Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0063] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third", "fourth", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", "third", "fourth", etc. may explicitly or implicitly include one or more features. In the description of the present application, "multiple" means two or more, and "several" means one or more, unless otherwise clearly and specifically defined.

[0064] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0065] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0066] The inventors have found through research that the traditional relationship extraction task uses a pipeline method to extract entities and their relationships. The pipeline method regards entity recognition and relationship classification as two independent subtasks, ignoring the interaction between the two subtasks, which is prone to error propagation and accumulation problems.

[0067] Based on this, in an embodiment of the present application, text information to be processed is obtained; an example search is performed on the text information to be processed to obtain a target example sample set; entity relationship extraction is performed on the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed. By using the target example sample set as a contextual prompt for entity relationship extraction, joint extraction of entity relationships can be achieved, thereby solving the problem of error propagation and accumulation that is prone to occur in pipeline methods.

[0068] The content of this application is further illustrated below through the description of embodiments in conjunction with the accompanying drawings.

[0069] This embodiment provides a text processing method, such as Figure 1 As shown in , the method includes:

[0070] Step S10: Obtain text information to be processed.

[0071] The text information to be processed is the text data from which entities and entity relationships need to be extracted in the relationship extraction task. The text information to be processed can be text data input by a user through a computer device (e.g., a smart phone), or text data obtained by converting speech to text from collected user conversations through a computer device. For example, the text information to be processed can be "Zhang San sang Rose Boy".

[0072] Step S20: perform example search on the text information to be processed to obtain a target example sample set.

[0073] The target example sample set is a sample set composed of example samples that are searched out and are similar to the text information to be processed. After obtaining the text information to be processed, this embodiment performs an example search on the text information to be processed, and searches out example samples that are similar to the text information to be processed to form a target example sample set, so that in subsequent steps, entity relationships can be extracted from the text information to be processed based on the target example sample set.

[0074] In a specific implementation, Figure 2 As shown, step S20 includes:

[0075] Step S21, obtaining a first example sample set;

[0076] Step S22: performing a first example search on the first example sample set based on the text information to be processed to obtain a second example sample set;

[0077] Step S23: performing a second example search on the first example sample set based on the text information to be processed to obtain a third example sample set;

[0078] Step S24: construct a target example sample set based on the second example sample set and the third example sample set.

[0079] The first example sample set includes multiple first example samples and entity relationship labels corresponding to each first example sample. For example, the first example sample set includes the first example sample "Li Si starred in the sea in my dream" and the entity relationship label "subject: Li Si; predicate: work; object: the sea in my dream" corresponding to the first example sample. The first example sample set can be expressed as: n represents the number of first example samples in the first example sample set, x i represents the first example sample in the first example sample set, y i Represents the entity relationship label corresponding to the i-th first example sample.

[0080] The second example sample set is an example sample set obtained by performing a first example search on the first example sample set based on the text information to be processed, the third example sample set is an example sample set obtained by performing a second example search on the first example sample set based on the text information to be processed, and the target example sample set is a sample set constructed based on the second example sample set and the third example sample set.

[0081] Taking into account that the accuracy of the target example sample set obtained by adopting a single example search method is not enough, when this embodiment performs an example search on the first example sample set based on the text information to be processed, firstly, a first example search is performed on the first example sample set based on the text information to be processed to obtain a second example sample set, and at the same time, a second example search is performed on the first example sample set based on the text information to be processed to obtain a third example sample set, and then based on the second example sample set and the third example sample set, a target example sample set is constructed, and multiple search methods are used to perform an example search on the first example sample set, which can effectively improve the accuracy of the entity relationship extraction results.

[0082] In a specific implementation, the first example sample set includes a plurality of first example samples, and step S22 includes:

[0083] Step S221, extracting and fusing features of each first example sample to obtain a first fused feature, and extracting and fusing features of the text information to be processed to obtain a second fused feature;

[0084] Step S222: performing feature segmentation on the first fused feature to obtain a plurality of first segmented features;

[0085] Step S223: clustering the multiple first segmentation features to obtain cluster center information of each first example sample;

[0086] Step S224, performing feature segmentation on the second fused feature to obtain a plurality of second segmented features;

[0087] Step S225: determining distance information between the text information to be processed and each first example sample based on the plurality of second segmentation features and cluster center information;

[0088] Step S226: Perform an example search on the first example sample set based on the distance information to obtain a second example sample set.

[0089] The first fusion feature is the word vector of each first example sample obtained by extracting and fusing features of each first example sample. The step of extracting and fusing features of each first example sample to obtain the first fusion feature may specifically include: extracting features of each first example sample to obtain the word vector of each first example sample; extracting features of the word vector of each first example sample to obtain the first global feature and the first local feature; fusing the first global feature and the first local feature to obtain the first fusion feature. For example, each first example sample may be input into a RoBERTa model, and feature extraction may be performed on each first example sample through the RoBERTa model to obtain the CLS word vector of each first example sample, and then the CLS word vector of each first example sample may be input into a Bi-LSTM model, and the first global feature may be output through the Bi-LSTM model. At the same time, the CLS word vector of each first example sample may be input into a Deep Pyramid Convolutional Neural Networks for Text Categorization (DPCNN), and the first local feature may be output through the DPCNN.

[0090] The second fused feature is the word vector of the text information to be processed obtained by extracting and fusing the features of the text information to be processed. The step of extracting and fusing the features of the text information to be processed to obtain the second fused feature may specifically include: extracting features of the text information to be processed to obtain the word vector of the text information to be processed; extracting features of the word vector of the text information to be processed to obtain the second global feature and the second local feature; fusing the second global feature and the second local feature to obtain the second fused feature. For example, the text information to be processed may be input into the RoBERTa model, and the RoBERTa model may be used to extract features of the text information to be processed to obtain the CLS word vector of the text information to be processed, and then the CLS word vector of the text information to be processed may be input into the Bi-LSTM model, and the Bi-LSTM model may output the second global feature, and at the same time, the CLS word vector of the text information to be processed may be input into the Deep Pyramid Convolutional Neural Networks for Text Categorization (DPCNN), and the DPCNN may output the second local feature.

[0091] The first segmentation features are features obtained by segmenting the first fusion features. For example, the first fusion feature can be represented as x i =(x i1 ,x i2 ,…,x i128 ), i = 1, 2, ..., n, the first fusion feature is divided into four first segmentation features, and the four first segmentation features can be expressed as (x i1 ,x i2 ,…,x i32 ), (x i33 ,x i34 ,…,x i64 ), (x i65 ,x i66 ,…,x i96 ), (x i97 ,x i98 ,…,x i128 ).

[0092] The cluster center information is the cluster center vector of each first example sample, and the step of clustering multiple first segmentation features to obtain the cluster center information of each first example sample may specifically include: clustering multiple first segmentation features to obtain multiple cluster centers; mapping multiple first segmentation features to multiple cluster centers to obtain the cluster center information of each first example sample. For example, assuming that the number of cluster centers is 256, 4*256 cluster centers can be obtained in the end, and then each first segmentation feature is mapped to its corresponding nearest cluster center ID, and a 128-dimensional first fusion feature becomes a vector composed of 4 cluster center IDs.

[0093] The second segmentation feature is a feature obtained by segmenting the second fused feature. After segmenting the second fused feature to obtain multiple second segmentation features, this embodiment determines the distance information between the text information to be processed and each first example sample based on the multiple second segmentation features and cluster center information. The calculation formula of the distance information is: x test Indicates the text information to be processed, x train represents the first example sample, u j (x) represents the jth second segmentation feature, u j (y) represents the jth first segmentation feature, q j (u j (t)) represents the first segmentation feature u j (y) The obtained cluster center information, j represents the segmentation score.

[0094] The second example sample set is an example sample set consisting of multiple first example samples in the first example sample set that are closest to the text information to be processed. After determining the distance information between the text information to be processed and each first example sample, this embodiment can filter out multiple first example samples that are closest to the text information to be processed from the first example sample set, thereby obtaining the second example sample set.

[0095] In a specific implementation, step S23 includes:

[0096] Step S231, encoding the text information to be processed to obtain first encoded information;

[0097] Step S232: performing encoding processing on each first example sample to obtain second encoding information;

[0098] Step S233: determining first similarity information between the text information to be processed and each first example sample based on the first encoding information and the second encoding information;

[0099] Step S234: perform an example search on the first example sample set based on the first similarity information to obtain a third example sample set.

[0100] The first coded information is the coded information obtained by encoding the text information to be processed. The first coded information can be expressed as E X (x), x represents the text information to be processed; the second encoding information is the encoding information obtained by encoding each first example sample in the first example sample set, and the second encoding information can be expressed as E P (x i ), x i represents the i-th first example sample in the first example sample set.

[0101] The first similarity information is used to characterize the degree of similarity between the text information to be processed and each first example sample, and the third example sample set is an example sample set composed of multiple first example samples that are most similar to the text information to be processed and are screened out from the first example sample set based on the first similarity information. When the present embodiment performs a second example search on the first example sample set based on the text information to be processed, the text information to be processed is first encoded to obtain first encoding information, and each first example sample is encoded to obtain second encoding information. Based on the first encoding information and the second encoding information, the first similarity information between the text information to be processed and each first example sample is determined, and based on the first similarity information, multiple first example samples that are most similar to the text information to be processed are screened out from the first example sample set to form the third example sample set.

[0102] The calculation formula of the first similarity information is: sim(x,x i )=E X (x) T E P (x i ), sim(x,x i ) represents the text information to be processed and the first example sample x in the first example sample set i The first similarity information, E X (x) represents the first coded information, E P (x i ) represents the second encoding information of the i-th first example sample in the first example sample set.

[0103] In a specific implementation, the first similarity information is obtained through a similarity prediction model, and the similarity prediction model includes a first encoder, a second encoder and a similarity calculation unit, and the first encoder and the second encoder are cascaded with the similarity calculation unit respectively. Accordingly, the step of determining the first similarity information between the text information to be processed and each first example sample may specifically include: inputting the text information to be processed into the first encoder, encoding the text information to be processed through the first encoder to obtain first encoding information; inputting each first example sample into the second encoder, encoding each first example sample through the second encoder to obtain second encoding information; inputting the first encoding information and the second encoding information into the similarity calculation unit, and outputting the first similarity information between the text information to be processed and each first example sample through the similarity calculation unit.

[0104] In a specific implementation, the step of determining the first similarity information between the text information to be processed and each first example sample includes: determining a candidate example set for each first example sample; determining a positive sample set and a negative sample set for each first example sample based on the candidate example set; constructing a training example set based on each first example sample, the positive sample set and the negative sample set, wherein the training example set includes the first example sample, the positive sample of the first example sample and multiple negative samples of the first example sample; inputting the training example set into a preset first network model, and outputting the second similarity information between the first example sample and the positive sample and the third similarity information between the first example sample and each negative sample through the first network model; determining a loss value based on the second similarity information, the third similarity information and the loss function of the first network model; when the loss value does not meet the preset conditions, correcting the model parameters of the first network model based on the preset parameter learning rate, and continuing to perform the step of inputting the training example set into the first network model until the loss value meets the preset conditions to obtain a similarity prediction model.

[0105] The candidate sample set is the sample set that is most similar to each first sample. The candidate sample set can be expressed as: For the first example sample set This embodiment traverses all the first example samples (x i ,y i )∈D, the unsupervised text similarity discrimination method BM25 based on the sparse retriever relying on surface text similarity is used to obtain the sample set that is most similar to each first example sample, and this sample set is used as the candidate sample set of the first example sample.

[0106] Furthermore, the positive sample set is a plurality of samples with the highest scores in the candidate sample set, and the negative sample set is a plurality of samples with the lowest scores in the candidate sample set. In order to determine the positive sample set and the negative sample set of each first example sample, this embodiment uses a large language model (LLM) to score each candidate example in the candidate sample set, and then selects a plurality of candidate examples with the highest scores from the candidate sample set as the positive sample set, and selects a plurality of candidate examples with the lowest scores from the candidate sample set as the negative sample set. The scoring formula for each candidate example is: represents the lth candidate example in the candidate example set, represents the score of the lth candidate example, represents a large language model, x i represents the first example sample of the i-th instance, y i represents the entity relationship label corresponding to the i-th first example sample, For a given x i and Large Language Model The output prediction is y i probability.

[0107] The training example set includes a first example sample, a positive sample of the first example sample, and multiple negative samples of the first example sample. The training example set can be expressed as x i represents the i-th first example sample, represents the positive sample of the i-th first example sample, which is extracted from the positive sample set of the i-th first example sample, and the remaining 2B-1 represent x i Negative samples, 1 negative sample is from x i The negative sample set is sampled, B-1 negative samples are positive samples of other first example samples in the same batch, and B-1 negative samples are negative samples of other first example samples in the same batch.

[0108] In a specific implementation, the loss function of the first network model can be expressed as: x i represents the i-th first example sample, Represents x i The positive sample of represents the second similarity information, Represents x i The jth negative sample of represents the third similarity information, and 2B-1 represents the number of negative samples.

[0109] In a specific implementation, step S24 includes:

[0110] Step S241, obtaining a fourth example sample set;

[0111] Step S242: construct a fifth example sample set based on the second example sample set, the third example sample set and the fourth example sample set;

[0112] Step S243: sorting the fifth example sample set to obtain a plurality of sorted example sample sets; each sorted example sample set includes a plurality of sorted example samples;

[0113] Step S244: determining a first prediction probability of each sorted example sample under a plurality of preset entity relationship labels;

[0114] Step S245: determining a second prediction probability of each sorted example sample set under each entity relationship label based on the first prediction probability;

[0115] Step S246: determining a first index value based on the first prediction probability, and determining a second index value based on the second prediction probability;

[0116] Step S247: determining a target ranking result of the fifth example sample set based on the first index value and the second index value;

[0117] Step S248: performing sample concatenation on the fifth sample set based on the target sorting result to obtain a target sample set.

[0118] The fourth example sample set is an example sample set composed of multiple random example samples. In order to increase the diversity of the target example sample set, this embodiment obtains the fourth example sample set, and determines the sample set composed of the second example sample set, the third example sample set, and the fourth example sample set as the fifth example sample set. For example, the second example sample set includes T example samples, the third example sample set includes G example samples, and the fourth example sample set includes D example samples. The example sample set composed of T example samples, G example samples, and D example samples is determined as the fifth example sample set.

[0119] There are multiple sorting methods for the example samples in the fifth example sample set, and different sorting methods will affect the results of entity relationship extraction. The target sorting result is the sorting method that can enable entity relationship extraction to obtain better performance among the multiple sorting methods. In order to improve the accuracy of the entity relationship extraction results, after constructing the fifth example sample set, this embodiment determines the target sorting result of the fifth example sample set, and performs sample splicing on the fifth example sample set based on the target sorting result to obtain the target example sample set.

[0120] The multiple sorting example sample sets are sorted fifth example sample sets obtained by using multiple different sorting methods to sort the fifth example sample set. That is, the sorting example samples included in the multiple sorting example sample sets are the same, and the difference between the multiple sorting example sample sets is that the sorting example samples are sorted in different ways. For example, sorting example sample set A includes sorting example sample A, sorting example sample B, sorting example sample C, and sorting example sample D; sorting example sample set B includes sorting example sample A, sorting example sample C, sorting example sample D, and sorting example sample B; and sorting example sample set C includes sorting example sample D, sorting example sample C, sorting example sample A, and sorting example sample B.

[0121] The first prediction probability is the probability of each sorted example sample output by the pre-training model under a preset entity relationship label after inputting multiple sorted example sample sets into the pre-training model; the second prediction probability is the probability of each sorted example sample set under each entity relationship label determined based on the first prediction probability. The calculation formula of the second prediction probability is: represents the second predicted probability of the sorted example sample set with sorting mode m under the entity relationship label v, represents the first predicted probability of the i-th sorted example sample under the entity relationship label v in the sorted example sample set with sorting mode m, and D represents the concatenation vector of each sorted example sample.

[0122] The first index value is used to characterize the balance of the predicted probabilities of different entity relationship labels under each sorting example sample set, and the second index value is used to characterize the distribution balance of entity relationship labels under each sorting example sample set. The calculation formula of the first index value is: The calculation formula of the second indicator value is: It represents the first predicted probability of the i-th sorted example sample under the entity relationship label v in the sorted example sample set with sorting mode m, represents the second predicted probability of the sorted example sample set with sorting mode m under the entity relationship label v, V represents the entity relationship label set, and D represents the concatenation vector of each sorted example sample.

[0123] Step S30: extracting entity relationships from the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

[0124] The entity relationship information is the entity and entity relationship information extracted from the text information to be processed based on the target example sample set. For example, the text information to be processed is "Zhang San sang Rose Boy", and the entity relationship information is "{subject: Zhang San; predicate: work; object: Rose Boy}". After obtaining the target example sample set, this embodiment extracts entity relationships from the text information to be processed based on the target example sample set to obtain the entity relationship information of the text information to be processed. The target example sample set is used as a context prompt for entity relationship extraction, so that the joint extraction of entity relationships can be achieved, and the problem of error propagation and accumulation that is prone to occur in the pipeline method can be solved.

[0125] In a specific implementation, continue to refer to Figure 2 As shown, step S30 includes:

[0126] Step S31, segmenting the target example sample set to obtain a processed target example sample set;

[0127] Step S32, fusing the processed target example sample set with the text information to be processed to obtain fused text information;

[0128] Step S33: Obtain preset task definition information, process the task definition information and the fused text information, and obtain entity relationship information of the text information to be processed.

[0129] The sample samples in the target sample set include the target sample and the entity relationship label corresponding to the target sample. The segmentation process refers to segmenting the target sample and the entity relationship label corresponding to the target sample. For example, i \ny i "The format of the i-th target example sample x i With the i-th target example sample x i The corresponding entity relationship label y i Perform segmentation processing.

[0130] Fusing the processed target example sample set with the text information to be processed refers to splicing the processed target example sample set with the text information to be processed. For example, the text information to be processed may be spliced ​​behind the processed target example sample set.

[0131] The task definition information is the output format information of the predefined text processing task. For example, the task definition information may be “{subject: entity 1; predicate: relationship; object: entity 2}; if there is no relationship, output no relationship”.

[0132] After obtaining the fused text information, this embodiment obtains the preset task definition information, inputs the task definition information and the fused text information into the pre-training model, and outputs the entity relationship information of the text information to be processed through the pre-training model. For example, the fused text information is "Example 1: Li Si played the role of the sea in the dream.\n{subject: Li Si; predicate: work; object: the sea in the dream}... Zhang San sang the rose boy.\n?", the task definition information and the fused text information are input into the pre-training model, and the pre-training model outputs {subject: Zhang San; predicate: work; object: rose boy}.

[0133] The pre-trained model is an already trained model. The pre-trained language model adopts a large language model (LLM). For example, the pre-trained model can adopt a generative pre-trained Transformer (GPT) model, such as a GPT-3.5 model.

[0134] It should be noted that the existing relationship extraction task adopts the traditional language model, uses independent pre-training and fine-tuning pipelines, and fine-tunes the dataset of the specific task in a fully supervised manner. The model has poor generalization ability, high time cost and large resource overhead. This embodiment adopts a large language model and uses the target example sample set as the prompt information for entity relationship extraction. It can convert the entity relationship extraction problem into a text-to-text problem, and avoid fine-tuning the large model by using a small number of labeled samples as prompts. It can not only save time and resource overhead, but also improve the entity relationship extraction performance of the pre-trained model and the generalization performance of the pre-trained model.

[0135] In a specific implementation, Figure 3As shown, the pre-trained model includes a multi-head attention layer, a first sum normalization layer, a feedforward neural network layer, a second sum normalization layer, a linear layer and an activation layer, which are cascaded in sequence. The steps of inputting the task definition information and the fused text information into the pre-trained model and outputting the entity relationship information of the text information to be processed through the pre-trained model specifically include: inputting the task definition information and the fused text information into the multi-head attention layer, performing multi-head attention processing on the task definition information and the fused text information through the multi-head attention layer to obtain the first processed information; inputting the first processed information into the first sum normalization layer, performing summation and normalization processing on the first processed information through the first sum normalization layer to obtain the second processed information; inputting the second processed information into the feedforward neural network layer, performing feature extraction on the second processed information through the feedforward neural network layer to obtain the first feature; inputting the first feature into the second sum normalization layer, performing summation and normalization processing on the first feature through the second sum normalization layer to obtain the third processed information; inputting the third processed information into the linear layer, performing linear transformation on the third processed information through the linear layer to obtain the second feature; inputting the second feature into the activation layer, performing feature mapping on the second feature through the activation layer to obtain the entity relationship information of the text information to be processed. Among them, the activation layer can use the SoftMax function.

[0136] In a specific implementation, the pre-trained model is obtained by training the second network model based on the training data set and the preset loss function of the second network model. In order to adapt to the relationship extraction task of multi-label classification, the loss function of the second network model can adopt a multi-label cross entropy loss function. Among them, the multi-label cross entropy loss function can be expressed as: S i Expressed as the score of category i, S j It is expressed as the score of category j, P represents the category set of positive samples, and N represents the category set of negative samples.

[0137] In a specific implementation, before training the second network model based on the training data set and the preset loss function of the second network model, the process includes: cleaning the training data set to remove meaningless symbols and punctuation, segmenting and annotating the cleaned training data set to improve the model's understanding of language structure and context, balancing and pruning the training data set after segmentation and annotation to avoid overfitting and data imbalance problems, and finally dividing the balanced and pruned training data set into a training set and a test set, and training the second network model using the training set and the test set to obtain a pre-trained model.

[0138] In order to better implement the text processing method in the embodiment of the present application, based on the text processing method, the embodiment of the present application also provides a text processing device, such as Figure 4 As shown, the text processing device includes:

[0139] The information acquisition unit 610 is used to acquire the text information to be processed;

[0140] An example search unit 620 is used to perform an example search on the text information to be processed to obtain a target example sample set;

[0141] The text processing unit 630 is used to extract entity relationships from the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

[0142] In an embodiment of the present application, an example search is performed on the text information to be processed to obtain a target example sample set, entity relationships are extracted from the text information to be processed based on the target example sample set, and the target example sample set is used as a context prompt for entity relationship extraction, so that joint extraction of entity relationships can be achieved, solving the problem of error propagation and accumulation that is prone to occur in pipeline methods.

[0143] In some embodiments of the present application, the example search unit 620 is specifically used to:

[0144] Obtaining a first example sample set;

[0145] Performing a first example search on the first example sample set based on the text information to be processed to obtain a second example sample set;

[0146] Performing a second example search on the first example sample set based on the text information to be processed to obtain a third example sample set;

[0147] Based on the second example sample set and the third example sample set, a target example sample set is constructed.

[0148] In some embodiments of the present application, the first example sample set includes multiple first example samples, and the example search unit 620 is specifically further used to:

[0149] Extracting and fusing features of each first example sample to obtain a first fused feature, and extracting and fusing features of the text information to be processed to obtain a second fused feature;

[0150] Performing feature segmentation on the first fusion feature to obtain a plurality of first segmentation features;

[0151] Clustering the multiple first segmentation features to obtain cluster center information of each first example sample;

[0152] Performing feature segmentation on the second fused feature to obtain a plurality of second segmentation features;

[0153] Based on the plurality of second segmentation features and cluster center information, determining distance information between the text information to be processed and each first example sample;

[0154] An example search is performed on the first example sample set based on the distance information to obtain a second example sample set.

[0155] In some embodiments of the present application, the example search unit 620 is further configured to:

[0156] Encoding the text information to be processed to obtain first encoded information;

[0157] Performing encoding processing on each first example sample to obtain second encoding information;

[0158] Based on the first encoding information and the second encoding information, determining first similarity information between the text information to be processed and each first example sample;

[0159] An example search is performed on the first example sample set based on the first similarity information to obtain a third example sample set.

[0160] In some embodiments of the present application, the example search unit 620 is further configured to:

[0161] Obtaining a fourth example sample set;

[0162] Based on the second example sample set, the third example sample set and the fourth example sample set, construct a fifth example sample set;

[0163] Performing example sorting on the fifth example sample set to obtain a plurality of sorted example sample sets; each sorted example sample set includes a plurality of sorted example samples;

[0164] Determine a first prediction probability of each sorted example sample under a plurality of preset entity relationship labels;

[0165] Based on the first prediction probability, determining a second prediction probability of each sorted example sample set under each entity relationship label;

[0166] determining a first indicator value based on the first predicted probability, and determining a second indicator value based on the second predicted probability;

[0167] Determining a target ranking result of the fifth example sample set based on the first index value and the second index value;

[0168] Based on the target sorting result, the fifth example sample set is concatenated to obtain a target example sample set.

[0169] In some embodiments of the present application, the text processing unit 630 is specifically used to:

[0170] Performing segmentation processing on the target example sample set to obtain a processed target example sample set;

[0171] Fusing the processed target example sample set with the text information to be processed to obtain fused text information;

[0172] Obtain preset task definition information, process the task definition information and fused text information, and obtain entity relationship information of the text information to be processed.

[0173] In some embodiments of the present application, the text processing unit 630 is further configured to:

[0174] Perform multi-head attention processing on the task definition information and the fused text information to obtain first processed information;

[0175] performing summing and normalization processing on the first processed information to obtain second processed information;

[0176] Performing feature extraction on the second processed information to obtain a first feature;

[0177] Performing summing and normalization processing on the first feature to obtain third processing information;

[0178] Performing a linear transformation on the third processed information to obtain a second feature;

[0179] Perform feature mapping on the second feature to obtain entity relationship information of the text information to be processed.

[0180] The embodiment of the present application further provides a computer device, which integrates any text processing device provided in the embodiment of the present application, and the computer device includes:

[0181] one or more processors;

[0182] Memory; and

[0183] One or more applications, wherein the one or more applications are stored in a memory and are configured to execute, by a processor, the steps of the text processing method in any of the above text processing method embodiments.

[0184] The present application also provides a computer device that integrates any text processing device provided in the present application. Figure 5 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0185] The computer device may include one or more processing core processors 801, one or more computer-readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will appreciate that Figure 5 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0186] The processor 801 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 802 and calling data stored in the memory 802, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0187] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0188] The computer device also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0189] The computer device may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0190] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 will run the application programs stored in the memory 802, thereby realizing various functions, as follows:

[0191] Get the text information to be processed;

[0192] Perform example search on the text information to be processed to obtain a target example sample set;

[0193] Entity relationship extraction is performed on the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

[0194] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0195] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any text processing method provided in the embodiment of the present application. For example, the computer program is loaded by a processor to execute the following steps:

[0196] Get the text information to be processed;

[0197] Perform example search on the text information to be processed to obtain a target example sample set;

[0198] Entity relationship extraction is performed on the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

[0199] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0200] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0201] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0202] The above is a detailed introduction to a text processing method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A text processing method, characterized in that: include: Get the text information to be processed; Performing an example search on the text information to be processed to obtain a target example sample set; Entity relationship extraction is performed on the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

2. The method according to claim 1, characterized in that The extracting entity relationships of the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed includes: Performing segmentation processing on the target example sample set to obtain a processed target example sample set; Fusing the processed target example sample set with the text information to be processed to obtain fused text information; Acquire preset task definition information, process the task definition information and the fused text information, and obtain entity relationship information of the text information to be processed.

3. The method according to claim 2, characterized in that The step of processing the task definition information and the fused text information to obtain entity relationship information of the text information to be processed includes: Performing multi-head attention processing on the task definition information and the fused text information to obtain first processed information; performing summing and normalization processing on the first processed information to obtain second processed information; performing feature extraction on the second processed information to obtain a first feature; performing summing and normalization processing on the first features to obtain third processing information; Performing a linear transformation on the third processed information to obtain a second feature; Perform feature mapping on the second feature to obtain entity relationship information of the text information to be processed.

4. The method according to claim 1, characterized in that The example search is performed on the text information to be processed to obtain a target example sample set, including: Obtaining a first example sample set; Performing a first example search on the first example sample set based on the text information to be processed to obtain a second example sample set; Performing a second example search on the first example sample set based on the text information to be processed to obtain a third example sample set; A target example sample set is constructed based on the second example sample set and the third example sample set.

5. The method according to claim 4, characterized in that The first example sample set includes a plurality of first example samples, and the first example search is performed on the first example sample set based on the text information to be processed to obtain a second example sample set, including: Extracting and fusing features of each of the first example samples to obtain a first fused feature, and extracting and fusing features of the text information to be processed to obtain a second fused feature; Performing feature segmentation on the first fused features to obtain a plurality of first segmented features; Clustering the plurality of the first segmentation features to obtain cluster center information of each of the first example samples; Performing feature segmentation on the second fused features to obtain a plurality of second segmented features; Based on the plurality of the second segmentation features and the cluster center information, determining distance information between the text information to be processed and each of the first example samples; An example search is performed on the first example sample set based on the distance information to obtain a second example sample set.

6. The method according to claim 4, characterized in that The performing a second example search on the first example sample set based on the text information to be processed to obtain a third example sample set includes: Encoding the text information to be processed to obtain first encoded information; Performing encoding processing on each of the first example samples to obtain second encoding information; Based on the first encoding information and the second encoding information, determining first similarity information between the text information to be processed and each of the first example samples; An example search is performed on the first example sample set based on the first similarity information to obtain a third example sample set.

7. The method according to claim 4, characterized in that The step of constructing a target example sample set based on the second example sample set and the third example sample set includes: Obtaining a fourth example sample set; constructing a fifth example sample set based on the second example sample set, the third example sample set and the fourth example sample set; Performing example sorting on the fifth example sample set to obtain a plurality of sorted example sample sets; each of the sorted example sample sets includes a plurality of sorted example samples; Determine a first prediction probability of each of the sorted example samples under a plurality of preset entity relationship labels; Based on the first prediction probability, determining a second prediction probability of each of the sorted example sample sets under each of the entity relationship labels; determining a first index value based on the first predicted probability, and determining a second index value based on the second predicted probability; Determining a target ranking result of the fifth example sample set based on the first index value and the second index value; Based on the target sorting result, the fifth example sample set is subjected to example concatenation to obtain a target example sample set.

8. A text processing device, characterized in that: include: An information acquisition unit, used for acquiring text information to be processed; An example search unit, used for performing an example search on the text information to be processed to obtain a target example sample set; The text processing unit is used to extract entity relationships of the text information to be processed based on the target example sample set to obtain entity relationship information of the text information to be processed.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the text processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the text processing method according to any one of claims 1 to 7.

Images