Text processing method, device, electronic device and computer-readable storage medium

Abstract

The embodiments of the present disclosure disclose a text processing method, device, electronic device and computer-readable storage medium. The text processing method includes a text processing method, which is characterized by comprising: obtaining a first text and a second text having an association relationship; obtaining a semantic vector sequence of the first text and a semantic vector sequence of the second text; interactively calculating and aggregating information on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain association points between the first text and the second text, thereby improving the accuracy of the acquired association points.

Description

Technical Field

[0001] The present disclosure relates to the field of computer application technology, and in particular to a text processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] Before a court hearing, it is often necessary to obtain the main dispute focus of the current case based on the objective factual text of the current case, such as the plaintiff's indictment and / or the defendant's defense statement, so that the judge can understand the overall summary of the current case based on the main dispute focus and ask questions to the plaintiff or defendant based on the main dispute focus during the court trial. Since the objective factual text is an unstructured text and does not have a topological structure such as a tree or mesh, how to extract the dispute focus from the unstructured objective factual text has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] In order to solve the problems in the related art, the embodiments of the present disclosure provide a text processing method, device, electronic device and computer-readable storage medium.

[0004] In a first aspect, a method for text processing is provided in an embodiment of the present disclosure.

[0005] Specifically, the text processing method includes:

[0006] Based on the first text data, a processor uses a first model to obtain a first semantic vector sequence {e 1 ,e 2 ,…,e m}, where e i is a vector, i=1~m;

[0007] Based on the second text data, the processor uses the first model to obtain a second semantic vector sequence {d 1 ,d 2 ,…,d n}, where d j is a vector, j=1~n;

[0008] Based on the first semantic vector sequence {e 1 ,e 2 ,…,e m}, using the processor to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}The corresponding third semantic vector sequence {q 1 ,q 2 ,…,q m}, where qi is a vector;

[0009] Based on the second semantic vector sequence {d 1 ,d 2 ,…,d n}, using the second model to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}The fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, where h j is a vector;

[0010] Based on the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data.

[0011] In combination with the first aspect, in a first implementation manner of the first aspect of the present disclosure, the first text data and the second text data contain content corresponding to each other, and the content of the target text data is associated with the corresponding content in the first text data and the second text data; and / or

[0012] The second text data includes a response to at least part of the content in the first text data, and the target text data includes text data related to the first text data and / or the second text data; and / or

[0013] The target text data is associated with text data that is controversial between the first text data and the second text data.

[0014] In combination with the first aspect, in a second implementation of the first aspect of the present disclosure, the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data, including:

[0015] The third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h2 ,…,h n} to perform splicing processing and obtain the splicing vector sequence {H 1 ,H 2 ,…,H r}, where H k is a vector, r=n+m, k=1~r;

[0016] Based on the splicing vector sequence {H 1 ,H 2 ,…,H r}, through the third model, obtain the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, where V k is a vector;

[0017] According to the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determine the target text data.

[0018] In combination with the second implementation of the first aspect, in a third implementation of the first aspect of the present disclosure, the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determining the target text data, including:

[0019] Based on the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, through the fourth model, obtain the first aggregation vector v 1 ;

[0020] The first aggregation vector v 1 Input the fully connected layer to obtain the second aggregate vector v 2 ;

[0021] The second aggregation vector v 2 After the activation function is applied, a probability distribution corresponding to the N candidate target text data is obtained, wherein the probability distribution includes the probability corresponding to each of the N candidate target text data;

[0022] The target text data is determined according to the probability distribution.

[0023] In combination with the third implementation method of the first aspect, the fourth implementation method of the first aspect of the present disclosure further includes pre-training the first model, the second model, the third model and the fourth model as a whole through at least one of the following tasks: predicting whether two paragraphs of text are continuous text, predicting the obscured part of the text, and predicting the operation performed on the text, wherein the overall loss function of the pre-training is determined according to the loss function of each task in the at least one task, and the parameters of the first model, the second model, the third model and the fourth model are adjusted according to the overall loss function.

[0024] In combination with the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect of the present disclosure, predicting whether two sections of text are continuous texts includes:

[0025] Acquire first training sample data, wherein the first training sample data includes first text sample data and second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0026] Based on the first training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0027] In combination with the fourth implementation of the first aspect, in a sixth implementation of the first aspect of the present disclosure, the masked portion of the predicted text includes:

[0028] Acquire second training sample data, wherein the second training sample data includes first text mask data acquired by processing the first text data in a preset mask manner and / or second text mask data acquired by processing the second text data in the preset mask manner;

[0029] Based on the second training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0030] In combination with the fourth implementation manner of the first aspect, in a seventh implementation manner of the first aspect of the present disclosure, the predicted operation performed on the text includes:

[0031] Acquire first training sample data, wherein the first training sample data includes first text sample data and / or second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0032] Acquire third training sample data according to the first training sample data, wherein the third training sample data includes third text sample data acquired by performing a specific action on the first text sample data and / or fourth text sample data acquired by performing the specific action on the second text sample data;

[0033] Based on the third training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0034] In combination with the first aspect, in an eighth implementation of the first aspect of the present disclosure, the first model includes any one of the following models: Word2vector model, Glove model, Fasttext model, BERT model.

[0035] In combination with the first aspect, in a ninth implementation of the first aspect of the present disclosure, the second model includes any one of the following models: CNN model, RNN model, LSTM model, Transformer model, BERT model, and attention model.

[0036] In combination with the second implementation manner of the first aspect, in the tenth implementation manner of the first aspect of the present disclosure, the third model includes any one of the following models: a Transformer model, an attention model.

[0037] In combination with the third implementation manner of the first aspect, in the eleventh implementation manner of the first aspect of the present disclosure, the fourth model includes an attention model.

[0038] In a second aspect, a method for text processing is provided in an embodiment of the present disclosure.

[0039] Specifically, the text processing method includes:

[0040] Obtaining a first text and a second text having an associated relationship;

[0041] Obtaining a semantic vector sequence of the first text and a semantic vector sequence of the second text;

[0042] Interactive calculation and information aggregation are performed on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the association points between the first text and the second text.

[0043] In conjunction with the second aspect, in a first implementation of the second aspect of the present disclosure, the interactive calculation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0044] splicing the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain a splicing vector;

[0045] A semantic vector sequence of the concatenated vector is calculated.

[0046] In combination with the first implementation manner of the second aspect, in a second implementation manner of the second aspect of the present disclosure, the information aggregation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0047] Obtaining an aggregate vector according to the semantic vector sequence of the concatenated vector;

[0048] Obtaining probability distribution corresponding to a plurality of candidate association points according to the aggregation vector;

[0049] According to the probability distribution corresponding to the plurality of candidate association points, association points between the first text and the second text are obtained.

[0050] In combination with the second implementation manner of the second aspect, in a third implementation manner of the second aspect of the present disclosure, obtaining a probability distribution corresponding to multiple candidate association points according to the aggregation vector includes:

[0051] The aggregated vector is input into the fully connected layer of the system model and is acted upon by an activation function to obtain a probability distribution corresponding to a plurality of candidate association points.

[0052] In combination with the third implementation of the second aspect, in a fourth implementation of the second aspect of the present disclosure, the method further includes:

[0053] Pre-training the system model;

[0054] The pre-training includes one or more of the following tasks: predicting whether two text segments are continuous texts, predicting masked parts of text, and predicting operations performed on text.

[0055] In combination with the fourth implementation manner of the second aspect, in a fifth implementation manner of the second aspect of the present disclosure, predicting whether two text segments are continuous texts includes:

[0056] Obtaining a first segment and a second segment in succession from a third text;

[0057] randomly selecting a third segment from the third text;

[0058] replacing the first segment or the second segment with the third segment;

[0059] Predict whether the third segment and the second segment, or the first segment and the third segment, are continuous text.

[0060] In combination with the fourth implementation of the second aspect, in a sixth implementation of the second aspect of the present disclosure, the masked portion of the predicted text includes:

[0061] Randomly selecting any text segment in the first text and / or the second text;

[0062] Processing the arbitrary text segment in a preset mask manner;

[0063] Predict the masked content in any processed text segment.

[0064] In combination with the fifth implementation of the second aspect, in a seventh implementation of the second aspect of the present disclosure, the predicted operation performed on the text includes:

[0065] In the text composed of the first segment and the second segment, and / or the text composed of the first segment and the third segment, and / or the text composed of the third segment and the second segment, select any word to perform a specific operation;

[0066] The specific operation is predicted, where the specific operation includes deletion, replacement, and no operation.

[0067] In a third aspect, a text processing device is provided in an embodiment of the present disclosure.

[0068] Specifically, the text processing device comprises:

[0069] The first acquisition module is configured to acquire, based on the first text data, a first semantic vector sequence {e 1 ,e 2 ,…,e m}, where e i is a vector, i=1~m;

[0070] The second acquisition module is configured to acquire, based on the second text data, a second semantic vector sequence {d 1 ,d 2 ,…,d n}, where d j is a vector, j=1~n;

[0071] The third acquisition module is configured to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}, using the processor to obtain the first semantic vector sequence {e 1 ,e2 ,…,e m}The corresponding third semantic vector sequence {q 1 ,q 2 ,…,q m}, where q i is a vector;

[0072] A fourth acquisition module is configured to obtain a plurality of semantic vectors based on the second semantic vector sequence {d 1 ,d 2 ,…,d n}, using the second model to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}The fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, where h j is a vector;

[0073] A determination module is configured to determine based on the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data.

[0074] In combination with the third aspect, in a first implementation manner of the third aspect of the present disclosure, the first text data and the second text data contain content corresponding to each other, and the content of the target text data is associated with the corresponding content in the first text data and the second text data; and / or

[0075] The second text data includes a response to at least part of the content in the first text data, and the target text data includes text data related to the first text data and / or the second text data; and / or

[0076] The target text data is associated with text data that is controversial between the first text data and the second text data.

[0077] In combination with the third aspect, in a second implementation of the third aspect of the present disclosure, the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data, including:

[0078] The third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n} to perform splicing processing and obtain the splicing vector sequence {H 1 ,H 2 ,…,H r}, where H k is a vector, r=n+m, k=1~r;

[0079] Based on the splicing vector sequence {H 1 ,H 2 ,…,H r}, through the third model, obtain the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, where V k is a vector;

[0080] According to the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determine the target text data.

[0081] In combination with the second implementation of the third aspect, in the third implementation of the third aspect of the present disclosure, the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determining the target text data, including:

[0082] Based on the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, through the fourth model, obtain the first aggregation vector v 1 ;

[0083] The first aggregation vector v 1 Input the fully connected layer to obtain the second aggregate vector v 2 ;

[0084] The second aggregation vector v 2 After the activation function is applied, a probability distribution corresponding to the N candidate target text data is obtained, wherein the probability distribution includes the probability corresponding to each of the N candidate target text data;

[0085] The target text data is determined according to the probability distribution.

[0086] In combination with the third implementation of the third aspect, the present disclosure, in a fourth implementation of the third aspect, further includes:

[0087] The first pre-training module is configured to pre-train the first model, the second model, the third model and the fourth model as a whole through at least one of the following tasks: predicting whether two paragraphs of text are continuous text, predicting the masked part of the text, and predicting the operation performed on the text, wherein the overall loss function of the pre-training is determined according to the loss function of each task in the at least one task, and the parameters of the first model, the second model, the third model and the fourth model are adjusted according to the overall loss function.

[0088] In combination with the fourth implementation of the third aspect, in a fifth implementation of the third aspect of the present disclosure, predicting whether two sections of text are continuous texts includes:

[0089] Acquire first training sample data, wherein the first training sample data includes first text sample data and second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0090] Based on the first training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0091] In combination with the fourth implementation of the third aspect, in a sixth implementation of the third aspect of the present disclosure, the masked portion of the predicted text includes:

[0092] Acquire second training sample data, wherein the second training sample data includes first text mask data acquired by processing the first text data in a preset mask manner and / or second text mask data acquired by processing the second text data in the preset mask manner;

[0093] Based on the second training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0094] In combination with the fourth implementation of the third aspect, in a seventh implementation of the third aspect of the present disclosure, the predicted operation on the text includes:

[0095] Acquire first training sample data, wherein the first training sample data includes first text sample data and / or second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0096] Acquire third training sample data according to the first training sample data, wherein the third training sample data includes third text sample data acquired by performing a specific action on the first text sample data and / or fourth text sample data acquired by performing the specific action on the second text sample data;

[0097] Based on the third training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0098] In combination with the third aspect, in an eighth implementation of the third aspect of the present disclosure, the first model includes any one of the following models: Word2vector model, Glove model, Fasttext model, BERT model.

[0099] In combination with the third aspect, in a ninth implementation of the third aspect of the present disclosure, the second model includes any one of the following models: CNN model, RNN model, LSTM model, Transformer model, BERT model, and attention model.

[0100] In combination with the second implementation manner of the third aspect, in the tenth implementation manner of the third aspect of the present disclosure, the third model includes any one of the following models: a Transformer model, an attention model.

[0101] In combination with the third implementation manner of the third aspect, in the eleventh implementation manner of the third aspect of the present disclosure, the fourth model includes an attention model.

[0102] In a fourth aspect, a text processing device is provided in an embodiment of the present disclosure.

[0103] Specifically, the text processing device comprises:

[0104] A fifth acquisition module is configured to obtain a first text and a second text having an associated relationship;

[0105] A sixth acquisition module is configured to obtain a semantic vector sequence of the first text and a semantic vector sequence of the second text;

[0106] The seventh acquisition module is configured to perform interactive calculation and information aggregation on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the association points between the first text and the second text.

[0107] In conjunction with the fourth aspect, in a first implementation of the fourth aspect of the present disclosure, the interactive calculation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0108] splicing the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain a splicing vector;

[0109] A semantic vector sequence of the concatenated vector is calculated.

[0110] In combination with the first implementation manner of the fourth aspect, in a second implementation manner of the fourth aspect of the present disclosure, the information aggregation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0111] Obtaining an aggregate vector according to the semantic vector sequence of the concatenated vector;

[0112] Obtaining probability distribution corresponding to a plurality of candidate association points according to the aggregation vector;

[0113] According to the probability distribution corresponding to the plurality of candidate association points, association points between the first text and the second text are obtained.

[0114] In combination with the second implementation manner of the fourth aspect, in a third implementation manner of the fourth aspect of the present disclosure, obtaining a probability distribution corresponding to multiple candidate association points according to the aggregation vector includes:

[0115] The aggregated vector is input into the fully connected layer of the system model and is acted upon by an activation function to obtain a probability distribution corresponding to a plurality of candidate association points.

[0116] In combination with the third implementation of the fourth aspect, in a fourth implementation of the fourth aspect of the present disclosure, the device further includes:

[0117] A second pre-training module is configured to pre-train the system model;

[0118] The pre-training includes one or more of the following tasks: predicting whether two text segments are continuous texts, predicting masked parts of text, and predicting operations performed on text.

[0119] In combination with the fourth implementation of the fourth aspect, in a fifth implementation of the fourth aspect of the present disclosure, predicting whether two text segments are continuous texts includes:

[0120] Obtaining a first segment and a second segment in succession from a third text;

[0121] randomly selecting a third segment from the third text;

[0122] replacing the first segment or the second segment with the third segment;

[0123] Predict whether the third segment and the second segment, or the first segment and the third segment, are continuous text.

[0124] In combination with the fourth implementation of the fourth aspect, in a sixth implementation of the fourth aspect of the present disclosure, the masked portion of the predicted text includes:

[0125] Randomly selecting any text segment in the first text and / or the second text;

[0126] Processing the arbitrary text segment in a preset mask manner;

[0127] Predict the masked content in any processed text segment.

[0128] In combination with the fifth implementation of the fourth aspect, in a seventh implementation of the fourth aspect of the present disclosure, the predicted operation performed on the text includes:

[0129] In the text composed of the first segment and the second segment, and / or the text composed of the first segment and the third segment, and / or the text composed of the third segment and the second segment, select any word to perform a specific operation;

[0130] The specific operation is predicted, where the specific operation includes deletion, replacement, and no operation.

[0131] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspect, the first to eleventh implementations of the first aspect, the second aspect, and the first to seventh implementations of the second aspect.

[0132] In the sixth aspect, a computer-readable storage medium is provided in an embodiment of the present disclosure, on which computer instructions are stored. When the computer instructions are executed by a processor, the method described in any one of the first aspect, the first to eleventh implementations of the first aspect, the second aspect, and the first to seventh implementations of the second aspect is implemented.

[0133] According to the technical solution provided by the embodiment of the present disclosure, a first text and a second text having an association relationship are obtained, a semantic vector sequence of the first text and a semantic vector sequence of the second text are obtained, and interactive calculation and information aggregation are performed on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the association points of the first text and the second text. The embodiment of the present disclosure can be applied to court trial application scenarios. Through the respective contextual semantic information and interactive semantic information of the indictment and the reply, the focus of dispute that can be recommended to the judge as a candidate is obtained, so that the obtained focus of dispute can correspond to the semantic information of the indictment and the reply, and the accuracy of the recommended focus of dispute can be improved.

[0134] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0135] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0136] Figure 1 A flowchart showing a text processing method according to an embodiment of the present disclosure;

[0137] Figure 2 The third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the schematic diagram of the target text data;

[0138] Figure 3 A schematic diagram showing pre-training the first model, the second model, the third model and the fourth model as a whole according to an embodiment of the present disclosure;

[0139] Figure 4 A schematic diagram showing an application scenario of the text processing method according to an embodiment of the present disclosure;

[0140] Figure 5 A flowchart showing a text processing method according to an embodiment of the present disclosure;

[0141] Figure 6 A schematic diagram showing interactive calculation and information aggregation of the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain association points of the first text and the second text according to an embodiment of the present disclosure;

[0142] Figure 7 A schematic diagram showing pre-training of the system model according to an embodiment of the present disclosure is shown;

[0143] Figure 8 A schematic diagram showing an application scenario of the text processing method according to an embodiment of the present disclosure;

[0144] Fig. 9 A structural block diagram of a text processing device according to an embodiment of the present disclosure is shown;

[0145] Fig.10 A structural block diagram of a text processing device according to an embodiment of the present disclosure is shown;

[0146] Fig.11 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0147] Fig.12 A schematic diagram showing the structure of a computer system suitable for implementing the text processing method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0148] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0149] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0150] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0151] The legal knowledge graph is the link between facts and legal knowledge. The graph stores legal knowledge nodes, such as principal disputes, interest agreements, mortgage guarantees, joint debts of husband and wife, etc. These nodes are important concepts and abstractions in the legal knowledge system. Usually, the legal knowledge graph has a tree-like or mesh-like topological structure, which is used to express complex legal relationships, such as interest-repayment of part of the loan interest, breach of contract loss-lawyer fees, investigation fees and other losses. The construction of the knowledge graph can be constructed automatically by the algorithm or manually. The generation of the focus of the dispute can be seen as mapping the contents of the indictment and the defense of the plaintiff and the defendant to a certain knowledge node in the graph, and then structuring the unstructured text information. Each node corresponds to a focus of the dispute. Since the objective fact text is an unstructured text and does not have a tree or mesh topological structure, how to obtain the focus of the dispute based on the unstructured objective fact text has become a technical problem that needs to be solved urgently.

[0152] It should be noted that the technical solution disclosed in the present invention can be used to process unstructured texts as well as structured texts.

[0153] Figure 1 1 is a flowchart showing a text processing method according to an embodiment of the present disclosure. Figure 1 As shown, the text processing method includes the following steps S101-S105:

[0154] In step S101, based on first text data, a processor uses a first model to obtain a first semantic vector sequence {e 1 ,e 2 ,…,e m}, where e i is a vector, i=1~m;

[0155] In step S102, based on the second text data, the processor uses the first model to obtain a second semantic vector sequence {d 1 ,d 2 ,…,d n}, where d j is a vector, j=1~n;

[0156] In step S103, based on the first semantic vector sequence {e 1 ,e 2 ,…,e m}, using the processor to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}The corresponding third semantic vector sequence {q 1 ,q2 ,…,q m}, where q i is a vector;

[0157] In step S104, based on the second semantic vector sequence {d 1 ,d 2 ,…,d n}, using the second model to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}The fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, where h j is a vector;

[0158] In step S105, based on the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data.

[0159] According to an embodiment of the present disclosure, the first text data and the second text data contain content corresponding to each other; and / or the second text data includes a response to at least part of the content in the first text data.

[0160] According to an embodiment of the present disclosure, first text data and second text data may be obtained, wherein the first text data and the second text data may contain content corresponding to each other, for example, containing the same or similar topics. Alternatively, the second text data may include a response to at least part of the content in the first text data, for example, the second text data may include reply text data of the first text data. For example, in a court trial application scenario, the first text data may include the plaintiff's indictment, and the second text data may include the defendant's reply.

[0161] According to an embodiment of the present disclosure, the first text data can be segmented to obtain a first segmentation data set, wherein the first segmentation data set includes the segmentation results of the first text data. The present disclosure does not specifically limit the segmentation processing method, which can be selected according to actual needs.

[0162] According to an embodiment of the present disclosure, in order to obtain semantic information of the first text data, the first model can be used to obtain a vector representation corresponding to each word in the first word segmentation data set, and obtain a first semantic vector sequence {e 1 ,e2 ,…,e m}, where e i is the word vector of the ith word segment, thereby introducing the semantic information of the first text data through the first model. The present disclosure does not specifically limit the first model. As long as the model can realize the conversion of text data into an embedding vector, it is within the protection scope of the embodiments of the present disclosure. For example, the first model may include any one of the following models: Word2vector model, Glove model, Fasttext model or BERT (Bidirectional Encoder Representations from Transformers) model, etc.

[0163] According to an embodiment of the present disclosure, the second text data can be segmented to obtain a second segmented data set, wherein the second segmented data set includes the segmentation results of the second text data. The present disclosure does not specifically limit the segmentation processing method, which can be selected according to actual needs. In order to obtain the semantic information of the second text data, the first model can be used to obtain the vector representation corresponding to each segmentation in the second segmented data set, and the second semantic vector sequence {d 1 ,d 2 ,…,d n}, where d j is the word vector of the jth word segment, thereby introducing the semantic information of the second text data through the first model.

[0164] According to an embodiment of the present disclosure, in order to obtain contextual semantic information of the first text data, a second model can be used to obtain contextual semantic information of the first text data based on the first semantic vector sequence {e 1 ,e 2 ,…,e m}, by learning m vectors e i The internal association relationship between them is used to obtain the third semantic vector sequence {q 1 ,q 2 ,…,q m}, where q iis a vector, thereby introducing the contextual semantic information of the first text data through the second model. The second model can be a trained deep learning model, which is not specifically limited in the present disclosure and can be selected according to actual needs. For example, the second model can include any one of the following models: CNN model, RNN model, LSTM model, Transformer model, BERT model, attention model, etc., wherein the attention model can include a self-attention model (Self Attention), a multi-head attention model (Multi-Head Attention) or a HAN (Hierarchical Attention Networks, multi-layer attention) model, etc.

[0165] According to an embodiment of the present disclosure, in order to obtain contextual semantic information of the second text data, a second model can be used to obtain contextual semantic information of the second text data based on a second semantic vector sequence {d 1 ,d 2 ,…,d n}, by learning n vectors d j The internal association relationship between them is used to obtain the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, where h j is a vector, thereby introducing the contextual semantic information of the second text data through the second model.

[0166] According to an embodiment of the present disclosure, the third semantic vector sequence {q 1 ,q 2 ,…,q m}, and a fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, obtain interactive semantic information between the first text data and the second text data, and determine the target text data based on the interactive semantic information between the first text data and the second text data.

[0167] According to an embodiment of the present disclosure, the content of the target text data is associated with the first text data and / or the second text data; and / or the target text data includes text data related to the first text data and / or the second text data; and / or the target text data is associated with text data in the first text data and the second text data that are controversial with each other.

[0168] According to an embodiment of the present disclosure, the content of the target text data is associated with the first text data and / or the second text data. For example, when the first text data and the second text data contain content corresponding to each other, the content of the target text data can be associated with the above-mentioned corresponding content.

[0169] Alternatively, the target text data may include text data related to the first text data and / or the second text data. For example, when the second text data includes a response to at least part of the content in the first text data, the target text data may include text data related to at least part of the above content and / or the response content.

[0170] Alternatively, the target text data may include and be associated with text data that is disputed between the first text data and the second text data. For example, when the first text data and the second text data contain text data that is disputed between them, the target text data may be associated with the above-mentioned text data that is disputed between them.

[0171] For example, in a court trial application scenario, the first text data may include the plaintiff's indictment, the second text data may include the defendant's reply, and the target text data may include multiple dispute focuses extracted from the indictment and the reply.

[0172] According to the technical solution provided by the embodiment of the present disclosure, based on the first text data, a processor uses a first model to obtain a first semantic vector sequence {e 1 ,e 2 ,…,e m}, based on the second text data, the processor uses the first model to obtain a second semantic vector sequence corresponding to the second text data {d 1 ,d 2 ,…,d n}, based on the first semantic vector sequence {e 1 ,e 2 ,…,e m}, the processor uses the second model to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}The corresponding third semantic vector sequence {q 1 ,q 2 ,…,q m}, based on the second semantic vector sequence {d 1 ,d 2 ,…,d n}, the processor uses the second model to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}The fourth semantic vector sequence {h1 ,h 2 ,…,h n}, based on the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data. The embodiment of the present disclosure obtains the target text data through the respective contextual semantic information and interactive semantic information of the first text data and the second text data, so that the obtained target text data can correspond to the semantic information of the first text data and the second text data, and the accuracy of the target text data can be improved. The embodiment of the present disclosure can be applied to court trial application scenarios, and the respective contextual semantic information and interactive semantic information of the indictment and the reply are used to obtain the focus of dispute that can be recommended to the judge as a candidate, so that the obtained focus of dispute can correspond to the semantic information of the indictment and the reply, and the accuracy of the recommended focus of dispute can be improved.

[0173] According to an embodiment of the present disclosure, the step S105, i.e. based on the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data, including:

[0174] The third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n} to perform splicing processing and obtain the splicing vector sequence {H 1 ,H 2 ,…,H r}, where H k is a vector, r=n+m, k=1~r;

[0175] Based on the splicing vector sequence {H 1 ,H 2 ,…,H r}, through the third model, obtain the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, where V k is a vector;

[0176] According to the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determine the target text data.

[0177] Figure 2 The third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, a schematic diagram for determining the target text data.

[0178] like Figure 2 As shown, in order to obtain the semantic information of the first text data and the second text data, the third semantic vector sequence {q 1 ,q 2 ,…,q m} and a fourth semantic vector sequence {h 1 ,h 2 ,…,h n} to perform splicing processing and obtain the splicing vector sequence {H 1 ,H 2 ,…,H r}.

[0179] According to an embodiment of the present disclosure, in order to obtain the interactive semantic information of the first text data and the second text data, a third model can be used based on a concatenated vector sequence {H 1 ,H 2 ,…,H r}, by learning r vectors H k The mutual correlation relationship between them is used to obtain the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, where the fifth semantic vector sequence {V 1 ,V 2 ,…,V r} can be used to represent at least one or more of the following semantic information: contextual semantic information of the first text data, contextual semantic information of the second text data, and interactive semantic information between the first text data and the second text data. The third model can be a trained deep learning model, which is not specifically limited in the present disclosure and can be selected according to actual needs. For example, the third model can include any one of the following models: Transformer model, attention model, etc., wherein the attention model can include a self-attention model, a multi-head attention model or a HAN model, etc.

[0180] According to an embodiment of the present disclosure, the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determining the target text data, including:

[0181] Based on the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, through the fourth model, obtain the first aggregation vector v 1 ;

[0182] The first aggregation vector v 1 Input the fully connected layer to obtain the second aggregate vector v 2 ;

[0183] The second aggregation vector v 2 After the activation function is applied, a probability distribution corresponding to the N candidate target text data is obtained, wherein the probability distribution includes the probability corresponding to each of the N candidate target text data;

[0184] The target text data is determined according to the probability distribution.

[0185] like Figure 2 As shown, in order to effectively extract the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, we can use the fourth model to obtain r vectors V respectively. k In the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, by calculating the corresponding weights of r vectors V k And the corresponding weights are weighted summed to obtain the first aggregation vector v 1The fourth model 202 may be a trained attention model, which is not specifically limited in the present disclosure and may be selected according to actual needs. For example, the fourth model 202 may include any one of the following models: a self-attention model, a multi-head attention model, or a HAN model.

[0186] According to an embodiment of the present disclosure, the first aggregate vector v including the semantic information of the first text data and the semantic information of the second text data may be 1 , and establish a mapping relationship with the preset text data. For example, N candidate target text data can be predetermined, and the first aggregation vector v 1 The target text data is determined from among the N candidate target text data.

[0187] According to an embodiment of the present disclosure, the first aggregation vector v 1 Input to the fully connected layer to obtain the second aggregate vector v 2 , where the second aggregation vector v 2 The dimension of the second aggregation vector v is the same as the number of candidate target text data. 2 The dimension is N.

[0188] For example, in the court trial application scenario, N dispute focuses can be determined on the topological structure nodes of the legal knowledge graph, such as "loan collection" and "joint debts of husband and wife". The dispute focuses in the legal knowledge graph can be used as candidate target text data, based on the first aggregation vector v 1 Target text data related to the indictment and the statement of defense, that is, the focus of dispute in the indictment and the statement of defense, are determined from the candidate target text data.

[0189] According to an embodiment of the present disclosure, the second aggregation vector v 2 After the activation function is applied, a probability distribution is obtained, wherein the probability distribution represents the probability that the N candidate target text data belong to the target text data. The greater the probability corresponding to a candidate target text data, the greater the possibility that the candidate target text data belongs to the target text data.

[0190] According to an embodiment of the present disclosure, K candidate target text data may be determined as target text data according to probability distribution. For example, the candidate target text data corresponding to the highest K probabilities in probability distribution may be determined as target text data, where 1≤K≤N is an integer.

[0191] For example, in a court trial application scenario, the first text data may include the plaintiff's indictment, and the second text data may include the defendant's reply. The indictment and the reply may be used to obtain the fifth semantic vector sequence {V 1 ,V 2 ,…,Vr}, and based on the fifth semantic vector sequence {V 1 ,V 2 ,…,V r} and the N controversial points in the legal knowledge graph, determine K controversial points.

[0192] Since the text processing method in the embodiment of the present disclosure involves the first model, the second model, the third model and the fourth model, when pre-training the first model, the second model, the third model and the fourth model, these models include a large number of parameters, and most of the parameters are randomly initialized. When the technical field or application scenario to which the text processing method in the embodiment of the present disclosure is applied includes less training data, for example, when the text processing method in the embodiment of the present disclosure is applied to the application scenario of a court trial, since there is not much training data such as an indictment or a reply that can be obtained, it is difficult to converge or fail to obtain the expected training effect when training the first model, the second model, the third model and the fourth model according to the randomly initialized parameters.

[0193] According to an embodiment of the present disclosure, the first model, the second model, the third model and the fourth model are pre-trained as a whole through at least one of the following tasks: predicting whether two paragraphs of text are continuous text, predicting the masked part of the text, and predicting the operation performed on the text, wherein the overall loss function of the pre-training is determined according to the loss function of each task in the at least one task, and the parameters of the first model, the second model, the third model and the fourth model are adjusted according to the overall loss function.

[0194] Figure 3 A schematic diagram is shown of pre-training the first model, the second model, the third model and the fourth model as a whole according to an embodiment of the present disclosure.

[0195] like Figure 3 As shown, in order to enable the first model, the second model, the third model and the fourth model involved in the text processing method in the embodiment of the present disclosure to obtain better initialization parameters and improve the model convergence efficiency and the expected training effect, the first model, the second model, the third model and the fourth model can be pre-trained as a whole through one or more of the following tasks: the first task 301 is to predict whether two paragraphs of text are continuous text, the second task 302 is to predict the obscured part of the text, and the third task 303 is to predict the operation performed on the text.

[0196] According to an embodiment of the present disclosure, the overall loss function of pre-training can be determined according to the loss function of each task in at least one task, for example, by summing the loss functions of each task to determine the overall loss function of pre-training. For example, assuming that the pre-training only includes the first task 301, the overall loss function of pre-training can be determined according to the loss function of the first task 301; for another example, assuming that the first task 301, the second task 302 and the third task 303 are included, the loss functions of the first task 301, the second task 302 and the third task 303 can be summed to determine the overall loss function of pre-training. The parameters of the first model, the second model, the third model and the fourth model can be adjusted according to the overall loss function so that the first model, the second model, the third model and the fourth model can obtain better initialization parameters, thereby improving the model convergence efficiency and the expected training effect.

[0197] According to an embodiment of the present disclosure, the predicting whether two paragraphs of text are continuous texts includes:

[0198] Acquire first training sample data, wherein the first training sample data includes first text sample data and second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0199] Based on the first training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0200] like Figure 3 As shown, in order to achieve the first task 301, ie, the training goal of predicting whether two text segments are continuous texts, first training sample data may be obtained. The present disclosure does not specifically limit the source and acquisition method of the first training sample data.

[0201] According to an embodiment of the present disclosure, the first training sample data can be obtained based on the first text data and / or the second text data. For example, the first text data includes an indictment A, and the second text data includes a statement of defense B. Two continuous text sample data (a1, b1) can be randomly extracted from the indictment A and / or the statement of defense B, so as to obtain the first text sample data. A section of other text sample data c1 can be randomly selected from the indictment A and / or the statement of defense B with a certain probability, and the text sample data a1 or the text sample data b1 can be replaced with the text sample data c1, so as to obtain two discontinuous text sample data (c1, b1) or (a1, c1), i.e., the second text sample data.

[0202] According to an embodiment of the present disclosure, the first training sample data may be obtained not based on the first text data and / or the second text data. For example, the first training sample data may be obtained from any one or more of the following text data: other indictments (excluding indictment A), other replies (excluding reply B), and other legal documents (excluding indictment and reply type files, such as public judgment documents, etc.), thereby increasing the amount of training sample data.

[0203] According to an embodiment of the present disclosure, the first model, the second model, the third model and the fourth model can be trained as a whole based on the first training sample data to obtain the mutual correlation between two texts, for example, the mutual correlation between sentences.

[0204] According to an embodiment of the present disclosure, the masked portion of the predicted text includes:

[0205] Acquire second training sample data, wherein the second training sample data includes first text mask data acquired by processing the first text data in a preset mask manner and / or second text mask data acquired by processing the second text data in the preset mask manner;

[0206] Based on the second training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0207] like Figure 3 As shown, in order to achieve the second task 302, ie, the training goal of predicting the masked part in the text, second training sample data may be obtained. The present disclosure does not specifically limit the source and acquisition method of the second training sample data.

[0208] According to an embodiment of the present disclosure, the second training sample data can be obtained based on the first text data and / or the second text data. For example, the first text data includes an indictment A, and the second text data includes a statement of defense B. A section of text sample data a2 can be randomly extracted from the indictment A, and a section of text sample data b2 can be randomly extracted from the statement of defense B. The text sample data a2 can be processed in a preset masking manner to obtain the first text mask data, for example, a preset proportion of segmented words in the text sample data a2 is randomly selected with a certain probability, and a mask is used to replace the original segmented words, thereby obtaining the first text mask data. The text sample data b2 can be processed in the above-mentioned preset masking manner to obtain the second text mask data.

[0209] According to an embodiment of the present disclosure, the first model, the second model, the third model and the fourth model can be trained as a whole based on the second training sample data, so as to obtain the mutual correlation between two word segments in the text, and / or the mutual correlation between the word segmentation and the text, for example, the mutual correlation between words, and / or the mutual correlation between words and sentences.

[0210] According to an embodiment of the present disclosure, the predicted operation performed on the text includes:

[0211] Acquire first training sample data, wherein the first training sample data includes first text sample data and / or second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0212] Acquire third training sample data according to the first training sample data, wherein the third training sample data includes third text sample data acquired by performing a specific action on the first text sample data and / or fourth text sample data acquired by performing the specific action on the second text sample data;

[0213] Based on the third training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0214] like Figure 3 As shown, in order to achieve the training goal of the third task 303, namely predicting the operation performed on the text, the third training sample data can be obtained based on the first training sample data. The present disclosure does not specifically limit the source and acquisition method of the first training sample data and the third training sample data.

[0215] According to an embodiment of the present disclosure, the first training sample data can be obtained based on the first text data and / or the second text data. For example, the first text data includes an indictment A, and the second text data includes a statement of defense B. Two continuous text sample data (a1, b1) can be randomly extracted from the indictment A and / or the statement of defense B, so as to obtain the first text sample data. A section of other text sample data c1 can be randomly selected from the indictment A and / or the statement of defense B with a certain probability, and the text sample data a1 or the text sample data b1 can be replaced with the text sample data c1, so as to obtain two discontinuous text sample data (c1, b1) or (a1, c1), i.e., the second text sample data.

[0216] According to an embodiment of the present disclosure, the first training sample data may be obtained not based on the first text data and / or the second text data. For example, the first training sample data may be obtained from any one or more of the following text data: other indictments (excluding indictment A), other replies (excluding reply B), and other legal documents (excluding indictment and reply type files, such as public judgment documents, etc.), thereby increasing the amount of training sample data.

[0217] According to an embodiment of the present disclosure, a third training sample can be obtained based on the first training sample data. A specific action can be performed on the first text sample data to obtain the third text sample data, and / or a specific action can be performed on the second text sample data to obtain the fourth text sample data, wherein the specific action includes but is not limited to "delete", "replace" or "no operation". For example, the word segmentation in the first text sample data (a1, b1) can be randomly deleted or replaced, or no operation is performed to obtain the third text sample data. Alternatively, the words in the second text sample data (c1, b1) or (a1, c1) can be randomly deleted or replaced, or no operation is performed to obtain the fourth text sample data.

[0218] According to an embodiment of the present disclosure, the first model, the second model, the third model and the fourth model can be trained as a whole based on the third training sample data. Without relying on manually labeled data, the parameters of the pre-trained first model, the second model, the third model and the fourth model can converge to a better range, thereby improving the robustness of the pre-trained first model, the second model, the third model and the fourth model.

[0219] Figure 4 A schematic diagram of an application scenario of the text processing method according to an embodiment of the present disclosure is shown. Figure 4 As shown, the application scenario includes a server 400. For the convenience of description, Figure 4 In the application scenario, only one server 400 is drawn. It should be understood that this example is only used as an example and is not a limitation of the present disclosure. The number and type of servers 400 in the present disclosure can be set according to actual needs, and the present disclosure does not make specific limitations on this. At the same time, the application scenario of the embodiment of the present disclosure will be described by taking a court trial as an example, but the present disclosure is not limited to this, but is also applicable to other application scenarios.

[0220] The server 400 may obtain the first text data (the indictment) and the second text data (the reply). In order to obtain the semantic information of the indictment and the reply, based on the indictment, the first model may be used to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}, where the first semantic vector sequence {e 1 ,e 2 ,…,e m} can be used to represent the semantic information of the indictment; based on the reply, the first model can be used to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}, where the second semantic vector sequence {d 1 ,d 2 ,…,d n} can be used to represent the semantic information of the reply statement.

[0221] According to an embodiment of the present disclosure, in order to obtain the contextual semantic information of the indictment and the reply, based on the first semantic vector sequence {e 1 ,e 2 ,…,e m}, the second model can be used to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}The corresponding third semantic vector sequence {q 1 ,q 2 ,…,q m}, where the third semantic vector sequence {q 1 ,q 2 ,…,q m} can be used to represent the contextual semantic information of the indictment; based on the second semantic vector sequence {d 1 ,d 2 ,…,d n}, the second model can be used to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}The fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, where the fourth semantic vector sequence {h 1 ,h 2 ,…,h n} can be used to represent the contextual semantic information of the reply statement.

[0222] According to an embodiment of the present disclosure, in order to obtain the interactive semantic information of the indictment and the reply, the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n} to perform splicing processing and obtain the splicing vector sequence {H 1 ,H 2 ,…,H r}. Based on the concatenated vector sequence {H 1 ,H 2 ,…,H r}, the third model can be used to obtain the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, where the fifth semantic vector sequence {V 1 ,V 2 ,…,V r} can be used to represent at least one or more of the following semantic information: contextual semantic information of the indictment, contextual semantic information of the reply, and interactive semantic information between the indictment and the reply.

[0223] According to an embodiment of the present disclosure, in order to effectively extract the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, the fourth model can be used to obtain the first aggregation vector v 1 The first aggregation vector v can be 1 Input to the fully connected layer to obtain the second aggregate vector v 2 , so that the first aggregation vector v 1 Mapped to the space of N dispute focal points in the legal knowledge graph. The second aggregation vector v 2 After the activation function is applied, the corresponding probability that the candidate dispute focus belongs to one of the N dispute focuses can be obtained, thereby determining K dispute focuses based on the above probability distribution.

[0224] The embodiments of the present disclosure can be applied to court trial application scenarios. Through the respective contextual semantic information and interactive semantic information of the indictment and the reply, the focus of dispute that can be recommended to the judge as a candidate is obtained, so that the obtained focus of dispute can correspond to the semantic information of the indictment and the reply, and the accuracy of the recommended focus of dispute can be improved.

[0225] Figure 5 1 is a flowchart showing a text processing method according to an embodiment of the present disclosure. Figure 5 As shown, the text processing method includes the following steps S501-S503:

[0226] In step S501, a first text and a second text having an associated relationship are obtained;

[0227] In step S502, a semantic vector sequence of the first text and a semantic vector sequence of the second text are obtained;

[0228] In step S503, interactive calculation and information aggregation are performed on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the association points between the first text and the second text.

[0229] According to an embodiment of the present disclosure, the first text and the second text contain content corresponding to each other; and / or the second text includes a response to at least part of the content in the first text.

[0230] According to an embodiment of the present disclosure, a first text and a second text having an associated relationship may be obtained, wherein the first text and the second text may contain content corresponding to each other, for example, containing the same or similar topics. Alternatively, the second text may include a response to at least part of the content in the first text, for example, the second text may include reply text data of the first text. For example, in a court trial application scenario, the first text may include the plaintiff's indictment, and the second text may include the defendant's statement of defense.

[0231] According to an embodiment of the present disclosure, the first text may be segmented to obtain a first segmentation set, wherein the first segmentation set includes the segmentation results of the first text. The present disclosure does not specifically limit the segmentation processing method, which may be selected according to actual needs.

[0232] According to an embodiment of the present disclosure, in order to obtain semantic information of the first text, the first model can be used to obtain a vector representation corresponding to each word in the first word set, and obtain a first semantic vector sequence {e 1 ,e 2 ,…e m}, where e i is the word vector of the i-th word segment, i=1-m, thereby introducing the semantic information of the first text through the first model. The present disclosure does not specifically limit the first model. As long as the model can realize the conversion of text data into an embedding vector, it is within the protection scope of the embodiments of the present disclosure. For example, the first model can include any one of the following models: Word2vector model, Glove model, Fasttext model or BERT (Bidirectional Encoder Representations from Transformers, bidirectional encoder representation based on transformer) model, etc.

[0233] According to an embodiment of the present disclosure, the second text may be segmented to obtain a second segmentation set, wherein the second segmentation set includes the segmentation results of the second text. The present disclosure does not specifically limit the segmentation processing method, and may be selected according to actual needs. In order to obtain the semantic information of the second text, the first model may be used to obtain a vector representation corresponding to each segmentation in the second segmentation set, and a second semantic vector sequence {d 1 ,d 2 ,…,d n}, where d j is the word vector of the jth word segment, j=1~n, thereby introducing the semantic information of the second text through the first model.

[0234] According to an embodiment of the present disclosure, in order to obtain contextual semantic information of the first text, the second model can be used to obtain contextual semantic information of the first text based on the first semantic vector sequence {e 1 ,e 2 ,…,e m}, by learning m vectors e i The internal association relationship between them is used to obtain the semantic vector sequence of the first text {q 1 ,q 2 ,…,q m}, where q i is a vector, thereby introducing the contextual semantic information of the first text through the second model. The second model can be a trained deep learning model, which is not specifically limited in the present disclosure and can be selected according to actual needs. For example, the second model can include any one of the following models: CNN model, RNN model, LSTM model, Transformer model, BERT model, attention model, etc., wherein the attention model can include a self-attention model (Self Attention), a multi-head attention model (Multi-Head Attention) or a HAN (Hierarchical Attention Networks, multi-layer attention) model, etc.

[0235] According to an embodiment of the present disclosure, in order to obtain contextual semantic information of the second text, a second model can be used to obtain contextual semantic information of the second text based on a second semantic vector sequence {d 1 ,d 2 ,…,d n}, by learning n vectors d j The internal association relationship between them is used to obtain the semantic vector sequence of the second text {h 1 ,h 2 ,…,h n}, where h jis a vector, thereby introducing the contextual semantic information of the second text through the second model.

[0236] According to an embodiment of the present disclosure, the semantic vector sequence {q 1 ,q 2 ,…,q m}, and a semantic vector sequence of the second text {h 1 ,h 2 ,…,h n}, perform interactive calculation and information aggregation to obtain interactive semantic information of the first text and the second text, and determine the connection point between the first text and the second text based on the interactive semantic information of the first text and the second text.

[0237] According to an embodiment of the present disclosure, the association point is associated with the first text and / or the second text; and / or the association point includes text related to the first text and / or the second text; and / or the association point is associated with texts in the first text and the second text that are controversial with each other.

[0238] According to an embodiment of the present disclosure, the association point is associated with the first text and / or the second text. For example, when the first text and the second text contain corresponding contents, the association point can be associated with the corresponding contents.

[0239] Alternatively, the association point may include text related to the first text and / or the second text. For example, when the second text includes a response to at least part of the content in the first text, the association point may include text data related to at least part of the above content and / or the response content.

[0240] Alternatively, the association point may include being associated with texts in the first text and the second text that are disputed with each other. For example, when the first text and the second text contain texts that are disputed with each other, the association point may be associated with the above-mentioned disputed texts.

[0241] For example, in a court trial application scenario, the first text may include the plaintiff's indictment, the second text may include the defendant's reply, and the related points may include multiple dispute points extracted from the indictment and the reply.

[0242] According to the technical solution provided by the embodiment of the present disclosure, a first text and a second text having an association relationship are obtained, and a semantic vector sequence of the first text and a semantic vector sequence of the second text are obtained. The association points of the first text and the second text are obtained by interactively calculating and aggregating information on the semantic vector sequence of the first text and the semantic vector sequence of the second text. The embodiment of the present disclosure obtains association points through the semantic information of the first text and the second text, so that the obtained association points can correspond to the semantic information of the first text and the second text, and the accuracy of the association points can be improved. The embodiment of the present disclosure can be applied to court trial application scenarios. Through the semantic information of the indictment and the reply, the focus of dispute that can be recommended to the judge as a candidate is obtained, so that the obtained focus of dispute can correspond to the semantic information of the indictment and the reply, and the accuracy of the recommended focus of dispute can be improved.

[0243] Figure 6 A schematic diagram is shown of interactively calculating and aggregating information on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain association points between the first text and the second text according to an embodiment of the present disclosure.

[0244] According to an embodiment of the present disclosure, the interactive calculation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0245] splicing the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain a splicing vector;

[0246] A semantic vector sequence of the concatenated vector is calculated.

[0247] like Figure 6 As shown, the semantic vector sequence {q 1 ,q 2 ,…,q m} and a semantic vector sequence of the second text {h 1 ,h 2 ,…,h n} to perform splicing processing and obtain the splicing vector {H 1 ,H 2 ,…,H r}, r=n+m.

[0248] According to an embodiment of the present disclosure, in order to obtain the interactive semantic information between the first text and the second text, the third model 601 can be used to obtain the interactive semantic information between the first text and the second text based on the concatenation vector {H 1 ,H 2 ,…,H r}, by learning r vectors H k The mutual correlation between them, k = 1 ~ r, obtain the semantic vector sequence of the concatenated vector {V 1 ,V 2 ,…,V r}, where the semantic vector sequence of the concatenated vector {V 1 ,V 2 ,…,V r} can be used to represent at least one or more of the following semantic information: contextual semantic information of the first text, contextual semantic information of the second text, and interactive semantic information between the first text and the second text. The third model can be a trained deep learning model, which is not specifically limited in the present disclosure and can be selected according to actual needs. For example, the third model can include any one of the following models: Transformer model, attention model, etc., wherein the attention model can include a self-attention model, a multi-head attention model or a HAN model, etc.

[0249] According to an embodiment of the present disclosure, the information aggregation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0250] Obtaining an aggregate vector according to the semantic vector sequence of the concatenated vector;

[0251] Obtaining probability distribution corresponding to a plurality of candidate association points according to the aggregation vector;

[0252] According to the probability distribution corresponding to the plurality of candidate association points, association points between the first text and the second text are obtained.

[0253] like Figure 6 As shown, in order to effectively extract the semantic vector sequence {V 1 ,V 2 ,…,V r}, we can use the fourth model to obtain r vectors V respectively. k In the semantic vector sequence of the concatenated vector {V 1 ,V 2 ,…,V r}, by calculating the corresponding weights of r vectors V k And the corresponding weights are weighted summed to obtain the aggregate vector v 1 The fourth model may be a trained attention model, which is not specifically limited in the present disclosure and may be selected according to actual needs. For example, the fourth model may include any one of the following models: a self-attention model, a multi-head attention model, or a HAN model.

[0254] According to the embodiments of the present disclosure, multiple candidate association points can be preset. For example, in a court trial application scenario, N dispute focuses can be determined on the topological structure nodes of the legal knowledge graph. For example, the dispute focuses such as "loan collection" and "joint debts of husband and wife" can be used as N candidate association points.

[0255] According to an embodiment of the present disclosure, the aggregate vector v including the interaction semantic information of the first text and the second text may be 1 , and establish a mapping relationship between the preset multiple candidate association points, so as to obtain the probability distribution corresponding to the multiple candidate association points, wherein the probability distribution represents the probability that the candidate association point belongs to the association point of the first text and the second text. The greater the corresponding probability of a candidate association point, the greater the possibility that the candidate association point belongs to the association point of the first text and the second text. For example, N candidate association points can be preset, and according to the aggregation vector v 1 , and obtain the probability distribution corresponding to N candidate association points. For example, in the court trial application scenario, N dispute points can be determined on the topological structure nodes of the legal knowledge graph, and the aggregation vector v 1 , and obtain the probability distribution corresponding to N controversial points.

[0256] According to an embodiment of the present disclosure, based on the probability distribution corresponding to multiple candidate association points, one or more of the candidate association points with larger probabilities can be determined as association points between the first text and the second text. For example, after obtaining the probability distribution corresponding to N candidate association points, the candidate association points corresponding to the largest K probabilities in the probability distribution can be determined as association points, where 1≤K≤N is an integer.

[0257] For example, in a court trial application scenario, the first text may include the plaintiff's indictment, and the second text may include the defendant's reply. The indictment and reply may be used to obtain the semantic vector sequence {V 1 ,V 2 ,…,V r}, and based on the semantic vector sequence {V 1 ,V 2 ,…,V r}Get the aggregate vector v 1 , according to the aggregation vector v 1 As well as the N dispute focuses of the legal knowledge graph, the probability distribution corresponding to the N dispute focuses is obtained, thereby determining the K connection points between the first text and the second text, that is, the K dispute focuses.

[0258] According to an embodiment of the present disclosure, obtaining a probability distribution corresponding to a plurality of candidate association points according to the aggregation vector includes:

[0259] The aggregated vector is input into the fully connected layer of the system model and is acted upon by an activation function to obtain a probability distribution corresponding to a plurality of candidate association points.

[0260] According to an embodiment of the present disclosure, the aggregation vector v 1 Input the fully connected layer of the system model to obtain the fully connected vector v 2 , where the fully connected vector v 2 The dimension of is the same as the number of candidate association points. For example, assuming there are N candidate association points, the fully connected vector v 2 The dimension is N. The fully connected vector v 2 After the activation function is applied, a probability distribution corresponding to multiple candidate association points is obtained. The present disclosure does not specifically limit the activation function, and it can be selected according to actual needs. For example, it can be a Sigmoid function, etc.

[0261] Since the text processing method in the embodiment of the present disclosure involves a system model, the system model may include the above-mentioned first model, second model, third model, fourth model and fully connected layer, etc., when pre-training the system model, the system model includes a large number of parameters, and most of the parameters are randomly initialized. When the technical field or application scenario to which the text processing method in the embodiment of the present disclosure is applied includes less training data, for example, when the text processing method in the embodiment of the present disclosure is applied to the application scenario of a court trial, since there is not much training data such as an indictment or a reply that can be obtained, when the system model is trained according to the randomly initialized parameters, it is difficult for the system model to converge or fail to obtain the expected training effect.

[0262] According to an embodiment of the present disclosure, the text processing method further includes:

[0263] Pre-training the system model;

[0264] The pre-training includes one or more of the following tasks: predicting whether two text segments are continuous texts, predicting masked parts of text, and predicting operations performed on text.

[0265] Figure 7 A schematic diagram of pre-training the system model according to an embodiment of the present disclosure is shown.

[0266] like Figure 7As shown, in order to enable the system model involved in the text processing method in the embodiment of the present invention to obtain better initialization parameters and improve the system model convergence efficiency and expected training effect, the system model can be pre-trained through one or more of the following tasks: the fourth task 701 is to predict whether two text segments are continuous texts, the fifth task 702 is to predict the obscured part of the text, and the sixth task 703 is to predict the operation performed on the text.

[0267] According to an embodiment of the present disclosure, the loss function of the pre-trained system model can be determined according to the loss function of each content in the at least one content mentioned above, for example, by summing the loss functions of each content to determine the loss function of the pre-trained system model. For example, assuming that the pre-training only includes the fourth task 701, the loss function of the pre-trained system model can be determined according to the loss function of the fourth task 701; for another example, assuming that the pre-training includes the fourth task 701, the fifth task 702 and the sixth task 703, the loss function of the pre-trained system model can be determined by summing the loss functions of the fourth task 701, the fifth task 702 and the sixth task 703. The parameters of the system model can be adjusted according to the loss function of the system model so that the system model can obtain better initialization parameters, thereby improving the convergence efficiency of the system model and the expected training effect.

[0268] According to an embodiment of the present disclosure, predicting whether two text segments are continuous texts includes:

[0269] Obtaining a first segment and a second segment in succession from a third text;

[0270] randomly selecting a third segment from the third text;

[0271] replacing the first segment or the second segment with the third segment;

[0272] Predict whether the third segment and the second segment, or the first segment and the third segment, are continuous text.

[0273] like Figure 7As shown, in order to achieve the fourth task 701, i.e., the training goal of predicting whether two text segments are continuous texts, the continuous first segment and the second segment (a1, b1) can be randomly obtained from the third text. The third segment c1 can be randomly selected in the third text with a certain probability, and the first segment a1 or the second segment b1 can be replaced with the third segment c1, so as to obtain two discontinuous text segments (c1, b1) or (a1, c1). The parameters of the system model can be adjusted by predicting whether the third segment and the second segment (c1, b1), or the first segment and the third segment (a1, c1) are continuous texts. For example, in a court trial application scenario, the third text may include an indictment, a reply or other legal documents (such as public judgment documents, etc.), thereby increasing the number of pre-training samples of the system model.

[0274] According to the technical solution provided by the embodiments of the present disclosure, based on the fourth task 701, i.e., predicting whether two text segments are continuous texts to train the overall model, the mutual correlation between the two segments of text can be obtained, for example, the mutual correlation between sentences.

[0275] According to an embodiment of the present disclosure, the masked portion of the predicted text includes:

[0276] Randomly selecting any text segment in the first text and / or the second text;

[0277] Processing the arbitrary text segment in a preset mask manner;

[0278] Predict the masked content in any processed text segment.

[0279] like Figure 7 As shown, in order to achieve the training goal of the fifth task 702, i.e., predicting the masked part of the content in the text, any text fragment can be randomly selected from the first text and / or the second text. For example, the first text includes an indictment A, and the second text includes a statement of defense B. A text fragment a2 can be randomly extracted from the indictment A, and a text fragment b2 can be randomly extracted from the statement of defense B. The text fragment a2 can be processed in a preset masking manner to obtain the processed arbitrary text fragment, for example, a preset proportion of word segments in the text fragment a2 is randomly selected with a certain probability, and a mask is used to replace the original word segmentation, so as to obtain the processed arbitrary text fragment a3. The text fragment b2 can be processed in the above-mentioned preset masking manner to obtain the processed arbitrary text fragment b3. The parameters of the system model can be adjusted by predicting the masked part of the processed arbitrary text fragment a3 and / or the processed arbitrary text fragment b3.

[0280] According to the technical solution provided by the embodiments of the present disclosure, based on the fifth task 702, i.e., training the overall model to predict the masked part of the content in the text, the mutual correlation between two word segments in the text and / or the mutual correlation between the word segment and the text can be obtained, for example, the mutual correlation between words and words, and / or the mutual correlation between words and sentences.

[0281] According to an embodiment of the present disclosure, the predicted operation performed on the text includes:

[0282] In the text composed of the first segment and the second segment, and / or the text composed of the first segment and the third segment, and / or the text composed of the third segment and the second segment, select any word to perform a specific operation;

[0283] The specific operation is predicted, where the specific operation includes deletion, replacement, and no operation.

[0284] like Figure 7 As shown, in order to achieve the training goal of the sixth task 703, i.e., predicting the operation performed on the text, any word can be selected to perform a specific operation in the text (a1, b1) composed of the first segment and the second segment, and / or the text (a1, c1) composed of the first segment and the third segment, and / or the text (c1, b1) composed of the third segment and the second segment, wherein the specific operation includes but is not limited to "delete", "replace" or "no operation". For example, the word segmentation in the text (a1, b1) composed of the first segment and the second segment, the text (a1, c1) composed of the first segment and the third segment, or the text (c1, b1) composed of the third segment and the second segment can be randomly deleted or replaced, or no operation can be performed. The parameters of the system model can be adjusted by predicting the above-mentioned specific operations.

[0285] According to the technical solution provided by the embodiments of the present disclosure, based on the sixth task 703, i.e., predicting the operations performed on the text, the overall model is trained. Without relying on manually labeled data, the parameters of the pre-trained overall model can be converged to a better range, thereby improving the robustness of the pre-trained overall model.

[0286] Figure 8 A schematic diagram of an application scenario of the text processing method according to an embodiment of the present disclosure is shown. Figure 8 As shown, the application scenario includes a server 800. For the convenience of description, Figure 8In the application scenario, only one server 800 is drawn. It should be understood that this example is only used for example and is not a limitation of the present disclosure. The number and type of servers 800 in the present disclosure can be set according to actual needs, and the present disclosure does not make specific limitations on this. At the same time, the application scenario of the embodiment of the present disclosure will be described by taking the court trial as an example, but the present disclosure is not limited to this, but is also applicable to other application scenarios.

[0287] The server 800 may obtain a first text (complaint) and a second text (reply) having an associated relationship. In order to obtain the semantic information of the complaint and the reply, based on the complaint, a first semantic vector sequence {e 1 ,e 2 ,…,e m}, where the first semantic vector sequence {e 1 ,e 2 ,…,e m} can be used to represent the semantic information of the indictment; based on the reply, the first model can be used to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}, where the second semantic vector sequence {d 1 ,d 2 ,…,d n} can be used to represent the semantic information of the reply statement.

[0288] According to an embodiment of the present disclosure, in order to obtain the contextual semantic information of the indictment and the reply, based on the first semantic vector sequence {e 1 ,e 2 ,…,e m}, the second model can be used to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}The semantic vector sequence of the first text corresponding to {q 1 ,q 2 ,…,q m}, where the semantic vector sequence of the first text {q 1 ,q 2 ,…,q m} can be used to represent the contextual semantic information of the indictment; based on the second semantic vector sequence {d 1 ,d 2 ,…,d n}, the second model can be used to obtain the second semantic vector sequence {d 1 ,d 2 ,…,d n}The semantic vector sequence of the second text corresponding to {h 1 ,h 2 ,…,h n}, where the semantic vector sequence of the second text {h 1 ,h 2 ,…,h n} can be used to represent the contextual semantic information of the reply statement.

[0289] According to an embodiment of the present disclosure, in order to obtain interactive semantic information of the indictment and the reply, the semantic vector sequence {q 1 ,q 2 ,…,q m} and the semantic vector sequence of the second text {h 1 ,h 2 ,…,h n} to perform splicing processing and obtain the splicing vector {H 1 ,H 2 ,…,H r}. Based on the splicing vector {H 1 ,H 2 ,…,H r}, the third model can be used to obtain the semantic vector sequence {V 1 ,V 2 ,…,V r}, where the semantic vector sequence of the concatenated vector {V 1 ,V 2 ,…,V r} can be used to represent at least one or more of the following semantic information: contextual semantic information of the indictment, contextual semantic information of the reply, and interactive semantic information between the indictment and the reply.

[0290] According to an embodiment of the present disclosure, in order to effectively extract the semantic vector sequence {V 1 ,V 2 ,…,V r}, we can use the fourth model to obtain the aggregate vector v 1 . The aggregation vector v can be 1 Input to the fully connected layer of the system model to obtain the fully connected vector v 2 , so that the aggregate vector v 1 Mapped to the space where N candidate association points (N dispute points) of the legal knowledge graph are located. Fully connected vector v 2 After the activation function is applied, the probability distribution corresponding to the N candidate association points (N dispute foci) can be obtained, and then K association points (K dispute foci) can be determined based on the above probability distribution.

[0291] The embodiments of the present disclosure can be applied to court trial application scenarios. Through the respective contextual semantic information and interactive semantic information of the indictment and the reply, the focus of dispute that can be recommended to the judge as a candidate is obtained, so that the obtained focus of dispute can correspond to the semantic information of the indictment and the reply, and the accuracy of the recommended focus of dispute can be improved.

[0292] Fig. 9 FIG. 1 is a block diagram showing a text processing device according to an embodiment of the present disclosure. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Fig. 9 As shown, the text processing device 900 includes a first acquisition module 910 , a second acquisition module 920 , a third acquisition module 930 , a fourth acquisition module 940 and a determination module 950 .

[0293] The first acquisition module 910 is configured to acquire, based on the first text data, a first semantic vector sequence {e 1 ,e 2 ,…,e m}, where e i is a vector, i=1~m;

[0294] The second acquisition module 920 is configured to acquire, based on the second text data, a second semantic vector sequence {d 1 ,d 2 ,…,d n}, where d j is a vector, j=1~n;

[0295] The third acquisition module 930 is configured to 1 ,e 2 ,…,e m}, using the processor to obtain the first semantic vector sequence {e 1 ,e 2 ,…,e m}The corresponding third semantic vector sequence {q 1 ,q 2 ,…,q m}, where q i is a vector;

[0296] The fourth acquisition module 940 is configured to: 1 ,d 2 ,…,d n}, using the second model to obtain the second semantic vector sequence {d1 ,d 2 ,…,d n}The fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, where h j is a vector;

[0297] The determining module 950 is configured to: 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data.

[0298] According to an embodiment of the present disclosure, the first text data and the second text data contain content corresponding to each other, and the content of the target text data is associated with the corresponding content in the first text data and the second text data; and / or

[0299] The second text data includes a response to at least part of the content in the first text data, and the target text data includes text data related to the first text data and / or the second text data; and / or

[0300] The target text data is associated with text data that is controversial between the first text data and the second text data.

[0301] According to an embodiment of the present disclosure, the third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n}, determine the target text data, including:

[0302] The third semantic vector sequence {q 1 ,q 2 ,…,q m} and the fourth semantic vector sequence {h 1 ,h 2 ,…,h n} to perform splicing processing and obtain the splicing vector sequence {H 1 ,H 2 ,…,H r}, where H k is a vector, r=n+m, k=1~r;

[0303] Based on the splicing vector sequence {H 1 ,H 2 ,…,H r}, through the third model, obtain the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, where V k is a vector;

[0304] According to the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determine the target text data.

[0305] According to an embodiment of the present disclosure, the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, determining the target text data, including:

[0306] Based on the fifth semantic vector sequence {V 1 ,V 2 ,…,V r}, through the fourth model, obtain the first aggregation vector v 1 ;

[0307] The first aggregation vector v 1 Input the fully connected layer to obtain the second aggregate vector v 2 ;

[0308] The second aggregation vector v 2 After the activation function is applied, a probability distribution corresponding to the N candidate target text data is obtained, wherein the probability distribution includes the probability corresponding to each of the N candidate target text data;

[0309] The target text data is determined according to the probability distribution.

[0310] According to an embodiment of the present disclosure, it also includes: a first pre-training module 960, which is configured to pre-train the first model, the second model, the third model and the fourth model as a whole through at least one of the following tasks: predicting whether two paragraphs of text are continuous text, predicting the masked part of the text, and predicting the operation performed on the text, wherein the overall loss function of the pre-training is determined according to the loss function of each task in the at least one task, and the parameters of the first model, the second model, the third model and the fourth model are adjusted according to the overall loss function.

[0311] According to an embodiment of the present disclosure, the predicting whether two paragraphs of text are continuous texts includes:

[0312] Acquire first training sample data, wherein the first training sample data includes first text sample data and second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0313] Based on the first training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0314] According to an embodiment of the present disclosure, the masked portion of the predicted text includes:

[0315] Acquire second training sample data, wherein the second training sample data includes first text mask data acquired by processing the first text data in a preset mask manner and / or second text mask data acquired by processing the second text data in the preset mask manner;

[0316] Based on the second training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0317] According to an embodiment of the present disclosure, the predicted operation performed on the text includes:

[0318] Acquire first training sample data, wherein the first training sample data includes first text sample data and / or second text sample data, the first text sample data includes two segments of continuous text sample data, and the second text sample data includes two segments of discontinuous text sample data;

[0319] Acquire third training sample data according to the first training sample data, wherein the third training sample data includes third text sample data acquired by performing a specific action on the first text sample data and / or fourth text sample data acquired by performing the specific action on the second text sample data;

[0320] Based on the third training sample data, the first model, the second model, the third model and the fourth model are trained as a whole.

[0321] According to an embodiment of the present disclosure, the first model includes any one of the following models: Word2vector model, Glove model, Fasttext model, BERT model.

[0322] According to an embodiment of the present disclosure, the second model includes any one of the following models: CNN model, RNN model, LSTM model, Transformer model, BERT model, and attention model.

[0323] According to an embodiment of the present disclosure, the third model includes any one of the following models: Transformer model, attention model.

[0324] According to an embodiment of the present disclosure, the fourth model includes an attention model.

[0325] Fig.10 FIG. 1 is a block diagram showing a text processing device according to an embodiment of the present disclosure. The device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Fig.10 As shown, the text processing device 1000 includes a fifth acquisition module 1010 , a sixth acquisition module 1020 and a seventh acquisition module 1030 .

[0326] The fifth acquisition module 1010 is configured to obtain a first text and a second text having an associated relationship;

[0327] The sixth acquisition module 1020 is configured to obtain a semantic vector sequence of the first text and a semantic vector sequence of the second text;

[0328] The seventh acquisition module 1030 is configured to perform interactive calculation and information aggregation on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the association points between the first text and the second text.

[0329] According to an embodiment of the present disclosure, the interactive calculation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0330] splicing the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain a splicing vector;

[0331] A semantic vector sequence of the concatenated vector is calculated.

[0332] According to an embodiment of the present disclosure, the information aggregation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes:

[0333] Obtaining an aggregate vector according to the semantic vector sequence of the concatenated vector;

[0334] Obtaining probability distribution corresponding to a plurality of candidate association points according to the aggregation vector;

[0335] According to the probability distribution corresponding to the plurality of candidate association points, association points between the first text and the second text are obtained.

[0336] According to an embodiment of the present disclosure, obtaining a probability distribution corresponding to a plurality of candidate association points according to the aggregation vector includes:

[0337] The aggregated vector is input into the fully connected layer of the system model and is acted upon by an activation function to obtain a probability distribution corresponding to a plurality of candidate association points.

[0338] According to an embodiment of the present disclosure, the device further includes:

[0339] A second pre-training module 1040 is configured to pre-train the system model;

[0340] The pre-training includes one or more of the following tasks: predicting whether two text segments are continuous texts, predicting masked parts of text, and predicting operations performed on text.

[0341] According to an embodiment of the present disclosure, predicting whether two text segments are continuous texts includes:

[0342] Obtaining a first segment and a second segment in succession from a third text;

[0343] randomly selecting a third segment from the third text;

[0344] replacing the first segment or the second segment with the third segment;

[0345] Predict whether the third segment and the second segment, or the first segment and the third segment, are continuous text.

[0346] According to an embodiment of the present disclosure, the masked portion of the predicted text includes:

[0347] Randomly selecting any text segment in the first text and / or the second text;

[0348] Processing the arbitrary text segment in a preset mask manner;

[0349] Predict the masked content in any processed text segment.

[0350] According to an embodiment of the present disclosure, the predicted operation performed on the text includes:

[0351] In the text composed of the first segment and the second segment, and / or the text composed of the first segment and the third segment, and / or the text composed of the third segment and the second segment, select any word to perform a specific operation;

[0352] The specific operation is predicted, where the specific operation includes deletion, replacement, and no operation.

[0353] The present disclosure also discloses an electronic device, Fig.11 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0354] like Fig.11 As shown, the electronic device 1100 includes a memory 1101 and a processor 1102; wherein,

[0355] The memory 1101 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 1102 to implement the method according to the embodiment of the present disclosure.

[0356] Fig.12 A schematic diagram showing the structure of a computer system suitable for implementing text processing according to an embodiment of the present disclosure is shown.

[0357] like Fig.12 As shown, the computer system 1200 includes a processing unit 1201, which can perform various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the system 1200 are also stored. The processing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0358] The following components are connected to the I / O interface 1205: an input part 1206 including a keyboard, a mouse, etc.; an output part 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1208 including a hard disk, etc.; and a communication part 1209 including a network interface card such as a LAN card, a modem, etc. The communication part 1209 performs communication processing via a network such as the Internet. The drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that the computer program read therefrom is installed into the storage part 1208 as needed. Among them, the processing unit 1201 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0359] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a readable medium thereof, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1209, and / or installed from the removable medium 1211.

[0360] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0361] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0362] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0363] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A text processing method, characterized in that: include: Obtaining a first text and a second text having an associated relationship, wherein the first text and the second text contain corresponding content to each other, and / or the second text includes a response to at least part of the content in the first text; Obtaining a semantic vector sequence of the first text and a semantic vector sequence of the second text; Interactive calculation and information aggregation are performed on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the association point between the first text and the second text, wherein the interactive calculation is used to splice the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the semantic vector sequence of the spliced ​​vector, and the semantic vector sequence of the spliced ​​vector includes: interactive semantic information between the first text and the second text, and the information aggregation is used to determine the probability distribution corresponding to the multiple candidate association points from the multiple candidate association points according to the semantic vector sequence of the spliced ​​vector, and determine the association point between the first text and the second text according to the probability distribution corresponding to the multiple candidate association points, and the association point between the first text and the second text is determined according to the interactive semantic information of the first text and the second text, and the probability distribution is used to represent the probability that the candidate association point belongs to the association point between the first text and the second text.

2. The method according to claim 1, characterized in that The interactive calculation of the semantic vector sequence of the first text and the semantic vector sequence of the second text includes: splicing the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain a spliced ​​vector; A semantic vector sequence of the concatenated vector is calculated.

3. The method according to claim 2, characterized in that The performing information aggregation on the semantic vector sequence of the first text and the semantic vector sequence of the second text includes: Performing weighted summation on the weights corresponding to the semantic vector sequence of the concatenated vector to obtain an aggregated vector; Obtaining probability distribution corresponding to a plurality of candidate association points according to the aggregation vector; According to the probability distribution corresponding to the plurality of candidate association points, the association points between the first text and the second text are obtained.

4. The method according to claim 3, characterized in that: The obtaining, according to the aggregation vector, a probability distribution corresponding to a plurality of candidate association points comprises: The aggregated vector is input into the fully connected layer of the system model and is acted upon by an activation function to obtain a probability distribution corresponding to a plurality of candidate association points.

5. The method according to claim 4, characterized in that The method further comprises: Pre-training the system model; The pre-training includes one or more of the following tasks: predicting whether two text segments are continuous texts, predicting masked parts of text, and predicting operations performed on text.

6. The method according to claim 5, characterized in that The predicting whether two text segments are continuous texts includes: Obtaining a first segment and a second segment in succession from a third text; randomly selecting a third segment from the third text; replacing the first segment or the second segment with the third segment; Predict whether the third segment and the second segment, or the first segment and the third segment, are continuous text.

7. The method according to claim 5, characterized in that The masked part of the predicted text includes: Randomly selecting any text segment in the first text and / or the second text; Processing the arbitrary text segment in a preset mask manner; Predict the masked content in any processed text segment.

8. The method according to claim 6, characterized in that The operations performed on the text by the prediction include: In the text composed of the first segment and the second segment, and / or the text composed of the first segment and the third segment, and / or the text composed of the third segment and the second segment, select any word to perform a specific operation; The specific operation is predicted, where the specific operation includes deletion, replacement, and no operation.

9. A text processing device, characterized in that: include: a fifth acquisition module, configured to obtain a first text and a second text having an associated relationship, wherein the first text and the second text contain content corresponding to each other, and / or the second text includes a response to at least part of the content in the first text; A sixth acquisition module is configured to obtain a semantic vector sequence of the first text and a semantic vector sequence of the second text; The seventh acquisition module is configured to perform interactive calculation and information aggregation on the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the association point between the first text and the second text, wherein the interactive calculation is used to splice the semantic vector sequence of the first text and the semantic vector sequence of the second text to obtain the semantic vector sequence of the spliced ​​vector, and the semantic vector sequence of the spliced ​​vector includes: interactive semantic information between the first text and the second text, the information aggregation is used to determine the probability distribution corresponding to the multiple candidate association points from the multiple candidate association points according to the semantic vector sequence of the spliced ​​vector, and determine the association point between the first text and the second text according to the probability distribution corresponding to the multiple candidate association points, the association point between the first text and the second text is determined according to the interactive semantic information of the first text and the second text, and the probability distribution is used to represent the probability that the candidate association point belongs to the association point between the first text and the second text.

10. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1-8.

11. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.

Images