Text relationship identification method, system, device and computer storage medium

Abstract

The application provides a kind of passage relationship identification method, system, equipment and computer storage medium, by obtaining the two texts of describing object;Two texts are input to pre-training language model BERT and are encoded, and the semantic representation of text is output;Two texts are carried out Brown clustering mapping, and Brown clustering vector is obtained;Brown clustering vector is input to the BERT model and is encoded, and Brown clustering semantic representation is output;The sum of the CLS vector in the semantic representation of text and the CLS vector in Brown clustering semantic representation is used to predict the passage relationship of two texts, and the passage relationship of two texts is obtained.In the present application, two texts and Brown clustering vector are encoded using pre-training language model BERT, and the sum of the CLS vector in the obtained semantic representation of text and the CLS vector in Brown clustering semantic representation is used to predict the passage relationship of two texts, so as to improve the accuracy of passage relationship identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a discourse relationship identification method, system, device and computer storage medium. BACKGROUND

[0002] The discourse relationship identification method refers to a task of identifying the discourse relationship between two given text segments. Specifically, two texts are input, for example: today the weather is very good, and yesterday it was still thunder and lightning. The discourse relationship between the two texts is predicted as: transition.

[0003] In the prior art, Brown clustering features are usually used to realize discourse relationship identification. However, the Brown clustering discourse relationship identification method adds Brown clustering as a discrete feature to a discourse relationship classifier, and the accuracy of discourse relationship identification is not high. SUMMARY

[0004] Therefore, the embodiments of the present application provide a discourse relationship identification method, system, device and computer storage medium to realize the purpose of improving the accuracy of discourse relationship identification.

[0005] To achieve the above object, the embodiments of the present application provide the following technical solutions.

[0006] The first aspect of the embodiments of the present application discloses a discourse relationship identification method, which comprises:

[0007] obtaining two texts describing an object;

[0008] inputting the two texts into a pre-trained language model BERT for encoding, and outputting a text semantic representation, wherein the text semantic representation at least comprises a classification CLS vector;

[0009] mapping the two texts by Brown clustering to obtain a Brown clustering vector;

[0010] inputting the Brown clustering vector into the pre-trained language model BERT for encoding, and outputting a Brown clustering semantic representation, wherein the Brown clustering semantic representation at least comprises a CLS vector;

[0011] predicting the discourse relationship of the two texts by using the sum of the CLS vector in the text semantic representation and the CLS vector in the Brown clustering semantic representation, and obtaining the discourse relationship of the two texts.

[0012] Optionally, the step of mapping the two texts by Brown clustering to obtain a Brown clustering vector comprises:

[0013] establishing a class index of Brown clustering pointing to the text for each of the texts;

[0014] obtaining a vector corresponding to each of the category indexes based on each of the category indexes of the Brown cluster;

[0015] obtaining a Brown cluster vector in combination with the vector corresponding to each of the category indexes.

[0016] Optionally, the obtaining of the vector corresponding to each of the category indexes based on each of the category indexes of the Brown cluster comprises:

[0017] obtaining a word vector with the same category index in the text based on each of the category indexes of the Brown cluster;

[0018] calculating a sum value of the word vector with the same category index in the text, and taking an average value of the sum value as the vector corresponding to each of the category indexes.

[0019] Optionally, the predicting of the discourse relationship of the two texts by using a sum value of a CLS vector in the text semantic representation and a CLS vector in the Brown cluster semantic representation comprises:

[0020] calculating the sum value of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation;

[0021] taking the sum value as an input of a Softmax of a logistic regression model to predict the discourse relationship, and outputting the discourse relationship of the two texts.

[0022] Optionally, the method further comprises:

[0023] taking the discourse relationship of the two texts and a standard category label as an input of a loss function to calculate a loss value, and obtaining the loss value of the discourse relationship of the two texts.

[0024] A second aspect of an embodiment of the present application discloses a discourse relationship identification system, and the system comprises:

[0025] an acquisition module configured to acquire two texts describing an object;

[0026] an encoding module configured to input the two texts into a pre-trained language model BERT to encode, and output a text semantic representation, the text semantic representation at least comprising a CLS vector, and input the Brown cluster vector into the pre-trained language model BERT to encode, and output a Brown cluster semantic representation, the Brown cluster semantic representation at least comprising a CLS vector;

[0027] a mapping module configured to perform Brown cluster mapping on the two texts to obtain a Brown cluster vector;

[0028] predicting the two texts' discourse relationship by using a sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation.

[0029] Optionally, the mapping module is specifically used for:

[0030] For each of the texts, a class index pointing to a Brown cluster of the text is established; based on the class index of each Brown cluster, a vector corresponding to each of the class indexes is obtained; and a Brown cluster vector is obtained by combining the vectors corresponding to each of the class indexes.

[0031] Optionally, the predicting module is specifically used for: calculating a sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation; and predicting the discourse relationship by taking the sum as an input of a Softmax of a logistic regression model, and outputting the two texts' discourse relationship.

[0032] A third aspect of the embodiment of the present application discloses an electronic device, which is used for running a program, wherein the program performs the discourse relationship identification method according to any one of the first aspect of the embodiment of the present application when running.

[0033] A fourth aspect of the embodiment of the present application discloses a computer storage medium, which comprises a program, wherein the program controls a device where the storage medium is located to perform the discourse relationship identification method according to any one of the first aspect of the embodiment of the present application when running.

[0034] Based on the above-mentioned discourse relationship identification method, system, device and computer storage medium provided by the embodiment of the present application, the method comprises: obtaining two texts describing an object; inputting the two texts into a pre-trained language model BERT for encoding, and outputting a text semantic representation, wherein the text semantic representation at least comprises a CLS vector; performing Brown cluster mapping on the two texts, and obtaining a Brown cluster vector; inputting the Brown cluster vector into the BERT model for encoding, and obtaining a Brown cluster semantic representation, wherein the Brown cluster semantic representation at least comprises a CLS vector; and predicting the two texts' discourse relationship by using a sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation, and obtaining the two texts' discourse relationship. In the present application, the pre-trained language model BERT is used to encode the two texts and the Brown cluster vector, and a sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation is used to predict the two texts' discourse relationship, thereby improving the accuracy of discourse relationship identification. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only are a part of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained based on the provided drawings without any creative effort.

[0036] Figure 1 A flowchart of a chapter relationship identification method provided by an embodiment of the present application is shown in FIG. 1.

[0037] Figure 2 A flowchart of obtaining a Brown clustering vector provided by an embodiment of the present application is shown in FIG. 2.

[0038] Figure 3 A flowchart of obtaining a vector corresponding to each category index provided by an embodiment of the present application is shown in FIG. 3.

[0039] Figure 4 A flowchart of obtaining a chapter relationship of two texts provided by an embodiment of the present application is shown in FIG. 4.

[0040] Figure 5 A flowchart of another chapter relationship identification method provided by an embodiment of the present application is shown in FIG. 5.

[0041] Figure 6 A structural diagram of a chapter relationship identification system provided by an embodiment of the present application is shown in FIG. 6.

[0042] Figure 7 A structural diagram of another chapter relationship identification system provided by an embodiment of the present application is shown in FIG. 7.

[0043] Figure 8 A structural diagram of an electronic device 80 provided by an embodiment of the present application is shown in FIG. 8.DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort fall within the scope of the present application.

[0045] In this application, the terms "comprising", "containing" or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0046] As can be known from the background art, when the existing Brown cluster chapter relationship recognition method is used to recognize the chapter relationship, the accuracy of chapter relationship recognition is not high.

[0047] Therefore, the embodiment of the present application provides a chapter relationship recognition method, system, device and computer storage medium, in the present scheme, the BERT (Bidirectional Encoder Representation from Transformers, pre-training language model) is used to encode two texts and Brown cluster vectors, and the sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation is used to predict the chapter relationship of the two texts, so as to improve the accuracy of chapter relationship recognition.

[0048] As Figure 1 shown, a flowchart of a chapter relationship recognition method provided by the embodiment of the present application is shown, which mainly includes the following steps:

[0049] Step S101: obtaining two texts describing an object.

[0050] In step S101, the object can be the same object or different objects, which is not limited by the present application.

[0051] In the process of implementing step S101, when the chapter relationship is recognized, it is necessary to determine the two texts for chapter relationship recognition, therefore, it is necessary to obtain the two texts describing the specific object.

[0052] Step S102: inputting the two texts into the pre-training language model BERT for encoding, and outputting the text semantic representation.

[0053] In step S102, the text semantic representation at least includes a classification CLS vector.

[0054] CLS (classification) is used for the classification task downstream. Specifically, for the text classification task, the BERT model inserts a [CLS] symbol in front of the text, and the output vector corresponding to the symbol is used as the semantic representation of the whole text, which is used for text classification.

[0055] The pre-trained language model BERT is a bidirectional conversion decoding model, which can utilize information of a previous word and information of a subsequent word.

[0056] The text semantic representation can be: wherein N is the total length of the two texts, that is, the number of words contained in the two texts, is a [CLS] vector.

[0057] In the process of specifically implementing the step S102, the obtained two texts are taken as inputs of the pre-trained language model BERT and are input into the pre-trained language model BERT for encoding to obtain the text semantic representation and output the text semantic representation.

[0058] Step S103: performing Brown clustering mapping on the two texts to obtain a Brown clustering vector.

[0059] In the process of specifically implementing the step S103, the obtained two texts are mapped to obtain a Brown clustering vector pointing to the two texts, and the Brown clustering vector is obtained based on the class indexes of the Brown clusters corresponding to all the two texts.

[0060] Optionally, the process of performing the step S103 to map the two texts to obtain the Brown clustering vector is as shown in Figure 2 which is a flowchart for obtaining a Brown clustering vector provided by an embodiment of the present application, mainly including the following steps:

[0061] Step S201: establishing, for each text, a class index of a Brown cluster pointing to the text.

[0062] In the process of specifically implementing the step S201, when establishing, for each text, the class index of the Brown cluster pointing to the text, the class index of the Brown cluster of the text needs to be introduced first, and then the class index of the Brown cluster pointing to each text is determined, so that the class index of the Brown cluster pointing to the text can be established.

[0063] Step S202: obtaining, based on the class index of each Brown cluster, a vector corresponding to each class index.

[0064] In the process of specifically implementing the step S202, according to the class index of each Brown cluster, a word vector with the same class index in the text needs to be obtained, and according to the word vector with the same class index in the text, a vector corresponding to each class index is obtained.

[0065] Optionally, the process of performing the step S202 to obtain, based on the class index of each Brown cluster, a vector corresponding to each class index is as shown in Figure 3As shown, a flowchart for obtaining a vector corresponding to each category index is provided, and mainly includes the following steps:

[0066] Step S301: Based on the category index of each Brown cluster, a word vector with the same category index in the text is obtained.

[0067] In the process of implementing step S301, according to the category index of each Brown cluster, the word vector with the same category index existing in the text is found, and the word vector with the same category index in the text is obtained.

[0068] For example, there are three words in the text, namely you, I, she and he, and the category index of the Brown cluster corresponding to you, I and he is 1, and the category index of the Brown cluster corresponding to she is 2. The words with the same category index found in this text are you, I and he, and the word vectors corresponding to you, I and he are e (you), e (I) and e (he) respectively.

[0069] Step S302: Calculate the sum value of the word vectors with the same category index in the text, and take the average value of the sum value as the vector corresponding to each category index.

[0070] In the process of implementing step S302, the word vectors with the same category index in the text are added to obtain a specific numerical value, and the average value of the numerical value is calculated, which is taken as the vector corresponding to each category index.

[0071] Taking the example in step S301, the word vectors of the words (you, I and he) with the same category index in the text are obtained, the three word vectors are added to obtain a specific numerical value, and the average value of the numerical value is calculated, which is taken as the vector corresponding to each category index.

[0072] Step S203: Combine the vector corresponding to each category index to obtain a Brown cluster vector.

[0073] In the process of implementing step S203, the vectors corresponding to each category index obtained are connected to obtain a Brown cluster vector.

[0074] Step S104: Input the Brown cluster vector into the pre-trained language model BERT for encoding, and output the Brown cluster semantic representation.

[0075] In step S104, the Brown cluster semantic representation at least includes a CLS vector.

[0076] The Brown cluster semantic representation can be: where N is the total length of the two texts, i.e. the number of words contained in the two texts, The [CLS] vector.

[0077] In the implementation of step S104, the obtained Brown clustering vector is taken as the pre-training language model BE Brown clustering semantic representation of the text semantic representation, and the Brown clustering semantic representation is output.

[0078] Step S105: predicting the discourse relationship between the two texts by using the sum of the [CLS] vector in the text semantic representation and the [CLS] vector in the Brown clustering semantic representation, to obtain the discourse relationship between the two texts.

[0079] In the implementation of step S105, based on the obtained [CLS] vector in the text semantic representation and the [CLS] vector in the Brown clustering semantic representation, the [CLS] vector in the text semantic representation and the [CLS] vector in the Brown clustering semantic representation are added to obtain a specific value, and the value is used to predict the discourse relationship between the two texts, to obtain the final discourse relationship between the two texts.

[0080] For example, the [CLS] vector in the text semantic representation is , the [CLS] vector in the Brown clustering semantic representation is , the sum of and is added to obtain a specific value, which is denoted as , and is used to predict the discourse relationship between the two texts, to obtain the final discourse relationship between the two texts .

[0081] Optionally, the process of step S105 of predicting the discourse relationship between the two texts by using the sum of the [CLS] vector in the text semantic representation and the [CLS] vector in the Brown clustering semantic representation, to obtain the discourse relationship between the two texts, is as shown in Figure 4 , which is a flowchart of obtaining the discourse relationship between the two texts provided by the embodiment of the present application, mainly comprising the following steps:

[0082] Step S401: calculating the sum of the [CLS] vector in the text semantic representation and the [CLS] vector in the Brown clustering semantic representation.

[0083] In the implementation of step S401, the [CLS] vector in the text semantic representation and the [CLS] vector in the Brown clustering semantic representation are added to obtain a specific value.

[0084] Step S402: taking the sum as the input of the Softmax of the logistic regression model to predict the discourse relationship, and outputting the discourse relationship between the two texts.

[0085] In the implementation of step S402, the obtained specific numerical value is taken as the input of the logistic regression model Softmax, and is input into the logistic regression model Softmax to predict the text relation, so as to obtain the text relation of the two texts and output the text relation of the two texts.

[0086] Taking the example of step S105, as the input of the logistic regression model Softmax, and is input into the logistic regression model Softmax to predict the text relation, so as to obtain the text relation of the two texts and output the text relation of the two texts .

[0087] Based on the method for identifying text relation provided in the embodiment of the application, two texts describing an object are obtained; the two texts are input into a pre-trained language model BERT for encoding, and a text semantic representation is output, the text semantic representation at least including a CLS vector; Brown clustering mapping is performed on the two texts, and a Brown clustering vector is obtained; the Brown clustering vector is input into the BERT model for encoding, and a Brown clustering semantic representation is obtained, the Brown clustering semantic representation at least including a CLS vector; the sum of the CLS vector in the text semantic representation and the CLS vector in the Brown clustering semantic representation is used to predict the text relation of the two texts, and the text relation of the two texts is obtained. In the application, the pre-trained language model BERT is used to encode the two texts and the Brown clustering vector, and the sum of the CLS vector in the obtained text semantic representation and the CLS vector in the Brown clustering semantic representation is used to predict the text relation of the two texts, so that the accuracy of the text relation identification is improved.

[0088] Based on the above-mentioned embodiment of the application Figure 1 The method for identifying text relation shown in the embodiment of the application is another method for identifying text relation provided in the embodiment of the application, and mainly includes the following steps: Figure 5

[0089] Step S501: obtaining two texts describing an object.

[0090] Step S502: inputting the two texts into a pre-trained language model BERT for encoding, and outputting a text semantic representation.

[0091] Step S503: performing Brown clustering mapping on the two texts, and obtaining a Brown clustering vector.

[0092] Step S504: inputting the Brown clustering vector into the pre-trained language model BERT for encoding, and outputting a Brown clustering semantic representation.

[0093] ​Step S505: predicting the discourse relationship of the two texts by using the sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation, to obtain the discourse relationship of the two texts.

[0094] The execution principles and processes of steps S501 to S505 are the same as those of steps S101 to S105 disclosed in Figure 1 , and details are not described herein again.

[0095] Step S506: taking the discourse relationship of the two texts and the standard category label as inputs of a loss function to calculate a loss value, to obtain the loss value of the discourse relationship of the two texts.

[0096] In the process of implementing step S506, the discourse relationship of the two texts and the standard category label are taken as inputs of a loss function, and are input into the loss function to calculate a loss value, to obtain the loss value of the discourse relationship of the two texts.

[0097] Based on the method for identifying the discourse relationship provided in the embodiment of the application, two texts describing an object are obtained; the two texts are input into a pre-trained language model BERT for encoding, to output a text semantic representation, which at least includes a CLS vector; Brown cluster mapping is performed on the two texts, to obtain a Brown cluster vector; the Brown cluster vector is input into the BERT model for encoding, to obtain a Brown cluster semantic representation, which at least includes a CLS vector; and the sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation is used to predict the discourse relationship of the two texts, to obtain the discourse relationship of the two texts. In the application, the pre-trained language model BERT is used to encode the two texts and the Brown cluster vector, and the sum of the CLS vector in the obtained text semantic representation and the CLS vector in the Brown cluster semantic representation is used to predict the discourse relationship of the two texts, and the obtained discourse relationship of the two texts is further subjected to loss value calculation, so as to measure the error degree or loss degree of the discourse relationship of the two texts, thereby improving the accuracy of the discourse relationship identification.

[0098] In order to better understand the method for identifying the discourse relationship provided in the above embodiment of the application, examples are given below.

[0099] For example, discourse 1 is “today the weather is very good”, and discourse 2 is “but I am in a bad mood”.

[0100] First, discourse 1 and discourse 2 are obtained, and are input into the pre-trained language model BERT for encoding, to obtain the semantic representation of discourse 1 and discourse 2 , and output the text semantic representation , wherein, is the CLS vector.

[0101] Next, the class indexes of the Brown clusters pointing to the paragraph 1 and the paragraph 2 are respectively established.

[0102] Suppose the class indexes of the Brown clusters corresponding to each word in the paragraph 1 are 1, 2, 2, 3, 4, 5 respectively, and the class indexes of the Brown clusters corresponding to each word in the paragraph 2 are 1, 2, 3, 4, 5, 6, 5 respectively.

[0103] According to the class indexes of each Brown cluster, the word vectors with the same class indexes existing in the paragraph 1 and the paragraph 2 are searched, and the words with the same class indexes searched are: today and but, day, day and is, air and I, very and heart, good, emotion and good, so the word vectors with the same class indexes in the paragraph 1 and the paragraph 2 are e (today) and e (but), e (day), e (day) and e (is), e (air) and e (I), e (very) and e (heart), e (good), e (emotion) and e (good).

[0104] The sum values of the word vectors with the same class indexes in the paragraph 1 and the paragraph 2 are calculated, and the average of the sum values is taken as the vector corresponding to each class index, which is:

[0105] The vector corresponding to the class index 1 is .

[0106] The vector corresponding to the class index 2 is .

[0107] The vector corresponding to the class index 3 is .

[0108] The vector corresponding to the class index 4 is .

[0109] The vector corresponding to the class index 5 is .

[0110] The vector corresponding to the class index 6 is .

[0111] The Brown cluster vector is obtained by combining the vector corresponding to each class index, which is: connecting the vector corresponding to each class index to obtain the Brown cluster vector .

[0112] The Brown cluster vector is input into the pre-trained language model BERT for encoding to obtain the Brown cluster semantic representation , and the Brown cluster semantic representation is output, wherein is the CLS vector.

[0113] The CLS vector in the text semantic representation is added to the CLS vector in the Brown cluster semantic representation, that is , and The prediction of the discourse relationship is taken as the input of the Softmax of the logistic regression model, and the discourse relationship of discourse 1 and discourse 2 is obtained as transition, and the discourse relationship of discourse 1 and discourse 2 is output.

[0114] According to the discourse relationship identification method provided in the embodiment of the application, the pre-trained language model BERT is used to encode the discourse 1 and the discourse 2 and the Brown cluster vector, and the sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation is used to predict the discourse relationship of the two texts, so that the accuracy of the discourse relationship identification is improved.

[0115] Corresponding to the above-mentioned embodiment of the application, the embodiment of the application also provides a discourse relationship identification system, as shown in Figure 6 The discourse relationship identification system comprises an acquisition module 61, an encoding module 62, a mapping module 63 and a prediction module 64.

[0116] The acquisition module 61 is used to acquire two texts describing an object.

[0117] The encoding module 62 is used to input the two texts into the pre-trained language model BERT for encoding, and output a text semantic representation, wherein the text semantic representation at least comprises a CLS vector, and input a Brown cluster vector into the pre-trained language model BERT for encoding, and output a Brown cluster semantic representation, wherein the Brown cluster semantic representation at least comprises a CLS vector.

[0118] The mapping module 63 is used to perform Brown cluster mapping on the two texts, and obtain a Brown cluster vector.

[0119] The prediction module 64 is used to use the sum of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation to predict the discourse relationship of the two texts, and obtain the discourse relationship of the two texts.

[0120] It should be noted that the specific principles and execution processes of each module in the above-mentioned embodiment of the application are the same as those of the above-mentioned embodiment of the application, and can be referred to the corresponding part of the above-mentioned embodiment of the application, which will not be repeated here.

[0121] The embodiment of the present application provides a kind of chapter relationship identification system, by obtaining the two texts of description object;Two texts are input to pre-training language model BERT and are encoded, and the output text semantic representation, text semantic representation at least includes CLS vector;Two texts are carried out Brown clustering mapping, and Brown clustering vector is obtained;Brown clustering vector is input to BERT model and is encoded, and Brown clustering semantic representation is obtained, and Brown clustering semantic representation at least includes CLS vector;The sum of CLS vector in text semantic representation and CLS vector in Brown clustering semantic representation is used to predict the chapter relationship of two texts, and the chapter relationship of two texts is obtained.In the present application, pre-training language model BERT is used to encode two texts and Brown clustering vector, and the sum of CLS vector in text semantic representation and CLS vector in Brown clustering semantic representation is used to predict the chapter relationship of two texts, to improve the accuracy of chapter relationship identification.

[0122] Optionally, based on the above Figure 6 The mapping module 63 shown is specifically used for:

[0123] For each text, the category index of the Brown cluster pointing to the text is established;Based on the category index of each Brown cluster, the vector corresponding to each category index is obtained;The vector corresponding to each category index is combined to obtain the Brown cluster vector.

[0124] The embodiment of the present application provides a kind of chapter relationship identification system, by carrying out Brown clustering mapping to two texts, and Brown clustering vector is obtained, to improve the accuracy of chapter relationship identification.

[0125] Optionally, based on the above Figure 6 The mapping module 63 shown is specifically used for:

[0126] Based on the category index of each Brown cluster, the word vector with the same category index in the text is obtained;The sum of the word vector with the same category index in the text is calculated, and the average of the sum is used as the vector corresponding to each category index.

[0127] The embodiment of the present application provides a kind of chapter relationship identification system, and the vector corresponding to each category index is obtained using the category index of each Brown cluster, to improve the accuracy of chapter relationship identification.

[0128] Optionally, based on the above Figure 6 The prediction module 64 shown is specifically used for:

[0129] The sum value of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation is calculated; and the sum value is taken as an input of a Softmax of a logistic regression model to predict the discourse relationship, and the discourse relationship of the two texts is output.

[0130] The system for identifying discourse relationship provided in the embodiment of the present application predicts the discourse relationship of the two texts by using the sum value of the CLS vector in the text semantic representation and the CLS vector in the Brown cluster semantic representation, thereby improving the accuracy of the discourse relationship identification.

[0131] Based on the above Figure 6 The system for identifying discourse relationship shown in the embodiment of the present application combines Figure 6 As shown in the embodiment of the present application, Figure 7 The system for identifying discourse relationship further comprises a loss value calculation module 65.

[0132] The loss value calculation module 65 is configured to calculate the loss value of the discourse relationship of the two texts by taking the discourse relationship of the two texts and the standard category label as inputs of a loss function, thereby obtaining the loss value of the discourse relationship of the two texts.

[0133] The system for identifying discourse relationship provided in the embodiment of the present application calculates the loss value of the discourse relationship of the two texts, and further measures the error degree or loss degree of the discourse relationship of the two texts, thereby improving the accuracy of the discourse relationship identification.

[0134] Based on the system for identifying discourse relationship disclosed in the embodiment of the present application, each module can be realized by a hardware device composed of a processor and a memory. Specifically, each module is stored in the memory as a program unit, and the processor executes the program unit stored in the memory to realize the discourse relationship identification.

[0135] The processor comprises a core, and the core retrieves the corresponding program unit from the memory. One or more cores can be provided, and the core parameters are adjusted to realize the discourse relationship identification.

[0136] The embodiment of the present application provides a computer storage medium, and the storage medium stores a program for identifying discourse relationship. When the program is executed by a processor, the program realizes Figure 1 The disclosed method for identifying discourse relationship.

[0137] The embodiment of the present application provides a processor, and the processor is used to run a program. When the program is run, the program executes Figure 1 The disclosed method for identifying discourse relationship.

[0138] The embodiment of the present application provides an electronic device 80, as shown in the embodiment of the present application, Figure 8 As shown in the embodiment of the present application, FIG. 1 is a structural schematic diagram of an electronic device 80 provided in the embodiment of the present application.

[0139] The electronic device in the embodiment of the application can be a server, a PC, a PAD, a mobile phone, etc.

[0140] The electronic device comprises at least one processor 801, at least one memory 802 connected with the processor, and a bus 803.

[0141] The processor 801 and the memory 802 complete mutual communication through the bus 803. The processor 901 is used for executing a program stored in the memory 802.

[0142] The memory 802 is used for storing a program, and the program is at least used for: obtaining two texts describing an object; inputting the two texts into a pre-trained language model BERT for encoding to output a text semantic representation; performing Brown clustering mapping on the two texts to obtain a Brown clustering vector; inputting the Brown clustering vector into the pre-trained language model BERT for encoding to output a Brown clustering semantic representation; and predicting a chapter relationship of the two texts by using a sum value of a CLS vector in the text semantic representation and a CLS vector in the Brown clustering semantic representation, to obtain the chapter relationship of the two texts.

[0143] The application also provides a computer program product suitable for executing a program of the following method steps when executed on an electronic device:

[0144] obtaining two texts describing an object; inputting the two texts into a pre-trained language model BERT for encoding to output a text semantic representation; performing Brown clustering mapping on the two texts to obtain a Brown clustering vector; inputting the Brown clustering vector into the pre-trained language model BERT for encoding to output a Brown clustering semantic representation; and predicting a chapter relationship of the two texts by using a sum value of a CLS vector in the text semantic representation and a CLS vector in the Brown clustering semantic representation, to obtain the chapter relationship of the two texts.

[0145] The application is described with reference to flowcharts and / or block diagrams according to the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks

[0146] In one typical arrangement, a device includes one or more processors (CPUs), memory, and a bus. The device can also include input / output interfaces, network interfaces, and the like.

[0147] The memory can include non-persistent memory in the form of random access memory (RAM) and / or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM), including at least one memory chip. The memory is an example of computer readable media.

[0148] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0149] Each of the embodiments described in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, it is described relatively simply, and the relevant part can be referred to the part of the method embodiment. The above described system and system embodiment is only illustrative, and the unit described as a separate component can be or can not be physically separated, and the component displayed as a unit can be or can not be a physical unit, that is, it can be located in one place, or it can be distributed to multiple network units. According to the actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0150] Those skilled in the art will further realize that the mechanisms of the various examples described herein are capable of being implemented using any number of combinations of the described features. Accordingly, these examples are not limited to the mechanisms described herein, but rather, the intent is to cover all modifications and alternatives equivalent thereto. The preceding description of the examples is illustrative, and not restrictive. Many other examples will be apparent to those of skill in the art upon reviewing the above description. The scope of the examples should, therefore, be determined not with reference to the above description, but instead should be given to the appended claims, along with their full scope of equivalents.

[0151] The above description of disclosed examples allows one of ordinary skill in the art to make and use the application. Modifications of those examples will be obvious to those of ordinary skill in the art, and the generic principles defined herein can be applied to other examples without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the examples shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying textual relationships, characterized in that, The method includes: Retrieve the two text segments describing the object; The two text segments are input into the pre-trained language model BERT for encoding, and the output text semantic representation includes at least a classification CLS vector. Perform Brownian clustering mapping on the two text segments to obtain Brownian clustering vectors; The Brownian clustering vector is input into the pre-trained language model BERT for encoding, and the Brownian clustering semantic representation is output, wherein the Brownian clustering semantic representation includes at least the CLS vector; The relationship between the two text segments is predicted by using the sum of the CLS vectors in the text semantic representation and the CLS vectors in the Brownian clustering semantic representation. The step of performing Brownian clustering mapping on the two text segments to obtain Brownian clustering vectors includes: For each segment of the text, create a Brownian clustering category index pointing to the text; Based on the category index of each Brownian cluster, a vector corresponding to each category index is obtained; By combining the vector corresponding to each category index, a Brownian clustering vector is obtained; The step of obtaining the vector corresponding to each category index based on each Brownian cluster includes: Based on the category index of each Brownian cluster, word vectors with the same category index in the text are derived; Calculate the sum of word vectors with the same category index in the text, and use the average of the sums as the vector corresponding to each category index.

2. The method according to claim 1, characterized in that, The step of predicting the discourse relationship between the two text segments by using the sum of the CLS vectors in the text semantic representation and the CLS vectors in the Brownian clustering semantic representation, and obtaining the discourse relationship between the two text segments, includes: Calculate the sum of the CLS vector in the text semantic representation and the CLS vector in the Brownian clustering semantic representation; The sum is used as input to the logistic regression model Softmax to predict the discourse relationship, and the discourse relationship between the two text segments is output.

3. The method according to claim 1, characterized in that, Also includes: The loss value of the textual relationship between the two text segments is calculated by using the textual relationship between the two text segments and the standard category label as input to the loss function.

4. A text relationship recognition system, characterized in that, The system includes: The acquisition module is used to retrieve two pieces of text describing the object; The encoding module inputs the two text segments into the pre-trained language model BERT for encoding and outputs a text semantic representation, which includes at least a CLS vector. The module also inputs the Brown clustering vector into the pre-trained language model BERT for encoding and outputs a Brown clustering semantic representation, which includes at least a CLS vector. The mapping module is used to perform Brownian clustering mapping on the two text segments to obtain Brownian clustering vectors; The prediction module is used to predict the discourse relationship between the two text segments by using the sum of the CLS vectors in the text semantic representation and the CLS vectors in the Brownian clustering semantic representation, thereby obtaining the discourse relationship between the two text segments. Specifically, the mapping module is used for: For each segment of text, a category index for Brownian clustering pointing to the text is established; based on the category index of each Brownian cluster, a vector corresponding to each category index is obtained; combining the vector corresponding to each category index, a Brownian cluster vector is obtained. The mapping module for obtaining the vector corresponding to each category index based on the category index of each Brownian cluster is specifically used for: Based on the category index of each Brownian cluster, word vectors with the same category index in the text are obtained; the sum of word vectors with the same category index in the text is calculated, and the average of the sums is used as the vector corresponding to each category index.

5. The system according to claim 4, characterized in that, The prediction module is specifically used to: calculate the sum of the CLS vector in the text semantic representation and the CLS vector in the Brownian clustering semantic representation; use the sum as the input of the logistic regression model Softmax to predict the text relationship, and output the text relationship between the two text segments.

6. An electronic device, characterized in that, The electronic device is used to run a program, wherein the program executes the document relationship recognition method as described in any one of claims 1 to 3.

7. A computer storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the chapter relationship identification method as described in any one of claims 1 to 3.

Images