Zero-reference resolution method, device, equipment and medium based on big data

Abstract

The present invention discloses a method for zero-reference resolution based on big data, the method comprising: performing vectorization processing on a sentence to be resolved and its preceding information to obtain a context vector representation of each word in the sentence to be resolved and the preceding information; inputting the context vector representation of each word in the sentence to be resolved and the preceding information into a bidirectional long short-term memory network to obtain an enhanced context vector representation of each word; traversing the enhanced context vector representation of each word, predicting the probability of the first word of the anaphoric item and the probability of the last word of the anaphoric item of each word according to the parameter vector in the BERT model; traversing each word, constructing a continuous text segment, and calculating the probability of the anaphoric item of the continuous text segment; selecting the continuous text segment with the largest probability of the anaphoric item as the anaphoric item of the sentence to be resolved. The present invention solves the problem that the existing zero-reference resolution technology is overly dependent on the candidate set of anaphoric items, and the resolution result is low in accuracy and unstable.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method, device, equipment and medium for zero reference resolution based on big data. Background Art

[0002] Co-reference resolution is one of the technologies in the field of natural language that has been studied for the longest time and has a wide range of application scenarios. Co-reference resolution is one of the most core technologies in customer service robots, conversational robots, and intelligent outbound call platforms. Co-reference resolution includes zero-reference resolution and coreference resolution.

[0003] In pronoun-deficient languages ​​such as Chinese, the part that can be inferred from the context is often omitted, and the omitted part plays a corresponding syntactic component in the sentence and refers back to a language unit in the previous text. The omitted part is called a zero pronoun. Zero pronoun resolution is to find the corresponding language unit in the previous text for the zero pronoun. The zero pronoun resolution task is usually divided into two subtasks: zero pronoun position detection and resolution.

[0004] The purpose of the resolution task is to identify the specific anaphora for the zero pronoun with the preceding anaphora based on the zero pronoun position detection results. The traditional resolution model usually constructs a set of anaphora candidates first, and then uses classification or sorting methods to select the most likely candidate from the set of anaphora candidates as the final recognition result. The construction of the anaphora candidate set is often composed of the largest noun phrase and the modifying noun phrase in the two sentences before the zero pronoun. The accuracy of this approach is highly dependent on the accuracy of the anaphora candidate set. If the set does not contain the correct anaphora, it will inevitably lead to subsequent recognition failure. Since the anaphora candidate set consists of only a few simple noun phrases, this makes the traditional resolution method highly unstable and less accurate.

[0005] Therefore, finding a method to solve the problem that the existing zero-reference resolution technology is too dependent on the candidate set of anaphoric terms and the resolution results are low in accuracy and unstable has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0006] The embodiments of the present invention provide a method, device, equipment and medium for zero-reference resolution based on big data to solve the problems that the existing zero-reference resolution technology is overly dependent on the candidate set of anaphoric items and the resolution results are low in accuracy and unstable.

[0007] A zero-reference resolution method based on big data, comprising:

[0008] Obtaining a sentence to be resolved and its context information, performing vectorization processing on the sentence to be resolved and its context information, and obtaining a context vector representation of each word in the sentence to be resolved and a context vector representation of each word in the context information;

[0009] Inputting the context vector representation of each word in the sentence to be resolved and the above information into a bidirectional long short-term memory network to enhance the context expression and position information of each word, and obtain an enhanced context vector representation of each word;

[0010] Traverse the enhanced context vector representation of each word, and predict the probability of the first word and the last word of the anaphoric item of each word according to the parameter vector in the BERT model;

[0011] Traversing each character, constructing a continuous text segment, and calculating the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each character;

[0012] A continuous text segment with the highest probability of an anaphora item is selected from the continuous text segments as the anaphora item of the sentence to be resolved.

[0013] Furthermore, performing vectorization processing on the sentence to be resolved and its context information to obtain a context vector representation of each word in the sentence to be resolved and a context vector representation of each word in the context information includes:

[0014] Representing each word in the sentence to be resolved and its context information in a one-hot form to obtain a high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and a high-dimensional discrete word representation matrix corresponding to its context information;

[0015] The word embedding method is used to embed the high-dimensional discrete word representation matrix corresponding to the sentence to be decomposed and the high-dimensional discrete word representation matrix corresponding to the above information into a low-dimensional dense representation matrix;

[0016] The low-dimensional dense representation matrix corresponding to the sentence to be resolved and its context information is input into the preset BERT model for bidirectional encoding to obtain the context vector representation of each word in the sentence to be resolved and its context information.

[0017] Further, the traversing the enhanced context vector representation of each word and predicting the probability of the first word of the anaphoric item and the probability of the last word of the anaphoric item of each word according to the parameter vector in the BERT model includes:

[0018] Get the first word parameter vector and the last word parameter vector in the BERT model;

[0019] Performing a dot product operation on the enhanced context vector representation of each word and the head word parameter vector, and performing a softmax process on the dot product operation result to obtain the head word probability of each word;

[0020] A dot product operation is performed on the enhanced context vector representation of each word and the tail word parameter vector, and a softmax process is performed on the dot product operation result to obtain the anaphoric tail word probability of each word.

[0021] Furthermore, the traversing each character, constructing a continuous text segment, and calculating the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each character includes:

[0022] Traverse each word, use the word as the head word of the anaphora item, use the word and the following words as the tail words of the anaphora item, and construct a continuous text segment;

[0023] The product of the anaphoric item head word probability of the head word and the anaphoric item tail word probability of the tail word in the continuous text segment is calculated to obtain the anaphoric item probability of the continuous text segment.

[0024] Furthermore, the selecting a continuous text segment with the highest probability of an anaphora item from the continuous text segments as the anaphora item of the sentence to be resolved includes:

[0025] Filtering out the continuous text segments that have intersections with the sentences to be resolved from the continuous text segments;

[0026] A continuous text segment with the highest probability of an anaphora item is selected from the filtered continuous text segments as the anaphora item of the sentence to be resolved.

[0027] A zero-reference elimination device based on big data, comprising:

[0028] A vectorization module, used to obtain a sentence to be resolved and its context information, perform vectorization processing on the sentence to be resolved and its context information, and obtain a context vector representation of each word in the sentence to be resolved and a context vector representation of each word in the context information;

[0029] An enhancement module, used for inputting the context vector representation of each word in the sentence to be resolved and the above information into a bidirectional long short-term memory network to enhance the context expression and position information of each word, and obtain an enhanced context vector representation of each word;

[0030] The prediction module is used to traverse the enhanced context vector representation of each word and predict the probability of the first word and the last word of the anaphoric item of each word according to the parameter vector in the BERT model;

[0031] A construction module, used for traversing each word, constructing a continuous text segment, and calculating the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each word;

[0032] The selection module is used to select a continuous text segment with the highest probability of anaphora from the continuous text segments as the anaphora of the sentence to be resolved.

[0033] Furthermore, the vectorization module includes:

[0034] A representation unit, used to represent each word in the sentence to be resolved and its context information in a one-hot form, to obtain a high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and a high-dimensional discrete word representation matrix corresponding to its context information;

[0035] An embedding unit, used for embedding the high-dimensional discrete word representation matrix corresponding to the sentence to be decomposed and the high-dimensional discrete word representation matrix corresponding to the above information into a low-dimensional dense representation matrix by using a word embedding method;

[0036] The encoding unit is used to input the low-dimensional dense representation matrix corresponding to the sentence to be resolved and its context information into a preset BERT model for bidirectional encoding to obtain a context vector representation of each word in the sentence to be resolved and its context information.

[0037] Furthermore, the prediction module includes:

[0038] The acquisition unit is used to obtain the first word parameter vector and the last word parameter vector in the BERT model;

[0039] A head word probability calculation unit, used for performing a dot product operation on the enhanced context vector representation of each word and the head word parameter vector, and performing a softmax process on the dot product operation result to obtain the head word probability of the anaphoric item of each word;

[0040] The tail word probability calculation unit is used to perform a dot product operation on the enhanced context vector representation of each word and the tail word parameter vector, and perform a softmax process on the dot product operation result to obtain the anaphoric tail word probability of each word.

[0041] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned zero reference resolution method based on big data when executing the computer program.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned zero reference resolution method based on big data.

[0043] The embodiment of the present invention performs vectorization processing on the sentence to be resolved and its preceding information to obtain the context vector representation of each word in the sentence to be resolved and the context vector representation of each word in the preceding information; then enhances the context expression and position information of each word through a bidirectional long short-term memory network to obtain the enhanced context vector representation of each word; then traverses the enhanced context vector representation of each word, predicts the anaphoric item head word probability and anaphoric item tail word probability of each word according to the parameter vector in the BERT model; constructs a continuous text segment according to the word, and calculates the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and anaphoric item tail word probability of each word; finally, selects the continuous text segment with the largest anaphoric item probability from the continuous text segment as the anaphoric item of the sentence to be resolved. The present invention is based on an extractive reading comprehension model, and can use all continuous segments in the previous text as candidate anaphoric items, without using rules to pre-construct a candidate set of anaphoric items, and the number and coverage of candidate anaphoric items are greater, which effectively improves the accuracy and reliability of the zero-reference resolution result. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0045] Figure 1 is a flow chart of a method for zero reference resolution based on big data in one embodiment of the present invention;

[0046] Figure 2 is a flow chart of step S101 in a method for zero reference resolution based on big data in another embodiment of the present invention;

[0047] Figure 3 is a flow chart of step S103 in the method for zero reference resolution based on big data in another embodiment of the present invention;

[0048] Figure 4 is a flow chart of step S104 in the method for zero reference resolution based on big data in another embodiment of the present invention;

[0049] Figure 5 is a flow chart of step S105 in the method for zero reference resolution based on big data in another embodiment of the present invention;

[0050] Figure 6 It is a principle block diagram of a zero reference resolution device based on big data in one embodiment of the present invention;

[0051] Figure 7 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The zero-reference resolution method based on big data provided in an embodiment of the present invention is applied to a server. The server can be implemented by an independent server or a server cluster composed of multiple servers. In one embodiment, Figure 1 As shown, a method for zero reference resolution based on big data is provided, comprising the following steps:

[0054] In step S101, a sentence to be resolved and its context information are obtained, and vectorization processing is performed on the sentence to be resolved and its context information to obtain a context vector representation of each word in the sentence to be resolved and a context vector representation of each word in the context information.

[0055] Here, the above information refers to the text information before the paragraph where the sentence to be resolved is located, which can be one or more sentences before the sentence to be resolved. In the application scenario of the customer service robot, the sentence to be resolved can be the text entered by the customer during the current inquiry and chat, or the result text after transliteration. The context information can be all the texts of the customer's inquiry and chat.

[0056] Each word corresponds to a context vector representation, and the context vector representation refers to the feature vector of each word. As a preferred example of the present invention, the embodiment of the present invention obtains the context vector representation of each word by vectorizing the sentence to be resolved and its context information, and introduces the correlation between words through the word embedding method and the BERT model, so that the context vector representation of each word is more accurate and the feature dimension in the context vector representation of each word is reduced. Optionally, as Figure 2 As shown, the vectorization process of the sentence to be resolved and its context information in step S101 to obtain the context vector representation of each word in the sentence to be resolved and the context vector representation of each word in the context information includes:

[0057] In step S201, each word in the sentence to be resolved and its context information is represented in one-hot form to obtain a high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and a high-dimensional discrete word representation matrix corresponding to its context information.

[0058] Here, the embodiment of the present invention uses a one-hot form to convert the sentence to be resolved and its context information into a mathematical representation, and obtains a high-dimensional discrete word representation matrix corresponding to the sentence to be resolved, and a high-dimensional discrete word representation matrix corresponding to its context information. Specifically, by pre-constructing a dictionary, the dictionary contains at least all the words in the sentence to be resolved, and each word is assigned a number. When encoding the sentence to be resolved, each word contained in the sentence to be resolved is converted into a one-hot form in which the corresponding position of the word number in the dictionary is 1. The processing logic for the above information is the same and will not be repeated here. The one-hot representation is very intuitive. The length of the one-hot form of each word is the length of the dictionary. For example, if the dictionary contains 10,000 words, then the one-hot form corresponding to each word is a 1*10,000 vector, and this vector has only one position of 1, and the rest are 0, which wastes space and is not conducive to calculation; in addition, the relationship between each word cannot be reflected in the one-hot form.

[0059] In view of this, the embodiment of the present invention further reduces the dimension of the high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and the high-dimensional discrete word representation matrix corresponding to the context information thereof and introduces correlation between words.

[0060] In step S202, a word embedding method is used to embed the high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and the high-dimensional discrete word representation matrix corresponding to the above information into a low-dimensional dense representation matrix.

[0061] Optionally, an embodiment of the present invention adopts the word2vec method in the word embedding method. Specifically, a preset shallow neural network is trained, and the dense feature vector of each word is learned through the shallow neural network to obtain a word vector representation that can reflect the relationship between any two words. Then, the one-hot form of each word in the sentence to be resolved is traversed, and the words in the sentence to be resolved are converted into corresponding dense feature vectors through the trained shallow neural network, and the dense feature vectors of all words are combined to obtain a low-dimensional dense representation matrix corresponding to the sentence to be resolved. The processing logic for context information is the same and will not be repeated here.

[0062] Here, the dense feature vector assigns a fixed-length vector representation to each word, and the fixed length can be set by oneself, such as 300, which is much smaller than the dictionary length in the one-hot form; and the relationship between two words can be represented by the angle value between the two words, which can be specifically represented by a simple cosine function. It can be seen that this embodiment introduces the correlation between words through the dense feature vector, and reduces the feature dimension in the sentence to be resolved and its previous information.

[0063] In step S203, the low-dimensional dense representation matrix corresponding to the sentence to be resolved and its context information is input into a preset BERT model for bidirectional encoding to obtain a context vector representation of each word in the sentence to be resolved and its context information.

[0064] Here, the context vector representation refers to a feature vector containing the previous and following information. The BERT model can perform deep bidirectional encoding on each word vector representation, and transform the word vector that originally does not contain context information into a context vector that introduces the previous and following information of the word. Specifically, the BERT model maps the input low-dimensional dense representation matrix to the latent space through a 24-layer transformation module Transformer-block. Among them, each Transformer-block is composed of modules such as a multi-head attention module, a residual network, a layer normalization, and a feedforward neural network in turn. In the multi-head attention module, the low-dimensional dense representation matrix can learn the interactive information between the contexts, and add position encoding, so that each word vector in the latent space obtained by the BERT model is a vector representation based on the context. Each word corresponds to a context vector representation, so that the vector representation of each word is more accurate.

[0065] In step S102, the context vector representation of each word in the sentence to be resolved and the above information is input into a bidirectional long short-term memory network to enhance the context expression and position information of each word, and obtain an enhanced context vector representation of each word.

[0066] As mentioned above, the BERT model uses a multi-head attention module to enable a low-dimensional dense representation matrix to learn the interactive information between contexts. Only the position encoding information is added at the time of input. After 24 layers of Transformer-block operations, the final output result will be insufficient in position information, and the context expression and position information are weak. In order to solve the above problems, the embodiment of the present invention inputs the context vector representation of each word in the sentence to be resolved and the above information into a bidirectional LSTM network, and the bidirectional LSTM network directly learns the dependency relationship between words. When the context vector representation of each word is input in the order in the sentence to be resolved, the context vector representation of each word has a strong position information encoding ability, so that the context expression and position information of each word can be further enhanced to obtain an enhanced context vector representation.

[0067] In step S103, the enhanced context vector representation of each word is traversed, and the probability of the first word of the anaphoric item and the probability of the last word of the anaphoric item of each word are predicted according to the parameter vector in the BERT model.

[0068] In the above step S203, the BERT model obtains the context vector representation of each word in the sentence to be resolved and its previous information by establishing two parameter vectors to be learned. The two parameter vectors to be learned are the first word parameter vector and the last word parameter vector, which are parameter vectors in the BERT model. The embodiment of the present invention further uses the parameter vectors in the BERT model to predict the probability of the first word of the anaphoric item and the probability of the last word of the anaphoric item for each word. Optionally, as Figure 3 As shown, the enhanced context vector representation of each word after traversing in step S103, predicting the probability of the first word of the anaphoric item and the probability of the last word of the anaphoric item of each word according to the parameter vector in the BERT model includes:

[0069] In step S301, the first word parameter vector and the last word parameter vector in the BERT model are obtained.

[0070] Here, the first word parameter vector and the last word parameter vector are two vectors randomly initialized in the BERT model, and can be continuously learned by optimizing the objective function.

[0071] In step S302, a dot product operation is performed on the enhanced context vector representation of each word and the head word parameter vector, and a softmax process is performed on the dot product operation result to obtain the head word probability of each word.

[0072] After obtaining the head parameter vector, traverse each word and calculate the dot product between the enhanced context vector representation of the word and the head parameter vector. In the BERT model, there are multiple head parameter vectors, each of which corresponds to a dot product, thereby obtaining multiple dot products corresponding to each word. This embodiment further numerically processes the multiple dot products through the Softmax function, converts the multiple dot products into relative probabilities, and selects the maximum value as the head probability of the anaphoric item of the word.

[0073] In step S303, a dot product operation is performed on the enhanced context vector representation of each word and the tail word parameter vector, and a softmax process is performed on the dot product operation result to obtain the anaphoric tail word probability of each word.

[0074] The calculation process of the tail word probability is the same as that of the head word probability. After obtaining the tail word vector parameter, traverse each word and calculate the dot product between the enhanced context vector representation of the word and the tail word parameter vector. In the BERT model, there are multiple tail word parameter vectors, each of which corresponds to a dot product, thereby obtaining multiple dot products corresponding to each word. This embodiment further performs numerical processing on the multiple dot products through the Softmax function, converts the multiple dot products into relative probabilities, and selects the maximum value therein as the anaphoric tail word probability of the word.

[0075] In step S104, each character is traversed to construct a continuous text segment, and the anaphoric item probability of the continuous text segment is calculated according to the anaphoric item head character probability and the anaphoric item tail character probability of each character.

[0076] Here, for each word, the continuous text segment is a continuous segment between the word as the head word of the anaphoric term and other words as the tail words of the anaphoric term. The anaphoric term probability of the continuous text segment refers to the product of the anaphoric term head word probability of the word as the head word of the anaphoric term and the anaphoric term tail word probability of other words as the tail words of the anaphoric term. Optionally, as Figure 4 As shown, the traversal of each character in step S104 to construct a continuous text segment, and the calculation of the anaphoric item probability of the continuous text segment according to the anaphoric item head character probability and the anaphoric item tail character probability of each character includes:

[0077] In step S401, each word is traversed, the word is used as the head word of the anaphora item, the word and the following words are used as the tail words of the anaphora item, and a continuous text segment is constructed.

[0078] Here, the continuous text segment can be a single character, or a word, a sentence, or a paragraph, thereby providing a new method for creating a set of anaphora candidate items. Any segment in the context of the sentence to be resolved can be used as a candidate item for the anaphora, and there is no need to manually create a candidate set for the anaphora, which effectively expands the scope of the candidate items for the anaphora.

[0079] In step S402, the product of the anaphoric item head word probability of the head word and the anaphoric item tail word probability of the tail word in the continuous text segment is calculated to obtain the anaphoric item probability of the continuous text segment.

[0080] For each continuous segment, this embodiment calculates the anaphoric item probability of the continuous text segment according to the first character and the last character in the continuous text segment. The value of the anaphoric item probability is the product of the anaphoric item first character probability of the first character and the anaphoric item last character probability of the last character in the continuous segment. The greater the product of the anaphoric item first character probability of the first character and the anaphoric item last character probability of the last character in the continuous segment, the greater the anaphoric item probability of the continuous segment, and vice versa.

[0081] In step S105, a continuous text segment with the highest probability of an anaphora item is selected from the continuous text segments as the anaphora item of the sentence to be resolved.

[0082] Alternatively, if Figure 5 As shown, the step S105 of selecting the continuous text segment with the highest probability of anaphora from the continuous text segments as the anaphora of the sentence to be resolved includes:

[0083] In step S501, the continuous text segments that have intersections with the sentence to be resolved are filtered out from the continuous text segments.

[0084] Here, if any part of the continuous text segment overlaps with the sentence to be resolved, the continuous text segment is deleted to complete the filtering of the continuous text segment. Specifically, it is possible to determine whether the first character and / or the last character of the continuous text segment falls within the sentence to be resolved, and delete the continuous text segment whose first character and / or the last character falls within the sentence to be resolved, to ensure that the first character and the last character of the retained continuous text segment are not in the current sentence to be resolved. Since the number of continuous text segments is very large, and the anaphoric item usually does not overlap with the sentence to be resolved, this embodiment deletes the continuous text segments that have an intersection with the sentence to be resolved, and filters out the continuous text segments that are clearly non-anaphoric items, so as to reduce the continuous text segments as candidates, improve the efficiency of selecting the anaphoric item of the sentence to be resolved, and then improve the efficiency of zero reference resolution.

[0085] In step S502, a continuous text segment with the highest probability of an anaphora item is selected from the filtered continuous text segments as the anaphora item of the sentence to be resolved.

[0086] Then, according to the anaphora item probability calculated in step S104, a continuous text segment with the largest anaphora item probability is selected from the retained continuous text segments as the anaphora item of the sentence to be resolved.

[0087] The embodiment of the present invention provides a new anaphora determination method. Based on an extractive reading comprehension model, all continuous segments in the previous text can be used as candidate anaphora items. There is no need to use rules to pre-construct a candidate set of anaphora items. The number and coverage of candidate anaphora items are greater, which effectively avoids the problem of recognition failure caused by the anaphora candidate set not containing the correct anaphora in the prior art, and solves the problem that the existing zero-reference resolution method is too dependent on the candidate set of anaphora items, thereby improving the accuracy and reliability of the zero-reference resolution result.

[0088] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0089] In one embodiment, a device for zero reference resolution based on big data is provided, and the device for zero reference resolution based on big data corresponds to the method for zero reference resolution based on big data in the above embodiment. Figure 6 As shown, the zero reference resolution device based on big data includes a vectorization module 61, an enhancement module 62, a prediction module 63, a construction module 64, and a selection module 65. The functional modules are described in detail as follows:

[0090] A vectorization module 61 is used to obtain a sentence to be resolved and its context information, perform vectorization processing on the sentence to be resolved and its context information, and obtain a context vector representation of each word in the sentence to be resolved and a context vector representation of each word in the context information;

[0091] An enhancement module 62, used for inputting the context vector representation of each word in the sentence to be resolved and the above information into a bidirectional long short-term memory network to enhance the context expression and position information of each word, and obtain an enhanced context vector representation of each word;

[0092] A prediction module 63 is used to traverse the enhanced context vector representation of each word and predict the probability of the first word of the anaphoric item and the probability of the last word of the anaphoric item of each word according to the parameter vector in the BERT model;

[0093] A construction module 64 is used to traverse each word, construct a continuous text segment, and calculate the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each word;

[0094] The selection module 65 is used to select a continuous text segment with the highest probability of anaphora from the continuous text segments as the anaphora of the sentence to be resolved.

[0095] Optionally, the vectorization module 61 includes:

[0096] A representation unit, used to represent each word in the sentence to be resolved and its context information in a one-hot form, to obtain a high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and a high-dimensional discrete word representation matrix corresponding to its context information;

[0097] An embedding unit, used for embedding the high-dimensional discrete word representation matrix corresponding to the sentence to be decomposed and the high-dimensional discrete word representation matrix corresponding to the above information into a low-dimensional dense representation matrix by using a word embedding method;

[0098] The encoding unit is used to input the low-dimensional dense representation matrix corresponding to the sentence to be resolved and its context information into a preset BERT model for bidirectional encoding to obtain a context vector representation of each word in the sentence to be resolved and its context information.

[0099] Optionally, the prediction module 63 includes:

[0100] The acquisition unit is used to obtain the first word parameter vector and the last word parameter vector in the BERT model;

[0101] A head word probability calculation unit, used for performing a dot product operation on the enhanced context vector representation of each word and the head word parameter vector, and performing a softmax process on the dot product operation result to obtain the head word probability of the anaphoric item of each word;

[0102] The tail word probability calculation unit is used to perform a dot product operation on the enhanced context vector representation of each word and the tail word parameter vector, and perform a softmax process on the dot product operation result to obtain the anaphoric tail word probability of each word.

[0103] Optionally, the building module 64 includes:

[0104] A construction unit, used for traversing each word, taking the word as the head word of the anaphora item, taking the word and the following words as the tail words of the anaphora item, and constructing a continuous text segment;

[0105] The calculation unit is used to calculate the product of the anaphoric item head word probability of the head word and the anaphoric item tail word probability of the tail word in the continuous text segment to obtain the anaphoric item probability of the continuous text segment.

[0106] Optionally, the selection module 65 includes:

[0107] A filtering unit, used to filter out the continuous text segments that have intersections with the sentence to be resolved from the continuous text segments;

[0108] The selection unit is used to select the continuous text segment with the highest probability of anaphora from the filtered continuous text segments as the anaphora of the sentence to be resolved.

[0109] For the specific definition of the zero-reference resolution device based on big data, please refer to the definition of the zero-reference resolution method based on big data above, which will not be repeated here. Each module in the above-mentioned zero-reference resolution device based on big data can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0110] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a zero reference resolution method based on big data is implemented.

[0111] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0112] Obtaining a sentence to be resolved and its context information, performing vectorization processing on the sentence to be resolved and its context information, and obtaining a context vector representation of each word in the sentence to be resolved and a context vector representation of each word in the context information;

[0113] Inputting the context vector representation of each word in the sentence to be resolved and the above information into a bidirectional long short-term memory network to enhance the context expression and position information of each word, thereby obtaining an enhanced context vector representation of each word;

[0114] Traverse the enhanced context vector representation of each word, and predict the probability of the first word and the last word of the anaphoric item of each word according to the parameter vector in the BERT model;

[0115] Traversing each character, constructing a continuous text segment, and calculating the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each character;

[0116] A continuous text segment with the highest probability of an anaphora item is selected from the continuous text segments as the anaphora item of the sentence to be resolved.

[0117] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0118] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0119] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A zero-reference resolution method based on big data, characterized in that: include: Obtain a sentence to be resolved and its context information, perform vectorization processing on the sentence to be resolved and its context information, obtain a context vector representation of each word in the sentence to be resolved, and a context vector representation of each word in the context information, represent each word in the sentence to be resolved and its context information in a one-hot form, and obtain a high-dimensional discrete word representation matrix corresponding to the sentence to be resolved, and a high-dimensional discrete word representation matrix corresponding to its context information; The word embedding method is used to embed the high-dimensional discrete word representation matrix corresponding to the sentence to be decomposed and the high-dimensional discrete word representation matrix corresponding to the above information into a low-dimensional dense representation matrix; Input the low-dimensional dense representation matrix corresponding to the sentence to be decomposed and its context information into the preset BERT model for bidirectional encoding to obtain the context vector representation of each word in the sentence to be decomposed and its context information; Inputting the context vector representation of each word in the sentence to be resolved and the above information into a bidirectional long short-term memory network to enhance the context expression and position information of each word, thereby obtaining an enhanced context vector representation of each word; Traverse the enhanced context vector representation of each word, and predict the probability of the first word and the last word of the anaphoric item of each word according to the parameter vector in the BERT model; Traversing each character, constructing a continuous text segment, and calculating the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each character; A continuous text segment with the highest probability of an anaphora item is selected from the continuous text segments as the anaphora item of the sentence to be resolved.

2. The method for zero reference resolution based on big data as claimed in claim 1, characterized in that: The enhanced context vector representation of each word is traversed, and the probability of the first word of the anaphoric item and the probability of the last word of the anaphoric item of each word are predicted according to the parameter vector in the BERT model, including: Get the first word parameter vector and the last word parameter vector in the BERT model; Performing a dot product operation on the enhanced context vector representation of each word and the head word parameter vector, and performing a softmax process on the dot product operation result to obtain the head word probability of each word; A dot product operation is performed on the enhanced context vector representation of each word and the tail word parameter vector, and a softmax process is performed on the dot product operation result to obtain the anaphoric tail word probability of each word.

3. The method for zero reference resolution based on big data as claimed in claim 2, characterized in that: The traversing of each character, constructing a continuous text segment, and calculating the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each character comprises: Traverse each word, use the word as the head word of the anaphora item, use the word and the following words as the tail words of the anaphora item, and construct a continuous text segment; The product of the anaphoric item head word probability of the head word and the anaphoric item tail word probability of the tail word in the continuous text segment is calculated to obtain the anaphoric item probability of the continuous text segment.

4. The method for eliminating zero reference based on big data as claimed in claim 3, characterized in that: The step of selecting the continuous text segment with the highest probability of anaphora from the continuous text segments as the anaphora of the sentence to be resolved comprises: Filtering out the continuous text segments that have intersections with the sentences to be resolved from the continuous text segments; A continuous text segment with the highest probability of an anaphora item is selected from the filtered continuous text segments as the anaphora item of the sentence to be resolved.

5. A zero-reference resolution device based on big data, characterized in that: include: A vectorization module, used to obtain a sentence to be resolved and its context information, perform vectorization processing on the sentence to be resolved and its context information, and obtain a context vector representation of each word in the sentence to be resolved and a context vector representation of each word in the context information; The vectorization module comprises: A representation unit is used to represent each word in the sentence to be resolved and its context information in a one-hot form, so as to obtain a high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and a high-dimensional discrete word representation matrix corresponding to its context information; an embedding unit is used to embed the high-dimensional discrete word representation matrix corresponding to the sentence to be resolved and the high-dimensional discrete word representation matrix corresponding to the context information into a low-dimensional dense representation matrix by using a word embedding method; an encoding unit is used to input the low-dimensional dense representation matrix corresponding to the sentence to be resolved and its context information into a preset BERT model for bidirectional encoding, so as to obtain a context vector representation of each word in the sentence to be resolved and its context information; An enhancement module, used for inputting the context vector representation of each word in the sentence to be resolved and the above information into a bidirectional long short-term memory network to enhance the context expression and position information of each word, and obtain an enhanced context vector representation of each word; The prediction module is used to traverse the enhanced context vector representation of each word and predict the probability of the first word and the last word of the anaphoric item of each word according to the parameter vector in the BERT model; A construction module, used for traversing each word, constructing a continuous text segment, and calculating the anaphoric item probability of the continuous text segment according to the anaphoric item head word probability and the anaphoric item tail word probability of each word; The selection module is used to select a continuous text segment with the highest probability of an anaphora item from the continuous text segments as the anaphora item of the sentence to be resolved.

6. The device for eliminating zero reference based on big data as claimed in claim 5, characterized in that: The prediction module comprises: The acquisition unit is used to obtain the first word parameter vector and the last word parameter vector in the BERT model; A head word probability calculation unit, used for performing a dot product operation on the enhanced context vector representation of each word and the head word parameter vector, and performing a softmax process on the dot product operation result to obtain the head word probability of the anaphoric item of each word; The tail word probability calculation unit is used to perform a dot product operation on the enhanced context vector representation of each word and the tail word parameter vector, and perform a softmax process on the dot product operation result to obtain the anaphoric tail word probability of each word.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for zero reference resolution based on big data as described in any one of claims 1 to 4 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for zero reference resolution based on big data as described in any one of claims 1 to 4 is implemented.

Images