Relationship extraction model establishment method and device, electronic equipment and storage medium

Abstract

The invention provides a relation extraction model establishment method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a text vector; the text vector is coded to obtain a dynamic word vector, and the dynamic word vector comprises a whole sentence vector and a feature word vector in the government and enterprise customer interview record; a directed multi-view is obtained according to the feature word vectors and a Gaussian graph generator, the directed multi-view comprises nodes and directed edges, the nodes represent the feature words, and the directed edges represent the relation between the feature words; determining a target node and a graph feature vector represented by the target node according to the nodes in the directed multi-view and directed edges among the nodes; performing splicing processing on the graph feature vector and the whole sentence vector to obtain a spliced vector; and obtaining an extraction relation classification result according to the splicing vector and the classifier. According to the method provided by the invention, the accuracy of relation extraction is improved.

Description

Technical Field

[0001] The present application relates to natural language processing technology, and in particular to a method, device, electronic device and storage medium for establishing a relationship extraction model. Background Art

[0002] For government and enterprise clients, especially VIPs, account managers will visit clients regularly and conduct in-depth interviews. The interview content includes feedback on product usage and collection of potential customer intentions.

[0003] At present, when processing the text of interview records of government and enterprise customers, the traditional method is for account managers to interview government and enterprise customers and record the interview content. Since the collected interview records are text-based unstructured data, the interview records need to be manually analyzed to identify the relationships between business stakeholders such as enterprises and merchants in the interview records.

[0004] However, existing methods are not effective in identifying relationships between relevant parties such as enterprises and merchants in interview records. Summary of the invention

[0005] The present application provides a method, device, electronic device and storage medium for establishing a relationship extraction model, so as to solve the problem that the existing methods are not effective in identifying the relationship between enterprises and merchants and other related parties in interview records.

[0006] In a first aspect, the present application provides a method for establishing a relationship extraction model, comprising:

[0007] Get the text vector. The text vector is the vector obtained by inputting the interview records of government and enterprise customers into the ALBERT model.

[0008] Perform ALBERT pre-training on the text vector to obtain dynamic word vectors, where the dynamic word vectors include the whole sentence vectors and feature word vectors in the government and enterprise customer interview records;

[0009] According to the feature word vector and the Gaussian graph generator, a directed multi-view is obtained, wherein the directed multi-view includes nodes and directed edges, the nodes represent the feature words, and the directed edges represent the relationship between the feature words;

[0010] Determine a target node and a graph feature vector representing the target node according to the nodes in the directed multi-view and the directed edges between the nodes;

[0011] Concatenate the graph feature vector and the whole sentence vector to obtain a concatenated vector;

[0012] According to the concatenated vector and the classifier, the extracted relation classification result is obtained.

[0013] In this application, obtaining a text vector includes:

[0014] Obtain interview records with government and enterprise customers;

[0015] Segment the interview records of government and enterprise customers to obtain segmentation sequences;

[0016] According to the embedding layer of the ALBERT model, the word segmentation sequence is transformed to obtain the text vector.

[0017] In this application, ALBERT pre-training is performed on the text vector to obtain a dynamic word vector, including:

[0018] The text vector is input into the Transformer layer of the ALBERT model to obtain the initial dynamic word vector. The Transformer layer includes multiple stacked Transformer units.

[0019] Perform feature extraction on the initial dynamic word vector to obtain the dynamic word vector.

[0020] In this application, the text vector is input into the Transformer layer of the ALBERT model to obtain the initial dynamic word vector, including:

[0021] Input the text vector into the multi-head self-attention module in the Transformer unit to obtain the word representation of all words in the text vector;

[0022] Input the text vector into the feed-forward network layer in the Transformer unit to obtain the word vector;

[0023] The word representations and word vectors of all words in the text vector are subjected to residual connection processing and normalization processing to obtain the initial dynamic word vector.

[0024] In the present application, according to the nodes in the directed multi-view and the directed edges between the nodes, the target node and the graph feature vector representing the target node are determined, including:

[0025] Input the directed multi-view into the densely connected graph convolution layer to determine the structural information of the directed multi-view, where the structural information represents the connection relationship between nodes;

[0026] Determine the in-degree and out-degree of the nodes in the directed multi-view according to the dynamic time warping pooling layer and the structural information of the directed multi-view;

[0027] Extracting a target directed multi-view from the directed multi-view according to the in-degree and out-degree of the nodes in the directed multi-view, wherein the target directed multi-view includes target nodes and target directed edges between the target nodes;

[0028] According to the target directed multi-view, a graph feature vector representing the target node is determined.

[0029] In the present application, after extracting the target directed multi-view from the directed multi-view according to the in-degree and out-degree of the nodes in the directed multi-view, the method further includes:

[0030] Input the target directed multi-view into the densely connected graph convolution layer to determine the structural information of the target directed multi-view, where the structural information represents the connection relationship between nodes;

[0031] Determine the in-degree and out-degree of the nodes in the target directed multi-view according to the dynamic time warping pooling layer and the structural information of the target directed multi-view;

[0032] Extracting a global target directed multi-view from the target directed multi-view according to the in-degree and out-degree of the nodes in the target directed multi-view;

[0033] According to the directed multi-views of the global target, the graph feature vector is determined.

[0034] In this application, the classifier is a SoftMax classifier. According to the concatenated vector and the classifier, the extracted relationship classification result is obtained, including:

[0035] Input the concatenated vector into the classifier to obtain the classification result;

[0036] The classification results are decoded to obtain the extracted relationship classification results.

[0037] In a second aspect, the present application provides a relationship extraction model establishment device, comprising:

[0038] The acquisition module is used to obtain the text vector. The text vector is the vector obtained after the government and enterprise customer interview records are input into the ALBERT model;

[0039] ALBERT pre-training module, used to perform ALBERT pre-training on text vectors to obtain dynamic word vectors, where the dynamic word vectors include the whole sentence vectors and feature word vectors in the government and enterprise customer interview records;

[0040] A first obtaining module is used to obtain a directed multi-view according to the feature word vector and the Gaussian graph generator, wherein the directed multi-view includes nodes and directed edges, the nodes represent the feature words, and the directed edges represent the relationship between the feature words;

[0041] A determination module, used to determine a target node and a graph feature vector representing the target node according to the nodes in the directed multi-view and the directed edges between the nodes;

[0042] A concatenation module is used to concatenate the graph feature vector and the whole sentence vector to obtain a concatenated vector;

[0043] The second module is used to obtain the extraction relationship classification result based on the splicing vector and the classifier.

[0044] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0045] Memory stores computer-executable instructions;

[0046] The processor executes the computer execution instructions stored in the memory to implement the relationship extraction model establishment method of the present application.

[0047] In a fourth aspect, a computer-readable storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the relationship extraction model establishment method of the present application.

[0048] The present application provides a method, device, electronic device and storage medium for establishing a relationship extraction model. The method obtains a text vector, wherein the text vector is a vector obtained after inputting a government and enterprise customer interview record into an ALBERT model, performs ALBERT pre-training on the text vector to obtain a dynamic word vector, wherein the dynamic word vector includes a whole sentence vector and a feature word vector in the government and enterprise customer interview record, obtains a directed multi-view based on the feature word vector and a Gaussian graph generator, wherein the directed multi-view includes nodes and directed edges, the nodes represent feature words, the directed edges represent the relationship between feature words, and determines a target node and a target node based on the nodes in the directed multi-view and the directed edges between the nodes. The represented graph feature vector is concatenated with the whole sentence vector to obtain a concatenated vector. According to the concatenated vector and the classifier, a means of extracting relationship classification results is obtained. The ALBERT model can be used to perform ALBERT pre-training according to the content of the interview record after text vectorization. Subsequently, a Gaussian generator is used to construct a directed multi-view according to the result representation of the ALBERT pre-training. After screening, the target graph representation in the directed multi-view is determined. Finally, the target graph representation in the directed multi-view and the result representation based on the ALBERT model are connected to perform relationship extraction, thereby avoiding manual relationship identification. Thus, the problem of poor effect of existing methods is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] Figure 1 A schematic diagram of a scenario of a relationship extraction model establishment system provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of a method for establishing a relationship extraction model provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of another method for establishing a relationship extraction model provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of the structure of a device for establishing a relationship extraction model provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0055] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0056] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0057] In the prior art, when processing the text of interview records of government and enterprise customers, the traditional method is that the account manager interviews the government and enterprise customers and records the interview content. Since the collected interview records are text-based unstructured data, the interview records need to be manually analyzed to identify the relationship between business stakeholders such as enterprises and merchants in the interview records. Among them, manual analysis of interview records usually requires a lot of time and effort, especially when processing a large amount of text, it is impossible to accurately cover all the content of the interview records.

[0058] In order to solve the above problems, the present application adopts a relational extraction model to establish a system for processing. The ALBERT model can be used to convert government and enterprise customer interview records into text vectors, and then the text vectors are pre-trained with ALBERT to obtain dynamic word vectors, wherein the dynamic word vectors include whole sentence vectors and feature word vectors in the government and enterprise customer interview records. According to the feature word vectors and the Gaussian graph generator, directed multi-views are obtained, wherein the directed multi-views include nodes and directed edges, the nodes represent feature words, and the directed edges represent the relationship between feature words, so as to facilitate subsequent relationship extraction. According to the nodes in the directed multi-view and the directed edges between the nodes, the target nodes and the graph feature vectors representing the target nodes are determined, the graph feature vectors and the whole sentence vectors are spliced ​​to obtain the spliced ​​vectors, and finally, according to the spliced ​​vectors and the classifier, the extracted relationship classification results are obtained for relationship extraction, which is more accurate than manual interview record recognition and has higher relationship extraction efficiency. Thus, the problem of poor effect of the existing methods is improved.

[0059] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0060] Figure 1 A schematic diagram of a scenario of a relationship extraction model establishment system provided in an embodiment of the present application, such as Figure 1As shown, the relationship extraction model building system includes: ALBERT (A Lite BERT) model unit, graph generation unit, graph processing unit and SoftMax classifier, wherein the graph generation unit is constructed by Gaussian Graph Generator (GGG) for directed multi-view construction, and the graph processing unit is composed of multiple layers of densely connected graph convolutional network and dynamic time warping pooling (DTWPool), and the densely connected graph convolutional network and DTWPool layer are used for graph feature extraction and sparseness. First, the Transformer layer in the ALBERT unit is used as a feature encoder to extract the input text and obtain the corresponding feature representation of the input text, including the whole sentence vector representation and dynamic word vector representation of the input text. The graph unit obtains the dynamic word vector representation from ALBERT and constructs a multi-view using the Gaussian graph generator. Then, the graph is refined through multiple interactions of the densely connected graph convolutional network and DTWPool to achieve target feature extraction. Finally, the potential multi-view after extracting the target feature and the whole sentence vector representation of the input text are input into the SoftMax classifier to predict the relationship type. The execution subject of the relationship extraction model establishment method provided in the embodiment of the present application can be a relationship extraction model establishment system. Among them, the relationship extraction model establishment system can be a mobile phone, a tablet, a computer and other devices. This embodiment does not impose any special restrictions on the implementation method of the execution subject, as long as the execution subject can obtain the text vector, the text vector is the vector obtained after the government and enterprise customer interview record is input into the ALBERT model; the text vector is pre-trained by ALBERT to obtain a dynamic word vector, wherein the dynamic word vector includes the whole sentence vector and the feature word vector in the government and enterprise customer interview record; according to the feature word vector and the Gaussian graph generator, a directed multi-view is obtained, wherein the directed multi-view includes nodes and directed edges, the nodes represent feature words, and the directed edges represent the relationship between feature words; according to the nodes in the directed multi-view and the directed edges between the nodes, the target node and the graph feature vector representing the target node are determined; the graph feature vector and the whole sentence vector are spliced ​​to obtain a spliced ​​vector; according to the spliced ​​vector and the classifier, the extracted relationship classification result is obtained.

[0061] Among them, ALBERT is a natural language processing (NLP) model, which is an improved version based on BERT. ALBERT adopts a parameter sharing strategy to share some parameters in BERT, thereby reducing the total number of parameters and improving the efficiency of the model; ALBERT introduces cross-layer parameter sharing, so that the attention matrix parameters of different layers can be shared, further reducing the scale of the model; ALBERT adopts a sentence embedding transformation strategy to reduce the dimension of sentence embedding through linear transformation, thereby reducing the computational cost.

[0062] Gaussian graph generator is a tool or algorithm for generating Gaussian graphs. Gaussian graphs are a class of graphical models used to model the network structure of random interactions in complex systems. In a Gaussian graph, nodes represent random variables and edges represent the correlations between these variables. Typically, the edges of a Gaussian graph correspond to conditional independence relations, where the conditional dependencies between nodes can be represented by the topological structure of the graph.

[0063] Dynamic time warping pooling is a technology used for time series data processing and analysis. It is mainly used to solve the similarity comparison and classification problems of time series data with different lengths, different speeds, and different amplitudes on the time axis. First, time series of different lengths are dynamically aligned to find the matching relationship between them for comparison, and then the dynamic programming algorithm is used to calculate the alignment path between time series to minimize the distance between them. Finally, the aligned time series are pooled to extract key feature information.

[0064] Figure 2 A flowchart of a method for establishing a relationship extraction model provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the relationship extraction model establishment method includes the following steps:

[0065] S201. Obtain a text vector, where the text vector is a vector obtained after inputting the government and enterprise customer interview records into the ALBERT model.

[0066] Among them, the text vector may refer to a vector obtained by mathematically vectorizing text data. In the embodiment of the present application, the text vector captures important feature information in the government and enterprise customer interview records, which helps to extract key information.

[0067] Government and enterprise customer interview records may refer to documents or records of information, conversation content, and key viewpoints recorded during face-to-face or remote communication with customers or users. In an embodiment of the present application, government and enterprise customer interview records refer to interview content recorded after an account manager interviews a government and enterprise customer.

[0068] In this embodiment of the present application, obtaining a text vector includes:

[0069] Obtain interview records with government and enterprise customers;

[0070] Segment the interview records of government and enterprise customers to obtain segmentation sequences;

[0071] According to the embedding layer of the ALBERT model, the word segmentation sequence is transformed to obtain the text vector.

[0072] Among them, the word segmentation sequence refers to splitting the original text (usually a natural language text) into words or vocabulary units to form an ordered word list or sequence, where each element represents a vocabulary unit in the text. For example, "I like to learn natural language processing." The word segmentation sequence after word segmentation of this sentence may be: "I", "like", "learn", "natural language processing", ".". Here, the sentence is divided into five vocabulary units and arranged into a sequence in the order in which they appear in the sentence. In the embodiment of the present application, the word segmentation sequence is the sequence obtained after word segmentation of the government and enterprise customer interview records as input text.

[0073] According to the embedding layer of the ALBERT model, the word segmentation sequence is converted to obtain a text vector. This means that in the embedding layer of the ALBERT model, the corresponding word vector is searched for each vocabulary unit in the word segmentation sequence, and all word vectors are aggregated or combined to generate a vector representation of the entire text, thereby obtaining a text vector.

[0074] S202. Perform ALBERT pre-training on the text vector to obtain a dynamic word vector, wherein the dynamic word vector includes a whole sentence vector and a feature word vector in the government and enterprise customer interview records.

[0075] Among them, dynamic word vector is a word vector representation method. Different from traditional static word vector, dynamic word vector has certain temporal or contextual dependence. The generation of dynamic word vector usually takes into account the changes of vocabulary in different contexts or time points. That is, the same word may have different word vector representations in different contexts to better capture its semantics.

[0076] The whole sentence vector is a text representation method that converts an entire sentence or text paragraph into a single vector. This vector captures the semantic information and features of the entire sentence. In the embodiment of the present application, the ALBERT pre-trained deep learning model is used to convert the entire sentence text into a vector representation, which can capture complex semantic information and contextual relationships.

[0077] The feature word vector can be used to convert the feature words in the government and enterprise customer interview records into word vector representations.

[0078] In the embodiment of the present application, ALBERT pre-training is performed on the text vector to obtain a dynamic word vector, including:

[0079] The text vector is input into the Transformer layer of the ALBERT model to obtain the initial dynamic word vector. The Transformer layer includes multiple stacked Transformer units.

[0080] Perform feature extraction on the initial dynamic word vector to obtain the dynamic word vector.

[0081] The Transformer layer of the ALBERT model refers to a deep learning neural network architecture, which is the basis of the ALBERT model and is used to process text data, especially in the pre-training stage. The main components of the Transformer architecture include: multi-head self-attention mechanism, position encoding, encoder (Encoder) and decoder (Decoder). In ALBERT, the encoder part is used to learn the contextual representation of text.

[0082] Performing feature extraction on the initial dynamic word vector to obtain the dynamic word vector may refer to determining the dynamic word vector from the initial dynamic word vector by pre-fine-tuning, wherein pre-fine-tuning refers to a pre-processing or pre-training step before the fine-tuning stage. In an embodiment of the present application, the dynamic word vector obtained by pre-fine-tuning is more suitable for the relationship extraction task.

[0083] In this embodiment of the present application, the text vector is input into the Transformer layer of the ALBERT model to obtain the initial dynamic word vector, including:

[0084] Input the text vector into the multi-head self-attention module in the Transformer unit to obtain the word representation of all words in the text vector;

[0085] Input the text vector into the feed-forward network layer in the Transformer unit to obtain the word vector;

[0086] The word representations and word vectors of all words in the text vector are subjected to residual connection processing and normalization processing to obtain the initial dynamic word vector.

[0087] Among them, the multi-head self-attention module is one of the core components of the Transformer unit. In the embodiment of the present application, the multi-head self-attention module enables the model to consider the information of all positions in the text vector while processing the input text vector. Each word can be encoded according to its context in the text, so as to better capture semantic and grammatical information.

[0088] The feed-forward network layer has multiple roles in the Transformer unit, including feature mapping, nonlinear transformation, position encoding, local feature capture, and parameterized transformation.

[0089] In an embodiment of the present application, inputting a text vector into a feedforward network layer in a Transformer unit to obtain a word vector may refer to performing data distribution fitting and capturing nonlinear relationships on the input text according to a nonlinear activation function in the feedforward network layer, so as to obtain a word vector with richer feature representation corresponding to the input text vector.

[0090] S203. Obtain a directed multi-view according to the feature word vector and the Gaussian graph generator, wherein the directed multi-view includes nodes and directed edges, the nodes represent the feature words, and the directed edges represent the relationship between the feature words.

[0091] Among them, directed edges represent the relationship between feature words, that is, there are directed connections between feature words, which are used to represent the dependency relationship or information transmission direction between views.

[0092] According to the feature word vector and Gaussian graph generator, directed multi-views are obtained, which refers to the initial nodes based on the potential graph. The Gaussian generator is used to encode the initial nodes according to the feature word vector to obtain the Gaussian distribution corresponding to the initial nodes, and then the adjacency matrix of the view is obtained according to the weight relationship between the nodes. Finally, the directed multi-views are constructed through the Gaussian graph generator.

[0093] S204: Determine a target node and a graph feature vector representing the target node according to the nodes in the directed multi-view and the directed edges between the nodes.

[0094] The graph feature vector refers to a numerical vector used to represent the features of nodes in the graph.

[0095] Among them, in the embodiment of the present application, determining the target node and the graph feature vector represented by the target node according to the nodes in the directed multi-view and the directed edges between the nodes includes:

[0096] Input the directed multi-view into the densely connected graph convolution layer to determine the structural information of the directed multi-view, where the structural information represents the connection relationship between nodes;

[0097] Determine the in-degree and out-degree of the nodes in the directed multi-view according to the dynamic time warping pooling layer and the structural information of the directed multi-view;

[0098] Extracting a target directed multi-view from the directed multi-view according to the in-degree and out-degree of the nodes in the directed multi-view, wherein the target directed multi-view includes target nodes and target directed edges between the target nodes;

[0099] According to the target directed multi-view, a graph feature vector representing the target node is determined.

[0100] The structural information may refer to information formed by connecting directed edges between nodes in multiple views and used to characterize the connection relationship between nodes.

[0101] In-degree is a metric that describes the degree to which a node receives information or is connected to other nodes. In a directed graph, the in-degree of a node indicates how many directed edges point to the node.

[0102] Out-degree is an indicator that describes the degree to which a node transmits information outward or connects to other nodes. In a directed graph, the out-degree of a node indicates how many directed edges point from the node to other nodes.

[0103] The target directed multi-view is determined and extracted by the ratio of out-degree to in-degree, wherein the ratio is set in advance according to the needs of the user, and the target node can be determined. The target node can represent a relatively important node and a node with a greater correlation with the context.

[0104] In the embodiment of the present application, after extracting the target directed multi-view from the directed multi-view according to the in-degree and out-degree of the node in the directed multi-view, the method further includes:

[0105] Input the target directed multi-view into the densely connected graph convolution layer to determine the structural information of the target directed multi-view, where the structural information represents the connection relationship between nodes;

[0106] Determine the in-degree and out-degree of the nodes in the target directed multi-view according to the dynamic time warping pooling layer and the structural information of the target directed multi-view;

[0107] Extracting a global target directed multi-view from the target directed multi-view according to the in-degree and out-degree of the nodes in the target directed multi-view;

[0108] According to the directed multi-views of the global target, the graph feature vector is determined.

[0109] Among them, according to the in-degree and out-degree of the nodes in the target directed multi-view, extracting the global target directed multi-view from the target directed multi-view means deleting unimportant nodes, that is, removing words in the text that are not important for the relationship extraction task. For example, "A boy Zhang San works in Company A", the relationship between Zhang San and Company A is predicted to be employment, and the words "a boy" are not important for the prediction. The extraction of the global target directed multi-view is achieved by screening the important words for relationship prediction.

[0110] S205: Concatenate the graph feature vector and the whole sentence vector to obtain a concatenated vector.

[0111] Among them, concatenating the graph feature vector and the whole sentence vector to obtain the concatenated vector may refer to simply connecting the graph feature vector and the whole sentence vector together to form a larger vector, for example: concatenated vector = [graph feature vector, whole sentence vector].

[0112] S206: Obtain an extraction relationship classification result based on the concatenated vector and the classifier.

[0113] In the embodiment of the present application, the classifier is a SoftMax classifier, and the extraction relationship classification result is obtained according to the concatenated vector and the classifier, including:

[0114] Input the concatenated vector into the classifier to obtain the classification result;

[0115] The classification results are decoded to obtain the extracted relationship classification results.

[0116] Among them, inputting the splicing vector into the classifier to obtain the classification result can refer to inputting the splicing vector to train the model, using the SoftMax function as the activation function of the output layer, obtaining the probability distribution of each possible relationship, and predicting the relationship classification result of the splicing vector.

[0117] The relationship extraction model establishment method provided in the embodiment of the present application can input the government and enterprise customer interview records into the ALBERT model to obtain a text vector, and then perform ALBERT pre-training on the text vector to obtain a dynamic word vector, wherein the dynamic word vector includes the whole sentence vector and the feature word vector in the government and enterprise customer interview record, and obtain a directed multi-view based on the feature word vector and the Gaussian graph generator, wherein the directed multi-view includes nodes and directed edges, and the target node and the graph feature vector representing the target node are determined based on the nodes in the directed multi-view and the directed edges between the nodes, and the graph feature vector represented by the target node is spliced, the graph feature vector and the whole sentence vector are spliced ​​to obtain a spliced ​​vector, and the extracted relationship classification result is obtained based on the spliced ​​vector and the classifier.

[0118] Figure 3 A flow chart of another method for establishing a relationship extraction model provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the relationship extraction model establishment method includes:

[0119] S301, obtain the text dialogue sequence X of the interview record of government and enterprise customers, where X={x0,x1,…,x N ,x N+1}.

[0120] Among them, x0 represents the sentence start [CLS] mark, {x1,…,x N} represents the content of the text conversation in the interview record of government and enterprise customers, x N+1 Indicates the sentence separator [SEP] of a text conversation.

[0121] S302: Input the text dialogue sequence X of the government and enterprise customer interview record into the ALBERT model for training to obtain the dynamic word vector U = {u0,u1,…,u N ,u N+1}, where u0 is the vector representation of the entire sentence, {u1,…,u N ,u N+1} is the dynamic vector representation of each word in the text conversation.

[0122] The text dialogue sequence X of the interview records of government and enterprise customers is input into the ALBERT model for training, and the dynamic word vector U = {u0,u1,…,u N ,u N+1} refers to the digital vectorization of the text dialogue sequence X of the government and enterprise customer interview records, which is then input into the ALBERT model and processed through the Transformer structure of the ALBERT model to obtain a dynamic word vector.

[0123] S303, using a Gaussian generator, based on the potential graph according to {u1,…,u N ,u N+1}, construct a complex directed multi-view MG.

[0124] In the embodiment of the present application, a Gaussian generator is used, based on the potential graph according to {u1,…,u N ,u N+1}, construct a complex directed multi-view MG, including:

[0125] Denote the initial node of the latent graph as Each of these nodes Correspondence represents a word;

[0126] Through the Gaussian graph generator, according to {u1,…,u N ,u N+1}, for each node Encode, where Get Node The corresponding Gaussian distribution

[0127] According to the Gaussian distribution between two nodes, KL divergence is introduced to model the weight of directed edges on multiple views. The weight calculation formula of the edge between the i-th node and the j-th node on the m-th view is:

[0128]

[0129] After calculating the edge weights between nodes on each view, we get the adjacency matrix {C 1 , C 2 , …, C m}, and finally get the directed multi-view MG = {K 0 , C 1 , C 2 , …, C m}.

[0130] S304: Input the directed multi-view into the graph unit for graph feature extraction and sparseness to obtain the final graph K, wherein the graph processing unit includes a 3-layer densely connected graph convolutional network and dynamic time warping pooling.

[0131] In the embodiment of the present application, the directed multi-view is input into the graph unit for graph feature extraction and sparseness to obtain the final graph K, including:

[0132] Extract graph features from directed multi-views through densely connected graph convolutional networks and update the representation of each node;

[0133] Select nodes to sparse the graph through dynamic time warping pooling;

[0134] The node representations of the intermediate graphs generated in the three pooling processes are concatenated to obtain the final graph K.

[0135] S305, use the maximum pooling neural network f to calculate the final graph representation K, and concatenate the calculation result with the whole sentence vector representation u0 to form the final representation: u fin =[u0;f(K)];u fin =[u0;f(K)]

[0136] S306, u fin = [u0; f(K)] and the whole sentence vector representation u0 are sent to the SoftMax classifier for classification to obtain the final relationship classification result.

[0137] Another method for establishing a relationship extraction model provided in an embodiment of the present application obtains word vectors for input text through an ALBERT pre-trained language model to improve the efficiency of obtaining word vectors, uses a Gaussian graph generator to construct potential directed multi-views as the basis for modeling indicator words for relationship extraction tasks, and effectively obtains inter-word weights. Finally, through graph feature extraction and sparsification, indicator words are effectively selected to guide entity relationship extraction, and the features of the indicative words to be extracted in the text are accurately and effectively extracted, thereby improving the performance of the entity relationship extraction algorithm in the government and enterprise customer interview text, so as to obtain an entity relationship extraction model for government and enterprise customer interview results, and classify the entity relationships of customer interview results through the model to improve the accuracy of entity relationship extraction.

[0138] Figure 4 This is a schematic diagram of the structure of the apparatus for establishing a relationship extraction model provided in the embodiment of the present application. Figure 4 As shown, the relationship extraction model building device 40 includes: an acquisition module 401, an ALBERT pre-training module 402, a first obtaining module 403, a determination module 404, a splicing module 405 and a second obtaining module 406. Among them:

[0139] The acquisition module is used to obtain the text vector. The text vector is the vector obtained after the government and enterprise customer interview records are input into the ALBERT model;

[0140] ALBERT pre-training module, used to perform ALBERT pre-training on text vectors to obtain dynamic word vectors, where the dynamic word vectors include the whole sentence vectors and feature word vectors in the government and enterprise customer interview records;

[0141] A first obtaining module is used to obtain a directed multi-view according to the feature word vector and the Gaussian graph generator, wherein the directed multi-view includes nodes and directed edges, the nodes represent the feature words, and the directed edges represent the relationship between the feature words;

[0142] A determination module, used to determine a target node and a graph feature vector representing the target node according to the nodes in the directed multi-view and the directed edges between the nodes;

[0143] A concatenation module is used to concatenate the graph feature vector and the whole sentence vector to obtain a concatenated vector;

[0144] The second module is used to obtain the extraction relationship classification result based on the splicing vector and the classifier.

[0145] In the embodiment of the present application, the acquisition module 401 may also be specifically used for:

[0146] Obtain interview records with government and enterprise customers;

[0147] Segment the interview records of government and enterprise customers to obtain segmentation sequences;

[0148] According to the embedding layer of the ALBERT model, the word segmentation sequence is transformed to obtain the text vector.

[0149] In the embodiment of the present application, the ALBERT pre-training module 402 may also be specifically used for:

[0150] The text vector is input into the Transformer layer of the ALBERT model to obtain the initial dynamic word vector. The Transformer layer includes multiple stacked Transformer units.

[0151] Perform feature extraction on the initial dynamic word vector to obtain the dynamic word vector.

[0152] In the embodiment of the present application, the ALBERT pre-training module 402 may also be specifically used for:

[0153] Input the text vector into the multi-head self-attention module in the Transformer unit to obtain the word representation of all words in the text vector;

[0154] Input the text vector into the feed-forward network layer in the Transformer unit to obtain the word vector;

[0155] The word representations and word vectors of all words in the text vector are subjected to residual connection processing and normalization processing to obtain the initial dynamic word vector.

[0156] In the embodiment of the present application, the determination module 404 may also be specifically used for:

[0157] Input the directed multi-view into the densely connected graph convolution layer to determine the structural information of the directed multi-view, where the structural information represents the connection relationship between nodes;

[0158] Determine the in-degree and out-degree of the nodes in the directed multi-view according to the dynamic time warping pooling layer and the structural information of the directed multi-view;

[0159] Extracting a target directed multi-view from the directed multi-view according to the in-degree and out-degree of the nodes in the directed multi-view, wherein the target directed multi-view includes target nodes and target directed edges between the target nodes;

[0160] According to the target directed multi-view, a graph feature vector representing the target node is determined.

[0161] In the embodiment of the present application, the determination module 404 may also be specifically used for:

[0162] Input the target directed multi-view into the densely connected graph convolution layer to determine the structural information of the target directed multi-view, where the structural information represents the connection relationship between nodes;

[0163] Determine the in-degree and out-degree of the nodes in the target directed multi-view according to the dynamic time warping pooling layer and the structural information of the target directed multi-view;

[0164] Extracting a global target directed multi-view from the target directed multi-view according to the in-degree and out-degree of the nodes in the target directed multi-view;

[0165] According to the directed multi-views of the global target, the graph feature vector is determined.

[0166] In the embodiment of the present application, the second obtaining module 406 may also be used for:

[0167] Input the concatenated vector into the classifier to obtain the classification result;

[0168] The classification results are decoded to obtain the extracted relationship classification results.

[0169] As can be seen from the above, the relationship extraction model establishment device of this embodiment is composed of an acquisition module 401, which is used to acquire a text vector, and the text vector is a vector obtained after inputting the government and enterprise customer interview record into the ALBERT model; an ALBERT pre-training module 402 is used to perform ALBERT pre-training on the text vector to obtain a dynamic word vector, wherein the dynamic word vector includes the whole sentence vector and the feature word vector in the government and enterprise customer interview record; a first acquisition module 403 is used to obtain a directed multi-view according to the feature word vector and the Gaussian graph generator, wherein the directed multi-view includes nodes and directed edges, the nodes represent feature words, and the directed edges represent the relationship between feature words; a determination module 404 is used to determine the target node and the graph feature vector represented by the target node according to the nodes in the directed multi-view and the directed edges between the nodes; a splicing module 405 is used to splice the graph feature vector and the whole sentence vector to obtain a splicing vector; and a second acquisition module 406 is used to obtain the extraction relationship classification result according to the splicing vector and the classifier. Thus, the effect of extracting model relationships is improved.

[0170] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 50 includes:

[0171] The electronic device 50 may include a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a communication component 503 and other components. The processor 501 , the memory 502 and the communication component 503 are connected via a bus 504 .

[0172] In the specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above relationship extraction model establishment method.

[0173] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0174] In the above Figure 5In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0175] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0176] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0177] In some embodiments, a computer program product is also proposed, including a computer program or instructions, which, when executed by a processor, implements the steps in any of the above-mentioned relationship extraction model establishment methods.

[0178] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0179] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0180] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions can be loaded by a processor to execute the steps in any one of the relationship extraction model establishment methods provided in the embodiment of the present application.

[0181] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0182] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.

[0183] Since the instructions stored in the storage medium can execute the steps in any one of the relationship extraction model establishment methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the relationship extraction model establishment methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0184] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0185] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for establishing a relationship extraction model, characterized in that: Applied to a natural language processing system, the method comprises: Obtain a text vector, wherein the text vector is a vector obtained by inputting the government and enterprise customer interview records into the ALBERT model; Performing ALBERT pre-training on the text vector to obtain a dynamic word vector, wherein the dynamic word vector includes a whole sentence vector and a feature word vector in the government and enterprise customer interview record; According to the feature word vector and the Gaussian graph generator, a directed multi-view is obtained, wherein the directed multi-view includes nodes and directed edges, the nodes represent feature words, and the directed edges represent the relationship between the feature words; Determine a target node and a graph feature vector represented by the target node according to the nodes in the directed multi-view and the directed edges between the nodes; Concatenating the graph feature vector and the whole sentence vector to obtain a concatenated vector; According to the concatenated vector and the classifier, an extraction relationship classification result is obtained.

2. The method according to claim 1, characterized in that: The obtaining of the text vector comprises: Obtain interview records with government and enterprise customers; Segmenting the interview records of the government and enterprise customers to obtain a segmentation sequence; According to the embedding layer of the ALBERT model, the word segmentation sequence is converted to obtain a text vector.

3. The method according to claim 1, characterized in that: The performing ALBERT pre-training on the text vector to obtain a dynamic word vector includes: Inputting the text vector into the Transformer layer of the ALBERT model to obtain an initial dynamic word vector, wherein the Transformer layer includes a plurality of stacked Transformer units; Perform feature extraction on the initial dynamic word vector to obtain a dynamic word vector.

4. The method according to claim 3, characterized in that The step of inputting the text vector into the Transformer layer of the ALBERT model to obtain an initial dynamic word vector includes: Input the text vector into the multi-head self-attention module in the Transformer unit to obtain word representations of all words in the text vector; Input the text vector into the feed-forward network layer in the Transformer unit to obtain a word vector; Residual connection processing and normalization processing are performed on the word representations of all words in the text vector and the word vector to obtain an initial dynamic word vector.

5. The method according to claim 1, characterized in that The determining, according to the nodes in the directed multi-view and the directed edges between the nodes, a target node and a graph feature vector represented by the target node comprises: Inputting the directed multi-view into a densely connected graph convolutional layer to determine structural information of the directed multi-view, wherein the structural information represents a connection relationship between the nodes; Determine the in-degree and out-degree of the nodes in the directed multi-view according to the dynamic time warping pooling layer and the structural information of the directed multi-view; Extracting a target directed multi-view from the directed multi-view according to the in-degree and out-degree of the nodes in the directed multi-view, wherein the target directed multi-view includes target nodes and target directed edges between the target nodes; According to the target directed multi-view, a graph feature vector representing the target node is determined.

6. The method according to claim 5, characterized in that After extracting the target directed multi-view from the directed multi-view according to the in-degree and out-degree of the nodes in the directed multi-view, the method further includes: Inputting the target directed multi-view into a densely connected graph convolutional layer to determine structural information of the target directed multi-view, wherein the structural information represents a connection relationship between the nodes; Determine the in-degree and out-degree of the node in the target directed multi-view according to the dynamic time warping pooling layer and the structural information of the target directed multi-view; Extracting a global target directed multi-view from the target directed multi-view according to the in-degree and out-degree of the nodes in the target directed multi-view; According to the global target directed multi-view, a graph feature vector is determined.

7. The method according to claim 1, characterized in that The classifier is a SoftMax classifier, and the extraction relationship classification result is obtained according to the splicing vector and the classifier, including: Inputting the concatenated vector into the classifier to obtain a classification result; The classification result is decoded to obtain an extracted relationship classification result.

8. A relationship extraction model building device, characterized in that: include: An acquisition module, used to acquire a text vector, wherein the text vector is a vector obtained after inputting the government and enterprise customer interview records into the ALBERT model; An ALBERT pre-training module, used to perform ALBERT pre-training on the text vector to obtain a dynamic word vector, wherein the dynamic word vector includes a whole sentence vector and a feature word vector in the government and enterprise customer interview record; A first obtaining module is used to obtain a directed multi-view according to the feature word vector and the Gaussian graph generator, wherein the directed multi-view includes nodes and directed edges, the nodes represent feature words, and the directed edges represent the relationship between the feature words; A determination module, configured to determine a target node and a graph feature vector representing the target node according to the nodes in the directed multi-view and the directed edges between the nodes; A concatenation module, used for concatenating the graph feature vector and the whole sentence vector to obtain a concatenated vector; The second obtaining module is used to obtain the extraction relationship classification result according to the splicing vector and the classifier.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the relationship extraction model establishment method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the relationship extraction model establishment method according to any one of claims 1 to 7.

Images