Text label extraction method, device and electronic device

Abstract

The present invention provides a text label extraction method, device and electronic device. The method comprises: obtaining a word vector matrix corresponding to a first text, wherein each word in the word vector matrix corresponds to a word embedding vector; determining a context shielding vector between each two word embedding vectors according to the semantic relevance between each two word embedding vectors; constructing a text semantic graph according to the word embedding vectors and the context shielding vectors; wherein each node in the text semantic graph corresponds to a word embedding vector, and each context shielding vector is an edge weight between two nodes; obtaining a label of the first text according to the text semantic graph, and relates to the field of communication technology. The scheme of the present invention can improve the accuracy of text label extraction.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a text tag extraction method, device and electronic equipment. Background Art

[0002] Document tag extraction is a key technology in the field of natural language processing. Its purpose is to automatically identify and extract key information from documents as tags. At present, document tag extraction technologies mainly include rule-based methods, machine learning-based methods, and deep learning-based methods. These methods rely on a large number of manually designed features and rules, which is not only time-consuming and labor-intensive, but also difficult to adapt to the diversity and complexity of text data, and the accuracy of text tag extraction is low. Summary of the invention

[0003] The purpose of the present invention is to provide a text label extraction method, device and electronic device, which solve the problem of low accuracy of extracted text labels.

[0004] To achieve the above object, an embodiment of the present invention provides a text tag extraction method, comprising:

[0005] Obtain a word vector matrix corresponding to the first text, wherein each word in the word vector matrix corresponds to a word embedding vector;

[0006] Determining a context masking vector between each two of the word embedding vectors according to the semantic relevance between each two of the word embedding vectors;

[0007] Constructing a text semantic graph according to the word embedding vector and the context shielding vector; wherein each node in the text semantic graph corresponds to one word embedding vector, and each context shielding vector is an edge weight between two nodes;

[0008] According to the text semantic graph, a label of the first text is obtained.

[0009] Optionally, determining the context masking vector between every two of the word embedding vectors according to the semantic relevance between every two of the word embedding vectors includes:

[0010] A context masking vector between every two of the word embedding vectors is determined according to the semantic relevance, a first adjustable parameter and a second adjustable parameter; wherein the first adjustable parameter is used to adjust the sensitivity of the semantic relevance to the masking strength, and the second adjustable parameter is used to adjust the baseline of the masking strength.

[0011] Optionally, determining the context masking vector between every two of the word embedding vectors according to the semantic relevance, the first adjustable parameter and the second adjustable parameter includes:

[0012] According to the formula: Generate a context mask vector between every two of the word embedding vectors;

[0013] in, is the context masking vector between word embedding vector i and word embedding vector j, σ is the sigmoid function, α is the first adjustable parameter, β is the second adjustable parameter, is the semantic relevance between word embedding vector i and word embedding vector j, where i and j are positive integers.

[0014] Optionally, acquiring a label of the first text according to the text semantic graph includes:

[0015] Using the text semantic graph as an input of a graph convolutional network, and outputting local features of the first text, wherein the local features are used to represent local context information between adjacent words in the first text;

[0016] Inputting the word vector matrix into a Transformer encoder, and outputting global features of the first text, wherein the global features are used to represent global context information of the first text;

[0017] Perform feature splicing on the local feature and the global feature to obtain a fusion feature;

[0018] The label is obtained according to the fusion feature.

[0019] Optionally, obtaining the label according to the fusion feature includes:

[0020] Using a Transformer decoder to decode the fused features to generate a decoded sequence;

[0021] Through the softmax function, predict the probability distribution of each word in the sequence;

[0022] The label is determined according to the probability distribution.

[0023] To achieve the above object, an embodiment of the present invention provides a text label extraction device, comprising:

[0024] A first acquisition module is used to obtain a word vector matrix corresponding to the first text, where each word in the word vector matrix corresponds to a word embedding vector;

[0025] A first determination module is used to determine a context masking vector between every two of the word embedding vectors according to the semantic relevance between every two of the word embedding vectors;

[0026] A second determination module is used to construct a text semantic graph according to the word embedding vector and the context shielding vector; wherein each node in the text semantic graph corresponds to one word embedding vector, and each context shielding vector is an edge weight between two nodes;

[0027] The second acquisition module is used to acquire a label of the first text according to the text semantic graph.

[0028] To achieve the above-mentioned purpose, an embodiment of the present invention provides an electronic device, including a processor and a transceiver, wherein the processor is used to obtain a word vector matrix corresponding to a first text, each word in the word vector matrix corresponds to a word embedding vector; according to the semantic relevance between each two of the word embedding vectors, a context shielding vector between each two of the word embedding vectors is determined; according to the word embedding vectors and the context shielding vectors, a text semantic graph is constructed; wherein each node in the text semantic graph corresponds to one of the word embedding vectors, and each of the context shielding vectors is the edge weight between two nodes; according to the text semantic graph, a label of the first text is obtained.

[0029] To achieve the above-mentioned purpose, an embodiment of the present invention provides an electronic device, including a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; when the processor executes the program or instruction, the text tag extraction method as described above is implemented.

[0030] To achieve the above objective, an embodiment of the present invention provides a readable storage medium having a program or instruction stored thereon, and the program or instruction, when executed by a processor, implements the steps in the above-mentioned text tag extraction method.

[0031] To achieve the above objective, an embodiment of the present invention provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above text tag extraction method.

[0032] The beneficial effects of the above technical solution of the present invention are as follows:

[0033] The scheme of the embodiment of the present invention is to obtain a word vector matrix corresponding to the first text, wherein each word in the word vector matrix corresponds to a word embedding vector; determine a context shielding vector between each two word embedding vectors according to the semantic relevance between each two word embedding vectors; construct a text semantic graph according to the word embedding vectors and the context shielding vectors; wherein each node in the text semantic graph corresponds to a word embedding vector, and each context shielding vector is an edge weight between two nodes; and obtain a label of the first text according to the text semantic graph. In this way, by generating a context shielding vector, constructing a text semantic graph based on the context shielding vector shielding, and further extracting labels based on the semantic graph, it is possible to shield noise information irrelevant to the topic, retain key local context features, and thus more accurately capture key information. Moreover, the application embodiment can improve the performance of document label extraction in long text and complex text scenarios, reduce dependence on a large amount of annotated data, and improve computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is one of the flowcharts of the text tag extraction method according to an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of updating a text semantic graph according to an embodiment of the present invention;

[0036] Figure 3 A flowchart of a text tag extraction method according to another embodiment of the present invention;

[0037] Figure 4 is a structural diagram of a text tag extraction device according to an embodiment of the present invention;

[0038] Figure 5 is a structural diagram of an electronic device according to an embodiment of the present invention;

[0039] Figure 6 FIG. 4 is a structural diagram of an electronic device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0041] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0042] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0043] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0044] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0045] like Figure 1 As shown, a text tag extraction method according to an embodiment of the present invention comprises the following steps:

[0046] Step 101: Obtain a word vector matrix corresponding to a first text, wherein each word in the word vector matrix corresponds to a word embedding vector.

[0047] In this step, the first text is divided into individual words or phrases by performing data cleaning, stop word removal and word segmentation on the first text; further, each word in the first text after word segmentation is mapped to a corresponding semantic vector according to a pre-trained word vector (such as Word2Vec, GloVe or FastText, etc.); at the same time, a position vector is generated for each word to represent the position of the word in the first text; wherein the position vector can be realized by encoding the position information of each word into a vector, which may include but is not limited to using fixed position encoding and learning position encoding; further, by concatenating or adding the word meaning vector and the position vector, the first text is mapped into a low-dimensional dense matrix containing word meaning information and position information, and the matrix size is n×d; wherein n represents the number of words in the first text, d represents the word vector dimension, and d represents the word embedding vector. In this way, the word vector matrix has certain semantic and positional connections.

[0048] Step 102: Determine a context masking vector between every two of the word embedding vectors according to the semantic relevance between every two of the word embedding vectors.

[0049] Optionally, the method for obtaining the semantic relevance between every two word embedding vectors includes: obtaining the cosine similarity between every two word embedding vectors; and determining the cosine similarity as the semantic relevance.

[0050] Specifically, the cosine similarity calculation formula between word embedding vectors is:

[0051] Among them, vec(w i) and vec(w j ) are word w i and w j The word embedding vector of .

[0052] Different texts correspond to different context masking vectors. The context masking vectors can mask noise information irrelevant to the topic and retain key local context features in the text.

[0053] Step 103: construct a text semantic graph according to the word embedding vector and the context shielding vector; wherein each node in the text semantic graph corresponds to one word embedding vector, and each context shielding vector is an edge weight between two nodes.

[0054] In this step, when constructing the text semantic graph, each word is represented as a node in the graph, and the edges between nodes are used to represent the semantic associations between words. Each node in the text semantic graph corresponds to a word in the first text, the word embedding vector is used to represent the node feature, and the dynamic context masking vector is used as the edge weight, that is: Among them, Edge i,j Represents the edge weight between node i and node j, that is, word w i and word w j The dynamic context shielding relationship between Represents the context mask vector between node i and node j.

[0055] Among them, the connections between nodes in the text semantic graph reflect the complex semantic structure within the text and the dynamic relationship between words.

[0056] Step 104: Obtain a label of the first text according to the text semantic graph.

[0057] In the above embodiment, by generating a context masking vector, constructing a text semantic graph based on the context masking vector masking, and further extracting labels based on the semantic graph, it is possible to mask noise information irrelevant to the topic and retain key local context features, thereby more accurately capturing key information. Moreover, the application embodiment can improve the performance of document label extraction in long text and complex text scenarios, reduce dependence on a large amount of annotated data, and improve computational efficiency.

[0058] In some embodiments, in the above step 102, determining the context masking vector between each two word embedding vectors according to the semantic relevance between each two word embedding vectors includes:

[0059] A context masking vector between every two of the word embedding vectors is determined according to the semantic relevance, a first adjustable parameter and a second adjustable parameter; wherein the first adjustable parameter is used to adjust the sensitivity of the semantic relevance to the masking strength, and the second adjustable parameter is used to adjust the baseline of the masking strength.

[0060] For example, a dynamic adjustment function is constructed according to the semantic relevance, the first adjustable parameter and the second adjustable parameter; and a context masking vector is generated between every two word embedding vectors according to the dynamic adjustment function.

[0061] In some embodiments, determining the context masking vector between every two of the word embedding vectors according to the semantic relevance, the first adjustable parameter, and the second adjustable parameter includes:

[0062] According to the formula: Generate a context mask vector between every two of the word embedding vectors;

[0063] in, is the context masking vector between word embedding vector i and word embedding vector j, σ is the sigmoid function, α is the first adjustable parameter, β is the second adjustable parameter, is the semantic correlation between word embedding vector i and word embedding vector j, where i and j are positive integers.

[0064] in, That is, a dynamic adjustment function, which is only an example here and is not limited to this. For example, parameters or values ​​can be added to obtain a deformed dynamic adjustment function.

[0065] In the above embodiment, α and β are used to control the shielding strength. Among them, α is mainly responsible for adjusting the sensitivity of semantic relevance to shielding strength. A larger α value will amplify the difference in semantic relevance, making the shielding vector between words with high relevance closer to 1 (i.e., retained), while the shielding vector between words with low relevance is closer to 0 (i.e., shielded); on the contrary, a smaller α value will reduce this difference, making the shielding vector change more gradually. β is mainly responsible for adjusting the baseline of shielding strength, that is, the default shielding strength when there is no influence of semantic relevance. Positive β will tend to retain more information, even if their semantic relevance is not very high; while negative β will tend to shield more information, even if they have a certain degree of semantic relevance.

[0066] In some embodiments, the step of obtaining a label of the first text according to the text semantic graph includes:

[0067] Using the text semantic graph as an input of a graph convolutional network, and outputting local features of the first text, wherein the local features are used to represent local context information between adjacent words in the first text;

[0068] Inputting the word vector matrix into a Transformer encoder, and outputting global features of the first text, wherein the global features are used to represent global context information of the first text;

[0069] Perform feature splicing on the local feature and the global feature to obtain a fusion feature;

[0070] The label is obtained according to the fusion feature.

[0071] Among them, Graph Convolutional Network (GCN) is a deep learning model used to process graph structured data.

[0072] In a specific implementation, the text semantic graph is used as the input of the graph convolutional network, and the local features of the first text are output, including: using the update function of the graph convolutional network to update the feature representation of the node according to the structure of the semantic graph. Figure 2 , assuming that the number of words included in the first text is 4, it shows the layer-by-layer updating process of the text semantic graph.

[0073] The update function is:

[0074]

[0075] in, is the adjacency matrix A plus the identity matrix I (to include self-connections), yes The diagonal matrix of the degree matrix, H (l) is the node feature matrix of the Lth layer, H (l+1) is the node feature matrix of the L+1th layer, W (l) is the weight matrix of the Lth layer, σ is the ReLU activation function, and L layers represents the number of training iterations.

[0076] It should be pointed out that the proximity matrix is ​​the core representation of the text semantic graph, which clarifies the relationship between nodes (such as words and sentences) in the semantic graph. A is the mathematical embodiment of the semantic graph. According to the node and edge relationships in the semantic graph, the adjacency matrix is ​​filled in. For example, in the adjacency matrix, if A[i][j] = 1, it means that there is a semantic association between node i and node j, and if A[i][j] = 0, it means that there is no direct association between node i and node j.

[0077] It should be noted that in GCN, the feature representation update process of the node is actually extracting and enhancing the local features of the first text. By aggregating the features of neighboring nodes, GCN can capture the information of the node in the local structure; as the number of GCN layers increases, the feature representation of the node will gradually integrate more information from neighboring nodes, thereby forming a more complex local feature representation. In other words, the feature representation of each node is updated by aggregating the features of its neighboring nodes. Through this update process, the feature representation of the node not only considers its own features, but also the features of its neighboring nodes, thereby forming a richer feature representation. Through the update process of GCN, the feature representation of the node is updated to contain information about its local context. This means that the local feature is not only the feature of the node itself, but also includes the information of the neighboring nodes directly connected to it. This local feature representation can better reflect the semantic information of the node in the graph structure.

[0078] In a specific implementation, the word vector matrix is ​​input into a Transformer encoder, and the global features of the first text are output, including: using a Transformer encoder structure and a multi-head self-attention mechanism to capture the complex relationship between different parts of the document and provide a global context feature representation for the document; wherein the multi-head self-attention mechanism is:

[0079]

[0080] MultiHead(Q,K,V)=concat(head1,…,head n )W;

[0081] Among them, Q, K, V are the representations of the word vector matrix mapped to the Query, Key and Value matrices; softmax is the activation function used to normalize the attention score; head n represents the nth attention head; W is the learnable matrix.

[0082] Here, d k is the hidden state dimension of Key and Query, indicating the dimension size of the feature vector of Query and Key, which is used to control the size of the feature space in the self-attention mechanism. k The size of d determines the representation capability of Query and Key. k It can capture richer features, but the computational cost is higher; smaller d k The amount of calculation is reduced, but some feature information may be lost.

[0083] Specifically, the word vector matrix is ​​mapped to the Query, Key, and Value matrices in the following ways:

[0084] Matrix Query (Q): Each word needs to find words related to itself from the entire context (i.e., it needs to be queried). Each row of Q is the query vector of the current word in the feature space.

[0085] Matrix Key(K): Each word provides its own features for other words to match (i.e., provide a key). Each row of K is the feature vector of the current word.

[0086] Matrix Value(V): Each word provides its own feature value as the content of the final information propagation. Each row of V is the value vector of the current word.

[0087] The similarities between Query and Key are calculated through Q*K, and Value is used to generate new feature representations using these similarities as weights.

[0088] In a specific implementation, the local features and the global features are spliced ​​to obtain fused features, including:

[0089] The local features extracted by GCN are concatenated with the global features output by the Transformer encoder to create a comprehensive feature representation, namely the fused features, and the labels are generated using the fused features.

[0090] F = Concat(H (L) ,G);

[0091] Among them, F represents the fusion feature, H (L) It is the node feature output by the last layer of GNN, the L layer represents the number of training iterations, G is the global feature representation, and Concat is the feature concatenation operation.

[0092] It should be pointed out that in Natural Language Processing (NLP), local features and global features usually refer to different levels of information representation in text data. Combining graph convolutional networks and Transformer encoders can better understand the meaning of these two features.

[0093] For example, in the sentence "The cat is sleeping in the sun", local features may include the relationship between "cat" and "sleeping", or the connection between "sunlight" and "sleeping". These features help capture the local dependencies between words. Global features refer to the overall information of the entire text, such as the emotion or intention conveyed by "The cat is sleeping in the sun". This feature can reflect the overall context of the text, not just the local lexical relationship.

[0094] In the above embodiment, label prediction is performed using local features extracted based on text semantic graphs and global features output by the Transformer encoder. This fusion strategy not only enhances the model's overall understanding of the text, but also improves the model's adaptability and generalization capabilities in different fields and styles of text, and can improve the accuracy and robustness of label extraction.

[0095] In some embodiments, obtaining the label according to the fusion feature includes:

[0096] Using a Transformer decoder to decode the fused features to generate a decoded sequence;

[0097] Through the softmax function, predict the probability distribution of each word in the sequence;

[0098] The label is determined according to the probability distribution.

[0099] For example, for the tth word y in the prediction sequence t , P vocab (y t )=softmax(W v F t +b v ), where P vocab (y t ) indicates the tth word y in the prediction sequence t The probability distribution in the vocabulary, F is the probability distribution of the tth word y in the prediction sequence t The corresponding fusion feature, W v and b v are learnable parameters.

[0100] Exemplarily, after obtaining the probability distribution corresponding to each word in the first text, one or more words with the largest probability value may be selected as labels, for example, the first K words with the largest probability may be selected as labels, where K is a positive integer.

[0101] For example, suppose the first text is: "I love natural language processing", the word segmentation result is: ["I", "love", "nature", "language", "processing"], and the vocabulary is: {I, love, nature, language, processing}. Assuming that the probability values ​​of the five words "I", "love", "nature", "language" and "processing" in the vocabulary are: 0.10, 0.15, 0.50, 0.20 and 0.05 respectively, then the first two words "nature" and "language" with the highest probability values ​​can be selected as labels.

[0102] As an example, Figure 3 As shown, the text label extraction method includes the following steps:

[0103] Step 301, obtaining a word vector matrix corresponding to the first text; wherein each word in the word vector matrix corresponds to a word embedding vector;

[0104] Step 302, determining a context masking vector between every two word embedding vectors according to the semantic relevance between every two word embedding vectors;

[0105] Step 303, constructing a text semantic graph according to the word embedding vector and the context shielding vector; wherein each node in the text semantic graph corresponds to one word embedding vector, and each context shielding vector is an edge weight between two nodes;

[0106] Step 304, using the text semantic graph as input of the graph convolutional network, and outputting local features of the first text;

[0107] Step 305, inputting the word vector matrix into a Transformer encoder, and outputting the global features of the first text;

[0108] Step 306, local features and global features are combined to obtain fused features;

[0109] Step 307, using a Transformer decoder to perform a decoding operation on the fused features to generate a decoded sequence;

[0110] Step 308, predicting the probability distribution of each word in the sequence through the softmax function;

[0111] Step 309: Determine a label of the first text according to the probability distribution.

[0112] It should be noted that the above steps 302 to 304 and step 305 are not limited in sequence.

[0113] As another embodiment, the step of acquiring a label of the first text according to the text semantic graph includes:

[0114] A graph convolutional network (GCN) is used to extract local context features of the first text from the text semantic graph;

[0115] The local context features are used as input to a machine learning or deep learning model, and a predicted label is output; wherein the machine learning or deep learning model can be obtained by training a classifier.

[0116] For example, all node features in the text semantic graph are average pooled, max pooled, or concatenated to obtain integrated features; the integrated features are input into a trained fully connected neural network or convolutional neural network to output predicted labels.

[0117] like Figure 4 As shown, a text tag extraction device 400 according to an embodiment of the present invention includes:

[0118] A first acquisition module 401 is used to acquire a word vector matrix corresponding to the first text, where each word in the word vector matrix corresponds to a word embedding vector;

[0119] A first determination module 402 is used to determine a context masking vector between each two of the word embedding vectors according to the semantic relevance between each two of the word embedding vectors;

[0120] A second determination module 403 is used to construct a text semantic graph according to the word embedding vector and the context shielding vector; wherein each node in the text semantic graph corresponds to one word embedding vector, and each context shielding vector is an edge weight between two nodes;

[0121] The second acquisition module 404 is used to acquire a label of the first text according to the text semantic graph.

[0122] Optionally, the first determining module 402 includes:

[0123] The first determination submodule is used to determine the context shielding vector between every two of the word embedding vectors based on the semantic relevance, a first adjustable parameter and a second adjustable parameter; wherein the first adjustable parameter is used to adjust the sensitivity of the semantic relevance to the shielding strength, and the second adjustable parameter is used to adjust the baseline of the shielding strength.

[0124] Optionally, the first determining submodule is specifically used for:

[0125] According to the formula: Generate a context mask vector between every two of the word embedding vectors;

[0126] in, is the context masking vector between word embedding vector i and word embedding vector j, σ is the sigmoid function, α is the first adjustable parameter, β is the second adjustable parameter, is the semantic correlation between word embedding vector i and word embedding vector j, where i and j are positive integers.

[0127] Optionally, the above device further includes:

[0128] A third acquisition module is used to obtain the cosine similarity between every two word embedding vectors;

[0129] The third determination module is used to determine the cosine similarity as the semantic relevance.

[0130] Optionally, the second acquisition module 404 includes:

[0131] A first acquisition submodule, used to use the text semantic graph as an input of a graph convolutional network, and output local features of the first text, where the local features are used to represent local context information between adjacent words in the first text;

[0132] A second acquisition submodule is used to input the word vector matrix into a Transformer encoder and output a global feature of the first text, where the global feature is used to represent global context information of the first text;

[0133] A third acquisition submodule is used to perform feature splicing on the local feature and the global feature to obtain a fusion feature;

[0134] The fourth acquisition submodule is used to obtain the label according to the fusion feature.

[0135] Optionally, the fourth acquisition submodule is specifically used for:

[0136] Using a Transformer decoder to decode the fused features to generate a decoded sequence;

[0137] Through the softmax function, predict the probability distribution of each word in the sequence;

[0138] The label is determined according to the probability distribution.

[0139] The text label extraction device 400 is a device corresponding to the above-mentioned text label extraction method. All implementation means in the above-mentioned method embodiment are applicable to the embodiment of the text label extraction device and can also achieve the same technical effect.

[0140] like Figure 5 As shown, an electronic device 500 according to an embodiment of the present invention includes a processor 510 and a transceiver 520, wherein the processor is used to:

[0141] Obtain a word vector matrix corresponding to a first text, wherein each word in the word vector matrix corresponds to a word embedding vector; determine a context shielding vector between each two word embedding vectors according to the semantic relevance between each two word embedding vectors; construct a text semantic graph according to the word embedding vectors and the context shielding vectors; wherein each node in the text semantic graph corresponds to a word embedding vector, and each context shielding vector is an edge weight between two nodes; obtain a label of the first text according to the text semantic graph.

[0142] Optionally, the processor is specifically used to: determine the context masking vector between every two of the word embedding vectors based on the semantic relevance, a first adjustable parameter and a second adjustable parameter; wherein the first adjustable parameter is used to adjust the sensitivity of the semantic relevance to the masking strength, and the second adjustable parameter is used to adjust the baseline of the masking strength.

[0143] Optionally, the processor is specifically configured to:

[0144] According to the formula: Generate a context mask vector between every two of the word embedding vectors;

[0145] in, is the context masking vector between word embedding vector i and word embedding vector j, σ is the sigmoid function, α is the first adjustable parameter, β is the second adjustable parameter, is the semantic relevance between word embedding vector i and word embedding vector j, where i and j are positive integers.

[0146] Optionally, the processor is further configured to:

[0147] Obtaining the cosine similarity between each two of the word embedding vectors;

[0148] The cosine similarity is determined as the semantic relevance.

[0149] Optionally, the processor is specifically configured to:

[0150] Using the text semantic graph as an input of a graph convolutional network, and outputting local features of the first text, wherein the local features are used to represent local context information between adjacent words in the first text;

[0151] Inputting the word vector matrix into a Transformer encoder, and outputting global features of the first text, wherein the global features are used to represent global context information of the first text;

[0152] Perform feature splicing on the local feature and the global feature to obtain a fusion feature;

[0153] The label is obtained according to the fusion feature.

[0154] Optionally, the processor is specifically configured to:

[0155] Using a Transformer decoder to decode the fused features to generate a decoded sequence;

[0156] Through the softmax function, predict the probability distribution of each word in the sequence;

[0157] The label is determined according to the probability distribution.

[0158] The electronic device of this embodiment can shield the noise information irrelevant to the topic and retain the key local context features by generating a context shielding vector and shielding and constructing a text semantic graph based on the context shielding vector, thereby more accurately capturing the key information. Moreover, the application embodiment can improve the performance of document tag extraction in long text and complex text scenarios, reduce the dependence on a large amount of annotated data, and improve the computing efficiency.

[0159] An electronic device according to another embodiment of the present invention, such as Figure 6 As shown, it includes a transceiver 610, a processor 600, a memory 620, and a program or instruction stored in the memory 620 and executable on the processor 600; when the processor 600 executes the program or instruction, the above-mentioned method for extracting text tags is implemented.

[0160] The transceiver 610 is used to receive and send data under the control of the processor 600 .

[0161] Among them, Figure 6 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 600 and various circuits of memory represented by memory 620 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 610 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 630 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0162] The processor 600 is responsible for managing the bus architecture and general processing, and the memory 620 can store data used by the processor 700 when performing operations.

[0163] A computer-readable storage medium according to an embodiment of the present invention stores a program or instruction thereon, and when the program or instruction is executed by a processor, the steps in the text tag extraction method described above are implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0164] To achieve the above objective, an embodiment of the present invention provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the above text tag extraction method.

[0165] It should be further explained that the electronic devices described in this specification include but are not limited to smart phones, tablet computers, etc., and many of the functional components described are called modules in order to more particularly emphasize the independence of their implementation methods.

[0166] In the embodiment of the present invention, module can be implemented with software so that it can be executed by various types of processors. For example, an executable code module of an identification can include one or more physical or logical blocks of computer instructions, for example, it can be constructed as an object, process or function. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different positions, and when these instructions are logically combined together, it constitutes a module and realizes the specified purpose of the module.

[0167] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed among different programs, and distributed across a plurality of memory devices. Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form. The operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0168] When a module can be implemented by software, considering the level of existing hardware technology, a person skilled in the art can build a corresponding hardware circuit to implement the corresponding function of the module that can be implemented by software without considering the cost. The hardware circuit includes a conventional very large scale integration (VLSI) circuit or gate array and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented by a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

[0169] The above exemplary embodiments are described with reference to the accompanying drawings, and many different forms and embodiments are feasible without departing from the spirit and teachings of the present invention. Therefore, the present invention should not be constructed as a limitation of the exemplary embodiments proposed herein. More specifically, these exemplary embodiments are provided so that the present invention will be perfect and complete, and the scope of the present invention will be conveyed to those who are familiar with the technology. In these figures, the component sizes and relative sizes may be exaggerated for clarity. The terms used here are only based on the purpose of describing specific exemplary embodiments and are not intended to be limiting. As used herein, unless the text clearly indicates otherwise, the singular forms "one", "an" and "the" are intended to include these multiple forms. It will be further understood that the terms "including" and / or "comprising" when used in this specification indicate the presence of the features, integers, steps, operations, components and / or components, but do not exclude the presence or increase of one or more other features, integers, steps, operations, components, components and / or their groups. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of that range and any subranges therebetween.

[0170] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A text label extraction method, characterized in that: include: Obtain a word vector matrix corresponding to the first text, wherein each word in the word vector matrix corresponds to a word embedding vector; Determining a context masking vector between each two of the word embedding vectors according to the semantic relevance between each two of the word embedding vectors; Constructing a text semantic graph according to the word embedding vector and the context shielding vector; wherein each node in the text semantic graph corresponds to one word embedding vector, and each context shielding vector is an edge weight between two nodes; According to the text semantic graph, a label of the first text is obtained.

2. The text label extraction method according to claim 1, characterized in that: The determining, according to the semantic relevance between each two of the word embedding vectors, a context masking vector between each two of the word embedding vectors comprises: A context masking vector between every two of the word embedding vectors is determined according to the semantic relevance, a first adjustable parameter and a second adjustable parameter; wherein the first adjustable parameter is used to adjust the sensitivity of the semantic relevance to the masking strength, and the second adjustable parameter is used to adjust the baseline of the masking strength.

3. The text label extraction method according to claim 2, characterized in that: The determining, according to the semantic relevance, the first adjustable parameter and the second adjustable parameter, a context masking vector between every two word embedding vectors comprises: According to the formula: Generate a context mask vector between every two of the word embedding vectors; in, is the context masking vector between word embedding vector i and word embedding vector j, σ is the sigmoid function, α is the first adjustable parameter, β is the second adjustable parameter, is the semantic relevance between word embedding vector i and word embedding vector j, where i and j are positive integers.

4. The text label extraction method according to claim 1, characterized in that: The step of obtaining a label of the first text according to the text semantic graph includes: Using the text semantic graph as an input of a graph convolutional network, and outputting local features of the first text, wherein the local features are used to represent local context information between adjacent words in the first text; Inputting the word vector matrix into a Transformer encoder, and outputting global features of the first text, wherein the global features are used to represent global context information of the first text; Perform feature splicing on the local feature and the global feature to obtain a fusion feature; The label is obtained according to the fusion feature.

5. The text label extraction method according to claim 4, characterized in that: The step of obtaining the label according to the fusion feature includes: Using a Transformer decoder to decode the fused features to generate a decoded sequence; Through the softmax function, predict the probability distribution of each word in the sequence; The label is determined according to the probability distribution.

6. A text label extraction device, characterized in that: include: A first acquisition module is used to obtain a word vector matrix corresponding to the first text, where each word in the word vector matrix corresponds to a word embedding vector; A first determination module is used to determine a context masking vector between every two of the word embedding vectors according to the semantic relevance between every two of the word embedding vectors; A second determination module is used to construct a text semantic graph according to the word embedding vector and the context shielding vector; wherein each node in the text semantic graph corresponds to one word embedding vector, and each context shielding vector is an edge weight between two nodes; The second acquisition module is used to acquire a label of the first text according to the text semantic graph.

7. An electronic device, characterized in that: include: transceivers and processors; The processor is used to obtain a word vector matrix corresponding to the first text, each word in the word vector matrix corresponds to a word embedding vector; According to the semantic relevance between each two of the word embedding vectors, a context shielding vector between each two of the word embedding vectors is determined; according to the word embedding vectors and the context shielding vectors, a text semantic graph is constructed; wherein each node in the text semantic graph corresponds to one of the word embedding vectors, and each of the context shielding vectors is an edge weight between two nodes; according to the text semantic graph, a label of the first text is obtained.

8. An electronic device comprising: A transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; characterized in that when the processor executes the program or instruction, the text label extraction method as described in any one of claims 1 to 5 is implemented.

9. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps in the text tag extraction method according to any one of claims 1 to 5 are implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the text tag extraction method according to any one of claims 1 to 5.

Images