A text analysis method and device, electronic equipment and storage medium

Abstract

The application discloses a text analysis method and device, electronic equipment and storage medium, comprising: obtaining a text to be analyzed, obtaining characters and words in the text after preprocessing the text; encoding the characters and words through a shared pre-training encoding layer to generate a word encoding result; extracting entities in the word encoding result to generate a text entity set; generating a target vector set after pre-training according to the text entity set, generating a first vector after maximum pooling operation on the target vector set; generating a second vector after maximum pooling operation on the word encoding result; after splicing the first vector and the second vector, inputting a full connection layer and a Softmax function to obtain a classification result of the text. The application integrates the features of the analysis and extraction task into the classification task, solves the high-order reasoning problem in the pledge and repurchase task, and improves the overall accuracy in the joint task of text analysis and text classification of the pledge and repurchase.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a text parsing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In text understanding, text classification and text parsing are the two most commonly used techniques. Their main purpose is to identify the intent and extract core elements from a text. For example, in financial transactions, the text "Overnight loan, 1.3e2.3% secured by a bond with a credit rating of AA+" is a text related to repurchase agreements. Its intent is "to provide funds," and the business category is "pricing process." The core elements are shown in Table 1 below.

[0003] Table 1

[0004] Category Direction Term Total Amount Rate Rating Quotation Out 1D 13000W 2.3％ China Bond Credit AA+

[0005] In text understanding, it is often necessary to transform unstructured text into standard structured text as shown above. This involves two directions of deep learning: text classification and text parsing.

[0006] In text classification and parsing tasks, there are two main technical approaches: pipeline and text parsing. Pipeline approaches separate classification and parsing tasks, using different algorithms for each sub-task. Text classification involves determining the category of a text or its intended meaning. This is a relatively mature task, with commonly used models including pre-trained BERT+Softmax and its variants, or faster methods like BiLSTM+Softmax and TextCNN+Softmax. These methods achieve good results for common classification problems. Text parsing involves extracting important information from text. Models for entity extraction are well-established, with common methods including BiLSTM+CRF, BiLSTM+Softmax, BERT+CRF, BERT+Softmax, and GAT+CRF. These methods also achieve good results for common problems.

[0007] The second type is the joint form, which combines text classification and feature parsing tasks into a single model. The common approach is to share pre-trained module parameters in the encoding layer and connect different decoding layers for downstream tasks, thus completing multiple tasks within a single model. There are no mature technical solutions for joint models; instead, targeted joint models are built to meet different task requirements.

[0008] Firstly, for the pipeline model, since two models are used to solve different problems, better results can be achieved on different tasks, but the waste of computing resources caused by the inability to share parameters and the error propagation problem of the pipeline mode.

[0009] Secondly, the joint model uses a unified encoding method, but the model is often complex, and the decoding layer starts to branch to perform decoding of different tasks, which cannot fully utilize the feature information of the text. In addition, there are certain differences between the sequence labeling task and the classification task in the requirement for the encoding layer, and the use of the same encoding layer will also lead to the reduction of the overall accuracy.

[0010] Therefore, the existing model has the defects of resource waste and low parsing accuracy when parsing the text.

[0011] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0012] In view of the above problems in the prior art, the present application provides a text parsing method and device, electronic equipment and storage medium, aiming at solving the problems of resource waste and low parsing accuracy of the model in the prior art when parsing the text.

[0013] The technical scheme of the present application is as follows:

[0014] The first embodiment of the present application provides a text parsing method, which comprises:

[0015] Obtaining the text to be parsed, and obtaining the characters and words in the text after preprocessing the text;

[0016] Encoding the characters and words by sharing the pre-training encoding layer to generate word encoding results;

[0017] Extracting the entities in the word encoding results to generate a text entity set;

[0018] Generating a target vector set after pre-training according to the text entity set, and generating a first vector after maximum pooling operation on the target vector set;

[0019] Generating a second vector after maximum pooling operation on the word encoding results;

[0020] Concatenating the first vector and the second vector, inputting them into a full connection layer and a Softmax function, and obtaining a classification result of the text.

[0021] Further, the encoding of the characters and words by sharing the pre-training encoding layer to generate the word encoding results comprises:

[0022] character feature coding is performed to generate a character vector corresponding to each character;

[0023] After the vocabulary is feature coded, a word vector corresponding to each word is generated;

[0024] The word vector and the word vector are jointly modeled to generate a word coding result.

[0025] Further, the character feature coding to generate a character vector corresponding to each character includes:

[0026] The character is feature coded by the BiLSTM network to generate a character vector corresponding to each character.

[0027] Further, the vocabulary feature coding to generate a word vector corresponding to each word includes:

[0028] The word2vec algorithm is used to feature code the vocabulary to generate a word vector corresponding to each word.

[0029] Further, the word vector and the word vector are jointly modeled to generate a word coding result, including:

[0030] According to the word vector, the word vector is input into the graph attention network layer to construct three different graphs, which are denoted as graph C, graph T and graph L, respectively;

[0031] Get the adjacency matrix corresponding to the graph C, the graph T and the graph L, and denote them as the adjacency matrix AC, the adjacency matrix AT and the adjacency matrix AL, respectively;

[0032] After feature learning of the adjacency matrix AC, the adjacency matrix AT and the adjacency matrix AL, the learned feature results are denoted as features GAT(AC), GAT(AT) and GAT(AL);

[0033] The features GAT(AC), GAT(AT) and GAT(AL) are spliced with the character vector to generate a word coding result.

[0034] Further, the entity in the word coding result is extracted to generate a text entity set, including:

[0035] The entity in the word coding result is extracted by the CRF model to generate a parsed sequence labeling result, and the sequence labeling result is denoted as a text entity set.

[0036] Further, after the text entity set is pre-trained, a target vector set is generated, and after maximum pooling operation on the target vector set, a first vector is generated, including:

[0037] The text entity set is spliced at a set interval to generate a text sequence;

[0038] According to the Bert pre-training model, the text sequence is encoded to generate a target vector set;

[0039] After maximum pooling operation is performed on the target vector set, a first vector is generated.

[0040] Another embodiment of the application provides a text analysis device, the device comprising:

[0041] A data preprocessing module is configured to obtain a text to be analyzed, and obtain characters and words in the text after preprocessing the text;

[0042] An encoding module is configured to encode the characters and words by using a shared pre-training encoding layer to generate a word encoding result;

[0043] An entity extraction module is configured to extract entities in the word encoding result to generate a text entity set;

[0044] A first pooling operation module is configured to generate a target vector set after pre-training according to the text entity set, and generate a first vector after maximum pooling operation is performed on the target vector set;

[0045] A second pooling operation module is configured to generate a second vector after maximum pooling operation is performed on the word encoding result;

[0046] A classification result generation module is configured to splice the first vector and the second vector, input the spliced first vector and second vector into a full connection layer and a Softmax function, and obtain a classification result of the text.

[0047] Another embodiment of the application provides an electronic device, the electronic device comprising at least one processor; and

[0048] A memory in communication connection with the at least one processor; wherein

[0049] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned text analysis method.

[0050] Another embodiment of the application also provides a non-volatile computer readable storage medium, the non-volatile computer readable storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to enable the one or more processors to execute the above-mentioned text analysis method.

[0051] Beneficial effects: The embodiment of the present application fuses the features of the parsing extraction task into the classification task, solves the high-order reasoning problem in the pledge repurchase task, and improves the overall accuracy in the joint task of text parsing and text classification of the pledge repurchase text. BRIEF DESCRIPTION OF DRAWINGS

[0052] The present application will be further described below in conjunction with the drawings and embodiments, wherein:

[0053] Figure 1 The flow chart of a preferred embodiment of the text parsing method of the present application is shown in the figure;

[0054] Figure 2 The text parsing model schematic diagram of the specific application embodiment of the text parsing method of the present application is shown in the figure;

[0055] Figure 3 The classification model schematic diagram of the specific application embodiment of the text parsing method of the present application is shown in the figure;

[0056] Figure 4a The schematic diagram of graph C in the graph attention network of the specific application embodiment of the text parsing method of the present application is shown in the figure;

[0057] Figure 4b The schematic diagram of graph T in the graph attention network of the specific application embodiment of the text parsing method of the present application is shown in the figure;

[0058] Figure 4c The schematic diagram of graph L in the graph attention network of the specific application embodiment of the text parsing method of the present application is shown in the figure;

[0059] Figure 5 The functional module schematic diagram of the preferred embodiment of the text parsing device of the present application is shown in the figure;

[0060] Figure 6 The hardware structure schematic diagram of the preferred embodiment of the electronic device of the present application is shown in the figure. DETAILED DESCRIPTION

[0061] To make the purpose, technical scheme and effect of the present application clearer and more explicit, the present application will be further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0062] For the task of financial pledge repurchase in the prior art, the classification and intent recognition problem is often dominated by a certain fine-grained keyword or key feature, but the existing model (such as attention structure) cannot learn this feature well.

[0063] Similarly, for classification problems, category determination does not rely solely on the text content itself but also requires consideration of industry characteristics. For example, whether the statement "overnight, 1.3e 2.3% pledged to a debt credit rating of AA+" indicates an "inquiry" or a "transaction" requires extracting "elements" from the information provided in the text and combining this with the business context to determine whether certain key elements are missing. However, within a sequence of text, the model cannot effectively learn and infer this logic.

[0064] Using the joint mode, the overall accuracy is low because the encoding features required for parsing and classification tasks are different, which prevents effective commercialization.

[0065] This invention provides a text parsing method for the joint task of classifying and parsing unstructured text in a pledged repurchase transaction. The scheme adopts a joint task model, and the overall framework includes an encoding layer, a graph network layer, a fusion layer, a parsing task layer, and a classification task layer. The main innovation is that it integrates the features extracted from the parsing task into the classification task, solving the high-order reasoning problem in the pledged repurchase task.

[0066] The embodiments of the present invention will be described below with reference to the accompanying drawings.

[0067] This invention provides a text parsing method; please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart illustrating a preferred embodiment of a text parsing method according to the present invention. Figure 1 As shown, it includes the following steps:

[0068] Step S100: Obtain the text to be parsed, and after preprocessing the text, obtain the characters and words in the text;

[0069] Step S200: Encode characters and words by sharing a pre-trained encoding layer to generate word encoding results;

[0070] Step S300: Extract entities from the word encoding results to generate a text entity set;

[0071] Step S400: After pre-training based on the text entity set, a target vector set is generated. After performing max pooling on the target vector set, the first vector is generated.

[0072] Step S500: After performing max pooling on the word encoding results, a second vector is generated;

[0073] Step S600: After concatenating the first vector and the second vector, input them into the fully connected layer and the Softmax function to obtain the text classification result.

[0074] In actual implementation, the embodiment of the application is mainly applied to a joint task of classifying and analyzing unstructured text in a pledge repurchase, a scheme adopts a joint mode, and the overall framework includes an encoding layer, a graph network layer, a fusion layer, an analysis task layer, and a classification task layer. The main innovation is to fuse the features of the analysis and extraction task into the classification task to solve the high-order reasoning problem in the pledge repurchase task.

[0075] First, a shared pre-training encoding layer is used to encode characters. In order to improve the boundary drift problem in the information extraction task, the scheme uses External Attention to strengthen the graph attention network GAT to jointly model the characters and words in the pre-training encoding layer, and improve the feature understanding of the word information. The encoding result is denoted as Text-Encoding. Then, CRF (Conditional Random Field) is used to extract entities to obtain the entity set of the text. The framework process of the analysis module is as shown in Figure 2 .

[0076] Secondly, for the text classification task, the scheme innovatively fuses the encoding information of the entity set and the encoding information of the characters to obtain high-order text encoding information, and then performs text classification. First, the text entity set is encoded by the pre-training module, and the vector set is converted into a vector V1 by Max Pooling, and the Text_encoding encoding information is MaxPooling to convert the vector set into V2. V1 and V2 are spliced, and then pass through a fully connected layer and Softmax to obtain the probability of each category corresponding to the Chinese text, thereby obtaining the classification result of the text. The model framework of the classification task is as shown in Figure 3 .

[0077] The classification task has the NER_encoding of the previous layer, and after Max pooling, the vector set is converted into a vector V1. The Text_encoding saved in the NER task is Max-pooled to obtain a vector V2. The two Attention operations are collectively referred to as Attention. V1 and V2 are spliced, and then connected to a fully connected layer and a Softmax operation, and finally the probability of the text belonging to each category is obtained. The process is called Classification.

[0078] In the GAT network layer in the embodiment of the application, the method of External Attention is used for optimization, and the embodiment of the application can be used not only in GAT, but also in any attention mechanism scheme.

[0079] In some other embodiments, the design of the training and prediction stages is not limited to financial services and is applicable to joint tasks of text parsing and text classification.

[0080] In one embodiment, the characters and words are encoded by sharing the pre-training encoding layer to generate word encoding results, including:

[0081] The characters are feature-encoded to generate a word vector corresponding to each word;

[0082] After feature-encoding the words, a word vector corresponding to each word is generated;

[0083] The word vectors and word vectors are jointly modeled to generate word encoding results.

[0084] In specific implementation, the scheme adopts a joint bottom layer encoding scheme for character encoding of sentence and word encoding of Match words. The characters are feature-encoded to generate a word vector corresponding to each word; after feature-encoding the words, a word vector corresponding to each word is generated; and the word vectors and word vectors are jointly modeled to generate word encoding results.

[0085] In one embodiment, the characters are feature-encoded to generate a word vector corresponding to each word, including:

[0086] The characters are feature-encoded by a BiLSTM network to generate a word vector corresponding to each word.

[0087] In specific implementation, for the character encoding layer, there are many mature technologies such as BiLSTM, TextCNN, BERT, etc. The scheme feature-encodes the characters based on BiLSTM, and finally each character {c1, c2, …, cm} forms m word vectors with a dimension of 300 (including but not limited to 300), denoted as SenEncoding.

[0088] In one embodiment, after feature-encoding the words, a word vector corresponding to each word is generated, including:

[0089] The words are feature-encoded by a word2vec algorithm to generate a word vector corresponding to each word.

[0090] In specific implementation, for the word encoding layer, first, the text is subjected to part-of-speech cutting (in this scheme, the jieba word segmentation tool optimized for financial industry data is adopted, i.e., many professional words in the financial industry are added to the dictionary of the jieba word segmentation tool), and word2vec is used to encode the word segmentation. For the numerical words, a word vector is set for the numerical value of each preset interval, so as to reflect the similarity of the numerical values in the same interval. Finally, each word {w1, w2, wn} forms a word vector with n dimensions of 300 (including but not limited to 300), denoted as WordEncoding.

[0091] In one embodiment, the word vector and the character vector are jointly modeled to generate a word-character encoding result, including:

[0092] According to the word vector, the word vector is input into the graph attention network layer, and three different graphs are constructed, denoted as graph C, graph T and graph L respectively.

[0093] The adjacency matrices corresponding to the graphs C, T and L are obtained, and are denoted as adjacency matrix AC, adjacency matrix AT and adjacency matrix AL respectively.

[0094] After feature learning is performed on the adjacency matrices AC, AT and AL, the learned feature results are denoted as features GAT(AC), GAT(AT) and GAT(AL) respectively.

[0095] The features GAT(AC), GAT(AT) and GAT(AL) are spliced with the word vector to generate a word-character encoding result.

[0096] In specific implementation, the graph attention network layer (Graph Layer) uses three different ways to construct three types of graphs (i.e., GAT_1, GAT_2, GAT_3), including Word-Character ContainingGraph, Word-CharacterTransition graph, Word-Character Lattice graph, which are represented by C, T and L respectively. The specific graph structure is shown in FIGS. 4a, 4b and 4c. Figure 4a 4b and 4c:

[0097] ​The same encoding mode is adopted for C, T and L. The encoding model adopted in the scheme is a graph attention network GAT, and the adjacency matrices corresponding to C, T and L are denoted as AC, AT and AL. Taking the above example as an example, the connection between characters and words is connected, if the position i and the position j in the graph C are connected, then AC[i,j]=AC[j,i]=1, it is assumed that the position i and the position i are connected, then AC[i,i]=1, and all other non-connected positions corresponding to AC are 0. Similarly, AT and AL are also obtained in this way.

[0098] Feature learning is performed on the adjacency matrices AC, AT and AL. The following represents the design mode of the GAT network of the adjacency matrices AC, AT and AL.

[0099] Taking the adjacency matrix AC as an example, in the GAT operation, the input of the t-th layer is a point set F t ={f1,f2,...,f N}, and there is an adjacency matrix G, and the GAT operation with multiple heads is used as follows:

[0100]

[0101]

[0102] wherein f j '∈R F represents the input feature of point j, the input of the feature is [SenEncoding:WordEncoding], that is, SenEncoding and WordEncoding are spliced in the length dimension; f i '∈R F′ represents the output feature of point i; || represents the splicing operation; σ represents an activation function, and a nonlinear activation function is generally selected; V i represents the adjacent vertex of i; represents the attention weight between point i and point j; M k ∈R F‘×F represents a linear transformation matrix; and are respectively a weight of a forward feedback neural network, and ∑* represents a summation operation.

[0103] An optimization here is that formula 1 is optimized into the following formula

[0104]

[0105] In formula 3, similar to the idea of External Attention in image recognition, V j, represents the prior knowledge of the data set, and all data sets share one V j In different layers of the GAT graph network, different V j There are as many V j The purpose of this is because the data itself has many similar features, using V j Combined with the f j of the above text itself, GAT can learn more text knowledge, rather than just its own knowledge.

[0106] The output of the last layer is obtained by t = 1, 2,..., N respectively Then calculate the AC corresponding to the result of GAT:

[0107]

[0108] Similarly, the adjacency matrix AM and the feature results GAT(AT) and GAT(AL) corresponding to the graph are obtained respectively.

[0109] The fusion layer uses a direct splicing method to splice, based on the GAT(AC), GAT(AT), and GAT(AL) obtained in the graph attention network layer, and the previously obtained character-level encoding SenEncoding. Assuming the length of the character is m, they are fused according to the following method.

[0110] Text_encoding = SenEncoding + GAT(AC)[:m] + GAT(AT)[:m] + GAT(AL)[:m]

[0111] Where GAT(AC)[:m], GAT(AT)[:m], GAT(AL)[:m] represent the selection of GAT(AC), GAT(AT), and GAT(AL) in the length direction. The first m vectors are obtained. The final text encoding Text_encoding is obtained.

[0112] In one embodiment, the entities in the word encoding result are extracted to generate a text entity set, including:

[0113] The entities in the word encoding result are extracted by the CRF model to generate the parsed sequence labeling result, and the sequence labeling result is recorded as the text entity set.

[0114] In specific implementation, mature CRF technology is used to decode Text_encoding to obtain the sequence labeling result of the parsing task, which is recorded as the text entity set. At the same time, Text_encoding is saved for subsequent classification tasks.

[0115] In a further embodiment, after pre-training according to the text entity set, a target vector set is generated, and after maximum pooling operation on the target vector set, a first vector is generated, including:

[0116] The text entity set is spliced at a set interval to generate a text sequence.

[0117] The text sequence is encoded according to the Bert pre-training model to generate a target vector set.

[0118] After maximum pooling operation on the target vector set, a first vector is generated.

[0119] In a specific implementation, the classification task encoding layer (Encoding of NER entity) ; For the classification task, the scheme does not directly use the text encoding information obtained by the fusion layer for decoding of the classification task, but innovatively uses the result of the analysis task again as the basis for the classification task. Based on the decoding layer of the analysis task, the entity set is obtained, and all entities are spliced with [SEP] as an interval to form a new text sequence.

[0120] For the above text sequence, the Bert pre-training model is used to encode the text sequence to form a vector set fixed to degrees, defined as NER_encoding. After maximum pooling operation on the target vector set, a first vector is generated.

[0121] As can be seen from the above method embodiment, the present application provides a text analysis method. In the GAT network design, the idea of external attention is used to add priori knowledge of text into the model to increase the connection between sample sets. In the joint task of analyzing and classifying the text of the pledge and repurchase, the overall accuracy is improved by more than 5%.

[0122] It should be noted that the above steps do not necessarily have a certain order, and those skilled in the art can understand from the description of the embodiments of the present application that the above steps can have different execution orders in different embodiments, that is, they can be executed in parallel, or they can be executed in exchange, etc.

[0123] Another embodiment of the present application provides a text analysis device, as shown in Figure 5 The device 1 includes:

[0124] The data preprocessing module 11 is used to acquire the text to be analyzed, and acquires characters and words in the text after preprocessing the text;

[0125] The encoding module 12 is used to encode the characters and words by sharing the pre-training encoding layer to generate a word encoding result.

[0126] The entity extraction module 13 is configured to extract entities in the word encoding result to generate a text entity set.

[0127] The first pooling operation module 14 is configured to generate a target vector set after pre-training according to the text entity set, and generate a first vector after maximum value pooling operation on the target vector set.

[0128] The second pooling operation module 15 is configured to generate a second vector after maximum value pooling operation on the word encoding result.

[0129] The classification result generation module 16 is configured to splice the first vector and the second vector, input the full connection layer and the Softmax function, and obtain the classification result of the text.

[0130] The specific implementation is described in the method embodiment, and will not be repeated here.

[0131] Another embodiment of the present application provides an electronic device, such as Figure 6 As shown in the figure, the electronic device 10 comprises:

[0132] one or more processors 110 and memories 120, Figure 6 In an embodiment, the processor 110 and the memory 120 can be connected through a bus or other means, Figure 6 In an embodiment, the connection through the bus is taken as an example.

[0133] The processor 110 is configured to complete various control logics of the electronic device 10, and can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single chip microcomputer, an ARM (Acorn RISC Machine), or other programmable logic devices, discrete gates or transistor logic, discrete hardware controls, or any combination of these components. In addition, the processor 110 can also be any conventional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, such as a combination of DSP and microprocessor, multiple microprocessors, one or more microprocessors combined with DSP cores, or any other such configuration.

[0134] The memory 120 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the text analysis method in the embodiment of the present application. The processor 110 executes various function applications and data processing of the device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implements the text analysis method in the above method embodiment.

[0135] The memory 120 can include a program storage area and a data storage area. The program storage area can store an operating system for operating the device, and application programs required for at least one function. The data storage area can store data created according to the use of the device 10, and the like. The memory 120 can include a high-speed random access memory, and can also include a nonvolatile memory such as at least one of a magnetic disk storage device, a flash memory device, or other nonvolatile solid-state storage device. In some embodiments, the memory 120 can optionally include a memory that is remotely located with respect to the processor 110, and can be connected to the device 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0136] One or more units are stored in the memory 120, and when executed by the one or more processors 110, perform the text analysis method of any of the above method embodiments, for example, perform the method steps S100 to S600 in the above described Figure 1

[0137] An embodiment of the present application provides a non-volatile computer readable storage medium, the computer readable storage medium stores computer executable instructions, the computer executable instructions are executed by one or more processors, for example, the method steps S100 to S600 in the above described Figure 1

[0138] By way of example, non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM), as external cache memory. By way of illustration, RAM can be available at many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM), etc. The disclosed memory control or memory of the operating environment described herein is intended to include one or more of these and / or any other suitable type of memory.

[0139] Another embodiment of the present application provides a computer program product, the computer program product includes a computer program stored on a non-volatile computer readable storage medium, the computer program includes program instructions, when the program instructions are executed by a processor, the processor executes the text analysis method of the above method embodiments. For example, the method steps S100 to S600 in the above described Figure 1 ​​The method steps S100 to S600 in the method.

[0140] The above-described embodiments are merely illustrative for describing the present application and the units described as separate parts can or can not be physically separate, and parts displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a general hardware platform, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product in essence or in the form of a part of the related technology. The computer software product can exist in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method of each embodiment or some part of the embodiment.

[0142] Conditional language such as, among others, "can," "could," "might" or "may," unless specifically stated otherwise, generally are intended to convey that a certain feature, element or operation can or can not be included in some or all examples of the present application. Thus, such conditional language generally is not intended to imply that a feature, element or operation is required in one or more implementations or that a feature, element or operation is necessary for one or more implementations. The

[0143] What has been described herein in the specification and drawings includes examples of methods and apparatuses that can provide text parsing. Of course, the application also contemplates methods and apparatuses that do not include, or secondarily to, all of the features that can be described in the specification and drawings. Thus, the application contemplates other embodiments that include the features described herein, and that are written or rendered in other ways. The application also contemplates embodiments that include any additional or alternative features that can be described in the specification and drawings. Thus, the application is not limited to the embodiments that have been described in the specification and drawings.

Claims

1. A method of text parsing, characterized by The method comprises: acquiring a text to be parsed, and acquiring characters and words in the text after preprocessing the text; encoding the characters and the words through a shared pre-training encoding layer to generate word encoding results; extracting entities in the word encoding results to generate a text entity set; generating a target vector set after pre-training according to the text entity set, and generating a first vector after maximum pooling operation on the target vector set; generating a second vector after maximum pooling operation on the word encoding results; concatenating the first vector and the second vector, inputting the first vector and the second vector into a full connection layer and a Softmax function, and obtaining a classification result of the text; the encoding of the characters and the words through the shared pre-training encoding layer to generate the word encoding results comprises: feature encoding of the characters to generate a word vector corresponding to each word; feature encoding of the words to generate a word vector corresponding to each word; joint modeling of the word vectors and the word vectors to generate the word encoding results; the joint modeling of the word vectors and the word vectors to generate the word encoding results comprises: according to the word vectors, inputting the word vectors into a graph attention network layer to construct three different graphs, denoted as graph C, graph T and graph L respectively; acquiring adjacency matrices corresponding to the graph C, the graph T and the graph L, and denoted as adjacency matrix AC, adjacency matrix AT and adjacency matrix AL respectively; feature learning on the adjacency matrix AC, the adjacency matrix AT and the adjacency matrix AL, and denoted as feature GAT(AC), feature GAT(AT) and feature GAT(AL) respectively after learning; concatenating the feature GAT(AC), the feature GAT(AT) and the feature GAT(AL) with the word vectors to generate the word encoding results; the extraction of the entities in the word encoding results to generate the text entity set comprises: extracting the entities in the word encoding results through a CRF model to generate a parsed sequence labeling result, and denoted as the text entity set; the generation of the target vector set after pre-training according to the text entity set, and the generation of the first vector after maximum pooling operation on the target vector set comprises: concatenating the text entity set at a set interval to generate a text sequence; encoding the text sequence according to a Bert pre-training model to generate the target vector set; generating the first vector after maximum pooling operation on the target vector set; wherein, the graph C captures the association relationship of 'word containing characters', the graph T captures the semantic transition relationship of 'characters - word', and the graph L captures the lattice association relationship of 'characters - word'; The attention network of the shared pre-training coding layer introduces dataset prior knowledge Vj to the node feature is optimized, and the formula is: , wherein represents the output feature of point i; represents a splicing operation; represents an activation function; represents the adjacent vertex of i; represents the attention weight between point i and point j; represents a linear transformation matrix, represents the input feature of point j, the general prior rule of the repurchase field text.

2. The method of claim 1, wherein, the feature encoding of the characters to generate a word vector corresponding to each word comprises: feature encoding of the characters through a BiLSTM network to generate a word vector corresponding to each word.

3. The method of claim 1, wherein, the feature encoding of the words to generate a word vector corresponding to each word comprises: feature encoding of the words through a word2vec algorithm to generate a word vector corresponding to each word.

4. A text analysis device, characterized by The device comprises: a data preprocessing module configured to acquire a text to be parsed, and acquire characters and words in the text after preprocessing the text; The encoding module is configured to encode characters and words by sharing a pre-trained encoding layer to generate word encoding results; The entity extraction module is configured to extract entities in the word encoding results to generate a text entity set; The first pooling operation module is configured to generate a target vector set after pre-training according to the text entity set, and generate a first vector after maximum value pooling operation on the target vector set; The second pooling operation module is configured to generate a second vector after maximum value pooling operation on the word encoding results; The classification result generation module is configured to splice the first vector and the second vector, input the full connection layer and the Softmax function, and obtain the classification result of the text; The encoding module is configured to encode characters and words by sharing a pre-trained encoding layer to generate word encoding results; The character is feature-encoded to generate a word vector corresponding to each word; The word vector is generated after feature encoding of the word; The word vector and the word vector are jointly modeled to generate the word encoding result; The word vector and the word vector are jointly modeled to generate the word encoding result, including: According to the word vector, input the word vector into the graph attention network layer to construct three different graphs, which are denoted as graph C, graph T and graph L respectively; Obtain the adjacency matrix corresponding to the graph C, the graph T and the graph L, and denote them as the adjacency matrix AC, the adjacency matrix AT and the adjacency matrix AL respectively; After feature learning on the adjacency matrix AC, the adjacency matrix AT and the adjacency matrix AL, the learned feature results are denoted as the feature GAT(AC), the feature GAT(AT) and the feature GAT(AL); The feature GAT(AC), the feature GAT(AT) and the feature GAT(AL) are spliced with the word vector to generate the word encoding result; The entity extraction module is configured to extract entities in the word encoding results to generate a text entity set; The entity in the word encoding result is extracted by the CRF model to generate a parsed sequence labeling result, and the sequence labeling result is denoted as the text entity set; The first pooling operation module is configured to generate a target vector set after pre-training according to the text entity set, and generate a first vector after maximum value pooling operation on the target vector set; The text sequence is generated by splicing the text entity set at a set interval; The target vector set is generated by encoding the text sequence according to the Bert pre-training model; The first vector is generated after maximum value pooling operation on the target vector set; The graph C captures the association relationship of 'word containing character', the graph T captures the semantic transition relationship of 'character - word', and the graph L captures the lattice association relationship of 'character - word'; The attention network of the shared pre-training coding layer introduces dataset prior knowledge Vj, and the node feature is optimized, and the formula is: wherein, represents the output feature of point i; represents a splicing operation; represents an activation function; represents the adjacent vertex of i; represents the attention weight between point i and point j; represents a linear transformation matrix, represents the input feature of point j, the general prior rule of the repurchase field text.

5. An electronic device, comprising: The electronic device includes at least one processor; and The memory is in communication connection with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the text parsing method of any one of claims 1-3.

6. A non-transitory computer readable storage medium, comprising: The non-volatile computer readable storage medium stores computer executable instructions, which, when executed by one or more processors, can cause the one or more processors to perform the text analysis method of any one of claims 1-3.

Images