A method, device and electronic equipment for extracting key information in text
By matching target labels to word chunks in text, the problem of low accuracy caused by noise interference in text processing models is solved, enabling efficient extraction and correction of key information and improving the training efficiency and accuracy of text processing models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2023-05-30
- Publication Date
- 2026-07-07
Smart Images
Figure CN116662545B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of language processing technology, and in particular to a method, apparatus and electronic device for extracting key information from text. Background Technology
[0002] The saying goes, "Data quality determines the upper limit of data mining, while algorithms merely approximate this limit." In deep learning tasks related to natural language processing, such as text generation and machine translation, improving data quality is crucial for task completion and enhancing the processing object. For example, in text processing involving the matching of data elements such as object class words, characteristic words, or representation words, data quality also significantly impacts the text processing results. Specifically, data element matching in text processing refers to calculating the similarity between the data element to be matched and data elements in the database, retrieving the data element with the highest similarity from the database as the matching data element for the data element to be matched. Therefore, the essence of data element matching tasks is text matching. The key to high-quality text processing and data element matching lies in the ability to achieve high-quality text vector representation. In other words, the better the text is represented by vectors, the better the final processing result.
[0003] However, in the process of text vectorization, the first thing to be processed is the original field annotations. These annotations are explanations and descriptions of the field settings to facilitate reader understanding. Although the original field annotations help with text matching, because they are artificially added for reader convenience, they often include redundant information such as rhetorical devices and analogies. When the text and its original field annotations are processed together in the language processing model, they often introduce a lot of noise, making it difficult to improve the accuracy of the language processing model's output.
[0004] To address this, existing technologies employ language processing models to identify and extract important segments from the original text to reduce noise. These text segments and their original annotations are then processed for alignment. However, this method cannot always identify and extract important segments. The large amount of redundant information still results in significant noise pressure on the language processing model, leading to uncontrollable results: difficulty in accurately and precisely identifying important segments from the original text, resulting in low accuracy of the obtained key information. Summary of the Invention
[0005] This application provides a method, apparatus, and electronic device for extracting key information from text, thereby improving the accuracy of key information extracted from text.
[0006] In a first aspect, embodiments of this application provide a method for extracting key information from text, including:
[0007] Obtain the text to be processed and input the text to be processed into the text processing model;
[0008] The text processing model matches target tags to word blocks in the text to be processed from a variety of preset tags; wherein the target tag indicates the editing action to be performed corresponding to the word block.
[0009] Based on the target tag, the word blocks in the text to be processed are processed to obtain the key information of the text to be processed.
[0010] In one possible implementation, the preset tags include categories such as delete, retain, replace, and add; wherein the replace includes the object to be replaced, and the add includes the object to be added.
[0011] One possible implementation involves matching target tags for word blocks in the text to be processed from a variety of preset tags using the text processing model, including:
[0012] The text to be processed is divided into word blocks, and position markers are added to the word blocks; wherein, the position markers indicate the relative positional relationship between the word blocks and other word blocks in the text to be processed;
[0013] Based on the word block and its position marker, self-attention encoding is performed on the word block to obtain a word block matrix corresponding to the word block; wherein, the number of row vectors in the word block matrix is equal to the number of hidden layers in the self-attention encoding calculation;
[0014] Based on the word block matrix, the target tag is matched for the word block in the preset tags.
[0015] One possible implementation involves performing self-attention encoding calculation on the word blocks based on the word blocks and their positional markers to obtain a word block matrix corresponding to the word blocks, including:
[0016] In the hidden layer of the text processing model, the word block is self-attention encoded, and a sub-matrix corresponding to the word block is obtained in each hidden layer.
[0017] The submatrix of each word block is input into the convolutional neural network in the text processing model to obtain the word block matrix corresponding to each word block.
[0018] One possible implementation includes, before inputting the text to be processed into the text processing model, the following steps:
[0019] The acquired training text is divided using a text processing model to be trained, resulting in training word blocks. The training text includes a correspondence between preset word blocks and standard labels, where the standard labels indicate the editing action to be performed corresponding to the word block.
[0020] Add position markers to the training word blocks, and match training labels to the training word blocks in the training text from the preset labels;
[0021] Based on the first error between the training word block and the preset word block, and the second error between the training label and the standard label, the parameters in the text processing model to be trained are adjusted until the accuracy of the training word block output by the text model to be trained is greater than a first set threshold, and the accuracy of the training label is greater than a second set threshold, thus obtaining the text processing model.
[0022] One possible implementation, wherein matching training labels for training word blocks in the training text from preset labels, includes:
[0023] Self-attention encoding is performed on the training word blocks, and a sub-matrix corresponding to the training word blocks is obtained in each hidden layer;
[0024] Feature information is extracted from the sub-matrix corresponding to the same training word block and fused to obtain a feature matrix corresponding to each training word block;
[0025] Based on the feature matrix, the training label is matched to the training word block in the preset label.
[0026] Secondly, embodiments of this application provide an apparatus for extracting key information from text, comprising:
[0027] The input unit is used to acquire the text to be processed and assign the text to the text processing model.
[0028] The matching unit is used to match target tags for word blocks in the text to be processed from a variety of preset tags using the text processing model; wherein the target tag indicates the editing action to be performed corresponding to the word block;
[0029] An information unit is used to process the word blocks in the text to be processed based on the target tag to obtain key information of the text to be processed.
[0030] In one possible implementation, the preset tags include categories such as delete, retain, replace, and add; wherein the replace includes the object to be replaced, and the add includes the object to be added.
[0031] In one possible implementation, the matching unit is specifically used to divide the text to be processed into word blocks and add position markers to the word blocks; wherein, the position markers indicate the relative positional relationship between the word blocks and other word blocks in the text to be processed; based on the word blocks and the position markers of the word blocks, self-attention encoding calculation is performed on the word blocks to obtain a word block matrix corresponding to the word blocks; based on the word block matrix, the target tag is matched for the word blocks in the preset tags.
[0032] In one possible implementation, the matching unit is specifically used to perform self-attention encoding calculation on the word block in the hidden layer of the text processing model, and obtain a sub-matrix corresponding to the word block in each hidden layer; and to assign the sub-matrix of each word block to the convolutional neural network matrix in the text processing model to obtain a word block matrix corresponding to each word block.
[0033] In one possible implementation, the apparatus further includes a training unit, specifically configured to use a text processing model to be trained to segment the acquired training text and obtain training word blocks; wherein the training text includes a correspondence between preset word blocks and standard labels, the standard labels indicating the editing action to be performed corresponding to the word blocks; adding position markers to the training word blocks and matching training labels to the training word blocks in the training text in the preset labels; adjusting the parameters in the text processing model to be trained based on a first error between the training word blocks and the preset word blocks, and a second error between the training labels and the standard labels, until the accuracy of the training word blocks output by the text processing model to be trained is greater than a first set threshold, and the accuracy of the training labels is greater than a second set threshold, thereby obtaining a text processing model.
[0034] In one possible implementation, the training unit is specifically used to perform self-attention encoding calculation on the training word block, obtain a sub-matrix corresponding to the training word block in each hidden layer; extract a feature matrix corresponding to the same training word block; and match the training label for the training word block in the preset label based on the feature matrix.
[0035] Thirdly, embodiments of this application provide a readable storage medium, including,
[0036] memory,
[0037] The memory is used to store a computer program that, when executed by a processor, causes the apparatus including the readable storage medium to perform the method as described in the first aspect and any possible implementation.
[0038] Fourthly, embodiments of this application also provide an electronic device, including:
[0039] Memory, used to store computer programs;
[0040] When a processor executes a computer program stored in the memory, it implements the method as described in the first aspect and any possible implementation.
[0041] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects:
[0042] By adding target labels indicating editing actions to tokens (i.e., the aforementioned word blocks) in the text, colloquial expressions, erroneous edits, or omissions in the text are promptly rewritten, effectively reducing noise in the text processing model when processing the text. Furthermore, errors in the text can be corrected through the matched target labels, thereby improving the accuracy of the final key information. Simultaneously, transforming key information extraction into a text rewriting task of token classification effectively reduces the amount of data processed by the text processing model after receiving the text. This avoids the slow convergence speed caused by manually matching word blocks with every character and word in the dictionary to determine match, enabling the text processing model provided in this embodiment to more efficiently determine the key information corresponding to the text. Based on this, the training efficiency of the text processing model provided in this embodiment is also effectively improved, effectively alleviating the training and usage pressure of the text processing model. Attached Figure Description
[0043] Figure 1 A flowchart illustrating a method for extracting key information from text, provided in an embodiment of this application;
[0044] Figure 2 This is a schematic diagram illustrating the connection between the hidden layer and the convolutional neural network in the text processing model provided in the embodiments of this application.
[0045] Figure 3 A schematic diagram of the structure of a device for extracting key information from text, provided in an embodiment of this application;
[0046] Figure 4 This is a schematic diagram of the structure of an electronic device for extracting key information from text, provided in an embodiment of this application. Detailed Implementation
[0047] To address the issue of low accuracy in extracting key information from text in existing technologies, this application provides a method for extracting key information from text: Utilizing a text processing model, target tags are matched to word blocks in the text to be processed from various types of preset tags to determine the editing action to be performed for each word block, i.e., determining the keyness of the word block or the key information it expresses. Then, each word block is edited according to the target tags to obtain the key information in the text to be processed. This allows erroneous word blocks in the text to be identified and corrected, thereby effectively improving the accuracy of the extracted key information and avoiding the low accuracy problem caused by existing technologies that extract subsets / fragments of text as key information, making it difficult to focus on erroneous content in the text.
[0048] It should be noted that the word blocks described in the embodiments of this application are the tokens in current language processing.
[0049] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.
[0050] Please refer to Figure 1 This application proposes a method for extracting key information from text, aiming to improve the accuracy of key information extracted from text. The method specifically includes the following implementation steps:
[0051] Step 101: Obtain the text to be processed and input the text to be processed into the text processing model.
[0052] Specifically, the text to be processed is already cleaned to initially reduce the interference of redundant information in extracting key information. Cleaning methods include deleting irrelevant text and fragments from the original text, such as deleting web text and symbols.
[0053] For example, if the original text is "Legal Representative_Person in Charge_ID Number", then the text to be processed after cleaning can be "Legal Representative_Person in Charge_ID Number".
[0054] Step 102: Using the text processing model, match target tags for word blocks in the text to be processed from a variety of preset tags.
[0055] The target tag indicates the editing action to be performed for the corresponding word block. The type of the editing action to be performed, i.e., the type of the target tag / preset tag, includes deletion.
[0056] The preset tags mentioned above include deletion, retention, replacement, and addition. Replacement includes the object to be replaced, and addition includes the object to be added. That is, adding a preset tag means inserting the object to be added before the corresponding token. For example, if I represents addition, Ia means inserting 'a' before the token corresponding to this tag as the target tag; then 'a' is the object to be added. If R represents replacement, Ra means deleting the token corresponding to this tag as the target tag and inserting 'a' in the position of the deleted token; then 'a' is the object to be replaced.
[0057] Furthermore, this text processing model can sequentially include a tokenizer, an encoder, and a convolutional neural network (Text CNN). Alternatively, the tokenizer can be placed first, in which case the text processing model sequentially includes an encoder and a convolutional neural network (Text CNN).
[0058] The tokenizer divides the text to be processed into tokens (i.e., word blocks). The encoder vectorizes the tokens, translating them into machine language. The convolutional neural network extracts and fuses key information from the vectorized tokens to obtain a representation H corresponding to each token. i It matches a unique target tag for each token among multiple preset tags, and determines whether to accept or reject the token based on the editing action indicated by the target tag, thereby obtaining the key information of the text to be processed.
[0059] Therefore, after inputting the text to be processed into the text processing model, it can first be tokenized using a tokenizer: that is, the text to be processed is divided into multiple word blocks (i.e., tokens) using a tokenizer. Then, a position marker is added to each token. This position marker indicates the relative positional relationship between the word block and other word blocks in the text to be processed.
[0060] Next, using an encoder, self-attention encoding is performed on the tokens in the text to be processed based on the tokens and their positional markers, resulting in a chunk matrix corresponding to the aforementioned word blocks. The number of row vectors in the chunk matrix is equal to the number of hidden layers in the self-attention encoding calculation. In other words, the tokens are self-attention encoded in different dimensions through hidden layers in the encoder. The chunk matrix obtained from each hidden layer indicates the meaning / feature of a word block from a preset dimension / angle. The number of embeddings in this hidden layer can be 12. Except for the first embedding layer, each subsequent hidden layer embedding represents the token from different angles, such as grammatical features, emotional connotations, etc., resulting in different representations for each hidden layer, i.e., different sub-matrices corresponding to the same token.
[0061] After obtaining the submatrix, the submatrix of each token is input into the convolutional neural network in the text processing model for feature calculation and feature fusion, resulting in a word block matrix corresponding one-to-one with the token. For the connection between the aforementioned hidden layer embedding and the convolutional neural network TextCNN, please refer to [link to documentation]. Figure 2 .
[0062] Finally, based on the word chunk matrix, the target tag is matched for the token within the preset tags. In fact, matching the target tag for the token here utilizes a classifier to categorize the token based on the editing action. For example, the action could be deleting the token, keeping the token, or replacing the token with another token.
[0063] Step 103: Process the word blocks in the text to be processed based on the target tags to obtain the key information of the text to be processed.
[0064] In the text processing model described above, after the convolutional neural network TextCNN and the classifier following it, the output can have various settings. These settings are used to decode the token and its target label, perform the editing action indicated by the target label on the token, and then output the result. Examples are provided below:
[0065] (1) Based on the attention mechanism, the encoder is connected to the decoder. It can be compared with the Seq2Seq architecture (Sequence to Sequence). Each hidden layer embedding in the encoder is connected to TextCNN, and the classifier is connected to TextCNN. Therefore, the encoder is connected to the classifier in the encoder.
[0066] (2) Using a fully connected architecture, an activation function (softmax) layer is added after the classifier to obtain the output of each token and its target label.
[0067] (3) Each time the classifier in the encoder outputs a result, the hidden state of the encoder is directly calculated based on the classifier's output to obtain the output of each token and its target label. This also belongs to autoregression.
[0068] The aforementioned text processing model is actually a language model trained to achieve an accuracy that meets a certain threshold. During training, the text processing model provided in this embodiment, particularly the encoder, can be based on BERT or XLNet. Specifically, the self-attention calculation for the embedding in the encoder corresponds to the pre-training phase of the BERT or XLNet model, while matching the target label to the token after the neural convolutional network outputs the hidden representation (Hi) of the token corresponds to the fine-tuning phase of the BERT or XLNet model. Compared to the BERT model, XLNet overcomes the problems of ignoring the correlation between masked tokens and the inconsistent data distribution between the pre-training and fine-tuning phases. Therefore, this embodiment preferably uses the XLNet model as its basis, i.e., the encoder is obtained by combining XLNet and Text CNN.
[0069] The following section further explains the method for training the text processing model to obtain the text processing model.
[0070] First, the tokenizer in the text processing model to be trained is used to divide the acquired training text into training word blocks. This tokenizer can be BPE (byte-pair encoding), WordPiece (word encoding), or SentencePiece (sentence encoding).
[0071] The training text includes a one-to-one correspondence between the preset word blocks and standard labels. The standard label indicates the editing action to be performed corresponding to the word block.
[0072] A preset dictionary is also generated during the word segmentation stage. In this embodiment, the vocabulary size in the preset dictionary is no greater than a preset vocabulary threshold L, and the vocabulary can be frequently used high-frequency words in the original annotations. Accordingly, preset tags are generated based on the vocabulary in the dictionary. For example, if the vocabulary size in the preset dictionary, i.e., the number of tokens, is V, and each token may be replaced or added, the number of preset tags related to addition and replacement is 2V, and the number of tags for retention and deletion is 2. Therefore, the number of preset tags is 2V+2.
[0073] Then, a position marker is added to the token, and training labels corresponding to each training word block in the training text are matched one-to-one with the preset labels. Specifically, the hidden layers in the text processing model are first used to perform self-attention encoding calculations on the aforementioned training word blocks, and a sub-matrix corresponding to the training word block is obtained in each hidden layer. Then, a convolutional neural network is used to extract and fuse the feature information from multiple sub-matrices of the same training word block to obtain a feature matrix corresponding to each training word block. Then, based on the feature matrix, training labels are matched to the training word blocks in the preset labels.
[0074] Next, the system outputs training word blocks with positional markers and training labels that correspond one-to-one with each training word block.
[0075] Finally, based on the positional markers, the first error between the training word blocks output by the training model and the preset word blocks, and the second error between the training labels and the standard labels are determined. Based on these first and second errors, the parameters in the text processing model are adjusted in reverse until the accuracy of the training tokens output by the text processing model exceeds its corresponding first threshold, and the accuracy of the training labels exceeds its corresponding second threshold, thus obtaining the text processing model.
[0076] The adjustment of the first error can precede the adjustment of the second error. Specifically, the parameters of the hidden layer in the text processing model to be trained are adjusted in reverse to reduce the first error until the accuracy of the training word blocks output by the text processing model exceeds a first set threshold. Then, the parameters in the hidden layer are fixed, and the parameters in the convolutional neural network of the text processing model to be trained are adjusted to reduce the second error until the accuracy of the training labels in the text processing model exceeds a second set threshold, thus obtaining the text processing model. Since the pre-training stage is unsupervised training, the first set threshold is generally smaller than the second set threshold.
[0077] Based on the method for extracting key information from text described in step 101 above, the following example illustrates the process.
[0078] Assume the original text is: Legal Representative_Responsible Person_ID Number. After data cleaning, the text to be processed is: Legal Representative / Responsible Person ID Number. Input this text into the text processing model.
[0079] In the text processing model, the text to be processed is divided into tokens. And the tokens are represented as (E1, E2, ..., E7) after being computed by self-attention encoding in each hidden layer embedding.
[0080] Each hidden layer is connected to a convolutional neural network in the text processing model. After processing by the aforementioned convolutional neural network, (E1, E2, ..., E7) obtain the hidden state (H1, H2, ..., H7) of the target label of each token.
[0081] The aforementioned hidden state outputs target word blocks with positional tags from the output side, along with target tags [delete,delete,delete,delete,delete,delete,delete,R] corresponding to each target word block. 公 ,I 民 Based on the output target word block with positional tags and target labels, the key information obtained is: citizen ID card number.
[0082] As can be seen, the method for extracting key information provided in this application, by eliminating redundant information in the original text annotations and assigning preset labels to each word block in the text through a text rewriting task, makes the extraction of key information (keywords) more flexible. This not only improves the accuracy of key information but also avoids the problem of slow convergence speed of the model corresponding to the generative task when extracting text information through generative tasks. Thus, it achieves the goal of efficiently obtaining key information.
[0083] Based on the same inventive concept, this application provides a device for extracting key information from text, which is similar to the aforementioned device. Figure 1 The method for extracting key information from text shown corresponds to the specific implementation of this device, which can be found in the description of the aforementioned method embodiments. Repeated descriptions will not be repeated here. Figure 3 The device includes:
[0084] The input unit 301 is used to acquire the text to be processed and classify the text to be processed as belonging to the text processing model.
[0085] The matching unit 302 is used to match target tags for word blocks in the text to be processed from a variety of preset tags using the text processing model.
[0086] The target tag indicates the editing action to be performed corresponding to the word block.
[0087] The preset tags include categories such as delete, keep, replace, and add; wherein, the replace category includes the object to be replaced, and the add category includes the object to be added.
[0088] The matching unit 302 is specifically used to divide the text to be processed into word blocks and add position markers to the word blocks; wherein, the position markers indicate the relative positional relationship between the word blocks and other word blocks in the text to be processed; based on the word blocks and the position markers of the word blocks, self-attention encoding calculation is performed on the word blocks to obtain a word block matrix corresponding to the word blocks; based on the word block matrix, the target label is matched for the word blocks in the preset labels.
[0089] The matching unit 302 is specifically used to perform self-attention encoding calculation on the word block in the hidden layer of the text processing model, and obtain a sub-matrix corresponding to the word block in each hidden layer; and to assign the sub-matrix of each word block to the convolutional neural network matrix in the text processing model to obtain a word block matrix corresponding to each word block.
[0090] Information unit 303 is used to process the word blocks in the text to be processed based on the target tag to obtain key information of the text to be processed.
[0091] The aforementioned device for extracting key information from text further includes a training unit, which is specifically used to use a text processing model to be trained to divide the acquired training text and obtain training word blocks; wherein, the training text includes a correspondence between preset word blocks and standard labels, and the standard labels indicate the editing action to be performed corresponding to the word blocks; add position markers to the training word blocks, and match training labels for the training word blocks in the training text in the preset labels; based on a first error between the training word blocks and the preset word blocks, and a second error between the training labels and the standard labels, adjust the parameters in the text processing model to be trained until the accuracy of the training word blocks output by the text processing model to be trained is greater than a first set threshold, and the accuracy of the training labels is greater than a second set threshold, thereby obtaining a text processing model.
[0092] The training unit is specifically used to perform self-attention encoding calculation on the training word block, obtain a sub-matrix corresponding to the training word block in each hidden layer; extract the feature matrix corresponding to the same training word block; and match the training label for the training word block in the preset label based on the feature matrix.
[0093] Based on the same inventive concept, embodiments of this application also provide a readable storage medium, including:
[0094] memory,
[0095] The memory is used to store a computer program that, when executed by a processor, causes the apparatus including the readable storage medium to perform the method described above for extracting key information from text.
[0096] Based on the same inventive concept as the method for extracting key information from text described above, this application also provides an electronic device that can implement the function of the aforementioned method for extracting key information from text. Please refer to [link / reference]. Figure 4 The electronic device includes:
[0097] At least one processor 401 and a memory 402 connected to at least one processor 401. In this embodiment, the specific connection medium between the processor 401 and the memory 402 is not limited. Figure 4 The example shown is the connection between processor 401 and memory 402 via bus 400. Bus 400 is... Figure 4 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The 400 bus can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 4 The term is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, processor 401 can also be called a controller; there is no restriction on the name.
[0098] In this embodiment, memory 402 stores instructions executable by at least one processor 401. By executing the instructions stored in memory 402, at least one processor 401 can perform the method for extracting key information from text as described above. Processor 401 can implement... Figure 3 The functions of each module in the device shown.
[0099] The processor 401 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 402 and calling data stored in memory 402, the processor can perform various functions and process data, thereby monitoring the device as a whole.
[0100] In one possible design, processor 401 may include one or more processing units. Processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 401. In some embodiments, processor 401 and memory 402 may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.
[0101] Processor 401 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method for extracting key information from text disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.
[0102] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 402 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 402 may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0103] By designing and programming the processor 401, the code corresponding to the method for extracting key information from text described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute it during operation. Figure 1The steps of the method for extracting key information from text are shown. How to design and program the processor 401 is a technique well-known to those skilled in the art and will not be elaborated here.
[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0105] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0106] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: Universal Serial Bus flash disks, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.
[0109] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for extracting key information from text, characterized in that, include: Obtain the text to be processed and input the text to be processed into the text processing model; The text to be processed is divided into word blocks, and position markers are added to the word blocks; wherein, the position markers indicate the relative positional relationship between the word blocks and other word blocks in the text to be processed; In the hidden layer of the text processing model, self-attention encoding calculations of different dimensions are performed on the word block, and a sub-matrix corresponding to the word block is obtained in each hidden layer; The submatrix of each word block is input into the convolutional neural network in the text processing model for feature calculation and feature fusion to obtain a word block matrix corresponding to each word block; wherein, the number of row vectors in the word block matrix is equal to the number of hidden layers in the self-attention encoding calculation; Based on the word block matrix, target tags are matched for the word blocks in preset tags; wherein, the target tag indicates the editing action to be performed corresponding to the word block; the types of preset tags include deletion, retention, replacement, and addition; wherein, replacement includes the object to be replaced, and addition includes the object to be added; the object to be replaced includes content other than the text to be processed; the object to be added includes content other than the text to be processed. Based on the target tag, the word blocks in the text to be processed are processed to obtain the key information of the text to be processed.
2. The method as described in claim 1, characterized in that, Before inputting the text to be processed into the text processing model, the method further includes: The acquired training text is divided using a text processing model to be trained, resulting in training word blocks. The training text includes a correspondence between preset word blocks and standard labels, where the standard labels indicate the editing action to be performed corresponding to the word block. Add position markers to the training word blocks, and match training labels to the training word blocks in the training text from the preset labels; Based on the first error between the training word block and the preset word block, and the second error between the training label and the standard label, the parameters in the text processing model to be trained are adjusted until the accuracy of the training word block output by the text model to be trained is greater than a first set threshold, and the accuracy of the training label is greater than a second set threshold, thus obtaining the text processing model.
3. The method as described in claim 2, characterized in that, The step of matching training tags for training word blocks in the training text from preset tags includes: Self-attention encoding is performed on the training word blocks, and a sub-matrix corresponding to the training word blocks is obtained in each hidden layer; Feature information is extracted from the sub-matrix corresponding to the same training word block and fused to obtain a feature matrix corresponding to each training word block; Based on the feature matrix, the training label is matched to the training word block in the preset label.
4. An apparatus for extracting key information from text, characterized in that, include: The input unit is used to acquire the text to be processed and assign the text to the text processing model. A matching unit is used to divide the text to be processed into word blocks and add position markers to the word blocks; wherein the position markers indicate the relative positional relationship between the word block and other word blocks in the text to be processed; self-attention encoding calculations of different dimensions are performed on the word blocks in the hidden layers of the text processing model, and a sub-matrix corresponding to the word block is obtained in each hidden layer; the sub-matrix of each word block is input into the convolutional neural network in the text processing model for feature calculation and feature fusion to obtain a word block matrix corresponding to each word block; wherein the number of row vectors in the word block matrix is equal to the number of hidden layers in the self-attention encoding calculation; based on the word block matrix, target tags are matched for the word blocks in preset tags; wherein the target tags indicate the editing action to be performed corresponding to the word block; the types of preset tags include deletion, retention, replacement, and addition; wherein replacement includes the object to be replaced, and addition includes the object to be added; the object to be replaced includes content other than the text to be processed; the object to be added includes content other than the text to be processed. An information unit is used to process the word blocks in the text to be processed based on the target tag to obtain key information of the text to be processed.
5. A readable storage medium, characterized in that, include, memory, The memory is used to store a computer program that, when executed by a processor, causes the apparatus including the readable storage medium to perform the method as described in any one of claims 1-3.
6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method as described in any one of claims 1-3.