Information extraction model training method, information extraction method, device and electronic equipment
By introducing domain knowledge rules and natural language understanding techniques into the information extraction model, matching text samples to update the loss function value, the problem of difficulty in utilizing human domain knowledge in the existing technology is solved, and the training efficiency and accuracy of the model are improved.
Patent Information
- Application Number
- CN202010839979.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-20
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2040-08-20
AI Technical Summary
Existing information extraction technologies are difficult to directly use human domain knowledge for model training, resulting in low efficiency and poor accuracy in distinguishing entities of different meanings.
By building a multi-layer deep learning network, combining pre-built domain knowledge rules and natural language understanding techniques, the text samples are matched and the loss function value is updated to introduce domain knowledge into the model training process and reduce sample training needs.
It improves the training efficiency and prediction accuracy of the information extraction model, and can learn domain knowledge without requiring a large number of samples to train.
Smart Images
Figure CN112052681B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of deep learning technology. Specifically, the present application relates to an information extraction model training method, an information extraction method, a device and an electronic device. Background Art
[0002] At present, in the field of information extraction research, the framework of sequence annotation is generally adopted, and then various deep learning models are used for modeling, and entities and relationships between entities are extracted in a data-driven way. However, in the whole process, human domain knowledge cannot be directly used in modeling. For example, "I ate an apple" and "Apples are very useful". The first apple refers to fruit, and the second apple refers to an iPhone. These two differences need to be learned during extraction, and a large amount of relevant distinguishing data needs to exist in the training data so that the model can learn the difference between the two. Summary of the invention
[0003] The present application provides an information extraction model training method, information extraction method, device and electronic device, which introduces domain knowledge into deep learning for information extraction, so that the trained model can learn domain knowledge without a large amount of sample training, thereby improving the efficiency of model training and the accuracy of model prediction. The technical solutions adopted in this application are as follows:
[0004] In a first aspect, a method for training an information extraction model is provided, comprising:
[0005] Step 1: construct an initial information extraction model, which includes a multi-layer deep learning network;
[0006] Step 2: Input the text sample sequence into the multi-layer neural network to obtain the prediction sequence;
[0007] Step 3: Determine a first loss function value based on the annotation sequence and the prediction sequence corresponding to the text sample;
[0008] Step 4: Match the text sample based on the pre-built domain knowledge rules, and determine the second loss function value based on the matching result;
[0009] Step 5: Determine a third loss function value based on the first loss function value and the second loss function value;
[0010] Step 6: Update the parameters of the multi-layer deep learning network based on the third loss function value;
[0011] The above steps are iteratively performed until the first loss function value converges, and a successfully trained target information extraction model is obtained.
[0012] Optionally, matching the text sample based on pre-built domain knowledge rules, and determining the second loss function value based on the matching result, includes:
[0013] Get the output vector of the previous layer of the last layer in the multi-layer deep learning network, and map the output vector to a transfer value;
[0014] Extract keywords from text samples based on natural language understanding technology. If the pre-built domain knowledge base includes the keywords, the match is successful.
[0015] The second loss function value is determined based on a predetermined successful matching score value and a transfer value.
[0016] Optionally, the method comprises:
[0017] Obtain text samples from a training text sample set;
[0018] A text sample sequence of the text sample is obtained based on the pre-trained word embedding model, and the text sample sequence is expressed as a vector.
[0019] Optionally, determining a third loss function value based on the first loss function value and the second loss function value includes:
[0020] Based on the predetermined weight values of the first loss function value and the second loss function, the first loss function value and the second loss function value are weighted to obtain a third loss function value.
[0021] Optionally, the weight value of the first loss function value and the second loss function is learned by a machine learning method.
[0022] Optionally, the annotation sequence corresponding to the text sample is a manually annotated sequence representing entities and / or a sequence of relations between entities.
[0023] In a second aspect, an information extraction method is provided, comprising:
[0024] Obtain a text sequence of the text to be extracted, where the text sequence is a text vector;
[0025] Input a text sequence of the text of the information to be extracted into the target information extraction model of any one of claims 1-5 to obtain the extracted target information.
[0026] Optionally, the extracted target information includes entity information and / or relationship information between entities.
[0027] Optionally, the text of the information to be extracted is a resume text.
[0028] In a third aspect, an information extraction model training device is provided, comprising:
[0029] A construction module, used to construct an initial information extraction model, the initial information extraction model includes a multi-layer deep learning network;
[0030] The first input module is used to input the text sample sequence into the multi-layer neural network to obtain a prediction sequence;
[0031] A first determination module, used to determine a first loss function value based on a label sequence and a prediction sequence corresponding to the text sample;
[0032] A second determination module is used to match the text sample based on the pre-built domain knowledge rules and determine the second loss function value based on the matching result;
[0033] A third determination module, used to determine a third loss function value based on the first loss function value and the second loss function value;
[0034] An updating module, used for updating parameters of the multi-layer deep learning network based on a third loss function value;
[0035] The iterative execution module is used to iteratively execute the functions of the first determination module, the second determination module, the third determination module, and the update module until the first loss function value converges, thereby obtaining a successfully trained target information extraction model.
[0036] Optionally, matching the text sample based on pre-built domain knowledge rules, and determining a second loss function value based on the matching result, includes:
[0037] An acquisition unit, used for acquiring an output vector of a deep learning network before a last layer in a multi-layer deep learning network, and mapping the output vector into a transfer value;
[0038] An extraction unit, used to extract keywords from a text sample based on natural language understanding technology, and if the pre-built domain knowledge base includes the keywords, the match is successful;
[0039] A determination unit is used to determine a second loss function value based on a predetermined successful matching score value and a transfer value.
[0040] Optionally, the device comprises:
[0041] A first acquisition module, used to acquire text samples from a training text sample set;
[0042] The word embedding module is used to obtain a text sample sequence of a text sample based on a pre-trained word embedding model, where the text sample sequence is a vector expression.
[0043] Optionally, determining a third loss function value based on the first loss function value and the second loss function value includes:
[0044] The weighted processing unit is used to perform weighted processing on the first loss function value and the second loss function value based on the predetermined weight value of the first loss function value and the second loss function to obtain a third loss function value.
[0045] Optionally, the device further comprises:
[0046] The machine learning module is used to learn the weight value of the first loss function and the second loss function through a machine learning device.
[0047] Optionally, the annotation sequence corresponding to the text sample is a manually annotated sequence representing entities and / or a sequence of relations between entities.
[0048] In a fourth aspect, an information extraction device is provided, the device comprising:
[0049] The second acquisition module is used to acquire a text sequence of the text of the information to be extracted, where the text sequence is a text vector;
[0050] The second input module is used to input the text sequence of the text of the information to be extracted into the target information extraction model shown in any one of the first aspects to obtain the extracted target information.
[0051] Optionally, the extracted target information includes entity information and / or relationship information between entities.
[0052] Optionally, the text of the information to be extracted is a resume text.
[0053] In a fifth aspect, an electronic device is provided, the electronic device comprising:
[0054] one or more processors;
[0055] Memory;
[0056] One or more applications, wherein one or more applications are stored in a memory and configured to be executed by one or more processors, and one or more programs are configured to: execute the information extraction model training method or information extraction method shown in the first aspect.
[0057] In a sixth aspect, a computer-readable storage medium is provided, which is used to store computer instructions. When the computer-readable storage medium is run on a computer, the computer can execute the information extraction model training method or the information extraction method shown in the first aspect.
[0058] The present application provides an information extraction model training method, an information extraction method, a device and an electronic device. Compared with training a deep learning model only through annotated training samples, the present application constructs an initial information extraction model through step one, and the initial information extraction model includes a multi-layer deep learning network; step two, inputting a text sample sequence into a multi-layer neural network to obtain a prediction sequence; step three, determining the first loss function value based on the annotation sequence and the prediction sequence corresponding to the text sample; step four, matching the text sample based on a pre-constructed domain knowledge rule, and determining the second loss function value based on the matching result; step five, determining the third loss function value based on the first loss function value and the second loss function value; step six, updating the parameters of the multi-layer deep learning network based on the third loss function value; iteratively executing the above steps until the first loss function value obtained converges, and obtaining a successfully trained target information extraction model. That is, domain knowledge is introduced into deep learning for information extraction, so that the trained model can learn domain knowledge without a large amount of sample training, thereby improving the efficiency of model training and improving the accuracy of model prediction.
[0059] Additional aspects and advantages of the present application will be partially given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0061] Figure 1 A flowchart of an information extraction model training method according to an embodiment of the present application;
[0062] Figure 2 A flowchart of an information extraction method according to an embodiment of the present application;
[0063] Figure 3 A schematic diagram of the structure of an information extraction model training device according to an embodiment of the present application;
[0064] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present application;
[0065] Figure 5 Schematic diagram of the structure of the information extraction device of the embodiment of the present application. DETAILED DESCRIPTION
[0066] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.
[0067] It will be understood by those skilled in the art that, unless otherwise stated, the singular forms "a", "an" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.
[0068] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0069] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0070] Information extraction: here refers to the extraction of natural language unstructured documents into structured information in the form of elements, mainly involving entity extraction and entity relationship extraction; entity extraction: refers to the entities and corresponding values of the entities extracted in the form of key-value, such as name-Jobs; entity relationship: the relationship type judgment between entities, for example, there are entities, company-Apple, name-Jobs, then the relationship between <company-Apple, name-Jobs> is 'create'.
[0071] Embodiment 1
[0072] The present application embodiment provides an information extraction model training method, such as Figure 1 As shown, the method may include the following steps:
[0073] Step S101, construct an initial information extraction model, the initial information extraction model includes a multi-layer deep learning network; wherein the multi-layer neural learning network can be modeled using a variety of neural network structures, such as LSTM, GRU, BiLSTM, etc.
[0074] Step S102: input the text sample sequence into the multi-layer neural network to obtain a prediction sequence; specifically, the text sample can be pre-processed and then input into the multi-layer neural network to obtain the prediction sequence, wherein the pre-processing includes word segmentation, stop word removal, etc.
[0075] Step S103, determining a first loss function value based on a labeling sequence and a prediction sequence corresponding to the text sample;
[0076] For example, for a text sequence X = {x1, x2, ..., xn} to be annotated, we need to predict a tag for each xi. First, define the tag set T = {t1, t2, ..., tm}. For example, the tag for word segmentation can be defined as {Begin, Middle, End, Single}, and the tag for named entity recognition can be defined as {personal name, place name, organization name, ...}.
[0077] Assume that the predicted sequence is Y = {y1, y2, ..., yn}, and the manually or automatically annotated sequence corresponding to the text sample sequence is L = {l1, l1, ..., ln}. The Loss can be calculated by using cross entropy on Y and L, and the parameters can be solved by gradient descent. The loss value predicted by the information extraction model can be obtained during training.
[0078] Step S104, matching the text sample based on the pre-built domain knowledge rule, and determining the second loss function value based on the matching result;
[0079] For example, there are mainly two types of rules, which are formalized as follows:
[0080] Rule 1: seg(Z=b)=>seg(X=a);
[0081] Rule 2: entity(X=c)+rel((X,Z)=l)=>entity(Z=d);
[0082] Rule 1 indicates that the type of sequence fragment Z is b, so it can be inferred that the type of sequence fragment X is a; Rule 2 indicates that the type of entity X is c, and the entity relationship between entity X and entity Z is l, so it can be inferred that the type of entity Z is d;
[0083] According to the two prototype rules, the domain knowledge of information extraction can be organized into different rules for different data and extraction businesses (for example, when Apple and Jobs appear at the same time, Apple means mobile phone instead of fruit; when Xiaomi and Lei Jun appear at the same time, Xiaomi means mobile phone instead of food; when Java and programmer appear at the same time, Java means programming language instead of place name).
[0084] Step S105, determine the third loss function value based on the first loss function value and the second loss function value; specifically, the first loss function value and the second loss function can be directly superimposed or the third loss function value can be obtained by weighted calculation based on the weight values of the first loss function value and the second loss function.
[0085] Step S106: updating the parameters of the multi-layer deep learning network based on the third loss function value;
[0086] The above steps S101-S106 are iteratively executed until the first loss function value is converged, thereby obtaining a successfully trained target information extraction model.
[0087] The embodiment of the present application provides a possible implementation method, matching text samples based on pre-built domain knowledge rules, and determining a second loss function value based on the matching result, including:
[0088] Get the output vector of the previous layer of the last layer in the multi-layer deep learning network, and map the output vector to a transfer value;
[0089] Extract keywords from text samples based on natural language understanding technology. If the pre-built domain knowledge base includes the keywords, the match is successful.
[0090] The second loss function value is determined based on a predetermined successful matching score value and a transfer value.
[0091] Specifically, the text sample is matched based on the pre-built domain knowledge rules to see if there is any corresponding knowledge matched. Taking the sentence "This apple tastes great" as an example, when doing entity recognition tasks, the keywords "taste" and "apple" can be extracted through natural language understanding technology, and then the keywords are matched using the pre-built domain knowledge base. It is found that "apple" and "taste" are used as matching units, and the match is successful in the knowledge base, which meets seg ("taste") = > seg ("apple" = fruit), then the output of the neural network in the rule module is Ye-1; on the contrary, if no corresponding knowledge is matched in the knowledge base, the output becomes Ye, which is the output of the corresponding neural network, artificially minus 1, which is equivalent to a certain degree of punishment for the output if it does not meet the pre-defined knowledge in the knowledge base, so that the knowledge that meets the knowledge base can be used as mandatory, so that the deep neural network can strengthen the memory. Among them, Ye is the transfer value, which can be obtained by mapping the vector of the text sample into a value through a simple neural network, wherein the neural network can be any layer other than the last layer in the multi-layer neural learning network in the information extraction model.
[0092] Optionally, the method comprises:
[0093] Obtain text samples from a training text sample set;
[0094] A text sample sequence of the text sample is obtained based on the pre-trained word embedding model, and the text sample sequence is expressed as a vector.
[0095] Optionally, determining a third loss function value based on the first loss function value and the second loss function value includes:
[0096] Based on the predetermined weight values of the first loss function value and the second loss function, the first loss function value and the second loss function value are weighted to obtain a third loss function value.
[0097] Optionally, the weight values of the first loss function value and the second loss function are learned by a machine learning method. Specifically, the weight values of the first loss function value and the second loss function are learned by a machine learning method, which can improve the accuracy of the prediction of the trained information extraction model compared with manually determining the weight values.
[0098] Optionally, the annotation sequence corresponding to the text sample is a manually annotated sequence representing entities and / or a sequence of relations between entities.
[0099] Specifically, the training of the information extraction model may be supervised learning, and the text sample training set is annotated in advance, wherein the annotation may be performed manually, automatically, or semi-automatically.
[0100] The present application provides a method for training an information extraction model. Compared with training a deep learning model only through annotated training samples, the present application comprises the following steps: step 1, constructing an initial information extraction model, the initial information extraction model comprising a multi-layer deep learning network; step 2, inputting a text sample sequence into a multi-layer neural network to obtain a prediction sequence; step 3, determining a first loss function value based on the annotation sequence and the prediction sequence corresponding to the text sample; step 4, matching the text sample based on a pre-constructed domain knowledge rule, and determining a second loss function value based on the matching result; step 5, determining a third loss function value based on the first loss function value and the second loss function value; step 6, updating the parameters of the multi-layer deep learning network based on the third loss function value; iteratively executing the above steps until the first loss function value obtained converges, and obtaining a successfully trained target information extraction model. That is, domain knowledge is introduced into deep learning for information extraction, so that the trained model can learn domain knowledge without a large amount of sample training, thereby improving the efficiency of model training and the accuracy of model prediction.
[0101] Embodiment 2
[0102] In a second aspect, an information extraction method is provided, comprising:
[0103] Obtain a text sequence of the text to be extracted, where the text sequence is a text vector;
[0104] The text sequence of the text of the information to be extracted is input into the target information extraction model of the embodiment shown in the first aspect to obtain the extracted target information.
[0105] Specifically, the extracted target information includes entity information and / or relationship information between entities.
[0106] Specifically, the text of the information to be extracted is a resume text. For example, the target information to be extracted can be entity information (name, age, school, company name, etc.), and relationship information between entities (such as the relationship between Zhang San and Microsoft, the relationship between Zhang San and Tsinghua University, and the relationship between Zhang San and Tsinghua University).
[0107] For the embodiments of the present application, currently in the field of resume automatic extraction, it is basically possible to extract corresponding information (i.e., keyword matching) from an unstructured resume according to predefined fields. Therefore, there is still a problem of low refinement in resume information extraction. The embodiments of the present application apply domain knowledge to the training of the information extraction model, which can improve the refinement and accuracy of information extraction.
[0108] Embodiment 3
[0109] The present application embodiment provides an information extraction model training device, such as Figure 3 As shown, the device 30 includes:
[0110] A construction module 301 is used to construct an initial information extraction model, wherein the initial information extraction model includes a multi-layer deep learning network;
[0111] A first input module 302, used to input a text sample sequence into a multi-layer neural network to obtain a prediction sequence;
[0112] A first determination module 303, configured to determine a first loss function value based on a labeling sequence and a prediction sequence corresponding to the text sample;
[0113] A second determination module 304 is used to match the text sample based on the pre-constructed domain knowledge rule and determine a second loss function value based on the matching result;
[0114] A third determination module 305, configured to determine a third loss function value based on the first loss function value and the second loss function value;
[0115] An updating module 306, configured to update the parameters of the multi-layer deep learning network based on the third loss function value;
[0116] The iterative execution module 307 is used to iteratively execute the functions of the first determination module, the second determination module, the third determination module, and the update module until the first loss function value converges, thereby obtaining a successfully trained target information extraction model.
[0117] Optionally, matching the text sample based on pre-built domain knowledge rules, and determining a second loss function value based on the matching result, includes:
[0118] An acquisition unit, used for acquiring an output vector of a deep learning network before a last layer in a multi-layer deep learning network, and mapping the output vector into a transfer value;
[0119] An extraction unit, used to extract keywords from a text sample based on natural language understanding technology, and if the pre-built domain knowledge base includes the keywords, the match is successful;
[0120] A determination unit is used to determine a second loss function value based on a predetermined successful matching score value and a transfer value.
[0121] Optionally, the device comprises:
[0122] A first acquisition module, used to acquire text samples from a training text sample set;
[0123] The word embedding module is used to obtain a text sample sequence of a text sample based on a pre-trained word embedding model, where the text sample sequence is a vector expression.
[0124] Optionally, determining a third loss function value based on the first loss function value and the second loss function value includes:
[0125] The weighted processing unit is used to perform weighted processing on the first loss function value and the second loss function value based on the predetermined weight value of the first loss function value and the second loss function to obtain a third loss function value.
[0126] Optionally, the device further comprises:
[0127] The machine learning module is used to learn the weight value of the first loss function and the second loss function through a machine learning device.
[0128] Optionally, the annotation sequence corresponding to the text sample is a manually annotated sequence representing entities and / or a sequence of relations between entities.
[0129] The present application provides a method for training an information extraction model. Compared with training a deep learning model only through annotated training samples, the present application comprises the following steps: step 1, constructing an initial information extraction model, the initial information extraction model comprising a multi-layer deep learning network; step 2, inputting a text sample sequence into a multi-layer neural network to obtain a prediction sequence; step 3, determining a first loss function value based on the annotation sequence and the prediction sequence corresponding to the text sample; step 4, matching the text sample based on a pre-constructed domain knowledge rule, and determining a second loss function value based on the matching result; step 5, determining a third loss function value based on the first loss function value and the second loss function value; step 6, updating the parameters of the multi-layer deep learning network based on the third loss function value; iteratively executing the above steps until the first loss function value obtained converges, and obtaining a successfully trained target information extraction model. That is, domain knowledge is introduced into deep learning for information extraction, so that the trained model can learn domain knowledge without a large amount of sample training, thereby improving the efficiency of model training and the accuracy of model prediction.
[0130] The specific implementation method and beneficial effects of the embodiment of the present application are the same as those of Embodiment 1 and will not be repeated here.
[0131] Embodiment 4
[0132] The present application embodiment provides an information extraction device, such as Figure 5 As shown, the device 50 comprises:
[0133] The second acquisition module 501 is used to acquire a text sequence of the text of the information to be extracted, where the text sequence is a text vector;
[0134] The second input module 502 is used to input a text sequence of the text of the information to be extracted into the target information extraction model shown in any one of the first aspects to obtain the extracted target information.
[0135] Optionally, the extracted target information includes entity information and / or relationship information between entities.
[0136] Optionally, the text of the information to be extracted is a resume text.
[0137] For the embodiments of the present application, currently in the field of resume automatic extraction, it is basically possible to extract corresponding information (i.e., keyword matching) from an unstructured resume according to predefined fields. Therefore, there is still a problem of low refinement in resume information extraction. The embodiments of the present application apply domain knowledge to the training of the information extraction model, which can improve the refinement and accuracy of information extraction.
[0138] For the embodiments of the present application, currently in the field of resume automatic extraction, it is basically possible to extract corresponding information (i.e., keyword matching) from an unstructured resume according to predefined fields. Therefore, there is still a problem of low refinement in resume information extraction. The embodiments of the present application apply domain knowledge to the training of the information extraction model, which can improve the refinement and accuracy of information extraction.
[0139] Embodiment 5
[0140] The present application embodiment provides an electronic device, such as Figure 4 As shown, Figure 4 The electronic device 40 shown includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as through a bus 402. Further, the electronic device 40 may also include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one, and the structure of the electronic device 40 does not constitute a limitation on the embodiments of the present application. The processor 401 is used in the embodiments of the present application to implement Figure 3 or Figure 5 Functions of the modules shown. The transceiver 404 includes a receiver and a transmitter.
[0141] Processor 401 may be a CPU, a general purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0142] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI bus or an EISA bus, etc. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0143] The memory 403 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
[0144] The memory 403 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 401. The processor 401 is used to execute the application code stored in the memory 403 to implement Figure 3 or Figure 5 The illustrated embodiments provide for the functionality of the device.
[0145] The embodiment of the present application provides an electronic device. Compared with training a deep learning model only through annotated training samples, the present application constructs an initial information extraction model through step one, and the initial information extraction model includes a multi-layer deep learning network; step two, inputting a text sample sequence into a multi-layer neural network to obtain a prediction sequence; step three, determining a first loss function value based on the annotation sequence and the prediction sequence corresponding to the text sample; step four, matching the text sample based on a pre-constructed domain knowledge rule, and determining a second loss function value based on the matching result; step five, determining a third loss function value based on the first loss function value and the second loss function value; step six, updating the parameters of the multi-layer deep learning network based on the third loss function value; iteratively executing the above steps until the first loss function value obtained converges, and a successfully trained target information extraction model is obtained. That is, domain knowledge is introduced into deep learning for information extraction, so that the trained model can learn domain knowledge without a large amount of sample training, thereby improving the efficiency of model training and improving the accuracy of model prediction.
[0146] The present application embodiment provides an electronic device applicable to the above method embodiment, which will not be described in detail here.
[0147] Implementation Four
[0148] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method shown in the above embodiment is implemented.
[0149] The embodiment of the present application provides a computer-readable storage medium. Compared with training a deep learning model only through annotated training samples, the present application constructs an initial information extraction model through step one, and the initial information extraction model includes a multi-layer deep learning network; step two, inputting a text sample sequence into a multi-layer neural network to obtain a prediction sequence; step three, determining a first loss function value based on the annotation sequence and the prediction sequence corresponding to the text sample; step four, matching the text sample based on a pre-constructed domain knowledge rule, and determining a second loss function value based on the matching result; step five, determining a third loss function value based on the first loss function value and the second loss function value; step six, updating the parameters of the multi-layer deep learning network based on the third loss function value; iteratively executing the above steps until the first loss function value obtained converges, and a successfully trained target information extraction model is obtained. That is, domain knowledge is introduced into deep learning for information extraction, so that the trained model can learn domain knowledge without a large amount of sample training, thereby improving the efficiency of model training and improving the accuracy of model prediction.
[0150] The present application embodiment provides a computer-readable storage medium applicable to the above method embodiment, which will not be described in detail here.
[0151] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0152] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for training an information extraction model, characterized in that: include: Step 1: construct an initial information extraction model, wherein the initial information extraction model includes a multi-layer deep learning network; Step 2: inputting the text sample sequence into the multi-layer neural network to obtain a prediction sequence; Step 3: determining a first loss function value based on the annotation sequence corresponding to the text sample and the prediction sequence; Step 4: matching the text sample based on the pre-built domain knowledge rules, and determining the second loss function value based on the matching result; Step 5: Determine a third loss function value based on the first loss function value and the second loss function value; Step 6: Update the parameters of the multi-layer deep learning network based on the third loss function value; Iterate the above steps until the first loss function value converges, and obtain a successfully trained target information extraction model; Matching the text sample based on pre-built domain knowledge rules, and determining a second loss function value based on the matching result, including: Obtain an output vector of any layer of deep learning network before the last layer in the multi-layer deep learning network, and map the output vector to a transfer value; Extracting keywords from the text sample based on natural language understanding technology, if the pre-built domain knowledge base includes the keywords, the match is successful; A second loss function value is determined based on a predetermined match success score value and the transfer value.
2. The method according to claim 1, characterized in that The method includes: Obtain text samples from a training text sample set; A text sample sequence of the text sample is obtained based on a pre-trained word embedding model, and the text sample sequence is a vector expression.
3. The method according to claim 1, characterized in that The determining a third loss function value based on the first loss function value and the second loss function value includes: Based on a predetermined first loss function value and a weight value of the second loss function, the first loss function value and the second loss function value are weighted to obtain a third loss function value.
4. The method according to claim 3, characterized in that The first loss function value and the weight value of the second loss function are learned by a machine learning method.
5. The method according to claim 1, characterized in that The annotation sequence corresponding to the text sample is a manually annotated sequence representing entities and / or a sequence of relations between entities.
6. An information extraction method, characterized in that: include: Obtaining a text sequence of the text of the information to be extracted, wherein the text sequence is a text vector; Input the text sequence of the text of the information to be extracted into the target information extraction model described in any one of claims 1 to 4 to obtain the extracted target information.
7. The method according to claim 6, characterized in that The extracted target information includes entity information and / or relationship information between entities.
8. The method according to claim 7, characterized in that The text of the information to be extracted is a resume text.
9. An information extraction model training device, characterized in that: include: A construction module, used to construct an initial information extraction model, wherein the initial information extraction model includes a multi-layer deep learning network; A first input module, used for inputting a text sample sequence into the multi-layer neural network to obtain a prediction sequence; A first determination module, configured to determine a first loss function value based on a label sequence corresponding to the text sample and the prediction sequence; A second determination module, used for matching the text sample based on pre-constructed domain knowledge rules, and determining a second loss function value based on the matching result; A third determination module, configured to determine a third loss function value based on the first loss function value and the second loss function value; An updating module, configured to update the parameters of the multi-layer deep learning network based on the third loss function value; An iterative execution module, used for iteratively executing the functions of the first determination module, the second determination module, the third determination module, and the update module until the obtained first loss function value converges, thereby obtaining a successfully trained target information extraction model; Matching the text sample based on pre-built domain knowledge rules, and determining a second loss function value based on the matching result, including: An acquisition unit, used to acquire an output vector of a deep learning network before a last layer in the multi-layer deep learning network, and map the output vector into a transfer value; An extraction unit, used for extracting keywords of the text sample based on natural language understanding technology, and if the pre-built domain knowledge base includes the keywords, the match is successful; A determination unit is used to determine a second loss function value based on a predetermined successful matching score value and the transfer value.
10. An information extraction device, characterized in that: include: A second acquisition module is used to acquire a text sequence of the text of the information to be extracted, wherein the text sequence is a text vector; The second input module is used to input the text sequence of the text of the information to be extracted into the target information extraction model described in any one of claims 1-4 to obtain the extracted target information.
11. The device according to claim 10, characterized in that The extracted target information includes entity information and / or relationship information between entities.
12. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the information extraction model training method or the information extraction method according to any one of claims 1 to 8.
13. A computer-readable storage medium, characterized in that: The computer storage medium is used to store computer instructions, and when it is run on a computer, it enables the computer to execute the information extraction model training method or information extraction method described in any one of claims 1 to 8.