A method and system for improving holographic archive extraction efficiency and quality
By combining regular expressions and BERT models, appropriate models are used for extraction for different types of documents, which solves the problems of long extraction time and low accuracy in the prior art, and achieves more efficient and accurate physical information extraction.
Patent Information
- Application Number
- CN202410680541.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-29
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-05-29
AI Technical Summary
When extracting physical contents of existing case templates based on knowledge graphs, large language models are used for both formatted and non-formatted documents, resulting in too long extraction time and low efficiency. It is easy to have hallucinations when combining complex template structures and simple cases, which reduces the Extraction accuracy and efficiency.
Regular expressions and pre-trained BERT model combined with industry-specific large language models, standard and non-standard content in formatted and non-formatted documents are extracted according to the document type, and node breadth priority strategy is used for layer-by-layer extraction, and During the extraction process, verify the correctness of the node entity content to avoid unnecessary extraction operations.
The accuracy and efficiency of formatted documents are improved, and different models are used for extraction for different document types are improved, and the accuracy and efficiency of physical content is enhanced.
Smart Images

Figure CN118503448B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of knowledge graph technology, and in particular to a method and system for improving the efficiency and quality of holographic archive extraction based on knowledge graph, which utilizes knowledge graph to construct case templates, and utilizes a combination of regular expressions, BERT models and large language models to extract content from documents and fill them into the case templates. Background Art
[0002] The case templates previously built based on the knowledge graph principle have the following defects when extracting and filling entity content: First, whether it is a formatted document or an unformatted document, a large language model is used to extract and fill the case template, resulting in a long extraction time and low extraction efficiency; Second, it is necessary to traverse all nodes in each branch and extract the corresponding entity content for filling. For complex case template structures, the extraction time is too long. For situations where the case template structure is complex but the case is simple, the model will be forced to produce hallucinations. Moreover, when there is a problem with the entity content extracted from a node or the entity content is not extracted from a node, the nodes to which the node belongs will continue to be extracted, which will reduce the extraction accuracy and efficiency. Summary of the invention
[0003] The present invention aims at the problems and shortcomings of the prior art and provides a method and system for improving the efficiency and quality of holographic archive extraction.
[0004] The present invention solves the above technical problems through the following technical solutions:
[0005] The present invention provides a method for improving the efficiency and quality of holographic archive extraction, wherein the case documents in the holographic archive of a certain case include formatted documents and unformatted documents, the formatted documents include standard content and non-standard content, and the unformatted documents include standard content and non-standard content. The method comprises the following steps:
[0006] S1. According to the case type of the case, a target case template is matched from the case template library. The target case template is constructed based on the knowledge graph principle;
[0007] S2, determining whether the case document to be extracted is a formatted document or an unformatted document, if it is a formatted document, proceeding to step S3, if it is an unformatted document, proceeding to step S4;
[0008] S3. For the target case template, first use the pre-set regular expression and the pre-trained BERT model to extract the standard content in the formatted document, extract the entity information from it and fill it into the corresponding node in the target case template, then use the industry-specific big model to extract the non-standard content in the formatted document, extract the entity information from it and fill it into the corresponding node in the target case template, and use the industry-specific big model to extract layer by layer according to the node breadth-first strategy in the target case template during the extraction process, and enter step S5;
[0009] Industry-specific large models are large language models that are fine-tuned and reinforced through learning based on the model base using industry-specific task designs and related datasets.
[0010] S4. For the target case template, directly use the industry-specific big model to extract the standard content and non-standard content in the non-formatted document, extract the entity information from it and fill it into the corresponding node in the target case template, and use the industry-specific big model to extract layer by layer according to the node breadth-first strategy in the target case template during the extraction process, and enter step S5;
[0011] S5. The filled target case template constitutes the latest version of the case map of the case and is stored in the holographic file.
[0012] The present invention also provides a system for improving the efficiency and quality of holographic archive extraction, wherein the case documents in the holographic archive of a certain case include formatted documents and unformatted documents, the formatted documents include standard content and non-standard content, and the unformatted documents include standard content and non-standard content, and the system includes a case template matching module, a document type judgment module, a first extraction and filling module, a second extraction and filling module, and a storage module;
[0013] The case template matching module is used to match a target case template from the case template library according to the case type of the case, and the target case template is constructed based on the knowledge graph principle;
[0014] The document type determination module is used to determine whether the case document to be extracted is a formatted document or an unformatted document, and to call the first extraction and filling module when it is a formatted document, and to call the second extraction and filling module when it is an unformatted document;
[0015] The first extraction and filling module is used to extract the standard content in the formatted document using a preset regular expression and a pre-trained BERT model for the target case template, extract entity information therefrom and fill it into the corresponding nodes in the target case template, and then use the industry-specific large model to extract the non-standard content in the formatted document, extract entity information therefrom and fill it into the corresponding nodes in the target case template, and perform extraction layer by layer according to the node breadth-first strategy in the target case template during the extraction process using the industry-specific large model;
[0016] Industry-specific large models are large language models that are fine-tuned and reinforced through learning based on the model base using industry-specific task designs and related datasets.
[0017] The second extraction and filling module is used to directly use the industry-specific large model to extract the standard content and non-standard content in the non-formatted document for the target case template, extract entity information from it and fill it into the corresponding node in the target case template, and extract layer by layer according to the node breadth-first strategy in the target case template during the extraction process using the industry-specific large model;
[0018] The storage module is used to store the latest version of the case map of the target case after filling in the template into the holographic file.
[0019] The positive and progressive effects of the present invention are:
[0020] In the present invention, for the standard content in the formatted document, regular expressions and BERT models are used to extract and fill the target case template, which has higher extraction accuracy and higher extraction efficiency. For the non-standard content in the formatted document, the standard content and non-standard content in the unformatted document, industry-specific large models are used to extract and fill the target case template. This operation of using different models to extract and fill the case template for different document types is conducive to improving extraction accuracy and efficiency.
[0021] In the present invention, when there is a problem with the entity content extracted from a certain node, the extraction and filling operation will no longer be performed on all the nodes to which the node belongs. When the entity content is not extracted from a certain node, the extraction and filling operation will no longer be performed on all the nodes to which the node belongs. This operation method further improves the extraction accuracy and extraction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 Flow chart of a method for improving holographic file extraction efficiency and quality according to a preferred embodiment of the present invention.
[0023] Figure 2 Schematic diagram of the structure of the target case template of a preferred embodiment of the present invention.
[0024] Figure 3 This is a structural block diagram of a system for improving the efficiency and quality of holographic file extraction according to a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0026] like Figure 1 As shown, this embodiment provides a method for improving the efficiency and quality of holographic file extraction. The case documents in the holographic file of a certain case include formatted documents and unformatted documents. The formatted documents include standard content and non-standard content, and the unformatted documents include standard content and non-standard content. The method includes the following steps:
[0027] Step 101: Match a target case template from a case template library according to the case type of the case. The target case template is constructed based on the knowledge graph principle.
[0028] Step 102 : Determine whether the case document to be extracted is a formatted document or an unformatted document. If it is a formatted document, proceed to step 103 ; if it is an unformatted document, proceed to step 104 .
[0029] Step 103: For the target case template, first use a pre-set regular expression and a pre-trained BERT model to extract the standard content in the formatted document, extract the entity information therefrom and fill it into the corresponding nodes in the target case template, then use the industry-specific big model to extract the non-standard content in the formatted document, extract the entity information therefrom and fill it into the corresponding nodes in the target case template, and use the industry-specific big model to extract layer by layer according to the node breadth-first strategy in the target case template, and then go to step 105.
[0030] Industry-specific large models are large language models obtained by fine-tuning and reinforcement learning based on the model base using industry-specific task design and related datasets.
[0031] In this step, for the target case template, a preset regular expression is used to extract the standard content in the formatted document before the quality of the BERT model reaches the preset requirements. The data extracted by the regular expression is accumulated as a data sample set for training the BERT model. After the quality of the BERT model reaches the preset requirements after sample training, the trained BERT model is used to extract the standard content in the formatted document.
[0032] This step 103 specifically includes the following steps:
[0033] Step 1031: For the target case template, use a pre-set regular expression and a pre-trained BERT model to extract the standard content in the formatted document, extract entity information from it and fill it into the corresponding node in the target case template.
[0034] Step 1032, determine whether all nodes in the i-th layer of the target case template have been filled with entity information, if so, proceed to step 1033, if not, proceed to step 1034, where 1≤i≤N, N is the number of layers of the target case template, and N is a positive integer.
[0035] Step 1033, i=i+1, repeat step 1032.
[0036] Step 1034: Use the industry-specific large model to extract the non-standard content in the formatted document, extract the entity information therefrom and fill it into the corresponding nodes in the target case template that are not filled in and not marked as not requiring extraction and filling operations.
[0037] In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy:
[0038] Traverse the layer, and for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when the entity content is extracted from the formatted document, fill it into the node. If the node is the tail node of a triple, reverse extraction verification is required. If they are consistent, it indicates that there is no problem with the entity content of the node, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. If they are inconsistent, it indicates that there is a problem with the entity content of the node, and there is no need to perform extraction and filling operations on all the nodes to which the node belongs. All the nodes to which the node belongs are marked as not requiring extraction and filling operations. When the entity content of the node is not extracted from the formatted document, there is no need to perform extraction and filling operations on all the nodes to which the node belongs. All the nodes to which the node belongs are marked as not requiring extraction and filling operations.
[0039] Step 1035, determine whether all nodes in the i-th layer of the target case template have been traversed, if so, proceed to step 1036, if not, repeat step 1034.
[0040] Step 1036, i=i+1, determine whether i≤N, if yes, repeat step 1034, if no, end the process.
[0041] For example: Figure 2 The target case template is a formatted document to be extracted. The standard content in the formatted document is extracted using regular expressions and the BERT model. The entity content of the first-layer node A, the entity content of the second-layer node B and the entity content of the node C are extracted and filled into the corresponding nodes in the target case template. Determine whether all nodes in the third layer of the target case template have been filled with entity information. If not, use the industry-specific large model to extract the non-standard content in the formatted document, extract the entity information of the third-layer node D and fill it into node D; extract the entity information of the third-layer node E and fill it into node E. If node E is the tail node of a triple, reverse extraction verification is required. If the verification is inconsistent, it means that there is a problem with the entity content of node E. There is no need to perform extraction and filling operations on all nodes to which node E belongs (i.e., node H). Node H is marked as a mark that does not require extraction and filling operations; extract the entity information of the third-layer node F and fill it into node F. ; extract the entity information of the third-layer node G and fill it into node G; extract the entity information of the third-layer node I and fill it into node I but fail to extract it, then there is no need to perform extraction and filling operations on all nodes to which node I belongs (i.e., node K, node O, and node P), and mark nodes K, node O, and node P as no need for extraction and filling operations; extract the entity information of the third-layer node J and fill it into node J. If node J is the tail node of a triple, reverse extraction verification is required. If the verification is consistent, it means that there is no problem with the entity content of node J, then it is necessary to perform extraction and filling operations on the next layer of nodes to which node J belongs (i.e., node L and node M). Entering the 4th layer, the only nodes that are not filled and not marked as not requiring extraction and filling operations are nodes L and M. The entity information of node L in the 4th layer is extracted and filled into node L. Node L is the tail node of a triple, so reverse extraction verification is required. If the verification is consistent, it means that there is no problem with the entity content of node L. Then, the next layer node to which node L belongs (i.e., node Q) needs to be extracted and filled; the entity information of node M in the 4th layer is extracted and filled into node M. Entering the 5th layer, the only node that is not filled and not marked as not requiring extraction and filling operations is node Q. The entity information of node Q in the 5th layer is extracted and filled into node Q, and the process ends.
[0042] Step 104: For the target case template, directly use the industry-specific big model to extract the standard content and non-standard content in the non-formatted document, extract the entity information therefrom and fill it into the corresponding nodes in the target case template. During the extraction process using the industry-specific big model, extract layer by layer according to the node breadth-first strategy in the target case template, and proceed to step 105.
[0043] Step 104 includes:
[0044] Step 1041, determine whether all nodes in the i-th layer of the target case template have been filled with entity information, if so, proceed to step 1042, if not, proceed to step 1043, where 1≤i≤N, N is the number of layers of the target case template, and N is a positive integer.
[0045] Step 1042, i=i+1, repeat step 1041.
[0046] Step 1043: directly use the industry-specific large model to extract the standard content and non-standard content in the non-formatted document, extract the entity information therefrom and fill it into the corresponding nodes in the target case template that are not filled in and not marked as not requiring extraction and filling operations.
[0047] In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy:
[0048] The layer is traversed, wherein for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when entity content is extracted from the formatted document, it is filled into the node. When the node is the head node of a triple, there is no need for reverse extraction verification, which directly indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. When the node is the tail node of a triple, reverse extraction verification is required. If they are consistent, it indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. If they are inconsistent, it indicates that there is a problem with the entity content of the node, and it is not necessary to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations. When the entity content of the node is not extracted from the formatted document, there is no need to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations.
[0049] Step 1044, determine whether all nodes in the i-th layer of the target case template have been traversed, if so, proceed to step 1045, if not, repeat step 1043.
[0050] Step 1045, i=i+1, determine whether i≤N, if yes, repeat step 1043, if no, end the process.
[0051] Step 105: The filled target case template constitutes the latest version of the case map of the case and is stored in the holographic file.
[0052] For example: Figure 2The target case template is the case document to be extracted, and the case document is an unformatted document. For the first layer, determine whether the node A of the first layer has been filled with entity information. If not, use the industry-specific large model to extract the standard content and non-standard content in the unformatted document, and extract the entity content of the first layer node A to fill it into node A. Then enter the second layer to determine whether each node in the second layer has been filled with entity information. If not, use the industry-specific large model to extract the unformatted document, extract the entity content of the second-layer node B and fill it into node B. If node B is the tail node of a triple, reverse extraction verification is required. If the verification is consistent, it means that there is no problem with the entity content of node B, then it is necessary to perform extraction and filling operations on the next layer of nodes to which node B belongs (i.e., node D, node E, node F, and node G); extract the entity content of the second-layer node C and fill it into node C. If node C is the tail node of a triple, reverse extraction verification is required. If the verification is consistent, it means that there is no problem with the entity content of node C, then it is necessary to perform extraction and filling operations on the next layer of nodes to which node C belongs (i.e., node I and node J). Then enter the third layer to determine whether all nodes in the third layer have been filled with entity information. If not, use the industry-specific large model to extract the non-formatted document, extract the entity information of the third-layer node D and fill it into node D; extract the entity information of the third-layer node E and fill it into node E. If node E is the tail node of a triple, reverse extraction verification is required. If the verification is inconsistent, it means that there is a problem with the entity content of node E. There is no need to perform extraction and filling operations on all nodes belonging to node E (that is, node H). Node H is marked as not requiring extraction and filling operations; extract the entity information of the third-layer node F and fill it into node F; The entity information of the third-layer node G is extracted and filled into node G; if the entity information of the third-layer node I is extracted and filled into node I but is not extracted, there is no need to perform extraction and filling operations on all nodes to which node I belongs (i.e., node K, node O, and node P), and nodes K, node O, and node P are marked as not requiring extraction and filling operations; the entity information of the third-layer node J is extracted and filled into node J. If node J is the tail node of a triple, reverse extraction verification is required. If the verification is consistent, it indicates that there is no problem with the entity content of node J, and it is necessary to perform extraction and filling operations on the next layer of nodes to which node J belongs (i.e., node L and node M). Entering the 4th layer again, the only nodes that have not been filled and marked as not requiring extraction and filling operations are node L and node M. The entity information of node L in the 4th layer is extracted and filled into node L. If node L is the tail node of a triplet, reverse extraction verification is required. If the verification is consistent, it means that there is no problem with the entity content of node L. Then, it is necessary to perform extraction and filling operations on the next layer of nodes to which node L belongs (i.e., node Q); extract the entity information of node M in the 4th layer and fill it into node M.Entering the 5th layer again, the only node that has not been filled and not marked as not requiring extraction and filling operation is node Q. Extract the entity information of node Q in the 5th layer and fill it into node Q, and end.
[0053] Moreover, if the node is the tail node of a triple, reverse extraction verification is required, specifically including: assuming that the triple is the head node-relationship-the node, which is the tail node, automatically reversely extracting the node, which is the tail node-relationship-head node, and using the industry-specific large model to determine whether the entity content filled in the head node is consistent with the reversely extracted entity content. If they are consistent, it indicates that there is no problem with the entity content of the node, and it is necessary to perform extraction and filling operations on the next layer of nodes to which the node belongs. If they are inconsistent, it indicates that there is a problem with the entity content of the node, and there is no need to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations.
[0054] In this solution, regular expressions and BERT models are used to extract and fill standard content in formatted documents into the target case template, which has higher extraction accuracy and efficiency. For non-standard content in formatted documents, standard content and non-standard content in unformatted documents, industry-specific large models are used to extract and fill them into the target case template.
[0055] like Figure 3 As shown, this embodiment also provides a system for improving the efficiency and quality of holographic archive extraction. The case documents in the holographic archive of a certain case include formatted documents and unformatted documents. The formatted documents include standard content and non-standard content, and the unformatted documents include standard content and non-standard content. The system includes a case template matching module 1, a document type judgment module 2, a first extraction and filling module 3, a second extraction and filling module 4 and a storage module 5.
[0056] The case template matching module 1 is used to match a target case template from a case template library according to the case type of the case. The target case template is constructed based on the knowledge graph principle.
[0057] The document type determination module 2 is used to determine whether the case document to be extracted is a formatted document or an unformatted document. If it is a formatted document, the first extraction and filling module 3 is called, and if it is an unformatted document, the second extraction and filling module 4 is called.
[0058] The first extraction and filling module 3 is used to extract the standard content in the formatted document for the target case template using a preset regular expression and a pre-trained BERT model, extract the entity information therefrom and fill it into the corresponding nodes in the target case template, and then use the industry-specific large model to extract the non-standard content in the formatted document, extract the entity information therefrom and fill it into the corresponding nodes in the target case template. During the extraction process using the industry-specific large model, extraction is performed layer by layer according to the node breadth-first strategy in the target case template.
[0059] Industry-specific large models are large language models obtained by fine-tuning and reinforcement learning based on the model base using industry-specific task design and related datasets.
[0060] The second extraction and filling module 4 is used to directly use the industry-specific large model to extract the standard content and non-standard content in the unformatted document for the target case template, extract the entity information therefrom and fill it into the corresponding nodes in the target case template, and extract layer by layer according to the node breadth-first strategy in the target case template during the extraction process using the industry-specific large model.
[0061] The storage module 5 is used to fill in the target case template to form the latest version of the case map of the case and store it in the holographic file.
[0062] The first extraction and filling module 3 includes a first extraction and filling unit 31 , a first judgment unit 32 , a first assignment unit 33 , a second extraction and filling unit 34 , a second judgment unit 35 and a first assignment judgment unit 36 .
[0063] The first extraction and filling unit 31 is used to extract the standard content in the formatted document for the target case template using a preset regular expression and a pre-trained BERT model, and extract entity information therefrom to fill in the corresponding nodes in the target case template.
[0064] The first judgment unit 32 is used to judge whether all nodes of the i-th layer in the target case template have been filled with entity information. If yes, the first assignment unit 33 is called, and if no, the second extraction and filling unit 34 is called, wherein 1≤i≤N, N is the number of layers of the target case template, and N is a positive integer.
[0065] The first assignment unit 33 is used to assign i=i+1 and repeatedly call the first judgment unit 32.
[0066] The second extraction and filling unit 34 is used to extract the non-standard content in the formatted document using the industry-specific large model, and extract the entity information therefrom to fill in the corresponding nodes in the target case template that are not filled in the layer and are not marked as not requiring extraction and filling operations.
[0067] In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy:
[0068] Traverse the layer, and for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when the entity content is extracted from the formatted document, fill it into the node. If the node is the tail node of a triple, reverse extraction verification is required. If they are consistent, it indicates that there is no problem with the entity content of the node, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. If they are inconsistent, it indicates that there is a problem with the entity content of the node, and there is no need to perform extraction and filling operations on all the nodes to which the node belongs. All the nodes to which the node belongs are marked as not requiring extraction and filling operations. When the entity content of the node is not extracted from the formatted document, there is no need to perform extraction and filling operations on all the nodes to which the node belongs. All the nodes to which the node belongs are marked as not requiring extraction and filling operations.
[0069] The second judgment unit 35 is used to judge whether all nodes of the i-th layer in the target case template have been traversed, and if so, calls the first assignment judgment unit 36, and if not, repeatedly calls the second extraction and filling unit 34.
[0070] The first value assignment judgment unit 36 is used to assign i=i+1 and judge whether i≤N. If so, the second extraction and filling unit 34 is repeatedly called, otherwise, the process ends.
[0071] The second extraction and filling module 4 includes a third judgment unit 41 , a second assignment unit 42 , a third extraction and filling unit 43 , a fourth judgment unit 44 and a second assignment judgment unit 45 .
[0072] The third judgment unit 41 is used to judge whether each node of the i-th layer in the target case template has been filled with entity information. If it is yes, the second assignment unit 42 is called, and if it is not, the third extraction and filling unit 43 is called, wherein 1≤i≤N, N is the number of layers of the target case template, and N is a positive integer.
[0073] The second assignment unit 42 is used to assign i=i+1 and repeatedly call the third judgment unit 41.
[0074] The third extraction and filling unit 43 is used to directly use the industry-specific large model to extract the standard content and non-standard content in the non-formatted document, and extract the entity information therefrom to fill in the corresponding nodes in the target case template that are not filled in this layer and are not marked as not requiring extraction and filling operations.
[0075] In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy:
[0076] The layer is traversed, wherein for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when entity content is extracted from the formatted document, it is filled into the node. When the node is the head node of a triple, there is no need for reverse extraction verification, which directly indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. When the node is the tail node of a triple, reverse extraction verification is required. If they are consistent, it indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. If they are inconsistent, it indicates that there is a problem with the entity content of the node, and it is not necessary to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations. When the entity content of the node is not extracted from the formatted document, there is no need to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations.
[0077] The fourth judgment unit 44 is used to judge whether all nodes of the i-th layer in the target case template have been traversed, and if so, calls the second assignment judgment unit 45, and if not, repeatedly calls the third extraction and filling unit 43.
[0078] The second value assignment judgment unit 45 is used to assign i=i+1 and judge whether i≤N. If so, the third extraction and filling unit 43 is repeatedly called, otherwise, the process ends.
[0079] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that these are only examples, and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.
Claims
1. A method for improving the efficiency and quality of holographic file extraction, characterized in that: The case documents in the holographic archive of the case include formatted documents and unformatted documents, the formatted documents include standard content and non-standard content, and the unformatted documents include standard content and non-standard content. The method includes the following steps: S1. According to the case type of the case, a target case template is matched from the case template library. The target case template is constructed based on the knowledge graph principle; S2, determining whether the case document to be extracted is a formatted document or an unformatted document, if it is a formatted document, proceeding to step S3, if it is an unformatted document, proceeding to step S4; S3. For the target case template, first use the pre-set regular expression and the pre-trained BERT model to extract the standard content in the formatted document, extract the entity information from it and fill it into the corresponding node in the target case template, then use the industry-specific big model to extract the non-standard content in the formatted document, extract the entity information from it and fill it into the corresponding node in the target case template, and use the industry-specific big model to extract layer by layer according to the node breadth-first strategy in the target case template during the extraction process, and enter step S5; Industry-specific large models are large language models that are fine-tuned and reinforced through learning based on the model base using industry-specific task designs and related datasets. S4. For the target case template, directly use the industry-specific big model to extract the standard content and non-standard content in the non-formatted document, extract the entity information from it and fill it into the corresponding node in the target case template, and use the industry-specific big model to extract layer by layer according to the node breadth-first strategy in the target case template during the extraction process, and enter step S5; S5. The filled target case template constitutes the latest version of the case map of the case and is stored in the holographic file; Step S3 includes: S31. For the target case template, use a pre-set regular expression and a pre-trained BERT model to extract the standard content in the formatted document, and extract entity information from it to fill in the corresponding node in the target case template; S32, determine whether all nodes in the target case template at the i-th level have been filled with entity information, if so, proceed to step S33, if not, proceed to step S34, where 1≤i≤N, N is the number of target case template layers, and N is a positive integer; S33, i=i+1, repeat step S32; S34, using the industry-specific large model to extract the non-standard content in the formatted document, extracting entity information therefrom and filling it into the corresponding nodes in the target case template that are not filled in the layer and are not marked as not requiring extraction and filling operations; In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy: Traverse the layer, wherein for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when entity content is extracted from the formatted document, fill it into the node; if the node is the tail node of a triple, reverse extraction verification is required; if they are consistent, it indicates that there is no problem with the entity content of the node, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs; if they are inconsistent, it indicates that there is a problem with the entity content of the node, and it is not necessary to perform extraction and filling operations on all nodes to which the node belongs; all nodes to which the node belongs are marked as not requiring extraction and filling operations; when the entity content of the node is not extracted from the formatted document, it is not necessary to perform extraction and filling operations on all nodes to which the node belongs; all nodes to which the node belongs are marked as not requiring extraction and filling operations; S35, determine whether all nodes in the target case template at the i-th level have been traversed, if yes, proceed to step S36, if no, repeat step S34; S36, i=i+1, determine whether i≤N, if so, repeat step S34, if not, end the process.
2. The method for improving the efficiency and quality of holographic file extraction according to claim 1, characterized in that: Step S4 includes: S41, determine whether all nodes in the target case template at the i-th level have been filled with entity information, if so, proceed to step S42, if not, proceed to step S43, where 1≤i≤N, N is the number of target case template layers, and N is a positive integer; S42, i=i+1, repeat step S41; S43, directly using the industry-specific large model to extract the standard content and non-standard content in the non-formatted document, extracting entity information therefrom and filling it into the corresponding nodes in the target case template that are not filled in the layer and are not marked as not requiring extraction and filling operations; In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy: Traverse the layer, wherein for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when entity content is extracted from the formatted document, fill it into the node. When the node is the head node of a triple, there is no need for reverse extraction verification, which directly indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. When the node is the tail node of a triple, reverse extraction verification is required. If they are consistent, it indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. If they are inconsistent, it indicates that there is a problem with the entity content of the node, and it is not necessary to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations. When the entity content of the node is not extracted from the formatted document, it is not necessary to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations. S44, determine whether all nodes in the target case template at the i-th level have been traversed, if yes, proceed to step S45, if no, repeat step S43; S45, i=i+1, determine whether i≤N, if so, repeat step S43, if not, end the process.
3. The method for improving the efficiency and quality of holographic file extraction according to claim 1 or 2, characterized in that: If the node is the tail node of a triple, reverse extraction verification is required: Suppose the triple is head node—relationship—the node, which is the tail node. Automatically reverse extract the node, which is the tail node—relationship—head node. Use the industry-specific large model to determine whether the entity content filled in the head node is consistent with the reversely extracted entity content. If they are consistent, it means that there is no problem with the entity content of the node, and it is necessary to perform extraction and filling operations on the next layer of nodes to which the node belongs. If they are inconsistent, it means that there is a problem with the entity content of the node, and there is no need to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations.
4. The method for improving the efficiency and quality of holographic file extraction according to claim 1, characterized in that: For the target case template, a preset regular expression is used to extract the standard content in the formatted document before the quality of the BERT model reaches the preset requirements. The data extracted by the regular expression is accumulated as a data sample set for training the BERT model. After the quality of the BERT model reaches the preset requirements after sample training, the trained BERT model is used to extract the standard content in the formatted document.
5. A system for improving the efficiency and quality of holographic file extraction, characterized in that: The case documents in the holographic archive of the case include formatted documents and unformatted documents, the formatted documents include standard content and non-standard content, and the unformatted documents include standard content and non-standard content. The system includes a case template matching module, a document type judgment module, a first extraction and filling module, a second extraction and filling module, and a storage module; The case template matching module is used to match a target case template from the case template library according to the case type of the case, and the target case template is constructed based on the knowledge graph principle; The document type determination module is used to determine whether the case document to be extracted is a formatted document or an unformatted document, and to call the first extraction and filling module when it is a formatted document, and to call the second extraction and filling module when it is an unformatted document; The first extraction and filling module is used to extract the standard content in the formatted document using a preset regular expression and a pre-trained BERT model for the target case template, extract entity information therefrom and fill it into the corresponding nodes in the target case template, and then use the industry-specific large model to extract the non-standard content in the formatted document, extract entity information therefrom and fill it into the corresponding nodes in the target case template, and perform extraction layer by layer according to the node breadth-first strategy in the target case template during the extraction process using the industry-specific large model; Industry-specific large models are large language models that are fine-tuned and reinforced through learning based on the model base using industry-specific task designs and related datasets. The second extraction and filling module is used to directly use the industry-specific large model to extract the standard content and non-standard content in the non-formatted document for the target case template, extract entity information from it and fill it into the corresponding node in the target case template, and extract layer by layer according to the node breadth-first strategy in the target case template during the extraction process using the industry-specific large model; The storage module is used to store the latest version of the case map of the target case after filling in the template into the holographic file; The first extraction and filling module includes a first extraction and filling unit, a first judgment unit, a first assignment unit, a second extraction and filling unit, a second judgment unit and a first assignment judgment unit; The first extraction and filling unit is used to extract standard content in the formatted document using a preset regular expression and a pre-trained BERT model for the target case template, and extract entity information therefrom to fill in corresponding nodes in the target case template; The first judgment unit is used to judge whether each node of the i-th layer in the target case template has been filled with entity information, and if yes, the first assignment unit is called, and if no, the second extraction and filling unit is called, wherein 1≤i≤N, N is the number of layers of the target case template, and N is a positive integer; The first assignment unit is used to assign i=i+1 and repeatedly call the first judgment unit; The second extraction and filling unit is used to extract the non-standard content in the formatted document by using the industry-specific large model, and extract entity information therefrom to fill in the corresponding nodes in the target case template that are not filled in the layer and are not marked as not requiring extraction and filling operations; In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy: Traverse the layer, wherein for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when entity content is extracted from the formatted document, fill it into the node; if the node is the tail node of a triple, reverse extraction verification is required; if they are consistent, it indicates that there is no problem with the entity content of the node, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs; if they are inconsistent, it indicates that there is a problem with the entity content of the node, and it is not necessary to perform extraction and filling operations on all nodes to which the node belongs; all nodes to which the node belongs are marked as not requiring extraction and filling operations; when the entity content of the node is not extracted from the formatted document, it is not necessary to perform extraction and filling operations on all nodes to which the node belongs; all nodes to which the node belongs are marked as not requiring extraction and filling operations; The second judgment unit is used to judge whether all nodes of the i-th layer in the target case template have been traversed, and if yes, call the first assignment judgment unit, and if no, repeatedly call the second extraction and filling unit; The first assignment judgment unit is used to assign i=i+1 and judge whether i≤N. If so, the second extraction and filling unit is repeatedly called, otherwise, the process ends.
6. The system for improving the efficiency and quality of holographic file extraction as claimed in claim 5, characterized in that: The second extraction and filling module includes a third judgment unit, a second assignment unit, a third extraction and filling unit, a fourth judgment unit and a second assignment judgment unit; The third judgment unit is used to judge whether all nodes of the i-th layer in the target case template have been filled with entity information, and call the second assignment unit if it is yes, and call the third extraction and filling unit if it is no, where 1≤i≤N, N is the number of layers of the target case template, and N is a positive integer; The second assignment unit is used to assign i=i+1 and repeatedly call the third judgment unit; The third extraction and filling unit is used to directly use the industry-specific large model to extract the standard content and non-standard content in the non-formatted document, extract entity information therefrom and fill it into the corresponding nodes in the target case template that are not filled in the layer and are not marked as not requiring extraction and filling operations; In the process of extracting the industry-specific large model, the nodes in the target case template are extracted layer by layer according to the breadth-first strategy: Traverse the layer, wherein for a node in the layer that is not filled and not marked as not requiring extraction and filling operations, when entity content is extracted from the formatted document, fill it into the node. When the node is the head node of a triple, there is no need for reverse extraction verification, which directly indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. When the node is the tail node of a triple, reverse extraction verification is required. If they are consistent, it indicates that the entity content of the node is fine, and it is necessary to perform extraction and filling operations on the nodes of the next layer to which the node belongs. If they are inconsistent, it indicates that there is a problem with the entity content of the node, and it is not necessary to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations. When the entity content of the node is not extracted from the formatted document, it is not necessary to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations. The fourth judgment unit is used to judge whether all nodes of the i-th layer in the target case template have been traversed, and if yes, call the second assignment judgment unit, and if no, repeatedly call the third extraction and filling unit; The second assignment judgment unit is used to assign i=i+1 and judge whether i≤N. If so, the third extraction and filling unit is repeatedly called, otherwise, the process ends.
7. The system for improving the efficiency and quality of holographic file extraction according to claim 5 or 6, characterized in that: If the node is the tail node of a triple, reverse extraction verification is required: Suppose the triple is head node—relationship—the node, which is the tail node. Automatically reverse extract the node, which is the tail node—relationship—head node. Use the industry-specific large model to determine whether the entity content filled in the head node is consistent with the reversely extracted entity content. If they are consistent, it means that there is no problem with the entity content of the node, and it is necessary to perform extraction and filling operations on the next layer of nodes to which the node belongs. If they are inconsistent, it means that there is a problem with the entity content of the node, and there is no need to perform extraction and filling operations on all nodes to which the node belongs. All nodes to which the node belongs are marked as not requiring extraction and filling operations.
8. The system for improving the efficiency and quality of holographic file extraction as claimed in claim 5, characterized in that: For the target case template, a preset regular expression is used to extract the standard content in the formatted document before the quality of the BERT model reaches the preset requirements. The data extracted by the regular expression is accumulated as a data sample set for training the BERT model. After the quality of the BERT model reaches the preset requirements after sample training, the trained BERT model is used to extract the standard content in the formatted document.
