Information extraction method and device, electronic equipment and computer readable storage medium

Abstract

The application provides an information extraction method and device, electronic equipment and computer readable storage medium. The method comprises the following steps: inputting a target document and a prompt text into a large language model; receiving content field values of each case basic information extraction item, start position field values and end position field values of each case entity content extraction item extracted from the target document according to the indication of the prompt text by the large language model; for each case entity content extraction item, taking the start position field value and the end position field value of the case entity content extraction item as a regular expression, matching the target document, and matching the content field value of the case entity content extraction item from the target document; and generating information extraction text of the target document according to the content field values of each case basic information extraction item and each case entity content extraction item. Through the method, the efficiency and accuracy of key information extraction are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an information extraction method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Currently, when extracting key information from legally binding documents (such as judicial documents), manual extraction or extraction using regular expressions is commonly used.

[0003] Due to the large number of documents, manual extraction would be time-consuming, inefficient, subjective, and inconsistent.

[0004] If extraction is done using regular expressions (which are essentially keyword matching), only specific keywords can be found. If the keywords in the document do not match the keywords corresponding to the regular expression, the key information cannot be successfully extracted from the document. Furthermore, regular expressions cannot identify missing key information. If the keyword corresponding to the regular expression does not appear in the document, this method will not be able to find it. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide an information extraction method, apparatus, electronic device and computer-readable storage medium to improve the efficiency and accuracy of key information extraction.

[0006] In a first aspect, embodiments of this application provide an information extraction method, including: The legally binding target document and the prompt text constructed for the target document are input into the large language model; wherein, the prompt text contains the name and content description of each case basic information extraction item, and the name, content description, start position marker information, end position marker information, and integrity extraction instruction of each case entity content extraction item; The large language model receives the content field values ​​of each case basic information extraction item, the start position field value, and the end position field value of each case entity content extraction item extracted from the target document according to the instructions of the prompt text. For each case entity content extraction item, the start position field value and end position field value of the case entity content extraction item are used as regular expressions to match the target document, and the content field value of the case entity content extraction item is matched from the target document. Based on the content field values ​​of each case basic information extraction item and each case entity content extraction item, the information extraction text of the target document is generated.

[0007] In conjunction with the first aspect, this application provides a first possible implementation of the first aspect, wherein the large language model is used to return a null value when the content field value, start position field value, or end position field value is not extracted from the target document; Before matching the start and end position field values ​​of each case entity content extraction item with a regular expression in the target document to extract the content field value of the case entity content extraction item from the target document, the method further includes: Determine whether there is a null value among the content field value, the start position field value, and the end position field value; If a null value exists, it means that the target document failed the initial quality verification. The target document will be discarded, and subsequent steps will be stopped. If no null value is found, it means that the target document has passed the initial quality verification and the subsequent steps will continue.

[0008] In conjunction with the first aspect, this application provides a second possible implementation of the first aspect, wherein, after generating the information extraction text of the target document based on the content field values ​​of each of the case basic information extraction items and each of the case entity content extraction items, the method further includes: For each extraction item in the case basic information extraction item and the case entity content extraction item, identify whether the beginning and end of the content field value of the extraction item in the information extraction text contain corresponding preset keywords; The number of extraction items that do not contain the corresponding preset keywords at the beginning and end is counted, and the probability of the information extraction text being unqualified is determined based on the number of such items. The extracted text containing information whose failure probability is greater than a preset probability threshold is discarded; The extracted information text with a failure probability less than or equal to the preset probability threshold is submitted to the manual verification process. If the manual verification passes, the extracted information text is retained; if the manual verification fails, the extracted information text is discarded.

[0009] In conjunction with the first aspect or the second possible implementation of the first aspect, embodiments of this application provide a third possible implementation of the first aspect, wherein the method further includes: The extracted information text is used as a training sample for the case search model, and the case search model is trained accordingly. The case search model is used to return the extracted information text of the case with the highest similarity to the user when the user inputs legally related text.

[0010] In conjunction with the first aspect, this application provides a fourth possible implementation of the first aspect, wherein the number of characters in the start position field value is greater than or equal to 10 and less than or equal to 15; and the number of characters in the end position field value is greater than or equal to 10 and less than or equal to 15.

[0011] Secondly, embodiments of this application also provide an information extraction device, comprising: The input module is used to input the legally binding target document and the prompt text constructed for the target document into the large language model; wherein, the prompt text contains the name and content description of each case basic information extraction item, as well as the name, content description, start position marker information, end position marker information, and integrity extraction instruction of each case entity content extraction item; The receiving module is used to receive the content field values ​​of each case basic information extraction item, the start position field value and the end position field value of each case entity content extraction item extracted from the target document by the large language model according to the instructions of the prompt text; The matching module is used to match the target document with the start position field value and end position field value of each case entity content extraction item as a regular expression, and to extract the content field value of the case entity content extraction item from the target document. The generation module is used to generate the information extraction text of the target document based on the content field values ​​of each case basic information extraction item and each case entity content extraction item.

[0012] In conjunction with the second aspect, this application provides a first possible implementation of the second aspect, wherein the large language model is used to return a null value when the content field value, start position field value, or end position field value is not extracted from the target document; The device further includes: The judgment module is used by the matching module to match the target document for each case entity content extraction item by using the start position field value and end position field value of the case entity content extraction item as a regular expression. Before matching the content field value of the case entity content extraction item from the target document, the module judges whether there is a null value in the content field value, the start position field value, and the end position field value. The stop module is used to discard the target document and stop executing subsequent steps if a null value exists, indicating that the initial quality verification of the target document has failed. The continue module is used to indicate that the target document has passed the initial quality verification if no null value exists, and to continue executing subsequent steps.

[0013] In conjunction with the second aspect, this application provides a second possible implementation of the second aspect, wherein the apparatus further includes: The identification module is used to identify whether the beginning and end of the content field value of the extraction item in the information extraction text of the target document contains corresponding preset keywords for each extraction item in the case basic information extraction item and each extraction item in the case entity content extraction item after the generation module generates the information extraction text of the target document based on the content field value of each extraction item in the case basic information extraction item and the case entity content extraction item. The statistics module is used to count the number of extraction items that do not contain the corresponding preset keywords at the beginning and end, and to determine the probability of the information extraction text being unqualified based on the number of such items. The discard module is used to discard the extracted text of information whose failure probability is greater than a preset probability threshold. The submission module is used to submit the extracted information text with a failure probability less than or equal to the preset probability threshold to the manual verification process. If the manual verification is successful, the extracted information text is retained; if the manual verification fails, the extracted information text is discarded.

[0014] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps in any of the possible implementations of the first aspect described above are performed.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps in any of the possible implementations of the first aspect described above.

[0016] The information extraction method, apparatus, electronic device, and computer-readable storage medium provided in this application's embodiments input a legally binding target document and a specially constructed prompt text into a large language model. Utilizing the natural language understanding capabilities of the large language model, and following the extraction requirements set in the prompt text, two types of information are initially extracted from the target document: one is the content field values ​​of basic case information extraction items, and the other is the start and end position field values ​​of case entity content extraction items. Subsequently, the start and end position field values ​​of each entity content extraction item are used as regular expressions to precisely match the target document, thereby extracting the actual content field values ​​of that entity content extraction item from the target document. Finally, the content field values ​​of all basic information items and entity content items are integrated to generate a complete information extraction text. The solution in this embodiment completely replaces the manual reading and extracting process, which helps to improve processing speed. At the same time, since all target documents use the same prompt text as the extraction basis, the large language model always maintains the same extraction standard for similar documents, which fundamentally avoids the drawbacks of subjective and inconsistent extraction results caused by differences in understanding or fatigue among different people.

[0017] Furthermore, the prompt text not only provides the name of each extraction item but also offers detailed content descriptions and semantic start and end position markers. Based on its understanding of natural language, the large language model can identify text blocks in the target document that have the same or similar descriptive functions. Even if the expressions in the target document are not entirely consistent with the preset keywords, it can accurately locate their start and end positions, thus overcoming the limitations of traditional regular expressions, which can only match fixed strings and miss variant expressions. Secondly, this solution does not directly use the text blocks returned by the large language model as the final extracted text. Instead, it uses the start and end position field values ​​identified by the large language model as anchor points and then uses regular expressions to precisely extract the corresponding content from the target document. This approach leverages the accuracy of the large language model's semantic understanding to locate boundaries while avoiding potential illusions or rewriting that might occur when the large language model generates text, ensuring word-for-word consistency between the extracted content and the target document, further improving the accuracy and reliability of the extraction results.

[0018] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of an information extraction method provided in an embodiment of this application is shown; Figure 2 A flowchart of another information extraction method provided in an embodiment of this application is shown; Figure 3 This paper shows a schematic diagram of the structure of an information extraction device provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] Currently, when extracting key information from legally binding documents (such as judicial documents), manual extraction or extraction using regular expressions is commonly used.

[0023] Due to the large number of documents, manual extraction would be time-consuming, inefficient, subjective, and inconsistent.

[0024] If extraction is done using regular expressions (which are essentially keyword matching), only specific keywords can be found. If the keywords in the document do not match the keywords corresponding to the regular expression, the key information cannot be successfully extracted from the document. Furthermore, regular expressions cannot identify missing key information. If the keyword corresponding to the regular expression does not appear in the document, this method will not be able to find it.

[0025] Based on this, embodiments of this application provide an information extraction method, apparatus, electronic device, and computer-readable storage medium to improve the efficiency and accuracy of key information extraction, which are described below through embodiments.

[0026] To facilitate understanding of this embodiment, a method for extracting information disclosed in this application will first be described in detail. For example... Figure 1 As shown, the process includes the following steps S101-S104: S101: Input the legally binding target document and the prompt text constructed for the target document into the large language model; wherein, the prompt text contains the name and content description of each case basic information extraction item, as well as the name, content description, start position marker information, end position marker information, and integrity extraction instruction of each case entity content extraction item.

[0027] In this embodiment, a legally binding target document is acquired for processing. For example, the target document can be a legally binding document such as a judicial judgment, ruling, or complaint, typically in Word or PDF format, and its content exists in unstructured natural language text form.

[0028] For each acquired target document, a dedicated prompt text is pre-constructed. This prompt text is a structured set of instructions designed to guide the large language model in information extraction, and its content is customized according to the type of target document and the information items to be extracted. For the same type of target document (e.g., a civil judgment), the same prompt text can be used to ensure consistency in extraction standards.

[0029] The prompt text contains settings information for two types of extraction items: The first category is basic case information extraction items. These items are used to obtain relatively fixed summary information from the target document, which can usually be obtained directly from the beginning or end of the document. For example, basic case information extraction items could be the case name and title.

[0030] For each case basic information extraction item, the prompt text specifies the name and content description of that item. For example, for the case basic information extraction item "Case Name", the name could be "Case Name" and the content description could be "Document Case Number or File Name"; for the case basic information extraction item "Title", the name could be "Title" and the content description could be "Standard case title summarized in the format of 'Plaintiff XX v. Defendant XX, YYY Dispute'".

[0031] The second category is case entity content extraction items. These items are used to retrieve long text paragraphs from the target document that contain core facts and legal judgments, such as "Case Summary," "Finding Facts," "The Court's Opinion," and "Judgment." For each case entity content extraction item, the prompt text includes the following information: the name of the extraction item (e.g., Case Summary); the content description of the extraction item (e.g., the plaintiff's claims, facts and reasons, and the defendant's response); start position marker information (e.g., similar example text such as "The plaintiff...filed claims with this court"); end position marker information (e.g., similar example text such as "Ascertained after trial" indicating the end of this section); and a completeness extraction instruction (requiring the large language model to extract complete field values ​​from the target document when extracting field values ​​for each case entity content extraction item, and to return a specific flag, such as null, if the content does not exist in the target document).

[0032] After the prompt text is constructed, it is used together with the full text of the target document as input to call the application programming interface (API) of the Large Language Model (LLM). The LLM will then analyze the target document according to the instructions in the prompt text, based on its ability to understand natural language.

[0033] S102: Receive the content field values ​​of each case basic information extraction item, the start position field value, and the end position field value of each case entity content extraction item extracted from the target document by the large language model according to the prompt text.

[0034] In this embodiment, after processing the input, the large language model returns a structured output (JSON data). This step receives this output, which contains two parts: For each case basic information extraction item, the large language model returns its content field value. For example, for the case basic information extraction item "Case Name", the large language model may return "(2020)Liaoning 0283 Civil First Instance No. 1666" as its content field value; for the case basic information extraction item "Title", the large language model may return "Plaintiff XXX v. Defendant XX in a divorce dispute" as its content field value.

[0035] For each extracted entity content item, the large language model does not directly return the full text content of the paragraph. Instead, it returns the "start position field value" and "end position field value" of the paragraph. These two field values ​​are key text fragments in the original target document that uniquely identify the start and end positions of the paragraph. For example, to ensure the accuracy of the location, the number of characters in the start position field value can be set to greater than or equal to 10 characters and less than or equal to 15 characters; the number of characters in the end position field value can also be set to greater than or equal to 10 characters and less than or equal to 15 characters.

[0036] For example, for the entity content extraction item "established facts," the large language model might return "After investigation, it was found that the plaintiff and defendant met through an introduction in 2006" as the start position field value and "request legal action." as the end position field value. When determining these position field values, the large language model relies on the start and end position markers given in the prompt text and utilizes its semantic understanding capabilities to identify text fragments with the same function and similar meaning in the original text, rather than performing mechanical keyword matching.

[0037] S103: For each case entity content extraction item, use the start position field value and end position field value of the case entity content extraction item as a regular expression to match the target document, and extract the content field value of the case entity content extraction item from the target document.

[0038] In this embodiment, since the start position field value and end position field value returned by the large language model are precise text fragments from the original target document, this step uses these two text fragments to construct a regular expression to perform string matching operations on the original target document.

[0039] Specifically, for each case entity content extraction item, the start position field value returned by the large speech model is used as a regular expression to match the starting point, and the end position field value is used as a regular expression to match the ending point. This searches the target document for all text content from the start point to the end point (or including the end point). In this way, the text content corresponding to the case entity content extraction item and identical to the target document word-for-word can be accurately extracted from the original target document and used as the final content field value for that extraction item.

[0040] For example, continuing from the previous example, we take the starting field value "It was found after trial that the plaintiff and defendant met through an introduction in 2006" as the starting marker and the ending field value "Request for legal processing." as the ending marker. We locate and extract all the text content from "It was found after trial that the plaintiff and defendant met through an introduction in 2006" to "Request for legal processing." in the target document. This part of the text is the complete content of "Finding Facts".

[0041] This step combines the accuracy of semantic localization from large language models with the precision of text matching using regular expressions, avoiding the illusions or content rewriting problems that may occur when directly using large language models to generate text.

[0042] S104: Generate the information extraction text of the target document based on the content field values ​​of the basic information extraction item and the entity content extraction item of each case.

[0043] In this embodiment, after obtaining the content field values ​​of all extracted items through the above steps, this step integrates these content field values ​​to generate a structured information extraction text.

[0044] For example, the content field values ​​of the extracted basic case information items (such as case name and title) can be used as the document's title; the content field values ​​of each extracted case entity content item (such as case summary, established facts, the court's opinion, and judgment) can be used as independent chapters, arranged sequentially. The integrated text can be stored in an easy-to-read and easily processed markup language format (such as Markdown), forming a final output that retains the original content while having a clear structure. This extracted text can be used for various downstream tasks such as subsequent judicial data analysis, training of similar case retrieval models, and knowledge base construction.

[0045] The reason this embodiment doesn't directly use the large language model to output the extracted text, but instead employs a two-step architecture of large language model localization and regular expression truncation, is to ensure the word-by-word accuracy and traceability of the extraction results. Large language models have an inherent "illusion" flaw when generating long texts, potentially unintentionally rewriting, summarizing, or rearranging the word order of the original text, generating content that appears fluent but actually deviates from the original. For legally binding target documents, any minor word substitution or sentence structure change can lead to a shift in legal semantics. For example, paraphrasing "as determined by trial" may lose its precise meaning in the legal context. This embodiment fundamentally limits the large language model's free generation space by only allowing it to output short text fragments (e.g., 10 to 15 characters) at the beginning and end positions, requiring it to only identify and reproduce the existing continuous characters in the original target document, thus minimizing the risk of illusion.

[0046] Building upon this, this embodiment further leverages the precise matching capabilities of regular expressions, using the start and end position field values ​​returned by the large language model as anchor points to extract corresponding content word by word from the original target document. This approach fully utilizes the complementary advantages of the large language model and regular expressions: the large language model handles semantic understanding, flexibly identifying variations of expressions such as "as determined by the court," "as determined by this court," and "the above facts are confirmed by…," accurately locating paragraph boundaries; regular expressions ensure that the extracted content is identical to the original document.

[0047] Compared to directly using large language models to generate extracted text, this solution achieves a better balance between intelligence and accuracy. It retains the ability of large language models to handle natural language variations while ensuring the originality and accuracy of the output content through secondary truncation. Furthermore, since the final content originates from the original text, each extracted result can be traced back to its exact location in the target document, facilitating subsequent manual review, cross-validation, and quality control, thus providing a reliable data foundation for building a high-quality legal knowledge base.

[0048] In one possible implementation, the large language model is used to return a null value when the content field value, start position field value, or end position field value is not extracted from the target document.

[0049] In this embodiment, the integrity extraction instruction contained in the prompt text is used to instruct the large language model to return a specific null value flag when it is unable to extract a certain case entity content extraction item from the target document. For example, the large language model is configured to return a null value for the missing item when it fails to extract the content field value of a certain case basic information extraction item from the target document, or fails to extract the start position field value or end position field value of a certain case entity content extraction item.

[0050] This mechanism enables the solution to automatically identify documents with missing content. For example, when a target document is missing the "assertional facts" section, the large language model will be unable to locate the starting field value, and will return a null value in the returned result, thus solving the problem that traditional regular expressions cannot detect missing information.

[0051] Accordingly, after receiving the content field values ​​of each case basic information extraction item returned by the large language model in step S102, as well as the start position field value and end position field value of each case entity content extraction item, before executing step S103, as follows: Figure 2 As shown, the specific steps S201-S203 can also be performed: S201: Determine if there is a null value in the content field, the start position field, and the end position field.

[0052] In this embodiment, all field values ​​received in step S102 are iterated and checked. For each case basic information extraction item, its content field value is checked to see if it is null; for each case entity content extraction item, its start position field value and end position field value are checked to see if they are null. If any field value is null, it is considered that a null value exists.

[0053] S202: If a null value exists, it means that the target document failed the initial quality verification. The target document will be discarded, and step S103 and subsequent steps will be stopped.

[0054] In this embodiment, the presence of a null value indicates that the target document has missing content and fails the initial quality verification. For example, if the start position field of the case entity content extraction item "judgment result" is null, it means that the large language model has failed to locate the starting position of the judgment result in the target document, and the target document may be missing the judgment result section. In this case, the target document is marked as poor quality and discarded, and subsequent steps S103 (i.e., stopping regular expression matching on the document) and S104 (i.e., stopping the generation of information extraction text) are stopped. This avoids generating erroneous results based on incomplete data and effectively saves computational resources.

[0055] S203: If no null value exists, it means that the target document has passed the initial quality verification. Continue to execute step S103 and subsequent steps.

[0056] In this embodiment, when it is determined that no null values ​​exist, it indicates that all case basic information extraction items and case entity content extraction items have successfully returned non-empty field values, and the target document has passed the initial quality verification. At this time, steps S103 and S104 are executed to perform subsequent content matching and structured text generation on the target document.

[0057] Through steps S201 to S203 described above, this embodiment introduces an automated quality screening mechanism based on null values. This mechanism utilizes the null value markers returned by the integrity extraction command from a large language model to quickly identify and remove low-quality documents before entering the precise truncation stage. This further ensures the overall quality of the final generated extracted text and improves the robustness and efficiency of the processing flow.

[0058] In one possible implementation, a secondary quality assessment process is also provided for the generated information extraction text. This process quantitatively evaluates the structural regularity and boundary accuracy of the text after its generation, thereby enabling further filtering and optimization of the output results. Specifically, after generating the information extraction text of the target document in step S104, the following steps S1051-S1054 can also be performed: S1051: For each extraction item in the case basic information extraction item and the case entity content extraction item, identify whether the beginning and end of the content field value of the extraction item in the information extraction text contain the corresponding preset keywords.

[0059] In this embodiment, corresponding opening and closing keywords are pre-defined for each extraction item. These pre-defined keywords (opening and closing keywords) are pre-defined based on the standardized expression methods of various legally binding target documents, and are used to verify whether the boundaries of the extracted content conform to the conventional format of this type of document.

[0060] For example, for the case content extraction item "This Court Holds," the preset keyword at the beginning can be set to "This Court Holds," and the preset keyword at the end can be set to a similar expression such as "Judgment is as follows" or "In accordance with the provisions of..."; for the case content extraction item "Judgment Result," the preset keyword at the beginning can be set to "Judgment is as follows" or "Ruling is as follows," and the preset keyword at the end can be set to a similar expression such as "If you disagree with this judgment" or "Case Acceptance Fee"; for the case basic information extraction item "Case Name," the preset keywords at the beginning and end can be set accordingly based on the case number format.

[0061] Subsequently, each extraction item in the extracted text is traversed, and each item's content field value is checked to see if the beginning and end of the value contain its corresponding preset keyword. If the beginning or end of the value does not contain the preset keyword, the extraction item is recorded as a boundary exception.

[0062] S1052: Count the number of extraction items that do not contain the corresponding preset keywords at the beginning and end, and determine the probability of the extracted text being unqualified based on the number.

[0063] In this embodiment, the number of all extracted items recorded as boundary anomalies is counted. Then, the probability of non-compliance of the extracted text is calculated based on the ratio of this number to the total number of extracted items contained in the extracted text.

[0064] For example, if the extracted information text contains six extraction items (including basic case information extraction items and case entity content extraction items), and three of these extraction items do not contain the corresponding preset keywords at the beginning or end, then the probability of non-compliance is 50%. This probability of non-compliance reflects the degree of risk that the extracted information text may have inaccurate boundary positioning or incomplete content extraction.

[0065] S1053: Discard the extracted text containing information with a non-compliance probability greater than a preset probability threshold.

[0066] In this embodiment, a probability threshold is preset to define the acceptable range of quality risk. If the calculated probability of non-compliance exceeds the preset probability threshold, the extracted text is deemed to have an excessively high quality risk, and the accuracy of its content boundaries cannot be effectively guaranteed. In this case, the extracted text is discarded directly and not subsequently stored or used.

[0067] S1054: Submit the extracted text of information with a non-compliance probability less than or equal to a preset probability threshold to the manual verification process. If the manual verification passes, the extracted text of information is retained; if the manual verification fails, the extracted text of information is discarded.

[0068] In this embodiment, if the calculated probability of non-compliance is less than or equal to a preset probability threshold, it indicates that the boundary anomalies of the extracted text are within a controllable range and have a certain quality foundation. At this point, the extracted text is submitted to a manual verification process for review by professionals to ultimately confirm its accuracy and completeness. If the manual verification passes, the extracted text is retained and stored in the database or used for subsequent tasks; if the manual verification fails, it is discarded to ensure that the quality of the data stored meets high standards.

[0069] Through steps S1051 to S1054 above, this embodiment introduces a statistical evaluation mechanism and manual review process based on preset keywords. This mechanism uses preset keywords to standardize the content boundaries of each extracted item in the information extraction text, and automates the graded screening of the output results through quantitative calculation of the probability of non-compliance. High-quality text is retained, text with questionable quality undergoes manual review, and text with excessively low quality is directly discarded. This constructs a multi-layered quality control system that can effectively identify and correct potential boundary errors based on automated processing, ensuring that the final generated information extraction text has reliable structural standardization and content accuracy.

[0070] In one possible implementation, after generating the information extraction text of the target document in step S104, or after manually verifying and confirming the retention of the information extraction text in step S1054, the following steps may also be performed: The extracted text is used as a training sample for the case search model, which is then used to train the model. The case search model is used to return the extracted text of the most similar case to the user when the user inputs legally related text.

[0071] In this embodiment, the extracted text is structured text extracted and generated from a legally binding target document. This extracted text contains content field values ​​for basic case information (such as case name and title) and content field values ​​for case entity content (such as case summary, established facts, the court's opinion, and judgment), presenting a clear structure and complete content. Compared to the original, unstructured target document, this extracted text features a unified format, clearly defined elements, and well-defined boundaries, making it more suitable as training data for machine learning models.

[0072] The case search model is a semantic retrieval model built on machine learning algorithms. Its core function is to perform semantic similarity matching from a pre-built case database when it receives legally related query text input by a user, and return the extracted information text corresponding to one or more cases with the highest similarity to the user. For example, this case search model can be built using a Siamese network structure in deep learning or a pre-trained language model based on Transformer. By learning from training samples, it masters the semantic associations and similarity calculation methods between case texts.

[0073] During model training, a large amount of extracted text is input as training samples into the case search model. By learning the content and structural features of different cases in these training samples, the model continuously adjusts its internal parameters, gradually improving its ability to understand the semantics of case texts and the accuracy of similarity calculations. After training, when a user inputs a case description, a legal question, or preliminary litigation materials, the case search model can efficiently search the database, find cases most similar to the user's input in terms of factual elements, applicable law, or points of contention, and return the corresponding extracted text, providing the user with accurate case references.

[0074] Based on the same technical concept, embodiments of this application also provide an information extraction device, such as... Figure 3 As shown, the device includes: The input module 301 is used to input a legally binding target document and a prompt text constructed for the target document into the large language model; wherein the prompt text includes the name and content description of each case basic information extraction item, and the name, content description, start position marker information, end position marker information, and integrity extraction instruction of each case entity content extraction item; The receiving module 302 is used to receive the content field values ​​of each case basic information extraction item, the start position field value and the end position field value of each case entity content extraction item extracted from the target document by the large language model according to the instructions of the prompt text; The matching module 303 is used to match the target document with the start position field value and end position field value of each case entity content extraction item as a regular expression, and to match the content field value of the case entity content extraction item from the target document. The generation module 304 is used to generate the information extraction text of the target document based on the content field values ​​of each case basic information extraction item and each case entity content extraction item.

[0075] Optionally, the large language model is configured to return a null value when the content field value, start position field value, or end position field value is not extracted from the target document; The device further includes: The judgment module is used by the matching module 303 to match the target document for each case entity content extraction item by using the start position field value and end position field value of the case entity content extraction item as a regular expression. Before matching the content field value of the case entity content extraction item from the target document, the judgment module determines whether there is a null value in the content field value, the start position field value, and the end position field value. The stop module is used to discard the target document and stop executing subsequent steps if a null value exists, indicating that the initial quality verification of the target document has failed. The continue module is used to indicate that the target document has passed the initial quality verification if no null value exists, and to continue executing subsequent steps.

[0076] Optionally, the device further includes: The identification module is used to identify whether the beginning and end of the content field value of the extraction item in the information extraction text of the target document contains corresponding preset keywords for each extraction item in the case basic information extraction item and the case entity content extraction item after the generation module 304 generates the information extraction text of the target document based on the content field value of each extraction item in the case basic information extraction item and the case entity content extraction item. The statistics module is used to count the number of extraction items that do not contain the corresponding preset keywords at the beginning and end, and to determine the probability of the information extraction text being unqualified based on the number of such items. The discard module is used to discard the extracted text of information whose failure probability is greater than a preset probability threshold. The submission module is used to submit the extracted information text with a failure probability less than or equal to the preset probability threshold to the manual verification process. If the manual verification is successful, the extracted information text is retained; if the manual verification fails, the extracted information text is discarded.

[0077] Optionally, the device further includes: The model training module is used to use the extracted information text as training samples for the case search model to train the case search model; the case search model is used to return the extracted information text of the case with the highest similarity to the user when the user inputs legally related text.

[0078] Optionally, the number of characters in the start position field value is greater than or equal to 10 and less than or equal to 15; the number of characters in the end position field value is greater than or equal to 10 and less than or equal to 15.

[0079] Figure 4A schematic diagram of an electronic device provided in this application embodiment includes: a processor 401, a memory 402, and a bus 403. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs the above-described information processing method, the processor 401 and the memory 402 communicate through the bus 403. The processor 401 executes the machine-readable instructions to perform the steps of the method described in Embodiment 1.

[0080] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps described in Embodiment 1.

[0081] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, electronic devices, and computer-readable storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, electronic devices, and computer-readable storage media can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or modules may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0085] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

1. An information extraction method, characterized in that, include: The legally binding target document and the prompt text constructed for the target document are input into the large language model; wherein, the prompt text contains the name and content description of each case basic information extraction item, and the name, content description, start position marker information, end position marker information, and integrity extraction instruction of each case entity content extraction item; The large language model receives the content field values ​​of each case basic information extraction item, the start position field value, and the end position field value of each case entity content extraction item extracted from the target document according to the instructions of the prompt text. For each case entity content extraction item, the start position field value and end position field value of the case entity content extraction item are used as regular expressions to match the target document, and the content field value of the case entity content extraction item is matched from the target document. Based on the content field values ​​of each case basic information extraction item and each case entity content extraction item, the information extraction text of the target document is generated.

2. The method according to claim 1, characterized in that, The large language model is used to return a null value when the content field value, start position field value, or end position field value is not extracted from the target document; Before matching the start and end position field values ​​of each case entity content extraction item with a regular expression in the target document to extract the content field value of the case entity content extraction item from the target document, the method further includes: Determine whether there is a null value among the content field value, the start position field value, and the end position field value; If a null value exists, it means that the target document failed the initial quality verification. The target document will be discarded, and subsequent steps will be stopped. If no null value is found, it means that the target document has passed the initial quality verification and the subsequent steps will continue.

3. The method according to claim 1, characterized in that, After generating the information extraction text of the target document based on the content field values ​​of each case basic information extraction item and each case entity content extraction item, the method further includes: For each extraction item in the case basic information extraction item and the case entity content extraction item, identify whether the beginning and end of the content field value of the extraction item in the information extraction text contain corresponding preset keywords; The number of extraction items that do not contain the corresponding preset keywords at the beginning and end is counted, and the probability of the information extraction text being unqualified is determined based on the number of such items. The extracted text containing information whose failure probability is greater than a preset probability threshold is discarded; The extracted information text with a failure probability less than or equal to the preset probability threshold is submitted to the manual verification process. If the manual verification passes, the extracted information text is retained; if the manual verification fails, the extracted information text is discarded.

4. The method according to claim 1 or 3, characterized in that, The method further includes: The extracted information text is used as a training sample for the case search model, and the case search model is trained accordingly. The case search model is used to return the extracted information text of the case with the highest similarity to the user when the user inputs legally related text.

5. The method according to claim 1, characterized in that, The number of characters in the start position field value is greater than or equal to 10 and less than or equal to 15; the number of characters in the end position field value is greater than or equal to 10 and less than or equal to 15.

6. An information extraction device, characterized in that, include: The input module is used to input the legally binding target document and the prompt text constructed for the target document into the large language model; wherein, the prompt text contains the name and content description of each case basic information extraction item, as well as the name, content description, start position marker information, end position marker information, and integrity extraction instruction of each case entity content extraction item; The receiving module is used to receive the content field values ​​of each case basic information extraction item, the start position field value and the end position field value of each case entity content extraction item extracted from the target document by the large language model according to the instructions of the prompt text; The matching module is used to match the target document with the start position field value and end position field value of each case entity content extraction item as a regular expression, and to extract the content field value of the case entity content extraction item from the target document. The generation module is used to generate the information extraction text of the target document based on the content field values ​​of each case basic information extraction item and each case entity content extraction item.

7. The apparatus according to claim 6, characterized in that, The large language model is used to return a null value when the content field value, start position field value, or end position field value is not extracted from the target document; The device further includes: The judgment module is used by the matching module to determine whether any null values ​​exist in the content field value, the start position field value, and the end position field value for each extracted case entity content item, using the start and end position field values ​​as regular expressions to match the target document. Before matching the content field value of the extracted case entity content item from the target document, the judgment module determines whether any null values ​​exist in the content field value, the start position field value, and the end position field value. The stop module is used to discard the target document and stop executing subsequent steps if a null value exists, indicating that the initial quality verification of the target document has failed. The continue module is used to indicate that the target document has passed the initial quality verification if no null value exists, and to continue executing subsequent steps.

8. The apparatus according to claim 6, characterized in that, The device further includes: The identification module is used to identify whether the beginning and end of the content field value of the extraction item in the information extraction text of the target document contains corresponding preset keywords for each extraction item in the case basic information extraction item and each extraction item in the case entity content extraction item after the generation module generates the information extraction text of the target document based on the content field value of each extraction item in the case basic information extraction item and the case entity content extraction item. The statistics module is used to count the number of extraction items that do not contain the corresponding preset keywords at the beginning and end, and to determine the probability of the information extraction text being unqualified based on the number of such items. The discard module is used to discard the extracted text of information whose failure probability is greater than a preset probability threshold. The submission module is used to submit the extracted information text with a failure probability less than or equal to the preset probability threshold to the manual verification process. If the manual verification is successful, the extracted information text is retained; if the manual verification fails, the extracted information text is discarded.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the memory via the bus, and the machine-readable instructions, when executed by the processor, perform the steps of the method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 5.

Images