A document risk intelligent identification method and device supporting real-time labeling

Abstract

The application discloses a document risk intelligent identification method and device supporting real-time labeling, which comprises the following steps: structurally analyzing a document, and comprehensively identifying and determining key chapter contents in the entire document according to chapter recognition features of multiple types in the document; preliminarily identifying risks in the key chapter contents in the entire document, obtaining risk candidate blocks and risk candidate paragraphs where the risk candidate blocks are located; verifying the accuracy of all the risk candidate blocks and the risk candidate paragraphs in batches by using a large language model based on batch processing prompts, obtaining determined risk blocks and paragraphs where the risk blocks are located; determining the positions of the risk blocks in the paragraphs where the risk blocks are located by using a sliding window algorithm, and converting the positions into PDF pixel-level coordinates, so as to provide accurate labeling starting and ending positions for risk point labeling operations. The application effectively improves the risk identification accuracy, provides real-time feedback and visualization, and improves the user experience.

Description

Technical Field

[0001] This invention relates to the field of document processing technology, and specifically to a method and apparatus for intelligent identification of document risks that supports real-time annotation. Background Technology

[0002] Currently, in business scenarios such as bidding and procurement, and contract review, document compliance review mainly relies on traditional rule-based or single-instance AI inference technical solutions.

[0003] (1) Risk identification stage: Traditional methods rely on single AI reasoning or fixed rules to judge compliance issues in documents through preset keyword rules or single AI model reasoning. This cannot cope with complex compliance scenarios (such as ambiguous expressions and multi-condition related compliance issues), resulting in an accuracy rate of generally less than 80% and a false positive rate of more than 25%.

[0004] (2) Result feedback mechanism: Most of them are in the "batch upload-offline processing-result export" mode, which cannot provide real-time feedback on risk points during the analysis process. Moreover, the identified risk point results are output in the form of a text list, which is not associated with the original document location. During manual review, it is necessary to repeatedly compare the document with the result list, which increases the review time of a single document by more than 30%, resulting in low efficiency. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method and apparatus for intelligent document risk identification that supports real-time annotation, effectively improving the accuracy of risk identification and enabling real-time annotation feedback, thereby improving the efficiency of document compliance review. The technical solution is as follows:

[0006] Firstly, a document risk intelligent identification method supporting real-time annotation is provided, including the following steps:

[0007] Receive documents to be analyzed;

[0008] The document is structured and parsed, and multiple types of chapter identification features are identified and extracted. Based on the multiple types of chapter identification features, the key chapter content in the entire document is identified and determined. The chapter identification features include multiple positive chapter identification features with different degrees of importance that have a positive orientation on whether the content is a key chapter and at least one negative chapter identification feature that has a negative orientation on whether the content is a key chapter. The negative orientation represents the chapter identification feature that leads to misjudgment as key chapter content.

[0009] Preliminary risk identification is performed on key chapters in the entire document to obtain risk candidate blocks and the risk candidate paragraphs where the risk candidate blocks are located;

[0010] The accuracy of all risk candidate phrases and paragraphs is verified by using a large language model (LLM) based on batch prompting (BatchPrompt) technology to obtain the identified risk phrases and paragraphs containing them.

[0011] The location of the risk block is determined by a sliding window algorithm within the paragraph containing the risk block, and the location is converted into PDF pixel-level coordinates, providing accurate start and end positions for risk point annotation operations.

[0012] In some implementations, each of the chapter identification features is configured with a contribution score to characterize the importance of the chapter identification feature in identifying whether the content is a key chapter.

[0013] In some implementations, the step of comprehensively identifying and determining the key chapter content in the entire document based on multiple types of chapter recognition features includes:

[0014] Different contribution scores are assigned to different chapters based on their characteristics and their importance in identifying whether a chapter is a key chapter.

[0015] The score for each chapter's identification feature is calculated by multiplying the numerical value of the identification feature of each chapter in the target local region of the entire document with the corresponding contribution score. The sum of the scores of all chapter identification features is used as the evaluation score for the target local region. The target local region consists of a preset target level heading and all the text under the target level heading.

[0016] The score determines whether a target local area is a key chapter in the entire document. The higher the score, the greater the probability that the target local area is a key chapter.

[0017] In some implementations, the positive chapter recognition features include:

[0018] Chapter title features are determined based on the text similarity between the title of the target local area and the titles in the preset chapter title library;

[0019] Content feature word density feature, determined based on the frequency of chapter content feature words appearing in the target local region;

[0020] Structured indicator features are determined based on whether structured markers appear in the target local region;

[0021] Chapter position layout features are determined based on the position of the target local area within the entire document;

[0022] Content length rationality features are determined based on the content length of the target local area;

[0023] Negative chapter identification features include distractor features such as headers and footers, page numbers, blank paragraphs, attachment identifiers, and notes identifiers.

[0024] In some implementations, the contribution scores of chapter title features, content feature word density features, structured indicator features, chapter position layout features, and content length rationality features gradually decrease, while the contribution of interference features is negative and the absolute value of the contribution score of interference features is between that of content feature word density features and chapter position layout features.

[0025] In some implementations, the contribution score of each chapter identification feature is preset to a dynamically adjustable value, and an adjustable range is set for the contribution score of each chapter identification feature. When dealing with different types of documents, the contribution score of each chapter identification feature is dynamically determined according to the document type.

[0026] In some implementations, the content feature word density feature, the chapter content feature words are determined based on a pre-built chapter content feature word library;

[0027] Based on the semantics of key chapter content, at least one type of key chapter is configured with a dedicated chapter content feature word library, risk keyword database, and risk identification rules. The dedicated chapter content feature word library, risk keyword database, and risk identification rules configured for the key chapter are used to perform content feature word density feature analysis by calling the dedicated chapter content feature word library for a single key chapter, and to perform preliminary risk identification by calling the dedicated risk keyword database and risk identification rules for a single key chapter.

[0028] In some implementations, the preliminary risk identification includes the following steps:

[0029] Extract the key chapter content of the document, compare the key chapter content with the preset risk keyword database, identify the paragraphs containing the preset risk keywords in the key chapter content, and identify them as the first risk candidate paragraphs; record the risk keywords contained in each first risk candidate paragraph as the first risk candidate block;

[0030] Extract the key chapter content of the document, use an AI model to perform a single inference to determine the paragraphs containing preset risk keywords in the key chapter content, and identify them as second risk candidate paragraphs; record the risk keywords contained in each second risk candidate paragraph as a second risk candidate block.

[0031] Based on the combination of the first risk candidate paragraph and its corresponding first risk candidate block, and the second risk candidate paragraph and its corresponding second risk candidate block, the preliminary risk identification results are determined.

[0032] In some implementations, the batch verification process includes the following steps:

[0033] The risk candidate blocks and risk candidate paragraphs in the preliminary risk identification results are split into multiple independent samples;

[0034] Based on the BatchPrompt technology, combined with random permutation and integrated BPE and self-reflective guided early stop SEAS technology, a portion of the samples are randomly selected from all independent samples, the sample order is randomly shuffled, and multiple rounds of sample selection are performed for batch risk verification, and the verification is completed through the early stop mechanism.

[0035] In some implementations, determining the position of the risky phrase within the paragraph containing the risky phrase using a sliding window algorithm and converting the position into PDF pixel-level coordinates includes the following steps:

[0036] Based on the paragraph containing the risky text block, the page containing the risky text block is determined using text hash matching technology throughout the document;

[0037] Based on the content feature word density in the paragraph containing the risky phrase, the size of the sliding window is dynamically determined; the higher the density, the smaller the window size.

[0038] The risk block is matched by sliding a window within the paragraph containing the risk block. Within each sliding window, the character index of the risk block is located by a binary search algorithm. Combined with the pre-acquired page coordinate matrix, the character index of the risk block is converted into the precise pixel-level coordinates of the PDF page.

[0039] Perform positional similarity assessment on the precise pixel-level coordinates of all risk phrases in the PDF pages. For at least two risk phrases with similar positions, merge the precise pixel-level coordinates into a single labeled coordinate.

[0040] In some implementations, the risk point marking operation includes the following steps: highlighting the risk block with a color and associating it with a floating annotation based on the PDF pixel-level coordinates. The annotation content includes the risk type, confidence level, compliance basis, and modification suggestions.

[0041] Secondly, a document risk intelligent identification device supporting real-time annotation is provided, the device comprising:

[0042] The document receiving unit is used to receive documents to be analyzed.

[0043] The document parsing unit is used to perform structured parsing of the document and identify and extract multiple types of chapter identification features in the document. Based on the multiple types of chapter identification features, the key chapter content in the entire document is identified and determined. The chapter identification features include multiple positive chapter identification features of different importance that have a positive orientation on whether the content is a key chapter and at least one negative chapter identification feature that has a negative orientation on whether the content is a key chapter. The negative orientation represents the chapter identification feature that leads to misjudgment as key chapter content.

[0044] The preliminary risk identification unit is used to perform preliminary risk identification on key chapters in the entire document and obtain risk candidate blocks and risk candidate paragraphs where the risk candidate blocks are located.

[0045] The batch risk verification unit is used to batch verify the accuracy of all risk candidate chunks and risk candidate paragraphs using a large language model (LLM) based on batch prompting (BatchPrompt) technology, and to obtain the identified risk chunks and the paragraphs in which the risk chunks are located.

[0046] The risk point precise location and annotation unit is used to determine the position of the risk block in the paragraph containing the risk block using a sliding window algorithm and convert the position into PDF pixel-level coordinates, providing accurate start and end positions for risk point annotation operations.

[0047] Thirdly, a computer-readable storage medium is provided having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0048] Fourthly, a document risk intelligent identification system supporting real-time annotation is provided. This system includes a client and a server, and the client and server perform the following communication process:

[0049] A two-way communication connection is established between the client and the server;

[0050] The server receives documents uploaded by the client and performs document parsing, preliminary risk identification, batch risk verification, and precise location and labeling of risk points in sequence based on the method described in the first aspect; and displays the risk identification progress to the client in real time.

[0051] The server sends a document containing risk point marking information to the client.

[0052] The present invention provides a document risk intelligent identification method and apparatus that supports real-time annotation, which has the following beneficial effects:

[0053] This invention employs a large language model (LLM) based on batch processing prompting (BatchPrompt) technology to batch verify the accuracy of all risk candidate phrases and risk candidate paragraphs, thereby obtaining the identified risk phrases and the paragraphs containing them.

[0054] This invention accurately determines the start and end positions of risk point annotation by locating risk points at pixel-level coordinates in the PDF, and can simultaneously highlight risk points in the PDF during the analysis process, with an annotation response time of <100ms. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating the document risk intelligent identification method in the embodiments of this application;

[0056] Figure 2 This is a flowchart illustrating the process of determining the PDF pixel-level coordinates of risk blocks based on the sliding window algorithm in an embodiment of this application.

[0057] Figure 3 This is a schematic diagram of the document risk intelligent identification device in the embodiments of this application. Detailed Implementation

[0058] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0059] See Figure 1 This application provides a document risk intelligent identification method that supports real-time annotation, the method including the following steps:

[0060] Step 1: Receive the document to be analyzed;

[0061] Step 2: Perform structured parsing on the document and identify and extract multiple types of chapter identification features in the document. Based on the multiple types of chapter identification features, comprehensively identify and determine the key chapter content in the entire document. The chapter identification features include multiple positive chapter identification features of different importance that have a positive orientation on whether the content is a key chapter and at least one negative chapter identification feature that has a negative orientation on whether the content is a key chapter. The negative orientation represents the chapter identification feature that leads to misjudgment as key chapter content.

[0062] Step 3: Perform preliminary risk identification on key chapters in the entire document to obtain risk candidate blocks and the risk candidate paragraphs where the risk candidate blocks are located;

[0063] Step 4: Use the Large Language Model (LLM) based on BatchPrompt technology to batch verify the accuracy of all risk candidate chunks and risk candidate paragraphs, and obtain the identified risk chunks and the paragraphs in which the risk chunks are located.

[0064] Step 5: Determine the position of the risk block in the paragraph using a sliding window algorithm and convert the position into PDF pixel-level coordinates to provide accurate start and end positions for risk point annotation.

[0065] In this embodiment, the document undergoes sequential processing including document parsing, preliminary risk identification, batch risk verification, and precise location and labeling of risk points. During document parsing, the input document (PDF / Word) is structured, extracting information such as chapter titles, content, and locations. Key chapter content is identified through a combination of multiple chapter identification features. This embodiment focuses on core risk areas, reducing unnecessary computation and allowing subsequent identification only of high-risk chapters, avoiding indiscriminate processing of the entire document and reducing unnecessary computation by over 50%. Furthermore, it adapts to differentiated risk rules, improving identification accuracy. Different chapters exhibit significant differences in risk types; after accurately locating a chapter, the corresponding chapter's risk identification rules and keyword library can be selectively invoked, avoiding confusion between risk types across chapters. In this embodiment, the BatchPrompt technology is used to batch verify risk points using a large language model (LLM), addressing the issue of low accuracy in single-sample inference and avoiding the high API call costs and low processing efficiency caused by repeated inference on single samples, thus preventing its inability to adapt to large-scale document batch processing scenarios. This application embodiment uses a sliding window algorithm to accurately locate the precise pixel coordinates of risk points on the PDF page, enabling real-time annotation.

[0066] It should be noted that, in this context, a phrase is the smallest unit in a sentence that can express a complete meaning, which can be a single word or a sequence of words in consecutive positions. In the embodiments of this application, a risk candidate phrase or a risk phrase is a word or a sequence of words in consecutive positions in a document that expresses risk, indicating that the text sequence in the phrase has a risk.

[0067] The preliminary risk identification in step 3 may be: using a general risk identification method to preliminarily identify risk points and candidate risk paragraphs where risk points are located, taking all key chapters in the entire document as the whole, and / or using a non-general risk identification method that matches the risk occurrence characteristics of the key chapter content to preliminarily identify risk points and candidate risk paragraphs where risk points are located, taking a single key chapter in the entire document as the analysis unit.

[0068] Specifically, after identifying the key chapters in the entire document in this application, a corresponding exclusive risk analysis method can be invoked for each key chapter. For example, different chapters can have a pre-set exclusive risk keyword database and exclusive risk identification rules. When performing preliminary risk identification on different chapters, the exclusive risk keyword database and risk identification rules corresponding to the target chapter can be invoked separately for a single target chapter to analyze whether there are risk keywords in the target chapter, thereby achieving preliminary risk identification of the target chapter.

[0069] In one implementation, during the process of identifying key chapter content in step 2 above, the chapter identification features include multiple positive chapter identification features of different importance that positively guide whether the content is key chapter content and at least one negative chapter identification feature that negatively guides whether the content is key chapter content. The negative guidance represents the chapter identification feature that leads to misjudgment as key chapter content.

[0070] Each of the chapter identification features is configured with a contribution score, which characterizes the importance of the chapter identification feature in identifying whether the content is a key chapter.

[0071] Specifically, positive identification features refer to features in a section of content that play a positive guiding role in identifying whether the content of that section is key. The more of these features there are, the greater their role in identifying whether the content of that section is key. Conversely, negative chapter identification features refer to features in a section of content that play a negative guiding role in identifying whether the content of that section is key. The presence of these features can lead to misclassifying a section that is not key as key.

[0072] In one implementation, step 2 above, which involves comprehensively identifying and determining the key chapter content in the entire document based on multiple types of chapter recognition features, includes the following steps:

[0073] Step 21: Identify features for different chapters and assign different contribution scores based on their importance in identifying whether a chapter is key content;

[0074] Step 22: The score for each chapter identification feature is obtained by multiplying the numerical value of the identification feature of each chapter in the target local area of ​​the entire document with the corresponding contribution score. The sum of the scores of all chapter identification features is used as the evaluation score of the target local area. The target local area consists of a preset target level heading and all the text under the target level heading.

[0075] Step 23: Determine whether the target local area is a key chapter in the entire document based on the evaluation score. The higher the evaluation score, the greater the probability that the target local area is a key chapter.

[0076] The target local area is a region containing a title and all the text under that title. In practical applications, the title level of the target local area for key chapter content identification can be determined as needed, and the key chapter content identification is performed by combining all the text under each title at that level with the title text at that level. That is, the text under which a certain title level is selected as a whole for key chapter content identification. For example, if the text under the third-level title is selected as a whole for key chapter content identification, then each first-level title, second-level title, and third-level title is obtained, and all the text under each third-level title is obtained. Based on all the text under each third-level title (if there are lower-level titles under the third-level title, these lower-level titles are not considered as titles for title feature analysis; instead, all the lower-level titles contained under the third-level title and their corresponding text are treated as a whole for key chapter content identification) and the first-level and second-level titles to which the third-level title belongs, the target local area is used to extract and analyze chapter identification features.

[0077] In one implementation, the chapter identification features in step 2 above include:

[0078] (1) Positive chapter identification features include:

[0079] 11) Chapter title features are determined based on the text similarity between the title of the target local area and the titles in the preset chapter title library (such as "Procurement Requirements" and "Evaluation Methods").

[0080] 12) Content feature word density feature, determined based on the frequency of chapter content feature words (such as "budget amount" and "supplier qualifications") appearing in the target local area;

[0081] 13) Structured indicator features: Based on whether structured markers appear in the target local area, such as "Chapter 1", "1.1", "I.", etc. The core function of structured indicators is to confirm the "chapter level" of the text. In document layout, chapters will almost always have such structured markers (such as "Chapter 3 Procurement Requirements" "2.1 Bidder Qualifications"). Text without markers is more likely to be paragraph notes, attachment descriptions, or other non-chapter content.

[0082] 14) Chapter position layout characteristics, determined based on the position of the target local area in the whole document. For example, according to common sense of document layout, the cover is on the first page, the table of contents is at the beginning, and the core chapters are mostly after the cover / table of contents and before the attachments.

[0083] 15) Content length reasonableness feature: determined based on the content length of the target local area. For example, text that is too short (<100 words) or too long (>10,000 words) is unlikely to be a chapter. The core function of content length reasonableness is to ensure that the selected chapters are "complete and meaningful text blocks". Chapters (regardless of risk level) need to contain sufficient information (such as the "procurement requirements" chapter, which will explain project requirements, technical parameters, etc., at least several hundred words); while text that is too short (such as "page 5" or "attachment 1") or too long (such as mistakenly splicing the entire document into one paragraph) is obviously not a chapter and does not need to be included in subsequent processing.

[0084] (2) Negative chapter identification features, including interference features such as headers and footers, page numbers, blank paragraphs, attachment identifiers, and notes identifiers. The purpose of negative chapter identification features is to target interference items that are "clearly not chapters" (headers, page numbers, blank paragraphs), regardless of their length or how high their other feature scores are, by "actively deducting points" to make their total score lower than the chapter determination threshold, thus achieving "precise rejection".

[0085] It should be noted that the positive chapter identification feature is used to identify whether a target local area is chapter content and whether the target local area is a key chapter for risk identification. Therefore, the title and chapter content feature words in the chapter title library are characteristic words belonging to key chapters mined from all historical documents. Specifically, when calculating chapter title features, the titles in the preset chapter title library can be obtained based on the chapter titles of all historically collected documents. For example, the chapter title library can be built by filtering out less common chapter titles from all historically collected documents. To further filter out chapter content irrelevant to risk, the titles in the preset chapter title library can also be based on the chapter titles of all historically collected documents, and then filtered through high-risk areas to select chapter titles from high-risk areas to build the chapter title library.

[0086] When calculating the content feature word density, similar to the construction of the aforementioned preset chapter title library, a chapter content feature word library can be constructed based on keywords from all historically collected documents. Furthermore, to accurately filter chapter content associated with risk, the chapter content feature word library can be obtained based on keywords from high-risk areas in all historically collected documents. Even further, the chapter content feature word library can determine multiple different sub-chapter content feature word libraries for different title themes. A matching sub-chapter content feature word library is determined based on the title of the target local area, and this sub-chapter content feature word library is used to analyze the content feature word density characteristics of the target local area. For example, if the title of Chapter 3 in a tender document is "Chapter 3 Procurement Requirements," then the content of this chapter will frequently contain keywords specific to this chapter, such as "budget amount" and "supplier qualifications."

[0087] Traditional chapter recognition relies on "single feature matching" (such as looking only at the title or keywords), which is prone to misjudgment due to the limitations of a single feature (e.g., paragraphs with irregular titles but dense keywords are missed). In this embodiment, multiple chapter recognition features are used. Through multi-dimensional information collaborative verification, the shortcomings of single features are made up for, and accurate positioning is achieved. Some paragraphs may not have obvious single features (e.g., titles are omitted but keywords are dense, or there are no structured indicators but the position is reasonable). Multi-feature weighting can form a strong judgment through "multiple weak features superimposed" (e.g., a paragraph with 40 points for keyword density + 15 points for position + 10 points for content length, the total score meets the standard and is judged as a chapter), avoiding the problem of "misjudgment due to missing single features", and ultimately reducing the chapter misjudgment rate to below 5%.

[0088] In one implementation, the contribution scores of the aforementioned chapter title features, content feature word density features, structured indicator word features, chapter position layout features, and content length rationality features gradually decrease (used to characterize the contribution of these chapter identification features to identifying whether it is key chapter content gradually decreases), and the contribution of the interference feature is negative and the absolute value of the contribution score of the interference feature is between the contribution of the content feature word density features and the chapter position layout features.

[0089] In one implementation, the contribution score of each chapter identification feature is preset to a dynamically adjustable value, and an adjustable range is set for the contribution score of each chapter identification feature. When dealing with different types of documents, the contribution score of each chapter identification feature is dynamically determined according to the document type.

[0090] In this system, each chapter identification feature within the actual target local area has a corresponding value. Combining the contribution score of each chapter identification feature, the score of that feature within the target local area can be determined. The contribution score of the chapter identification feature can be in the form of a weighted ratio to indicate the importance of each feature in identifying whether it is key chapter content. Alternatively, the contribution score configuration can be to assign a maximum score to each chapter identification feature. Naturally, the greater the contribution of a chapter identification feature in identifying whether it is key chapter content, the higher its maximum score. For example, the contribution scores of chapter title features, content feature word density features, structured indicator word features, chapter position layout features, content length rationality features, and negative chapter identification features can be configured as follows: 70 points, 45 points, 30 points, 20 points, 15 points, and -40 points, respectively. Furthermore, the contribution score of each chapter identification feature is set as a dynamically adjustable value, with an adjustable range for each feature to adapt to specific scenario requirements. For example, the adjustable ranges for the contribution scores of chapter title features, content feature word density features, structured indicator word features, chapter position layout features, content length rationality features, and negative chapter identification features are set as follows: 50-70 points, 40-45 points, 25-30 points, 10-20 points, 10-15 points, and 20-40 points for negative features, respectively. Furthermore, the contribution score of each chapter identification feature can be flexibly adjusted when dealing with different types of documents. For example, the weight of the chapter title "Procurement Requirements" in a tender document can be increased to 70 points.

[0091] It should be noted that in configuring the contribution scores of chapter title features, content feature word density features, structured indicator word features, chapter position layout features, content length rationality features, and interference features (i.e., negative exclusion features), the following considerations are taken into account: The title is the most direct and reliable identifier of a chapter, with the lowest probability of misjudgment and the highest contribution to chapter identification; therefore, it is given the highest weight. Chapter content feature words reflect the core content of a chapter (e.g., "budget amount" and "performance deadline" are specific keywords for a particular chapter). The higher the density, the stronger the correlation between the paragraph and the chapter theme; its contribution is second only to the title, therefore, it is given the second highest weight. Structured indicator words such as "Chapter 1" and "1.1" are standardized markers for document layout and can directly reflect the "chapter level" of a paragraph; however, some chapters omit these indicator words. In cases where subheadings are unnumbered, the weight of positional features is lower than that of titles and keyword density. Based on common document layout principles (core chapters are usually after the cover / table of contents and before attachments), but with instances of non-standard document formatting (such as disordered chapter order), the reliability of positional features is low, hence their lower weight. Content length rationality features are only used to assist in filtering abnormal paragraphs (e.g., a short sentence of 10 characters cannot be a complete chapter), and cannot be used as a sole basis for chapter judgment, thus contributing the least and having the lowest weight. Negative exclusion features, targeting headers, footers, page numbers, and other "clearly non-chapter content," require rapid deduction of points with high negative weight to prevent them from being mistakenly identified as chapters due to accidental title / keyword matching (e.g., page number "Page 5" might accidentally trigger title matching, but the negative weight directly deducts 20 points, causing the total score to fall below the target).

[0092] Each chapter's identified features have corresponding values ​​in a local region of the target. The value of the chapter title feature can be obtained based on the text similarity between the title of the target local area and the titles in the preset chapter title library. The value of the content feature word density can be determined based on the distribution density of the chapter content feature words in the target local area (the proportion of their frequency of occurrence to the total number of words in the target local area). The value of the structured indicator word feature can be determined based on whether structured markers appear in the target local area; if they appear, the value can be 1, and if they do not appear, the value can be 0. The value of the chapter position layout feature can be determined based on the position of the target local area in the entire document. It can be understood that core chapters are mostly after the cover / table of contents and before the appendices. The value of the chapter position layout feature for different areas can be set according to the probability that different areas in the entire document belong to core chapters. For example, it can be set as follows: when the target local area is in the first 1 / 3 of the entire document, the value of the chapter position layout feature of the target local area is 0.8; when the target local area is in the middle of the entire document, the value of the chapter position layout feature of the target local area is 0.5; and when the target local area is in the last 1 / 3 of the entire document, the value of the chapter position layout feature of the target local area is set to 0.1. The value of the content length reasonableness feature can be determined based on the content length of sections with the same heading level as the target local area in different documents collected historically. This determines the reasonableness of different content lengths, and further, the reasonableness of the content length of the target local area is determined based on its content length. For example, the content length reasonableness feature value of the target local area with content length that is too short (<100 words) or too long (>10,000 words) can be set to a value close to 0, while the reasonableness probability can be determined for content lengths between 100 and 10,000 words. Among them, negative chapter identification features, such as some interfering features, such as headers and footers, page numbers, blank paragraphs, attachment markers, and note marker areas, can be understood as the lines containing headers and footers, the lines containing page numbers, blank paragraphs, the "attachment" marker areas, and the "note" marker paragraphs, etc. These contents are definitely not core chapter content, nor are they high-risk areas of the document, and need to be filtered out. These features can lead to misjudgment of chapters. For "clearly non-chapter content" such as headers and footers and page numbers, the values ​​of negative chapter identification features can be set, for example, to 1 if these features exist in the target local area, and 0 otherwise.

[0093] In one implementation, the content feature word density feature, the chapter content feature words are determined based on a pre-built chapter content feature word library;

[0094] The chapter content feature word library is constructed based on the content feature words in all historically collected documents, or it is obtained based on the content feature words of high-risk areas in all historically collected documents, or it is based on all historically collected documents, analyzing the exclusive content feature words in the main text content under each title and constructing the sub-chapter content feature word library corresponding to the title chapter based on the exclusive content feature words. The matching sub-chapter content feature word library is determined according to the title of the target local area to analyze the content feature word density features of the target local area.

[0095] Based on the same concept, different key chapters can be pre-configured with their own chapter content feature thesaurus, risk keyword database, and risk identification rules. That is:

[0096] Based on the semantics of key chapter content, at least one type of key chapter is configured with a dedicated chapter content feature word library, risk keyword database, and risk identification rules. The dedicated chapter content feature word library, risk keyword database, and risk identification rules configured for the key chapter are used to perform content feature word density feature analysis by calling the dedicated chapter content feature word library for a single key chapter, and to perform preliminary risk identification by calling the dedicated risk keyword database and risk identification rules for a single key chapter.

[0097] Specifically, based on historical document data, key chapters with the same semantic meaning are mined and constructed to create matching chapter content feature lexicons, risk keyword databases, and risk identification rules. For example, based on all historically collected documents, risk identification rules can be analyzed in the text content under each title, and different risk identification rule libraries can be constructed for different title chapters. Then, after comprehensively identifying and determining the key chapter content in the entire document based on multiple types of chapter identification features, the corresponding risk identification rule library can be called to identify risks for different chapter content.

[0098] In one implementation, the preliminary risk identification in step 3 above includes the following steps:

[0099] Step 31: Extract the key chapter content of the document, compare the key chapter content with the preset risk keyword database, identify the paragraphs containing the preset risk keywords in the key chapter content, and identify them as the first risk candidate paragraphs; record the risk keywords contained in each first risk candidate paragraph as the first risk candidate block;

[0100] Step 32: Extract the key chapter content of the document, use the AI ​​model to perform single-step reasoning to determine the paragraphs containing preset risk keywords in the key chapter content, and identify them as second risk candidate paragraphs; record the risk keywords contained in each second risk candidate paragraph as a second risk candidate phrase block;

[0101] Step 33: Based on the combination of the first risk candidate paragraph and its corresponding first risk candidate block, and the second risk candidate paragraph and its corresponding second risk candidate block, determine the preliminary risk identification result.

[0102] It is understandable that the preset risk keyword database can be a single database storing all risk keywords, with key chapter content compared to the preset risk keyword database; alternatively, based on historical document data, different risk keyword sub-databases and risk identification rule sub-databases can be constructed for different title chapters, with each chapter content calling the corresponding sub-database for risk comparison analysis. In this embodiment, preliminary risk identification quickly filters out "suspected risk candidate samples" (paragraphs / keywords), providing high-quality input for the next step of "batch verification," avoiding indiscriminate processing of the entire document text, and improving overall process efficiency. In one implementation, the above-mentioned method of comparing the "preset risk keyword database" and the method of single-step inference using the AI ​​model can be used in combination, or one of them can be used alone, or they can be used simultaneously. Alternatively, the method of comparing the "preset risk keyword database" can be used as the first step, and based on the risk locations identified in the first step, the method of single-step inference using the AI ​​model can be used to further optimize and improve the preliminary risk identification results. Preferably, this embodiment adopts a hybrid algorithm approach of "rule matching (keyword database) + AI semantic understanding":

[0103] (1) Rule matching (keyword library): responsible for "fast filtering", utilizing the efficiency of hash matching (millisecond response) to quickly narrow down the risk range, avoid AI models processing massive amounts of risk-free text, and improve efficiency;

[0104] (2) AI semantic understanding: responsible for "precise filtering", making up for the limitations of rule matching and ensuring the quality of candidate samples;

[0105] (3) Multi-threaded parallel processing: Multi-threaded synchronous processing of texts from multiple core chapters further improves the filtering speed (e.g., the initial identification of a 120-page document takes only 6 seconds).

[0106] In one implementation, the batch verification process in step 4 above includes the following steps:

[0107] Step 41: Divide the risk candidate blocks and risk candidate paragraphs in the preliminary risk identification results into multiple independent samples;

[0108] Step 42: Based on the batch processing prompt BatchPrompt technology, combined with random permutation and integrated BPE and self-reflective guided early stop SEAS technology, a portion of the samples are randomly selected from all independent samples, the sample order is randomly shuffled, and multiple rounds of sample selection are performed for batch risk verification, and the verification is completed through the early stop mechanism.

[0109] Specifically, in this embodiment of the application, during the batch verification process, risk points are split into independent samples to ensure that each sample focuses on a single risk candidate region, avoiding model inference bias caused by excessively long text. Step 42, "BatchPrompt+BPE+SEAS," includes the following steps:

[0110] Random shuffling: Before each round of batch inference, the sample order is shuffled by a random shuffling algorithm (e.g., the sample order in the first round is [sample A, sample B, sample C], and in the second round it is [sample C, sample A, sample B]), which completely eliminates the model's dependence on the sample position and avoids positional deviations from affecting the recognition results. Further multi-round batch inference is performed, with a fixed number of voting rounds (5 rounds by default, dynamically configurable). Each round, a non-repeating subset of samples is randomly selected from all samples (20%-30% of the total samples per round). This subset is input into the AI ​​risk identification model for batch inference. The model simultaneously outputs the risk type (e.g., "eligibility requirements out of scope") and single-round confidence score (0-1 points) for each sample. Then, the results are initially aggregated. After each round of inference, the total confidence score for each sample is accumulated, providing data support for the early termination judgment of the subsequent SEAS mechanism. An "average confidence score" (average confidence score = total accumulated confidence score / number of rounds executed) is calculated for each sample to avoid misjudgments caused by fluctuations in single-round confidence scores. Based on the accuracy requirements of risk identification for different documents, multi-level average confidence score thresholds are preset, for example, setting a three-level average confidence score threshold, including:

[0111] High confidence threshold (0.85-0.9): If the average confidence of the samples is greater than or equal to this threshold, it is judged as a "high confidence result", and subsequent rounds of reasoning are stopped directly, and the current result is output;

[0112] Low confidence threshold (0.6-0.65): If the average confidence level of a sample is still less than this threshold after all preset rounds have been completed, it is marked as a "low confidence sample" and a manual review process is triggered.

[0113] The middle range (0.65-0.85): The sample needs to complete all preset rounds of reasoning, and the accuracy is ensured by the results of multiple rounds of voting.

[0114] It should be noted that the embodiments of this application set dynamic confidence thresholds, which support the adaptation of thresholds according to document type (such as bidding documents and other high compliance requirements, where the high confidence threshold is increased to 0.9 and the low confidence threshold is increased to 0.65; the threshold can be appropriately reduced in ordinary contract scenarios), thus balancing accuracy and efficiency.

[0115] For example, the batch verification process can be:

[0116] (1) Step 1: Sample splitting and batch processing. The initially identified risk points are split into 3-8 per batch. The sample order is randomly shuffled before each round of voting (to eliminate positional bias).

[0117] (2) Second step: multi-round inference and confidence accumulation. In each round, the AI ​​model is called to infer the batch of samples, obtain the "single-round confidence (vote_confidence_i, range 0-1)" of each sample, and accumulate the total score (Σvote_confidence_i).

[0118] (3) Third step: Dynamic stop judgment. Through the SEAS mechanism (self-reflection-guided early stop), if the average confidence of all samples in the batch is greater than or equal to the threshold (0.7-0.9), then stop subsequent voting (to avoid over-inference).

[0119] (4) Fourth step: Calculate the final confidence level. The formula is confidence_ratio = cumulative confidence level / actual number of voting rounds (e.g., the confidence levels of a sample in 3 rounds of voting are 0.9, 0.92 and 0.88 respectively, the cumulative confidence level is 2.7, the actual number of rounds is 3, and the final confidence level = 2.7 / 3 = 0.9).

[0120] See Figure 2 In one implementation, step 5 above, which involves determining the position of the risky phrase within the paragraph containing the risky phrase using a sliding window algorithm and converting the position into PDF pixel-level coordinates, includes the following steps:

[0121] Step 51: Based on the paragraph where the risky text block is located, determine the page of the paragraph where the risky text block is located in the entire document using text hash matching technology, and record it as the risk page where the risky text block is located, so as to achieve pre-screening of risk-free pages;

[0122] Step 52: Based on the content feature word density in the paragraph where the risk block is located, dynamically determine the size of the sliding window; the higher the density, the smaller the window size.

[0123] Step 53: Based on the sliding window, slide to match the risk block in the paragraph where the risk block is located. Within each sliding window, locate the character index of the risk block using a binary search algorithm. Combined with the pre-acquired page coordinate matrix, convert the character index of the risk block into precise pixel-level coordinates of the PDF page.

[0124] Step 54: Perform position similarity judgment on the precise pixel-level coordinates of all risk phrases in the PDF pages. For at least two risk phrases with similar positions, the precise pixel-level coordinates are merged into one labeled coordinate.

[0125] In this embodiment, the process of locating and marking risk points is optimized through four levels: pre-screening, dynamic windowing, binary search, and deduplication and merging, achieving accurate and efficient keyword positioning. For example, the specific implementation process can be as follows:

[0126] 1) Risk-free page pre-screening: Based on text hash matching technology, quickly scan all pages of the document and filter pages that do not contain the target risk keywords (such as pages in the bidding document that do not contain keywords such as "budget amount" or "specific region"), reducing more than 50% of the invalid calculations;

[0127] 2) Dynamic window adjustment: Abandoning fixed window size, the window size is adaptively adjusted according to keyword density.

[0128] Keyword density ≥2% (core risk area): narrow the window to 3-4 words, focus on core keywords, and avoid irrelevant text interference;

[0129] Keyword density 1%-2% (medium-risk area): Keep the window at 5-8 words to balance targeting accuracy and coverage;

[0130] Keyword density <1% (low-risk area): Expand the window to 10-15 words (maximum 15) to ensure no scattered keywords are missed;

[0131] 3) Binary search acceleration: For the text in each dynamic window, the character index of the keyword is quickly located by the binary search algorithm (3-5 times faster than linear search). Then, combined with the page coordinate matrix obtained in the document parsing stage, the character index is converted into the precise pixel coordinates of the PDF page (accuracy of ±2px on the x-axis and ±2px on the y-axis).

[0132] 4) Deduplication and merging: The similarity of multiple keyword coordinates corresponding to the same risk point is judged. If the coordinate distance is less than or equal to the preset pixel spacing threshold, which can be set between 5px and 8px (dynamically adjusted according to the document resolution), they are determined to be the same risk area and merged into one labeled coordinate to avoid duplicate labeling.

[0133] In this embodiment, compared to the traditional fixed window design, the dynamic sliding window improves the positioning speed by 3-5 times, with a timeout rate of less than 2% for processing 500-page PDFs. Risk-free page pre-screening avoids loading full-page text, reducing memory usage by 40-60%. Coordinate error is less than 2px, and annotation accuracy reaches 96.5%. Multi-level optimization through "pre-screening - dynamic window - binary search - deduplication and merging" improves positioning efficiency and accuracy, solving the problems of "timeout and duplicate annotation in large document keyword positioning."

[0134] In one implementation, the risk point marking operation in step 5 above includes the following steps: highlighting the risk block with a color and associating it with a floating annotation based on the PDF pixel-level coordinates. The annotation content includes the risk type, confidence level, compliance basis, and modification suggestions.

[0135] Specifically, non-destructive visual annotation is performed on the document based on the pixel-level coordinates of risk points, without modifying the original document data. The annotation rules strictly follow the differentiated configuration based on risk level: high risk (confidence ≥ 0.9) is highlighted in red with a transparency of 0.6; medium risk (0.7 ≤ confidence < 0.9) is highlighted in orange with a transparency of 0.5; and low risk (0.6 ≤ confidence < 0.7) is highlighted in yellow with a transparency of 0.4, ensuring clear color differentiation without obscuring the original text. Each highlighted area is associated with a clickable floating annotation, which includes four core pieces of information: risk type (e.g., "qualification requirements exceed the scope"), confidence level (accurate to three decimal places, e.g., 0.925), compliance basis, and modification suggestions, ensuring that reviewers can carry out rectification without additional documentation.

[0136] In summary, the beneficial effects of the present invention are as follows:

[0137] (1) In view of the problem that traditional methods rely on single AI reasoning or fixed rules, resulting in low recognition accuracy and high false negative rate, this invention adopts a large language model LLM based on batch processing prompt BatchPrompt technology to batch verify the accuracy of all risk candidate chunks and risk candidate paragraphs. Combined with random permutation and integrated BPE and self-reflective guided early stop SEAS technology, the positional bias is eliminated and the reasoning accuracy is improved by randomly shuffling the sample order and voting aggregation in multiple rounds.

[0138] (2) In view of the problems that existing systems can only output results after the entire document is processed, and there is no original document annotation, the manual review is inefficient, lacks real-time feedback and visualization, and has a poor user experience, this invention adopts "real-time annotation + WebSocket streaming push", which can simultaneously highlight risk points in PDF and push progress during the analysis process, with annotation response time <100ms.

[0139] (3) In traditional methods, AI output only contains a binary judgment of "whether it is risky" without confidence assessment, which means that all results need to be manually reviewed, resulting in low review efficiency (average daily processing volume per person < 10 documents). This invention proposes "three-level confidence level processing", which automatically marks high confidence (≥0.9), prompts review for medium confidence (0.6-0.9), and requires expert review for low confidence (<0.6), reducing more than 60% of invalid review work.

[0140] (4) In traditional methods, when processing PDFs with more than 500 pages, the keyword location timeout rate exceeds 15% and the memory usage exceeds 12GB. When processing in batches, the recognition accuracy of samples in the later positions of prompt words is 20%-30% lower than that in the earlier positions. This invention improves the location efficiency by 3-5 times through the "optimized sliding window algorithm" (pre-screening + dynamic window + binary search), and eliminates position deviation by combining the BPE mechanism, reducing the impact of deviation by more than 80%.

[0141] See Figure 3 This application provides a document risk intelligent identification device that supports real-time annotation, the device comprising:

[0142] The document receiving unit is used to receive documents to be analyzed.

[0143] The document parsing unit is used to perform structured parsing of the document and identify and extract multiple types of chapter identification features in the document. Based on the multiple types of chapter identification features, the key chapter content in the entire document is identified and determined. The chapter identification features include multiple positive chapter identification features of different importance that have a positive orientation on whether the content is a key chapter and at least one negative chapter identification feature that has a negative orientation on whether the content is a key chapter. The negative orientation represents the chapter identification feature that leads to misjudgment as key chapter content.

[0144] The preliminary risk identification unit is used to perform preliminary risk identification on key chapters in the entire document and obtain risk candidate blocks and risk candidate paragraphs where the risk candidate blocks are located.

[0145] The batch risk verification unit is used to batch verify the accuracy of all risk candidate chunks and risk candidate paragraphs using a large language model (LLM) based on batch prompting (BatchPrompt) technology, and to obtain the identified risk chunks and the paragraphs in which the risk chunks are located.

[0146] The risk point precise location and annotation unit is used to determine the position of the risk block in the paragraph containing the risk block using a sliding window algorithm and convert the position into PDF pixel-level coordinates, providing accurate start and end positions for risk point annotation operations.

[0147] Specific limitations regarding the document risk intelligent identification device can be found in the limitations of the document risk intelligent identification method described above, and will not be repeated here. Each unit in the aforementioned document risk intelligent identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each unit.

[0148] This application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the document risk intelligent identification method supporting real-time annotation as described above. The computer-readable storage medium includes: permanent and non-permanent, removable and non-removable media, which are tangible devices capable of retaining and storing instructions for use by an instruction execution device. The computer-readable storage medium includes: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination thereof.

[0149] This application provides a document risk intelligent identification system that supports real-time annotation. The system includes a client and a server, and the client and server perform the following communication process:

[0150] (A1) A two-way communication connection is established between the client and the server;

[0151] (A2) The server receives the document uploaded by the client. Based on the document risk intelligent identification method that supports real-time annotation as described above, the server performs document parsing, preliminary risk identification, batch risk verification, precise location and annotation of risk points in sequence; and displays the risk identification progress to the client in real time.

[0152] (A3) The server sends a document containing risk point marking information to the client.

[0153] The client and server use WebSocket for bidirectional collaborative interaction. During the bidirectional communication between the client and server, heartbeat packets are sent at preset intervals to avoid disconnection.

[0154] This invention is not limited to the specific embodiments described above. Any modifications made by those skilled in the art based on the above concept without creative effort are within the scope of protection of this invention.

Claims

1. A document risk intelligent identification method supporting real-time annotation, characterized in that, Includes the following steps: Receive documents to be analyzed; The document is structured and parsed, and multiple types of chapter identification features are identified and extracted. Based on the multiple types of chapter identification features, the key chapter content in the entire document is identified and determined. The chapter identification features include multiple positive chapter identification features with different degrees of importance that have a positive orientation on whether the content is a key chapter and at least one negative chapter identification feature that has a negative orientation on whether the content is a key chapter. The negative orientation represents the chapter identification feature that leads to misjudgment as key chapter content. Preliminary risk identification is performed on key chapters in the entire document to obtain risk candidate blocks and the risk candidate paragraphs where the risk candidate blocks are located; The accuracy of all risk candidate phrases and paragraphs is verified by using a large language model (LLM) based on batch prompting (BatchPrompt) technology to obtain the identified risk phrases and paragraphs containing them. The process involves determining the location of a risky phrase within a paragraph using a sliding window algorithm and converting that location into pixel-level coordinates in the PDF file, providing accurate start and end points for risk point annotation. This process includes the following steps: First, based on the paragraph containing the risky phrase, determine the page containing that paragraph using text hash matching technology throughout the entire document. Second, dynamically determine the size of the sliding window based on the content feature word density within the paragraph; the higher the density, the smaller the window size. Third, slide the sliding window within the paragraph to match the risky phrase; within each sliding window, locate the character index of the risky phrase using a binary search algorithm; and, combined with a pre-acquired page coordinate matrix, convert the character index of the risky phrase into precise pixel-level coordinates on the PDF page. Finally, perform positional similarity judgment on the precise pixel-level coordinates of all risky phrases on the PDF page, merging the precise pixel-level coordinates of at least two similar risky phrases into a single annotation coordinate.

2. The document risk intelligent identification method according to claim 1, characterized in that, Each of the chapter identification features is configured with a contribution score, which characterizes the importance of the chapter identification feature in identifying whether the content is a key chapter.

3. The document risk intelligent identification method according to claim 2, characterized in that, The method of comprehensively identifying and determining key chapter content in the entire document based on multiple types of chapter recognition features includes: Different contribution scores are assigned to different chapters based on their characteristics and their importance in identifying whether a chapter is a key chapter. The score for each chapter's identification feature is calculated by multiplying the numerical value of the identification feature of each chapter in the target local region of the entire document with the corresponding contribution score. The sum of the scores of all chapter identification features is used as the evaluation score for the target local region. The target local region consists of a preset target level heading and all the text under the target level heading. The score determines whether a target local area is a key chapter in the entire document. The higher the score, the greater the probability that the target local area is a key chapter.

4. The document risk intelligent identification method according to claim 2, characterized in that, Among the chapter identification features, the positive chapter identification features include: Chapter title features are determined based on the text similarity between the title of the target local area and the titles in the preset chapter title library; Content feature word density feature, determined based on the frequency of chapter content feature words appearing in the target local region; Structured indicator features are determined based on whether structured markers appear in the target local region; Chapter position layout features are determined based on the position of the target local area within the entire document; Content length rationality features are determined based on the content length of the target local area; Negative chapter identification features include distractor features such as headers and footers, page numbers, blank paragraphs, attachment identifiers, and notes identifiers.

5. The document risk intelligent identification method according to claim 4, characterized in that, The contribution scores of chapter title features, content feature word density features, structured indicator word features, chapter position layout features, and content length rationality features gradually decrease. The contribution of interference features is negative, and the absolute value of the contribution score of interference features is between that of content feature word density features and chapter position layout features.

6. The document risk intelligent identification method according to claim 2, characterized in that, The contribution score of each chapter's identification feature is pre-set to a dynamically adjustable value, and an adjustable range is set for the contribution score of each chapter's identification feature. When dealing with different types of documents, the contribution score of each chapter's identification feature is dynamically determined according to the document type.

7. The document risk intelligent identification method according to claim 4, characterized in that, The content feature word density feature, the chapter content feature words are determined based on a pre-built chapter content feature word library; Based on the semantics of key chapter content, at least one type of key chapter is configured with a dedicated chapter content feature word library, risk keyword database, and risk identification rules. The dedicated chapter content feature word library, risk keyword database, and risk identification rules configured for the key chapter are used to perform content feature word density feature analysis by calling the dedicated chapter content feature word library for a single key chapter, and to perform preliminary risk identification by calling the dedicated risk keyword database and risk identification rules for a single key chapter.

8. The document risk intelligent identification method according to claim 1, characterized in that, The preliminary risk identification includes: Extract the key chapter content of the document, compare the key chapter content with the preset risk keyword database, identify paragraphs containing preset risk keywords in the key chapter content, and identify them as first risk candidate paragraphs; record the risk keywords contained in each first risk candidate paragraph as a first risk candidate block; Extract the key chapter content of the document, use an AI model to perform a single inference to determine the paragraphs containing preset risk keywords in the key chapter content, and identify them as second risk candidate paragraphs; record the risk keywords contained in each second risk candidate paragraph as a second risk candidate block. Based on the combination of the first risk candidate paragraph and its corresponding first risk candidate block, and the second risk candidate paragraph and its corresponding second risk candidate block, the preliminary risk identification results are determined.

9. The document risk intelligent identification method according to claim 1, characterized in that, The batch verification process includes the following steps: The risk candidate blocks and risk candidate paragraphs in the preliminary risk identification results are split into multiple independent samples; Based on the BatchPrompt technology, combined with random permutation and integrated BPE and self-reflective guided early stop SEAS technology, a portion of the samples are randomly selected from all independent samples, the sample order is randomly shuffled, and multiple rounds of sample selection are performed for batch risk verification, and the verification is completed through the early stop mechanism.

10. The document risk intelligent identification method according to claim 1, characterized in that, The risk point annotation operation includes the following steps: highlighting the risk block with a color and associating it with a floating annotation based on the PDF pixel-level coordinates. The annotation content includes the risk type, confidence level, compliance basis, and modification suggestions.

11. A document risk intelligent identification device supporting real-time annotation, characterized in that, include: The document receiving unit is used to receive documents to be analyzed. The document parsing unit is used to perform structured parsing of the document and identify and extract multiple types of chapter identification features in the document. Based on the multiple types of chapter identification features, the key chapter content in the entire document is identified and determined. The chapter identification features include multiple positive chapter identification features of different importance that have a positive orientation on whether the content is a key chapter and at least one negative chapter identification feature that has a negative orientation on whether the content is a key chapter. The negative orientation represents the chapter identification feature that leads to misjudgment as key chapter content. The preliminary risk identification unit is used to perform preliminary risk identification on key chapters in the entire document and obtain risk candidate blocks and risk candidate paragraphs where the risk candidate blocks are located. The batch risk verification unit is used to batch verify the accuracy of all risk candidate chunks and risk candidate paragraphs using a large language model (LLM) based on batch prompting (BatchPrompt) technology, and to obtain the identified risk chunks and the paragraphs in which the risk chunks are located. The risk point precise location and annotation unit is used to determine the position of the risk block within the paragraph containing the risk block using a sliding window algorithm and convert the position into PDF pixel-level coordinates, providing accurate start and end positions for risk point annotation operations. The process of determining the position of the risk block within the paragraph using a sliding window algorithm and converting the position into PDF pixel-level coordinates includes the following steps: Based on the paragraph containing the risk block, determine the page of the paragraph containing the risk block using text hash matching technology throughout the document; dynamically determine the size of the sliding window based on the content feature word density in the paragraph containing the risk block (the higher the density, the smaller the window size); slide and match the risk block within the paragraph using the sliding window, and within each sliding window, locate the character index of the risk block using a binary search algorithm, and convert the character index of the risk block into precise pixel-level coordinates of the PDF page using a pre-acquired page coordinate matrix; perform position similarity judgment on the precise pixel-level coordinates of all risk blocks in the PDF page, and merge the precise pixel-level coordinates of at least two risk blocks with similar positions into one annotation coordinate.

12. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1-10.

13. A document risk intelligent identification system supporting real-time annotation, characterized in that, This includes a client and a server, which perform the following communication process: A two-way communication connection is established between the client and the server; The server receives documents uploaded by the client and performs document parsing, preliminary risk identification, batch risk verification, and precise location and labeling of risk points in sequence based on the method described in any one of claims 1-10. And display the risk identification progress to the client in real time; The server sends a document containing risk point marking information to the client.

Images