Method for intelligently analyzing policy content by using atlas model

Abstract

The invention relates to the technical field of artificial intelligence-natural language processing and knowledge graph construction, in particular to a method for intelligently analyzing policy content by using a graph model. The invention discloses a method for intelligently analyzing policy contents by using a graph model. The method comprises the following steps of: increment acquisition; a format decoding step; an entity identification step; a relation extraction step; a body mapping step; a map writing step; a hybrid reasoning step; and a service output step. By means of the above step cooperation mechanism, automatic grabbing, accurate analysis, tense modeling and intelligent reasoning are achieved in a multi-format and easily-changed policy data scene, the data integrity and query timeliness are remarkably improved, and the problems of missing grabbing, missing extraction and version mixing in the prior art are completely solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence - natural language processing and knowledge graph construction technology, and particularly relates to a method for intelligently analyzing policy content by using a graph model. BACKGROUND

[0002] Existing policy information processing mostly relies on two separate processes: one end uses a timed task to poll and capture each page of a government or industry website, and then extracts limited fields from the web page by using regular or keyword scripts; the other end manually supplements and regularly corrects the extraction results by manually supplementing and regularly correcting the extraction results in an Excel or semi-structured database. When the site style mutates and the announcement format is various (HTML + PDF + scan), the script is easily invalidated, resulting in missed or incorrect capture. The collected "field table" cannot intuitively express the version replacement and time effective relationship, and cannot support semantic connection across sentences and paragraphs, so the final query link still needs manual comparison with the original text.

[0003] Therefore, how to automatically, accurately and time-traceably convert policy content into searchable structured knowledge in the case of frequent changes in the site and multiple document formats has become a subject to be studied by those skilled in the art. SUMMARY

[0004] The present application aims to provide a method for intelligently analyzing policy content by using a graph model to address the shortcomings of the prior art, and to automatically, accurately and time-traceably convert policy content into searchable structured knowledge in the case of frequent changes in the site and multiple document formats.

[0005] The present application achieves the above-mentioned purpose by the following technical solution: a method for intelligently analyzing policy content by using a graph model, comprising: Incremental acquisition step: performing structure change detection on a preset policy publishing source, and if the page structure entropy difference reaches a threshold, capturing the page; Format decoding step: performing HTML parsing, text layer PDF extraction and scan OCR recognition on the captured page respectively to obtain the policy original text; Entity recognition step: using a hierarchical soft label sequence labeling model (BERT-BiLSTM-CRF model) to extract entities including at least a publishing subject, an applicable object, a support amount and a reporting deadline; Relationship extraction step: extracting policy relationships between entities within a cross-sentence sliding window by using a pointer network; Ontology mapping step: mapping the entities and relationships into time triples carrying effective time and invalid time attributes; Graph writing step: writing the time triples into a knowledge graph database and recording the version; Hybrid reasoning step: using rule set and TransE-GCN embedding model to complete corresponding triples into knowledge graph and detect conflicts, update knowledge graph; Service output step: based on the updated knowledge graph, provide natural language query and API service interface output structured policy information.

[0006] Further, the structure change detection comprises: Calculate the structure entropy difference between the current page and the last period page by formula one; Compare the structure entropy difference with the threshold value, when the structure entropy difference is greater than or equal to the threshold value, determine that the page has structure change and trigger crawling; Formula one: ; In formula one, The structure entropy of the page DOM fingerprint; The probability of the first Class node in the fingerprint; The current and last period fingerprints respectively; The structure entropy difference; The threshold value, when Crawl the page; The incremental collection step further comprises: writing the URL to be crawled into a priority queue, and assigning crawler threads to crawl according to the weight from high to low.

[0007] Further, the scanning OCR identification process further comprises confidence quality judgment, which comprises: Generate a confidence matrix for the recognition result And calculate the average confidence by formula two ; Compare the average confidence With the threshold value, the threshold value is 0.90, if the average confidence is less than the threshold value, that is Then add the text segment to the artificial review queue Formula two: ; In formula two, The average confidence; The probability of the first Candidate box being the first Character; The number of candidate boxes; The number of character categories; when Add the text segment to the artificial review queue; The format decoding step further comprises: using binarization + adaptive threshold to re-identify once for the image block with OCR score lower than the threshold value to improve accuracy.

[0008] Further, the loss function of the sequence labeling model used in the entity recognition step is defined as the following formula three; Formula three: ; In formula three, is the total loss; is the loss of the i-th position encoding vector; is the position encoding vector; is the weight and bias of the label; is the true label; is the hard label distribution; is the soft label distribution; is the hierarchical weight; is the regularization coefficient; is the regularization coefficient; The entity recognition step further comprises: manually checking and labeling the entity whose recognition confidence is lower than 0.6.

[0009] Further, in the relation extraction step, the following determination rule is used to determine whether there is a relation between the entity numbered i and the entity numbered j in the relation extraction step: Attention calculation: within the cross-sentence sliding window width Ws=5, the attention weight of the two entities is calculated by formula four; Threshold determination: if formula five is met within the range of , output the corresponding entity relation triple; Formula four: ; Formula five: ; In formula four and formula five, is the entity pointing to the entity attention; are the context vectors, respectively; is the weight matrix; is the number of window entities, is the cross-sentence sliding window width and is specifically 5; The relation extraction step further comprises: adding a window hedging strategy separated by a period and a semicolon to the cross-paragraph relation to improve recall rate.

[0010] Further, in the ontology mapping step, when it is determined that a new version v' of a policy replaces an old version v, the time-based triple is updated according to the following formula six; Formula six: ; In the formula, is the invalid time of the old version ; is the effective time of the new version ; and write new tense triplets to explicitly record the replacement relationship and the validity period of the new version.

[0011] Further, the mixed reasoning step is: for the missing candidate triplet 〈h, r, t〉 in the knowledge graph, first apply the rule set to generate a derivable relationship, and then use the TransE-GCN embedding model to calculate the confidence score of the candidate triplet; when the score meets the completion threshold condition, the corresponding triplet is completed into the knowledge graph and conflict detection is performed; wherein the confidence of the missing candidate triplet 〈h, r, t〉 is calculated according to formula seven: Formula seven: ; In formula seven, are the embedding vectors of the head entity, the relationship and the tail entity respectively; is the L1 norm; is a coupling coefficient for balancing the scores of the two terms; is the adjacency weighted matrix learned by the graph convolution network; When , the candidate triplet 〈h, r, t〉 is determined as a trusted missing relationship and is completed into the knowledge graph; all completed relationships are sorted in ascending order of score, and the top 5% of relationship edges are marked with "reasoning source" for subsequent explainability display and manual review. Specifically,

[0012] Further, in the mixed reasoning step, if there are two mutually exclusive relationships for the same head and tail entities and , the conflict confidence is calculated using formula eight; Formula eight: ; In formula eight, is a preset relationship mutual exclusion coefficient; and are the triplet scores; wherein when , it is marked as a conflict relationship, and the conflict record is written into the audit log and pushed for manual review.

[0013] Further, in the service output step, the retrieval ranking of the user natural language question is calculated according to formula nine: Formula nine: ; In formula nine, is the question vector; are the embedding vectors of the entities ; is a temperature parameter; The system outputs the ranking probability before The entity is displayed and its adjacency relationship is automatically expanded for visualization.

[0014] Furthermore, the incremental collection steps are as follows: perform structural change detection on the web pages of the preset policy release source; when the page structure entropy difference ΔH reaches the threshold η, crawl the page and write it into the parsing queue, and at the same time dynamically adjust the collection bandwidth or thread concurrency of the crawler in this cycle according to the intensity of change of each site. Among them, the acquisition bandwidth Based on Formula 10, dynamic allocation: Formula 10: ; In the formula, For real-time bandwidth; This is the maximum bandwidth limit; For the first Policy site weight; This represents the current periodic structural entropy difference at the site. This serves as the reference structural entropy for the site. The decay index is preferred. ; The bandwidth allocation results are refreshed at regular intervals (preferably 30 minutes) to adjust the number of concurrent crawler threads in the next cycle.

[0015] Furthermore, the spectral database adopts Second-level partitions: Level is based on administrative level (country, province, city, county), and Year is based on the year the policy took effect; when querying across partitions, priority is given to searching the partitions of the same level that have been in the most recent two years, and if no match is found, the search is expanded outwards.

[0016] A policy content intelligent parsing device includes a collection module, a parsing module, a semantic extraction module, a mapping and writing module, a reasoning module, and a service module, configured to execute the corresponding steps in any of the methods of claims 1-11.

[0017] The beneficial effects of this invention are: By using incremental data collection steps triggered by structural entropy differences, data is only collected when the page actually changes. This approach can adapt to frequent site layout adjustments, ensuring data freshness while saving bandwidth.

[0018] By employing a format decoding process, different parsing, direct extraction, and OCR links are selected for HTML, text-layer PDF, and scanned documents respectively, avoiding the failure of a single path and achieving unified original text acquisition from multiple source documents.

[0019] By employing deep sequence labeling models and pointer networks in entity recognition and relation extraction steps, a cross-sentence sliding window captures the semantic relationships between "issuing entity - application target - support amount - term," solving the problems of missed extraction and mismatch in traditional keyword scripts.

[0020] The ontology mapping step transforms the extracted elements into temporal triples with effective / expiration times, recording not only the facts but also their effective intervals, naturally supporting policy iteration and failure management.

[0021] The step graph writing process incorporates temporal triples into the knowledge graph, and uses rules and embedding to complete missing relationships and mark conflicts, resulting in a complete and consistent graph that eliminates the need for manual proofreading.

[0022] The service output steps directly provide natural language retrieval and APIs on the updated graph, allowing users to obtain real-time, structured answers without comparing the original text.

[0023] Through the collaborative mechanism described above, this invention achieves automatic capture, accurate parsing, temporal modeling, and intelligent reasoning in scenarios with multi-format and easily changing policy data, significantly improving data integrity and query timeliness, and completely solving the problems of missed capture, missed extraction, and version mixing in existing technologies. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It is understood that the accompanying drawings are provided for reference and illustration only, and are not intended to limit the present invention. The connection relationships shown in the accompanying drawings are only for clear description and do not limit the connection method.

[0026] like Figure 1 As shown, this embodiment of the invention provides a method for intelligent analysis of policy content using a graph model, including: Incremental data collection steps: Perform structural change detection on the preset policy release source; if the page structure entropy difference reaches the threshold, then crawl the page. Format decoding steps: Perform HTML parsing, PDF text extraction, and OCR recognition on the captured page to obtain the original policy text; Entity identification steps: Use a hierarchical soft-label sequence labeling model (BERT-BiLSTM-CRF model) to extract entities that include at least the issuing entity, applicable objects, support amount, and application period; Relationship extraction steps: Extract policy relationships between entities through a pointer network within a cross-sentence sliding window; Body mapping step: mapping entities and relations into temporal triples carrying validity time and invalidity time attributes; Graph writing step: writing temporal triples into knowledge graph database and recording version; Mixed reasoning step: using rule set and TransE-GCN embedding model to complete corresponding triples into knowledge graph and perform conflict detection, updating knowledge graph; Service output step: providing natural language query and API service interface output structured policy information based on updated knowledge graph. Specifically: The crawler and parsing service runs in the Kubernetes cluster, the knowledge graph database uses NebulaGraph, and the algorithm service exposes the interface with gRPC. Through Airflow, the incremental collection DAG is scheduled every 5 minutes, and the parsing and extraction DAG is triggered by the message queue (KafkaTopic="PolicyRaw"). A unique trace-id is generated for each step, and when an exception occurs, the original page, parsed text, triple version and reasoning record can be traced back along the link.

[0027] In this scheme, the whole link is automatically run without manual script maintenance at each station; any node error can be located in seconds, improving system operation and maintenance and stability.

[0028] In an embodiment, the structure change detection includes: Calculate the structure entropy difference between the current page and the last period page using Formula One; Compare the structure entropy difference with the threshold value, and when the structure entropy difference is greater than or equal to the threshold value, determine that the page has a structure change and trigger crawling; Formula One: ; In Formula One, is the structure entropy of the page DOM fingerprint; is the probability of the first node in the fingerprint; are the current and last period fingerprints, respectively; is the structure entropy difference; is the threshold value, and when the page is crawled; The incremental collection step further includes: writing the URL to be grabbed into a priority queue, and assigning crawler threads to grab according to the weight from high to low. Specifically: DOM fingerprint: only select label name + depth + sibling number triplets to construct hash, and the fingerprint size is <1KB; Threshold value η: dynamically take the average of the past 7 days + 1.5 times the standard deviation, and adapt to different site stability; Priority queue: weight , where α:β:γ=3:4:3.

[0029] Furthermore, only pages that have actually changed are crawled, saving an average of 25% of traffic per day; priority weight ensures that high-popularity, high-level sites are crawled first.

[0030] In one embodiment, the OCR recognition process of the scanned document further includes performing a confidence quality assessment, which includes: Generate a confidence matrix from the recognition results. And calculate the average confidence level according to Formula 2. ; average confidence Compared to a threshold of 0.90, if the average confidence level is less than the threshold, i.e. If the text fragment is not found, it will be added to the manual review queue. Formula 2: ; In Formula 2, The average confidence level; For the first Candidate boxes were identified as the first The probability of a character; The number of candidate boxes; Number of character categories; The text fragment should then be added to the manual review queue. The format decoding step also includes: binarizing and then re-identifying image blocks with OCR scores below a threshold using an adaptive threshold to improve accuracy. Specifically: Character set n: contains 6000 categories including simplified Chinese, traditional Chinese, numbers, and common symbols; Confidence matrix block division: Calculation of long PDFs with more than 20 lines of text by page block. Only low-confidence segments are reviewed. Binarization parameters: Otsu threshold + local mean filtering; adaptive threshold window 25×25px.

[0031] Furthermore, automatic verification accounts for only 8% of the total scanned pages, reducing manual work by 92%, but improving the overall OCR accuracy rate to over 95%.

[0032] In one embodiment, the loss function of the sequence labeling model used in the entity recognition step is defined as the following formula three; Formula 3: ; In Formula 3, Total loss; For the first Position encoding vector; For tags the weights and biases of the; is a true label; is a hard label distribution; is a soft label distribution; is a hierarchical weight; is a regularization coefficient; The entity recognition step further comprises: triggering manual verification labeling for entities with an identification confidence lower than 0.6. Specifically: Soft label source: 3036 categories of policy field vocabulary, mapped by administrative level -> funding indicator -> time limit in three layers; λ value: 0.3-0.7 grid search to verify the best F1 of 0.5 on the validation set; Manual verification interface: identification confidence bar + shortcut label button, average correction <30 seconds per announcement.

[0033] Furthermore, soft label distillation reduces incorrect labeling caused by hierarchical differences; manual work only handles a small number of key entities with a confidence of <0.6, and overall efficiency is improved by 4 times.

[0034] In an embodiment, in the relationship extraction step, the following determination rule is used to determine whether there is a relationship between the entity numbered i and the entity numbered j in the relationship extraction step: Attention calculation: within a cross-sentence sliding window width Ws=5, the attention weight of the two entities is calculated using Formula Four; Threshold determination: if Formula Five is satisfied within , output the corresponding entity relationship triple; Formula Four: ; Formula Five: ; In Formula Four and Formula Five, is the entity points to the entity attention; are the context vectors respectively; is the weight matrix; is the number of window entities, is the cross-sentence sliding window width and specifically 5; The relationship extraction step further comprises: adding a window hedging strategy separated by periods and semicolons for cross-paragraph relationships to improve recall. Specifically: Context vector: from the same RoBERTa-large encoder, parameters shared with entity recognition; Sliding window compensation: if the period separation causes the distance between i and j to be >5, but there is a reference mark (“the file” “the above enterprise”) within the same paragraph, a soft window hedging is added.

[0035] Window entity number N: dynamically equal to the number of entities appearing in the sliding window, at most 20.

[0036] Further, the matching success rate of the scattered point distribution of the supporting object and the amount of money is improved from 72% to 86%, which significantly improves the relationship integrity.

[0037] In an embodiment, in the ontology mapping step, when it is determined that a new version v' of a policy replaces an old version v, the temporal triple is updated according to the following formula six; Formula six: ; In the formula, is the invalid time of the old version ; is the effective time of the new version ; and a new temporal triple is written to explicitly record the replacement relationship and the valid period of the new version. Specifically: Version identification: regular + BERT-CLS multi-classification judgment of "revision" and "abolition" words; Time source: if the official text is missing the invalid date, fill in 23:59:59 on the same day; Triple writing: replace the relationship with an independent edge-type "replaces" to facilitate visual display of the version chain.

[0038] Further, it prevents the coexistence of multiple versions of the same policy, so that users always obtain the latest valid text and reduce compliance risks.

[0039] In an embodiment, the mixed reasoning step is: for the missing candidate triple 〈h, r, t〉 in the knowledge graph, first apply the rule set to generate derivable relationships, and then use the TransE-GCN embedding model based on the candidate triple confidence score; When the score meets the completion threshold condition, the corresponding triple is completed into the knowledge graph and conflict detection is performed; Wherein, the confidence of the missing candidate triple 〈h, r, t〉 is calculated according to formula seven: Formula seven: ; In formula seven, are the embedding vectors of the head entity, the relationship and the tail entity respectively; is the L1 norm; is the coupling coefficient, used to balance the scores of the two items; is the adjacency weighted matrix learned by the graph convolution network; When When the candidate triple is identified as a reliable missing relation, it is completed to the knowledge graph. The top 5% of the relations are marked with a "reasoning source" label to facilitate subsequent explainability display and manual review. Specifically: Vector dimension: TransE embedding 200 dimensions, GCN2 layer hidden 128 dimensions; μ parameter tuning: experiment between 0.1-1.0, μ=0.3 when the comprehensive MAP is best; Completion threshold: 1.0 fixed, matching true rate 91%, false positive rate 6%.

[0040] Furthermore, automatic completion improves the coverage of "fund pool-amount" information from 58% to 89%; 5% high-confidence edges are marked with reasoning sources to increase data explainability.

[0041] In an embodiment, in the mixed reasoning step, if there are two mutually exclusive relations for the same head and tail entities Existence of two mutually exclusive relations And , the conflict confidence is calculated using formula eight; Formula eight: ; In formula eight, is the preset relationship mutual exclusion coefficient; And is the score of the triple; Wherein, when , it is marked as a conflict relation, and the conflict record is written into the audit log and pushed to manual review. Specifically: Mutually exclusive matrix : generated by manual rules and data statistics, set weight 1 for "valid-after" vs "invalid-after" relations; Audit process: conflict edges are written into KafkaTopic "PolicyConflict", which is assigned for review by the background ticket system.

[0042] Furthermore, about 300 contradictory relations are actively discovered and resolved every week, avoiding conflicting answers in front-end retrieval.

[0043] In an embodiment, in the service output step, the retrieval ranking of the user natural language question is calculated according to formula nine: Formula nine: ; In formula nine, is the question vector; is the embedding vector of the entity ; is the temperature parameter; The system outputs the ranking probability before The entities are identified, and their adjacency relationships are automatically expanded for visualization. Specifically: Question vector Uses legal Chinese E5-base encoding; β temperature: 0.06, which makes the probability distribution smoother; Visualization: Front-end Vue + vis-network, which by default expands the entity nodes within the 2-hop adjacency.

[0044] Furthermore, the TOP-10 hit rate was 92%, and the average number of interactions per user decreased from 4.2 to 1.7; the node graph allows non-technical personnel to intuitively see the full picture of "policy-funding-conditions".

[0045] In one embodiment, the incremental collection step is as follows: perform structural change detection on the webpage of the preset policy release source; when the page structure entropy difference ΔH reaches the threshold η, crawl the page and write it into the parsing queue, and dynamically adjust the collection bandwidth or thread concurrency of the crawler in this cycle according to the change intensity of each site. Among them, the acquisition bandwidth Based on Formula 10, dynamic allocation: Formula 10: ; In the formula, For real-time bandwidth; This is the maximum bandwidth limit; For the first Policy site weight; This represents the current periodic structural entropy difference at the site. This serves as the reference structural entropy for the site. The decay index is preferred. ; The bandwidth allocation results are refreshed at regular intervals (preferably 30 minutes) to adjust the concurrency of crawler threads in the next cycle. Specifically: Data collection period: 30-minute sliding window statistics Writes to Redis in real time; B_{\max}: Set according to an outbound bandwidth of 100Mbps; Thread concurrency mapping: 1 thread for every 5Mbps, minimum 2 threads, maximum 16 threads.

[0046] Furthermore, the 30-minute adaptive refresh avoids "hot sites overload" or "cold sites idle for a long time", increasing the capture success rate by 18% and reducing network costs by 25% under the same bandwidth.

[0047] In one embodiment, the spectral database adopts... Secondary partition: Level according to administrative level (national, provincial, municipal, county), Year according to policy effective year; when cross-partition query, prefer to search in the same level in the last two years, if not hit, expand outward.

[0048] A policy content intelligent analysis device, comprising an acquisition module, an analysis module, a semantic extraction module, a mapping and writing module, an inference module and a service module, configured to perform the corresponding steps in any of claims 1-11.

[0049] Technical principle: This scheme constructs a closed loop link around the six links of "web crawling - text analysis - information extraction - ontology mapping - knowledge reasoning - intelligent service", realizes the automatic analysis and instant service of policy content. First, the system constructs the DOM structure fingerprint of the web page of each policy publishing source, and calculates the structural entropy difference of adjacent periods in real time. When the entropy difference ΔH exceeds the threshold η, it means that the page format or content has changed significantly, and the corresponding URL is immediately placed in the high-priority queue, and the network bandwidth or crawler thread concurrency of the current period is dynamically adjusted according to the change intensity of each site to ensure that the hot site is preferentially crawled. The crawled page will enter multi-branch format decoding: regular HTML direct analysis, PDF containing text layer extracts embedded text, and pure scanned copy enters OCR recognition; generate a confidence matrix for OCR output, and automatically binarize and re-identify image blocks with an average confidence less than 0.90, and add a manual review cache to ensure the recognition accuracy of scanned announcements.

[0050] The parsed original text is sent to the entity recognition model. This model uses the BERT-BiLSTM-CRF sequence labeling framework and introduces the "hierarchical soft label distillation" mechanism: hard labels ensure accuracy, and soft labels pass on the category information to improve recall. The policy entities output by the model (such as the publishing subject, applicable object, support amount, and reporting period) then enter the relationship extraction stage. The system uses pointer networks to calculate the attention between entities within a cross-sentence sliding window; when the attention weight is greater than 0.35, it is considered that there is a semantic relationship between them. To avoid text fragmentation caused by periods or semicolons, a window hedging strategy is also adopted for cross-paragraph relationships, thereby improving the recall rate in cross-sentence and cross-paragraph scenarios.

[0051] The extracted entities and relationships are mapped to the predefined policy ontology: each triple carries the effective time t_s and the invalid time t_e. If a new version v' of the same policy is detected to replace the old version v, the old clause t_e will be automatically updated to before t_s(v'), and the "replaces" relationship will be written to ensure that only the valid version chain is retained in the graph. The time-aware triples that complete ontology mapping are written to the knowledge graph database (NebulaGraph), and are stored according to the 〈administrative level, effective year〉 secondary partition to obtain high-concurrency read-write capability.

[0052] To make up for the implicit relationship that the original extraction is difficult to cover, the system performs hybrid reasoning at the graph level: on the one hand, the potential triples are derived through logical rules; on the other hand, the confidence f(h, r, t) of the candidate triple is calculated by means of the TransE-GCN embedding model, and when the score is less than or equal to 1.0, it is automatically completed into the graph. When the same head and tail entities appear in a mutually exclusive relationship, the system calculates the conflict confidence Conf; if Conf exceeds 0.8, an audit log is automatically recorded and manual review is pushed. Finally, based on the continuously updated knowledge graph, the system provides a natural language query interface: the user query vector is matched with the entity embedding vector in soft temperature, and the top ten entities are output, and their adjacent relationships are automatically expanded in the front end to present a clear policy knowledge view.

[0053] Beneficial effects: Through the above closed-loop design, the present application solves the core problems of traditional policy information processing in multiple dimensions. First, the structural entropy difference detection cooperates with the dynamic bandwidth scheduling, so that the system only initiates the crawling when the webpage is actually changed, and at the same time, the bandwidth is concentrated to the high-heat and high-level sites, and the average delay is shortened from tens of minutes to about two minutes, significantly improving the timeliness of policy information. The multi-branch format decoding and the OCR confidence self-checking strategy make the character recognition accuracy of scanned announcements improve to more than 95%, and the manual review amount accounts for only eight percent of all scanned pages, greatly reducing the labor cost.

[0054] At the information extraction level, hierarchical soft label distillation not only maintains the accuracy of the existing CRF model, but also supplements the semantic generalization ability through soft labels, so that the entity F1 value of the key slot is improved as a whole; the pointer network plus the sliding window hedging strategy effectively captures the supporting relationship across paragraphs, and the relationship recall rate is improved by nearly eight percentage points, thereby ensuring the completeness of the knowledge graph. The temporal ontology mapping and the automatic version replacement rule eliminate the risk of "old text new interpretation", and ensure that users always get effective clauses within the current time window.

[0055] Hybrid reasoning uses TransE-GCN embedding and rule engine to complement each other, automatically supplements missing relationships, and improves the completeness of the graph from 80% to more than 90%; at the same time, the conflict confidence calculation and audit mechanism controls potential logical contradictions to less than one in ten thousand, ensuring the consistency of knowledge. The question and answer retrieval part sorts the vectors by probability and visualizes the adjacency, providing non-technical users with a "one-question-at-a-time, visual and traceable" user experience, reducing the average interaction steps by more than half.

[0056] In summary, through the integrated technical path of "real-time detection, accurate analysis, deep extraction, time mapping, intelligent completion, and interpretable service", the application realizes faster speed, higher accuracy, more complete coverage, and lower operation and maintenance cost of policy information acquisition, and exhibits outstanding technical progress and industrial application value in government information transparency and enterprise compliance management scenarios.

[0057] In detail: The application passes through a complete set of end-to-end links to pass through the technical process of "policy webpage -> original text analysis -> knowledge graph -> service output", compared with the existing manual search and dispersed script mode, the following comprehensive improvements and quantifiable benefits are brought: Significant improvement in timeliness: using structural entropy difference detection and dynamic bandwidth scheduling, once the structure of the webpage changes significantly, high-priority crawling is triggered, and the number of crawler threads automatically increases or decreases with the change intensity. In the actual measurement of 200+ publishing source scenarios, the average policy onboarding delay is shortened from ≥15min of traditional polling to ≤2min, ensuring that enterprises can learn about the latest support or regulatory policies in the first time.

[0058] Improvement of original text analysis accuracy: through multi-branch format decoding (HTML, PDF text layer, OCR scan), and superimposing OCR confidence quality judgment + adaptive secondary recognition, the Top-1 accuracy of character recognition of scanned announcements is improved from 85% to more than 95%. Low-confidence segments are automatically initiated for manual review, greatly reducing the entity extraction bias caused by "wrong or missing characters".

[0059] Sensitive and reliable entity and relationship extraction: the hierarchical soft label sequence labeling model introduces soft label distillation constraints on important slots ("publishing subject / declaration period / amount"), and the F1 value is improved by 3.4pp compared with pure CRF. Pointer network with cross-sentence window and period separation hedging strategy, the recall rate of "declaration subject-support method" relationship across paragraphs is improved by about 8%.

[0060] Entities with a confidence score of less than 0.6 are automatically marked in red, reducing the cost of manual re-labeling by half.

[0061] Temporal triple + version mapping eliminates the risk of "old text with new interpretation": the version replacement formula automatically modifies the expiration date of old clauses, and generates the replaces relationship, so that the knowledge graph always maintains a unique and valid version chain. The retrieval interface always returns the clauses that are effective in the current time window, avoiding the misuse of expired policies by users.

[0062] Mixed reasoning completes the information blind area: TransE-GCN combines graph structure adjacency weight to score missing triples, and completes the "support direction-amount of money" type implicit relationship with the rule set. The overall graph completeness is improved from 82% to 93%. The top 5% confidence edges are marked as "reasoning source", which not only ensures explainability, but also facilitates later review.

[0063] Conflict detection daemon data consistency: When the same entity pair appears in a mutually exclusive relationship, calculate the conflict confidence. If the threshold is above 0.8, trigger manual review to eliminate logical conflicts such as "the same policy being effective and ineffective at the same time", and reduce the graph consistency detection rate found by manual inspection to <0.5‰.

[0064] Intelligent question and answer experience optimization: Use a temperature-softened vector retrieval formula, and the average precision of TOP-10 recall reaches 92%. The adjacent nodes are automatically expanded for visualization, and non-technical users can also locate "who can apply, and when is the deadline" in one step.

[0065] Higher resource utilization: The bandwidth dynamic formula allocates network resources according to the entropy difference of each site, and the bandwidth of popular sites can be enlarged by 3x, while the bandwidth of cold sites is automatically contracted. The overall crawling traffic is reduced by ≈28%, but the high-value page crawling timeliness is improved.

[0066] Full-link scalability and easy maintenance: Key thresholds (entropy difference η, OCR 0.90, attention 0.35, score 1.0, conflict 0.8, bandwidth decay κ=2) can be dynamically adjusted according to actual business indicators. The system can be smoothly migrated to more industry or regional policy sources, reducing the difficulty of subsequent operation and maintenance.

[0067] In summary, this scheme introduces quantitative decision thresholds, deep learning models, and graph computing fusion algorithms at various stages such as collection, analysis, extraction, mapping, reasoning, and service, achieving faster speed, higher accuracy, more complete knowledge coverage, and lower operation and maintenance costs for policy information acquisition. It solves the outstanding pain points of "multiple release sources, mixed formats, slow updates, many conflicts, and difficult queries" in existing technologies, and has significant technical progress and industrial application value.

[0068] Finally, it should be noted that: the skilled in the art will cross-reference or superimpose the various embodiments of the present scheme, which still belongs to the original disclosure range of the present scheme. In addition, the above only describes the preferred embodiments of the present application, and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features, as long as they are within the spirit and principles of the present application. Any modification, equivalent replacement, improvement, etc. made shall be included in the protection scope of the present application.

Claims

1. A method for intelligent analysis of policy content using a graph model, characterized in that, include: Incremental data collection steps: Perform structural change detection on the preset policy release source; if the page structure entropy difference reaches the threshold, then crawl the page. Format decoding steps: Identify the captured page to obtain the original policy text; Entity recognition steps: Extract the required entities using a hierarchical soft-label sequence labeling model; Relationship extraction steps: Extract policy relationships between entities through a pointer network within a cross-sentence sliding window; Ontology mapping steps: Map entities and relations to temporal triples carrying effective time and expiration time attributes; Knowledge graph writing steps: Write temporal triples into the knowledge graph database and record the version; Hybrid reasoning steps: The corresponding triples are completed into the knowledge graph using rule sets and embedding models, and conflict detection is performed to update the knowledge graph; Service output steps: Based on the updated knowledge graph, output structured policy information.

2. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, The structural change detection includes: Calculate the structural entropy difference between the current page and the previous period's page using Formula 1; The structural entropy difference is compared with the threshold. When the structural entropy difference is greater than or equal to the threshold, it is determined that the page has undergone structural changes and crawling is triggered. Formula 1: ; In Formula 1, The structural entropy of the page's DOM fingerprint; For the first The probability of a class node in the fingerprint; These are the fingerprints for the current and previous cycles, respectively. The difference is the structural entropy. As a threshold, when Timely page capture; The incremental collection steps also include: writing the URLs to be crawled into a priority queue and allocating crawler threads to crawl them according to their weight from high to low.

3. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, The OCR recognition process for the scanned document includes a confidence quality assessment, which includes: Generate a confidence matrix from the recognition results. And calculate the average confidence level according to Formula 2. ; average confidence If the average confidence level is less than the threshold, the text segment is added to the manual review queue. Formula 2: ; In Formula 2, The average confidence level; For the first Candidate boxes were identified as the first The probability of a character; The number of candidate boxes; Number of character categories; The format decoding step also includes: binarizing and adaptively thresholding image blocks with OCR scores below the threshold for re-recognition to improve accuracy.

4. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, The loss function of the sequence labeling model used in the entity recognition step is defined as the following formula three; Formula 3: ; In Formula 3, Total loss; For the first Position encoding vector; For tags Weights and biases; True label; The distribution is based on hard labels; Distribution of soft labels; Hierarchical weights; The regularization coefficient; The entity recognition step further includes: triggering manual verification and labeling for entities with a recognition confidence level below 0.

6.

5. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, In the relation extraction step, the following rule is used to determine whether there is a relationship between entity i and entity j: Within the width Ws of the cross-sentence sliding window, the attention weights of the two entities are calculated using Formula 4; If in If Formula 5 is satisfied within the specified range, then the corresponding entity relation triplet will be output. Formula 4: ; Formula 5: ; In formulas four and five, For entities Pointing to entity attention; These are the context vectors; This is the weight matrix; For the number of window entities, The width of the cross-sentence sliding window is 5; The relation extraction step also includes adding a window hedging strategy of using periods and semicolons to separate cross-paragraph relations in order to improve recall.

6. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, In the ontology mapping step, when it is determined that a new version v′ of a policy replaces the old version v, the temporal triplet is updated according to the following formula six; Formula Six: ; In the formula, Older version Failure time; This refers to the effective date of the new version; And write the new temporal triplet. This is to clearly record the replacement relationship and the validity period of the new version.

7. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, The hybrid reasoning steps are as follows: For the missing candidate triples <h,r,t> in the knowledge graph, first apply the rule set to generate derivable relations, then use the embedding model to calculate the confidence score of the candidate triples; when the score meets the completion threshold condition, the corresponding triples are completed into the knowledge graph and conflict detection is performed. The confidence level of the missing candidate triple <h,r,t> is calculated according to Formula 7: Formula 7: ; In Formula 7, These are the embedding vectors for the head entity, relation, and tail entity, respectively. It is an L1 norm; This is the coupling coefficient, used to balance the scores of the two items; The adjacency weighting matrix learned by the graph convolutional network; when When the candidate triple <h,r,t> is identified as a reliable missing relation and is added to the knowledge graph, all completed relations are sorted in ascending order of score, and the top 5% of relation edges are marked.

8. The method for intelligent policy content analysis using a graph model according to claim 7, characterized in that, In the hybrid reasoning step, if the same head and tail entities are involved... There are two mutually exclusive relations and Then, Formula 8 is used to calculate the conflict confidence level; Formula 8: ; In Formula 8, These are the preset mutual exclusion coefficients for the relationships; Score the triples; Among them, when When a conflict occurs, it is marked as a conflict relationship, the conflict record is written to the audit log, and then pushed for manual review.

9. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, In the service output step, the user's natural language question... The search ranking is calculated according to Formula Nine: Formula Nine: ; In Formula Nine, The vector of questions; For entities The embedding vector; For temperature parameters; The system outputs the ranking probability before The entity is displayed and its adjacency relationship is automatically expanded for visualization.

10. The method for intelligent policy content analysis using a graph model according to claim 1, characterized in that, The incremental data collection steps are as follows: perform structural change detection on the webpages of the preset policy release sources; when the page structure entropy difference... Reaching the threshold The system captures pages and writes them to the parsing queue, while dynamically adjusting the crawler's collection bandwidth or thread concurrency based on the intensity of changes at each site. Among them, the acquisition bandwidth Based on Formula 10, dynamic allocation: Formula 10: ; In the formula, For real-time bandwidth; This is the maximum bandwidth limit; For the first Policy site weight; This represents the current periodic structural entropy difference at the site. This serves as the reference structural entropy for the site. The decay index; The bandwidth allocation results are refreshed at regular intervals to adjust the number of concurrent crawler threads in the next cycle.

Images