Method and system for compliance review of large language model based on multi-source data fusion

Abstract

The application discloses a large language model compliance review method and system based on multi-source data fusion, comprising: performing hierarchical structure analysis and semantic block on multi-source heterogeneous regulation documents, binding metadata to the blocks according to regulation validity levels, obtaining a regulation knowledge base, searching the to-be-reviewed documents based on the regulation knowledge base, weighting and sorting the search results according to regulation validity levels and timeliness, obtaining a regulation constraint set, dynamically splicing the to-be-reviewed documents and the regulation constraint set to a prompt word template, injecting mandatory reference constraint instructions, inputting a large language model to perform compliance reasoning, performing confidence evaluation on the compliance state, generating an audit evidence chain between the to-be-reviewed documents and the regulation constraint set according to the evaluation result, and obtaining a compliance review result. The method utilizes multi-source regulation validity weighted search and beam search constraint pruning to improve heterogeneous regulation search accuracy and compliance determination reliability, and improves the explainability of the review result through bidirectional character-level tracing.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a method and system for compliance review of large language models based on multi-source data fusion. Background Technology

[0002] In recent years, with the acceleration of global digital transformation, the demand for certification and oversight work, including qualification certification, compliance audits, ISO quality standard certification, and environmental, social, and governance (ESG) compliance reviews, has exploded across various industries. Traditional certification oversight mainly relies on human auditors to compare compliance documents, contracts, operational logs, and supporting materials submitted by companies item by item. Due to the large number of regulatory provisions in national laws, industry standards, local standards, and internal management regulations, their frequent updates, and overlapping hierarchical relationships, manual audits are not only time-consuming, labor-intensive, and costly, but also prone to omissions or misjudgments due to limitations in individual professional knowledge. To address the limitations of manual audits, the industry has begun to introduce the concept of digital auditors for certification, attempting to build automated certification oversight systems using NLP (Natural Language Processing) technology. With the rise of LLM (Limited Language Management), leveraging its powerful semantic understanding and text generation capabilities for assisted auditing has become a current research hotspot.

[0003] In certification oversight applications, digital auditors not only need to determine whether documents under review are compliant, but also need to strictly adhere to multi-source and multi-level regulatory provisions to output legally valid, traceable, and logically rigorous audit opinions. However, due to the inherent illusion phenomenon of large language models and insufficient logical constraints on rigid rules, reliable audit evidence chains cannot be provided. How to achieve digital certification oversight with high confidence, strong constraints, and two-way traceability has become an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for compliance review of large language models based on multi-source data fusion.

[0005] To achieve the above objectives, the present invention is implemented according to the following technical solution:

[0006] The first aspect of this invention provides a compliance review method for large language models based on multi-source data fusion, including:

[0007] Hierarchical structure parsing and semantic segmentation are performed on multi-source heterogeneous regulatory documents, and metadata is bound to the segments according to the level of regulatory effectiveness to obtain a regulatory knowledge base;

[0008] Obtain the document to be reviewed, perform semantic and keyword searches on the document to be reviewed based on the legal knowledge base, and sort the search results by weight according to the level of legal validity and timeliness to obtain the legal constraint set;

[0009] The document to be reviewed and the set of legal constraints are dynamically concatenated into the prompt word template, and a mandatory reference constraint instruction is injected to generate constraint prompt words;

[0010] The constraint prompts are input into a large language model for compliance reasoning. If a regulatory conflict is detected, the conflict is resolved according to the level of regulatory effectiveness to determine the compliance status.

[0011] A confidence level assessment is performed on the compliance status, and a two-way traceable audit evidence chain is generated between the document to be reviewed and the set of regulatory constraints based on the assessment results to obtain the compliance review results.

[0012] Furthermore, the method for obtaining the aforementioned regulatory knowledge base includes:

[0013] Obtain multi-source heterogeneous regulatory documents, perform optical character recognition and hierarchical parsing on the source files of the regulatory documents to obtain a hierarchical tree structure. The nodes of the hierarchical tree structure represent the structural identifiers of chapters, sections, articles, and clauses, and the edges of the hierarchical tree structure represent inclusion relationships.

[0014] The text content is split into a sentence sequence, and the sentences in the sentence sequence are transformed into dense vectors through a pre-trained sentence vector model. The cosine similarity between adjacent sentences is calculated sequentially based on the dense vectors. If the cosine similarity is less than a preset semantic mutation threshold and the current cumulative text character length is greater than or equal to a preset minimum length, then the block boundary is defined at the corresponding position. The pre-trained sentence vector model is a dual encoder architecture.

[0015] The sentence sequence is segmented based on the segmentation boundary to obtain legal text blocks. The block text, block vector, and metadata vector corresponding to the legal text blocks are packaged into triples according to the hierarchical tree structure. The triples are persistently stored in the vector database to obtain the legal knowledge base. The block vector is obtained by encoding the block text by the pre-trained sentence vector model. The metadata vector includes the legal validity level coefficient and the publication time.

[0016] The multi-source heterogeneous regulatory documents include regulatory source files from different levels of validity. These source files include one or more of the following: national laws, administrative regulations, departmental rules, industry standards, local standards, and enterprise internal management systems. These source files from different levels of validity have different document formats and different hierarchical arrangement structures.

[0017] Furthermore, the method for obtaining the set of regulatory constraints includes:

[0018] Key sentences are extracted from the document to be reviewed. The key sentences are input into a pre-trained sentence vector model to obtain a query vector. The semantic similarity score between the query vector and each legal text block in the legal knowledge base is calculated. Candidate legal text blocks are recalled based on the semantic similarity score. The word frequency retrieval score between the document to be reviewed and the candidate legal text blocks is calculated using the BM25 algorithm based on the key sentences. The semantic similarity score and the word frequency retrieval score are weighted and fused to obtain a mixed retrieval score. If the mixed retrieval score is lower than the preset domain matching threshold, it is determined that there is no matching legal constraint between the document to be reviewed and the legal knowledge base. A domain mismatch prompt message is output, and the compliance review is terminated.

[0019] Based on the legal knowledge base, the legal validity level coefficient and publication time are obtained from the metadata corresponding to each legal text block. The validity level correction value is determined, and the timeliness decay value is determined according to the time difference between the publication time and the current time. The smaller the legal validity level coefficient, the higher the validity level of the corresponding legal source document. The validity level correction value increases as the legal validity level coefficient decreases, and the timeliness decay value decreases as the time difference increases.

[0020] The search score is calculated based on the hybrid search score, the effectiveness level correction value, and the timeliness decay value. The legal text blocks are sorted in descending order according to the search score, and the top-K legal text blocks are selected to obtain the legal constraint set.

[0021] Furthermore, the method for obtaining the constraint prompt word includes:

[0022] Based on the key sentences of the document to be reviewed, the context window is expanded forward and backward to extract the text fragments to be reviewed. Based on the text fragments to be reviewed, the block texts and legal clause identifiers corresponding to each legal text block are obtained through the legal constraint set. The legal clause identifiers are generated based on the hierarchical path of the corresponding node in the hierarchical tree structure. The hierarchical path includes a hierarchical combination of chapter identifiers, section identifiers, article identifiers and clause identifiers.

[0023] The text fragment to be reviewed is embedded into the text slot of the preset prompt word template, the block text and the legal clause identifier corresponding to each legal text block are embedded into the legal constraint slot of the preset prompt word template, and the preset compliance reasoning instruction is embedded into the reasoning instruction slot of the preset prompt word template to obtain the assembled prompt word.

[0024] Mandatory citation constraint instructions are injected into the assembly prompts to obtain constraint prompts. The mandatory citation constraint instructions include citation identification instructions, reasoning boundary instructions, external knowledge blocking instructions, and validity determination instructions. The citation identification instructions require that the output results must contain a unique identifier of the cited regulatory block. The reasoning boundary instructions limit the reasoning scope to the set of regulatory constraints. The external knowledge blocking instructions prohibit the introduction of common sense conclusions outside the set of regulatory constraints. The validity determination instructions require that compliance determinations that do not cite regulatory clauses be considered invalid determinations.

[0025] Furthermore, the method for compliant reasoning includes:

[0026] The constraint prompts are input into a large language model, and compliance reasoning is performed through bundle search. The bundle search includes taking the lexical sequence corresponding to the constraint prompts as the initial candidate path, setting the bundle width according to the preset bundle width parameter, and performing autoregressive decoding on the initial candidate path through the large language model under the bundle width constraint. The decoding step includes predicting the conditional probability distribution of the next lexical based on each candidate prefix in the current candidate path set, obtaining the extended lexical corresponding to each candidate prefix, and forming an extended path set based on the combination of each candidate prefix and the corresponding extended lexical.

[0027] Calculate the cumulative conditional probability of each extended path in the extended path set, apply probability penalties to extended paths in the extended path set that do not contain regulatory clause identifiers, sort the extended path set in descending order based on the cumulative conditional probability after penalties, retain the top-ranked candidate prefixes with a preset bundle width parameter as the candidate path set for the next decoding step, repeat the decoding steps until a terminator is generated, and select the constraint-complete path with the highest overall cumulative conditional probability as the compliance analysis reasoning chain.

[0028] Furthermore, the method for obtaining the compliance status includes:

[0029] Based on the compliance analysis reasoning chain, the compliance judgment status and the entity of the behavior to be judged corresponding to each regulatory text block are extracted. The entity of the behavior to be judged is extracted based on the text fragment to be reviewed. The entity of the behavior to be judged is semantically aligned with the entity of the behavior to be reviewed. The set of regulatory text blocks pointing to the same behavior to be reviewed is determined. If the regulatory text blocks at different levels in the set of regulatory text blocks give contradictory compliance judgment statuses, the conflict resolution mechanism is activated.

[0030] Based on the comparison of the regulatory effectiveness level coefficients of the regulatory text blocks according to the conflicting compliance judgment statuses, if the regulatory effectiveness level coefficients are different, the one with the smaller regulatory effectiveness level coefficient is selected as the final compliance judgment status.

[0031] If the coefficients of the legal force level are the same, the publication time of the legal text blocks are compared, and the one with the newer publication time is selected as the final compliance status.

[0032] If the coefficient of the legal force level and the publication time are the same, the one with the larger status value of the legal text block is selected, and the corresponding compliance judgment status is taken as the final compliance status. The status value is assigned according to the preset regulatory strictness. The compliance judgment status includes the violation status and the compliance status. The status value of the violation status is greater than the status value of the compliance status.

[0033] If the legal force level coefficient, publication time, and status value are all the same, a conflict flag will be output.

[0034] Furthermore, the method for obtaining the evaluation results includes:

[0035] Based on the compliance analysis reasoning chain, an entity set is extracted, and entities are extracted from the regulatory constraint set and the document to be reviewed to obtain a regulatory entity set and a document entity set, respectively. The joint overlap degree is calculated based on the entity set, the regulatory entity set, and the document entity set to obtain the entity alignment degree. The joint overlap degree is the ratio of the number of entities in the entity set that exist in both the regulatory entity set and the document entity set to the total number of entities in the entity set.

[0036] The average log-likelihood is calculated based on the generation conditional probability of each term in the compliance analysis reasoning chain to obtain the generation probability score. The chain is then checked to determine whether it contains legal clause citations and to identify the citation integrity index.

[0037] The entity alignment, generation probability score, and citation integrity index are weighted and fused to obtain a confidence score, which is used as the evaluation result. The confidence score is positively correlated with the entity alignment, generation probability score, and citation integrity index.

[0038] Furthermore, the method for obtaining the compliance review results includes:

[0039] Based on the compliance status, extract non-compliant text fragments from the documents pending review as non-compliant anchors, and determine the corresponding regulatory text blocks by reference identifiers according to the compliance analysis reasoning chain;

[0040] Based on the violation anchor points and corresponding legal text blocks, a local sequence optimal comparison is performed using the Smith-Waltman algorithm. A semantic scoring matrix is ​​constructed based on the cosine similarity between the violation anchor points and the semantic units within the corresponding legal text blocks. The scoring function of the semantic scoring matrix is:

[0041] ;

[0042] in The matching score for the semantic scoring matrix, The first of the non-compliant anchor points A semantic unit, The first block of the regulatory text A semantic unit, and These are the word vectors corresponding to semantic units. To preset the semantic matching threshold, It is an indicator function;

[0043] The scoring function is used as the matching score input for dynamic programming recursion, and the locally optimal aligned segment is determined through dynamic programming recursion. The dynamic programming recursion formula is as follows:

[0044] ;

[0045] in For the th in the dynamic programming matrix Line number The local optimal cumulative similarity of the column. This is the gap penalty value. For bandwidth, ;

[0046] The system generates a character offset range for violation anchor points in the document to be reviewed based on the locally optimal alignment fragments, and sets a bidirectional traceable pointer based on the character offset range of the corresponding regulatory text blocks in the regulatory source file. A structured compliance review report is generated based on the confidence score and the bidirectional traceable pointer as the compliance review result. The structured compliance review report includes compliance status, confidence score and bidirectional traceable pointer.

[0047] A second aspect of this invention provides a compliance review system for large language models based on multi-source data fusion, comprising:

[0048] The regulatory knowledge base generation module is used to perform hierarchical structure parsing and semantic segmentation of multi-source heterogeneous regulatory documents, and bind metadata to the segments according to the level of regulatory effectiveness to obtain a regulatory knowledge base.

[0049] Regulatory constraint set generation module: used to obtain the document to be reviewed, perform semantic and keyword searches on the document to be reviewed based on the legal knowledge base, and weight and sort the search results according to the level of legal validity and timeliness to obtain the regulatory constraint set;

[0050] Constraint prompt word generation module: used to dynamically concatenate the document to be reviewed and the set of legal constraints to the prompt word template, and inject mandatory reference constraint instructions to generate constraint prompt words;

[0051] Compliance status determination module: It is used to input the constraint prompt words into the large language model for compliance reasoning. If a regulatory conflict is detected, the conflict is resolved according to the level of regulatory effectiveness to determine the compliance status.

[0052] Compliance review result output module: used to assess the confidence level of the compliance status, generate a two-way traceable audit evidence chain between the document to be reviewed and the set of regulatory constraints based on the assessment results, and obtain the compliance review results.

[0053] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0054] This invention performs hierarchical structure parsing and semantic segmentation on multi-source heterogeneous regulatory documents, binding the regulatory validity level and publication time to segmented metadata. Based on regulatory validity, the search results are tilted towards high-efficiency, newly published core constraint clauses, improving the recall accuracy and hierarchical conflict resolution capabilities of heterogeneous regulations. In the inference stage, the document to be reviewed and the regulatory constraint set are dynamically concatenated to the prompt word template and a mandatory citation constraint instruction is injected. Probabilistic penalties and pruning are applied to candidate paths that do not contain regulatory clause identifiers, strictly limiting the generation space of the large language model to the range of real regulations obtained from the search, suppressing model illusion and semantic drift. In the output stage, the confidence score of the output of the large language model is calculated based on entity alignment, generation probability, and citation integrity. Local sequence optimal comparison is performed between violation anchors and regulatory text segments to generate bidirectional traceable pointers, enabling bidirectional traceability of the quantitative evaluation results of compliance review conclusions and reducing the cost of manual review. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating the steps of the compliance review method for a large language model based on multi-source data fusion in an embodiment of the present invention. Detailed Implementation

[0056] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0057] Reference Figure 1 As shown, this invention provides a compliance review method for large language models based on multi-source data fusion, including:

[0058] Hierarchical structure parsing and semantic segmentation are performed on multi-source heterogeneous regulatory documents, and metadata is bound to the segments according to the level of regulatory effectiveness to obtain a regulatory knowledge base;

[0059] In the actual assessment, the system imported three source regulatory documents from a multi-source heterogeneous regulatory database: Document 1, Document 2, and Document 3. Document 1, "GHG Protocol Corporate Accounting and Reporting Standard," was a PDF scan with a validity level coefficient of L=1 and a publication date of July 2015. Document 2, ISO 14064-1:2018, "Greenhouse gases – Part 1: Specifications and guidance for the quantification and reporting of greenhouse gas emissions and removals at the organizational level," was a PDF scan with L=3 and a publication date of December 2018. Document 3, "Administrative Measures for Carbon Emission Accounting and Reporting," was a Word document with L=6 and a publication date of January 15, 2024. The system performed OCR text recognition on the source regulatory documents and used a document parser to identify the hierarchical markers of each regulation, constructing a hierarchical tree structure T=(V,E). Node V of Document 1 contained "Chapter 5: Setting Organizational Boundaries" and its sub-node "Operational..." "Boundaries", including definitions Scope1,2,3, where edge E represents the inclusion relationship between chapters and sections, performs semantic segmentation on the source documents of regulations, and splits the regulatory text into sentence sequences S={s1,s2,…,s...} according to punctuation. n The sentence sequence is input into a pre-trained sentence vector model BGE-base with a dual encoder architecture, which outputs a 768-dimensional dense vector, encoding each sentence into a vector v. i And calculate the cosine similarity between adjacent sentences, the formula for which is:

[0060] ;

[0061] like Furthermore, when the current cumulative text length is greater than or equal to 50 characters, block boundaries are defined. In document 1, the similarity between "Companies shall account for and report all Scope 1 and Scope 2 emissions" and the following sentence "A minimum of Scope 1 and Scope 2 is required…" is 0.82, which is higher than the threshold, so they are merged into the same block. However, the similarity between "Scope 3 emissions are optional but recommended…" and the following sentence is 0.38, which is lower than the threshold, and the cumulative length has reached 120 characters, so it is divided into an independent regulatory text block C at this point. k Pack each chunk into a triple (Text k ,e k M k Textk For block text, e k The text is segmented into vectors, which are generated by the sentence vector model. k Obtained by re-encoding, M k For including the coefficient L of the level of legal force k The metadata vector with publication time t is used to persist the triple to the Milvus vector database to obtain the regulatory knowledge base;

[0062] Obtain the document to be reviewed, perform semantic and keyword searches on the document to be reviewed based on the legal knowledge base, and sort the search results by weight according to the level of legal validity and timeliness to obtain the legal constraint set;

[0063] In the actual assessment, the user uploaded the document to be reviewed, "2025 ESG Report" (hereinafter referred to as Report 1), through the client. Section 4.2, "Carbon Emission Management," included the statement: "Based on actual operating conditions, our company has only quantified and disclosed the direct emissions of fossil fuel combustion from Scope 1, totaling 12,500 tons of CO2 equivalent in 2024. Emissions from Scope 2 and Scope 3 are not included in this disclosure due to difficulties in data acquisition." The system extracted key sentences from the paragraph and input them into the sentence vector model to obtain the query vector e. q A mixed retrieval is performed in the vector database based on the query vector. Files 1 and 2 correspond to Report 1, each containing 3 blocks, while file 3 corresponds to 1 block, containing a total of 112 semantic units. The calculation e... q The cosine similarity with the vectors of each regulatory text block, and the semantic similarity score S with the "Scope 1 and Scope 2 required" block of document 1-1. dense =0.2197, S in the block of file 1-2 "Scope 3 optional" dense =0.1510, which is consistent with the S value of the enterprise system block in document 3-1. dense =0.4985. Due to differences in terminology between Chinese and English, the phrase "Scope 1" does not appear directly in any of the blocks of file 2, resulting in a semantic similarity of 0.0000. The word frequency relevance between the key sentence and the candidate blocks is calculated based on the BM25 algorithm, where k1=1.5 and b=0.75. File 3-1 contains high-frequency words such as "Scope 1", "accounting", and "disclosure", so the word frequency retrieval score S is high. BM25 =30.3678, S in file 1-1 BM25 =7.9740, S in files 1-2 BM25 =5.8875, weighted fusion is performed according to the system-configured retrieval weight coefficient α=0.7, and the maximum value of the BM25 score is normalized. The formula for calculating the hybrid retrieval score is as follows:

[0064] ;

[0065] in For mixed search scores, For semantic similarity, the word frequency retrieval score is... ;

[0066] The combined search score for document 1-1 is 0.2326, for document 1-2 it is 0.1638, and for document 3-1 it is 0.6490. After adjusting the combined search scores for effectiveness and timeliness, a time decay constant λ = 0.001 days is taken, and the review date is June 30, 2025. The L... d =1, Δt≈3652 days, then the effectiveness correction value Given that this standard is a core overarching specification in the ESG field, the system will address L... d For regulations with a value ≤2, a time decay exemption mechanism is activated, with λ=0.00001, and the time decay value e is adjusted accordingly. −0.00001×3652 ≈0.9641, final S corefinal =0.2326×1.0×0.9641≈0.2243, for file 1-2 S corefinal =0.1638×1.0×0.9641≈0.1573, for each block L of file 2 d =3, Δt≈2403 days, effectiveness correction value The timeliness decay value is 0.0904. Since the mixed retrieval score is zero, the final S corefinal =0.0000, for file 3-1, L d =6, Δt≈532, effectiveness correction value The time-related decay value is 0.5874, and the final S corefinal =0.6490×0.4893×0.5874≈0.1866, the combined search scores of files 1-1, 1-2, and 3-1 are all higher than the preset domain matching threshold of 0.10. Each block of file 2 is filtered out due to its low score. (Please click S...) corefinal Sort the data in descending order, select the top K=5 blocks of regulatory texts, construct a set of regulatory constraints, and recall 3 blocks, including file 1-1 (0.2243), file 3-1 (0.1866), and file 1-2 (0.1573).

[0067] The document to be reviewed and the set of legal constraints are dynamically concatenated into the prompt word template, and a mandatory reference constraint instruction is injected to generate constraint prompt words;

[0068] In the actual assessment, the system extracted the text fragment to be reviewed: "Based on actual operating conditions, our company has only quantified and disclosed direct emissions of fossil fuel combustion within Scope 1, totaling 12,500 tons of CO2 equivalent in 2024. Scope 2 and Scope 3 emissions are not included in this disclosure due to data acquisition difficulties." Based on regulatory constraints, the system centrally retrieved the corresponding text blocks and regulatory clause identifiers. Text 1's ID: GHG_1_1 corresponds to "Companies shall account for and report all Scope 1 and Scope 2 emissions.", with the hierarchical path being Chapter 5 - Operational Boundaries. Text 2's ID: F1_GRP_3_1 corresponds to "Each subsidiary of the group only needs to conduct annual accounting for direct emissions within Scope 1 and disclose them in the ESG report. Scope 2 and Scope 3 emissions are not included in the mandatory disclosure scope due to high data acquisition costs.", with the hierarchical path being Chapter 3 - Section 2. Text 3's ID: GHG_1_2 corresponds to "Scope 3 emissions are optional but recommended for completeness.", with the hierarchical path being Chapter 3. 5-Scope 3Guidance invokes a preset prompt word template to embed the text fragment to be reviewed into the text slot to be reviewed, specifying Text1, Text2, and Text3 as the text slots. 3和 The ID is embedded in the regulatory constraint slot, and the preset compliance reasoning instruction is embedded in the reasoning instruction slot to obtain the assembly prompt word. Then, the mandatory reference constraint instruction "The output result must contain the unique identifier of the referenced regulatory block", the reasoning boundary instruction "The reasoning scope is limited to the above regulatory constraint set", the external knowledge blocking instruction "It is forbidden to introduce common sense conclusions outside the regulatory constraint set", and the validity judgment instruction "Compliance judgments that do not cite regulatory clause identifiers are considered invalid judgments" are injected to generate the constraint prompt word P. rompt ;

[0069] The constraint prompts are input into a large language model for compliance reasoning. If a regulatory conflict is detected, the conflict is resolved according to the level of regulatory effectiveness to determine the compliance status.

[0070] In actual assessment, the constraint cue word P will be used. romptThe input is a large language model LLaMA-70B deployed on a computing server, and compliance inference is performed through beam search. A preset beam width parameter B=3 is set. In each autoregressive decoding step, the model predicts the conditional probability distribution of the next lexical based on the current candidate prefix. When generating the first analysis lexical, the cumulative log probability of candidate prefix 1 "Upon review, the pending report only discloses Scope 1 emissions, which does not comply with [REF:GHG_1_1] requirements..." is -1.8; the cumulative log probability of candidate prefix 2 "According to analysis, this disclosure complies with the group's internal requirements..." is -2.1 (containing the identifier, it is retained); and the cumulative log probability of candidate prefix 3 "From the perspective of ESG best practices..." is -2.0 (not containing the regulatory clause identifier, triggering a probability penalty, set to -∞, and pruning). The system retains the three candidate prefixes with the highest conditional probabilities containing the identifier for the next decoding step, repeating the inference process until a terminator is generated. The complete constraint path with the highest overall cumulative conditional probability is selected as the compliance analysis inference chain. "Upon review, the pending ESG report only discloses Scope 1 direct emissions, without disclosing Scope..." 2. Emissions. According to [REF:GHG_1_1], the organization must account for and report all Scope 1 and Scope 2 emissions. Therefore, disclosing only Scope 1 does not comply with the requirements of this international standard. Meanwhile, [REF:F1_GRP_3_1] allows for the disclosure of only Scope 1, but this standard is an internal corporate policy. Judgment: Non-compliant. Based on the aforementioned chain, the compliance judgment status is extracted. Due to the omission of Scope 2, GHG_1_1 is non-compliant, with a status value of 2. Due to compliance with internal group requirements, F1_GRP_3_1 is compliant, with a status value of 0. The entity of the judged behavior, "disclosing only Scope 1 emissions," is extracted. The entity of the behavior to be reviewed, "quantified and disclosed only direct emissions of Scope 1," is extracted from the text fragment to be reviewed. Semantic alignment is performed, and the entity vector cosine similarity is 0.97, confirming that they point to the same behavior to be reviewed. The conflict resolution mechanism is initiated, and the regulatory validity level coefficient is compared. GHG =1 <L F1-GRP =6, based on the principle that higher-level laws prevail over lower-level laws, select document 1 with the lower level of legal force coefficient, and output "non-compliant" as the final compliance status;

[0071] A confidence level assessment is performed on the compliance status, and a two-way traceable audit evidence chain is generated between the document to be reviewed and the set of regulatory constraints based on the assessment results to obtain the compliance review results.

[0072] In actual assessments, entity set E is extracted based on the compliance analysis reasoning chain. out ={This Company, Scope1, Scope2, Scope3, Accounting, Disclosure, 12,500 tons, CO2 equivalent}, extracting the regulatory entity set E based on the regulatory constraint set. law={Scope1,Scope2,Scope3,account,report,Scope1,Scope2,Scope3,Accounting,Disclosure}, extract the document entity set E from the document to be reviewed. doc ={Company, Scope1, Scope2, Scope3, Direct Emissions, Accounting, Disclosure, 12,500 tons, CO2 Equivalent, Fossil Fuel}, calculate the joint overlap ratio, the formula for calculating the joint overlap ratio is:

[0073] ;

[0074] Extract each term from the inference chain, calculate the average log-likelihood and normalize it. The typical range of each term is [−2,0]. The generation probability score is 0.9250. The chain contains [REF:GHG_1_1] and [REF:F1_GRP_3_1]. The citation integrity index I=1. Weighted fusion is performed according to the fusion weights configured in the system, where the generation probability weight w1=0.3, entity alignment weight w2=0.4, and citation integrity weight w3=0.3. The confidence score CS=0.3×0.9250+0.4×0.6250+0.3×1=0.8275. Based on the final compliance status "non-compliant", the violation anchor point is located from the text fragment to be reviewed as "Only the direct emissions of fossil fuel combustion from Scope 1 were quantitatively calculated and disclosed, totaling 12,500 tons of CO2 equivalent in 2024. Scope 2 and Scope..." 3. Due to difficulties in data acquisition, emissions data is not included in this disclosure. Based on the citation identifier, the corresponding regulatory text block is identified as document 1-1, GHG_1_1. The violation anchor points and target regulatory text blocks are semantically segmented to obtain semantic unit sequences. The input sentence vector model is encoded as word vectors, and a semantic scoring matrix is ​​constructed based on the word vectors. A scoring function is then calculated, with a preset semantic matching threshold. The cosine of anchor point “Scope 1” and regulation “Scope 1” is 1.0, and S is 1.0; the cosine of “emissions” and “emissions” is 0.88, and S is 0.94; the cosine of “12500 tons” and “shall” is 0.05 and < 0.6, and S is 0. The S score is used as the matching score input, and the local optimal alignment is determined recursively through dynamic programming, where the gap penalty value W... gap=0.5, bandwidth limit is 10, recursively obtain the optimal path, the optimal path passes through alignment points such as "Scope 1↔Scope 1", "emissions↔emissions", "accounting↔account", etc. Based on the local optimal alignment fragment, generate the character offset range [12,43] of the violation anchor point in the document to be reviewed, corresponding to "only Scope 1 direct emissions... were quantitatively calculated and disclosed". According to the character offset range [43,50] of the regulatory text block in the regulatory source file, set a bidirectional traceable pointer. Clicking "Scope 1" in Section 4.2 of Report 1 will jump to the original text of Chapter 5 of GHG Protocol in file 1. Clicking "Scope 1" in the original regulatory text will jump back to Section 4.2 of Report 1, generating a structured compliance review report. The structured compliance review report includes "Compliance status: non-compliant, confidence score: 0.8275, medium to high risk, it is recommended to supplement the disclosure of Scope 2 data, violation anchor point: only Scope 1 in Section 4.2 of Report 1" 1 Direct emissions…”, highlighted, cited regulation: [REF:GHG_1_1], two-way traceability pointer: Report 1 [12:43] ↔ Document 1 [43:50], conflict resolution record: corporate policy document 3 allows disclosure of only Scope 1, which conflicts with the superior international standard document 1. Document 1 has been given priority according to its level of effectiveness and has been determined to be non-compliant.

[0075] In this embodiment, the method for obtaining the regulatory knowledge base includes:

[0076] Obtain multi-source heterogeneous regulatory documents, perform optical character recognition and hierarchical parsing on the source files of the regulatory documents to obtain a hierarchical tree structure. The nodes of the hierarchical tree structure represent the structural identifiers of chapters, sections, articles, and clauses, and the edges of the hierarchical tree structure represent inclusion relationships.

[0077] The text content is split into a sentence sequence, and the sentences in the sentence sequence are transformed into dense vectors through a pre-trained sentence vector model. The cosine similarity between adjacent sentences is calculated sequentially based on the dense vectors. If the cosine similarity is less than a preset semantic mutation threshold and the current cumulative text character length is greater than or equal to a preset minimum length, then the block boundary is defined at the corresponding position. The pre-trained sentence vector model is a dual encoder architecture.

[0078] The sentence sequence is segmented based on the segmentation boundary to obtain legal text blocks. The block text, block vector, and metadata vector corresponding to the legal text blocks are packaged into triples according to the hierarchical tree structure. The triples are persistently stored in the vector database to obtain the legal knowledge base. The block vector is obtained by encoding the block text by the pre-trained sentence vector model. The metadata vector includes the legal validity level coefficient and the publication time.

[0079] The multi-source heterogeneous regulatory documents include regulatory source files from different levels of validity. These source files include one or more of the following: national laws, administrative regulations, departmental rules, industry standards, local standards, and enterprise internal management systems. These source files from different levels of validity have different document formats and different hierarchical arrangement structures.

[0080] In this embodiment, the method for obtaining the set of regulatory constraints includes:

[0081] Key sentences are extracted from the document to be reviewed. The key sentences are input into a pre-trained sentence vector model to obtain a query vector. The semantic similarity score between the query vector and each legal text block in the legal knowledge base is calculated. Candidate legal text blocks are recalled based on the semantic similarity score. The word frequency retrieval score between the document to be reviewed and the candidate legal text blocks is calculated using the BM25 algorithm based on the key sentences. The semantic similarity score and the word frequency retrieval score are weighted and fused to obtain a mixed retrieval score. If the mixed retrieval score is lower than the preset domain matching threshold, it is determined that there is no matching legal constraint between the document to be reviewed and the legal knowledge base. A domain mismatch prompt message is output, and the compliance review is terminated.

[0082] Based on the legal knowledge base, the legal validity level coefficient and publication time are obtained from the metadata corresponding to each legal text block. The validity level correction value is determined, and the timeliness decay value is determined according to the time difference between the publication time and the current time. The smaller the legal validity level coefficient, the higher the validity level of the corresponding legal source document. The validity level correction value increases as the legal validity level coefficient decreases, and the timeliness decay value decreases as the time difference increases.

[0083] The search score is calculated based on the hybrid search score, the effectiveness level correction value, and the timeliness decay value. The legal text blocks are sorted in descending order according to the search score, and the top-K legal text blocks are selected to obtain the legal constraint set.

[0084] In this embodiment, the method for obtaining the constraint prompt word includes:

[0085] Based on the key sentences of the document to be reviewed, the context window is expanded forward and backward to extract the text fragments to be reviewed. Based on the text fragments to be reviewed, the block texts and legal clause identifiers corresponding to each legal text block are obtained through the legal constraint set. The legal clause identifiers are generated based on the hierarchical path of the corresponding node in the hierarchical tree structure. The hierarchical path includes a hierarchical combination of chapter identifiers, section identifiers, article identifiers and clause identifiers.

[0086] The text fragment to be reviewed is embedded into the text slot of the preset prompt word template, the block text and the legal clause identifier corresponding to each legal text block are embedded into the legal constraint slot of the preset prompt word template, and the preset compliance reasoning instruction is embedded into the reasoning instruction slot of the preset prompt word template to obtain the assembled prompt word.

[0087] Mandatory citation constraint instructions are injected into the assembly prompts to obtain constraint prompts. The mandatory citation constraint instructions include citation identification instructions, reasoning boundary instructions, external knowledge blocking instructions, and validity determination instructions. The citation identification instructions require that the output results must contain a unique identifier of the cited regulatory block. The reasoning boundary instructions limit the reasoning scope to the set of regulatory constraints. The external knowledge blocking instructions prohibit the introduction of common sense conclusions outside the set of regulatory constraints. The validity determination instructions require that compliance determinations that do not cite regulatory clauses be considered invalid determinations.

[0088] In this embodiment, the compliant reasoning method includes:

[0089] The constraint prompts are input into a large language model, and compliance reasoning is performed through bundle search. The bundle search includes taking the lexical sequence corresponding to the constraint prompts as the initial candidate path, setting the bundle width according to the preset bundle width parameter, and performing autoregressive decoding on the initial candidate path through the large language model under the bundle width constraint. The decoding step includes predicting the conditional probability distribution of the next lexical based on each candidate prefix in the current candidate path set, obtaining the extended lexical corresponding to each candidate prefix, and forming an extended path set based on the combination of each candidate prefix and the corresponding extended lexical.

[0090] Calculate the cumulative conditional probability of each extended path in the extended path set, apply probability penalties to extended paths in the extended path set that do not contain regulatory clause identifiers, sort the extended path set in descending order based on the cumulative conditional probability after penalties, retain the top-ranked candidate prefixes with a preset bundle width parameter as the candidate path set for the next decoding step, repeat the decoding steps until a terminator is generated, and select the constraint-complete path with the highest overall cumulative conditional probability as the compliance analysis reasoning chain.

[0091] In this embodiment, the method for obtaining the compliance status includes:

[0092] Based on the compliance analysis reasoning chain, the compliance judgment status and the entity of the behavior to be judged corresponding to each regulatory text block are extracted. The entity of the behavior to be judged is extracted based on the text fragment to be reviewed. The entity of the behavior to be judged is semantically aligned with the entity of the behavior to be reviewed. The set of regulatory text blocks pointing to the same behavior to be reviewed is determined. If the regulatory text blocks at different levels in the set of regulatory text blocks give contradictory compliance judgment statuses, the conflict resolution mechanism is activated.

[0093] Based on the comparison of the regulatory effectiveness level coefficients of the regulatory text blocks according to the conflicting compliance judgment statuses, if the regulatory effectiveness level coefficients are different, the one with the smaller regulatory effectiveness level coefficient is selected as the final compliance judgment status.

[0094] If the coefficients of the legal force level are the same, the publication time of the legal text blocks are compared, and the one with the newer publication time is selected as the final compliance status.

[0095] If the coefficient of the legal force level and the publication time are the same, the one with the larger status value of the legal text block is selected, and the corresponding compliance judgment status is taken as the final compliance status. The status value is assigned according to the preset regulatory strictness. The compliance judgment status includes the violation status and the compliance status. The status value of the violation status is greater than the status value of the compliance status.

[0096] If the legal force level coefficient, publication time, and status value are all the same, a conflict flag will be output.

[0097] In this embodiment, the method for obtaining the evaluation result includes:

[0098] Based on the compliance analysis reasoning chain, an entity set is extracted, and entities are extracted from the regulatory constraint set and the document to be reviewed to obtain a regulatory entity set and a document entity set, respectively. The joint overlap degree is calculated based on the entity set, the regulatory entity set, and the document entity set to obtain the entity alignment degree. The joint overlap degree is the ratio of the number of entities in the entity set that exist in both the regulatory entity set and the document entity set to the total number of entities in the entity set.

[0099] The average log-likelihood is calculated based on the generation conditional probability of each term in the compliance analysis reasoning chain to obtain the generation probability score. The chain is then checked to determine whether it contains legal clause citations and to identify the citation integrity index.

[0100] The entity alignment, generation probability score, and citation integrity index are weighted and fused to obtain a confidence score, which is used as the evaluation result. The confidence score is positively correlated with the entity alignment, generation probability score, and citation integrity index.

[0101] In this embodiment, the method for obtaining the compliance review result includes:

[0102] Based on the compliance status, extract non-compliant text fragments from the documents pending review as non-compliant anchors, and determine the corresponding regulatory text blocks by reference identifiers according to the compliance analysis reasoning chain;

[0103] Based on the violation anchor points and corresponding legal text blocks, a local sequence optimal comparison is performed using the Smith-Waltman algorithm. A semantic scoring matrix is ​​constructed based on the cosine similarity between the violation anchor points and the semantic units within the corresponding legal text blocks. The scoring function of the semantic scoring matrix is:

[0104] ;

[0105] in The matching score for the semantic scoring matrix, The first of the non-compliant anchor points A semantic unit, The first block of the regulatory text A semantic unit, and These are the word vectors corresponding to semantic units. To preset the semantic matching threshold, It is an indicator function;

[0106] The scoring function is used as the matching score input for dynamic programming recursion, and the locally optimal aligned segment is determined through dynamic programming recursion. The dynamic programming recursion formula is as follows:

[0107] ;

[0108] in For the th in the dynamic programming matrix Line number The local optimal cumulative similarity of the column. This is the gap penalty value. For bandwidth, ;

[0109] The system generates a character offset range for violation anchor points in the document to be reviewed based on the locally optimal alignment fragments, and sets a bidirectional traceable pointer based on the character offset range of the corresponding regulatory text blocks in the regulatory source file. A structured compliance review report is generated based on the confidence score and the bidirectional traceable pointer as the compliance review result. The structured compliance review report includes compliance status, confidence score and bidirectional traceable pointer.

[0110] A second aspect of this invention also provides a compliance review system for large language models based on multi-source data fusion, comprising:

[0111] The regulatory knowledge base generation module is used to perform hierarchical structure parsing and semantic segmentation of multi-source heterogeneous regulatory documents, and bind metadata to the segments according to the level of regulatory effectiveness to obtain a regulatory knowledge base.

[0112] Regulatory constraint set generation module: used to obtain the document to be reviewed, perform semantic and keyword searches on the document to be reviewed based on the legal knowledge base, and weight and sort the search results according to the level of legal validity and timeliness to obtain the regulatory constraint set;

[0113] Constraint prompt word generation module: used to dynamically concatenate the document to be reviewed and the set of legal constraints to the prompt word template, and inject mandatory reference constraint instructions to generate constraint prompt words;

[0114] Compliance status determination module: It is used to input the constraint prompt words into the large language model for compliance reasoning. If a regulatory conflict is detected, the conflict is resolved according to the level of regulatory effectiveness to determine the compliance status.

[0115] Compliance review result output module: used to assess the confidence level of the compliance status, generate a two-way traceable audit evidence chain between the document to be reviewed and the set of regulatory constraints based on the assessment results, and obtain the compliance review results.

[0116] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A compliance review method for large language models based on multi-source data fusion, characterized in that, Includes the following steps: Hierarchical structure parsing and semantic segmentation are performed on multi-source heterogeneous regulatory documents, and metadata is bound to the segments according to the level of regulatory effectiveness to obtain a regulatory knowledge base; Obtain the document to be reviewed, perform semantic and keyword searches on the document to be reviewed based on the legal knowledge base, and sort the search results by weight according to the level of legal validity and timeliness to obtain the legal constraint set; The document to be reviewed and the set of legal constraints are dynamically concatenated into the prompt word template, and a mandatory reference constraint instruction is injected to generate constraint prompt words; The constraint prompts are input into a large language model for compliance reasoning. If a regulatory conflict is detected, the conflict is resolved according to the level of regulatory effectiveness to determine the compliance status. A confidence level assessment is performed on the compliance status, and a two-way traceable audit evidence chain is generated between the document to be reviewed and the set of regulatory constraints based on the assessment results to obtain the compliance review results.

2. The compliance review method for large language models based on multi-source data fusion according to claim 1, characterized in that, Methods for obtaining the aforementioned regulatory knowledge base include: Obtain multi-source heterogeneous regulatory documents, perform optical character recognition and hierarchical parsing on the source files of the regulatory documents to obtain a hierarchical tree structure. The nodes of the hierarchical tree structure represent the structural identifiers of chapters, sections, articles, and clauses, and the edges of the hierarchical tree structure represent inclusion relationships. The text content is split into a sentence sequence, and the sentences in the sentence sequence are transformed into dense vectors through a pre-trained sentence vector model. The cosine similarity between adjacent sentences is calculated sequentially based on the dense vectors. If the cosine similarity is less than a preset semantic mutation threshold and the current cumulative text character length is greater than or equal to a preset minimum length, then the block boundary is defined at the corresponding position. The pre-trained sentence vector model is a dual encoder architecture. The sentence sequence is segmented based on the segmentation boundary to obtain legal text blocks. The block text, block vector, and metadata vector corresponding to the legal text blocks are packaged into triples according to the hierarchical tree structure. The triples are persistently stored in the vector database to obtain the legal knowledge base. The block vector is obtained by encoding the block text by the pre-trained sentence vector model. The metadata vector includes the legal validity level coefficient and the publication time. The multi-source heterogeneous regulatory documents include regulatory source files from different levels of validity. These source files include one or more of the following: national laws, administrative regulations, departmental rules, industry standards, local standards, and enterprise internal management systems. These source files from different levels of validity have different document formats and different hierarchical arrangement structures.

3. The compliance review method for large language models based on multi-source data fusion according to claim 1, characterized in that, The method for obtaining the set of regulatory constraints includes: Key sentences are extracted from the document to be reviewed. The key sentences are input into a pre-trained sentence vector model to obtain a query vector. The semantic similarity score between the query vector and each legal text block in the legal knowledge base is calculated. Candidate legal text blocks are recalled based on the semantic similarity score. The word frequency retrieval score between the document to be reviewed and the candidate legal text blocks is calculated using the BM25 algorithm based on the key sentences. The semantic similarity score and the word frequency retrieval score are weighted and fused to obtain a mixed retrieval score. If the mixed retrieval score is lower than the preset domain matching threshold, it is determined that there is no matching legal constraint between the document to be reviewed and the legal knowledge base. A domain mismatch prompt message is output, and the compliance review is terminated. Based on the legal knowledge base, the legal validity level coefficient and publication time are obtained from the metadata corresponding to each legal text block. The validity level correction value is determined, and the timeliness decay value is determined according to the time difference between the publication time and the current time. The smaller the legal validity level coefficient, the higher the validity level of the corresponding legal source document. The validity level correction value increases as the legal validity level coefficient decreases, and the timeliness decay value decreases as the time difference increases. The search score is calculated based on the hybrid search score, the effectiveness level correction value, and the timeliness decay value. The legal text blocks are sorted in descending order according to the search score, and the top-K legal text blocks are selected to obtain the legal constraint set.

4. The compliance review method for large language models based on multi-source data fusion according to claim 1, characterized in that, The method for obtaining the constraint prompt words includes: Based on the key sentences of the document to be reviewed, the context window is expanded forward and backward to extract the text fragments to be reviewed. Based on the text fragments to be reviewed, the block texts and legal clause identifiers corresponding to each legal text block are obtained through the legal constraint set. The legal clause identifiers are generated based on the hierarchical path of the corresponding node in the hierarchical tree structure. The hierarchical path includes a hierarchical combination of chapter identifiers, section identifiers, article identifiers and clause identifiers. The text fragment to be reviewed is embedded into the text slot of the preset prompt word template, the block text and the legal clause identifier corresponding to each legal text block are embedded into the legal constraint slot of the preset prompt word template, and the preset compliance reasoning instruction is embedded into the reasoning instruction slot of the preset prompt word template to obtain the assembled prompt word. Mandatory citation constraint instructions are injected into the assembly prompts to obtain constraint prompts. The mandatory citation constraint instructions include citation identification instructions, reasoning boundary instructions, external knowledge blocking instructions, and validity determination instructions. The citation identification instructions require that the output results must contain a unique identifier of the cited regulatory block. The reasoning boundary instructions limit the reasoning scope to the set of regulatory constraints. The external knowledge blocking instructions prohibit the introduction of common sense conclusions outside the set of regulatory constraints. The validity determination instructions require that compliance determinations that do not cite regulatory clauses be considered invalid determinations.

5. The compliance review method for large language models based on multi-source data fusion according to claim 1, characterized in that, The method of compliant reasoning includes: The constraint prompts are input into a large language model, and compliance reasoning is performed through bundle search. The bundle search includes taking the lexical sequence corresponding to the constraint prompts as the initial candidate path, setting the bundle width according to the preset bundle width parameter, and performing autoregressive decoding on the initial candidate path through the large language model under the bundle width constraint. The decoding step includes predicting the conditional probability distribution of the next lexical based on each candidate prefix in the current candidate path set, obtaining the extended lexical corresponding to each candidate prefix, and forming an extended path set based on the combination of each candidate prefix and the corresponding extended lexical. Calculate the cumulative conditional probability of each extended path in the extended path set, apply probability penalties to extended paths in the extended path set that do not contain regulatory clause identifiers, sort the extended path set in descending order based on the cumulative conditional probability after penalties, retain the top-ranked candidate prefixes with a preset bundle width parameter as the candidate path set for the next decoding step, repeat the decoding steps until a terminator is generated, and select the constraint-complete path with the highest overall cumulative conditional probability as the compliance analysis reasoning chain.

6. The compliance review method for large language models based on multi-source data fusion according to claim 1, characterized in that, The method for obtaining the compliance status includes: Based on the compliance analysis reasoning chain, the compliance judgment status and the entity of the behavior to be judged corresponding to each regulatory text block are extracted. The entity of the behavior to be judged is extracted based on the text fragment to be reviewed. The entity of the behavior to be judged is semantically aligned with the entity of the behavior to be reviewed. The set of regulatory text blocks pointing to the same behavior to be reviewed is determined. If the regulatory text blocks at different levels in the set of regulatory text blocks give contradictory compliance judgment statuses, the conflict resolution mechanism is activated. Based on the comparison of the regulatory effectiveness level coefficients of the regulatory text blocks according to the conflicting compliance judgment statuses, if the regulatory effectiveness level coefficients are different, the one with the smaller regulatory effectiveness level coefficient is selected as the final compliance judgment status. If the coefficients of the legal force level are the same, the publication time of the legal text blocks are compared, and the one with the newer publication time is selected as the final compliance status. If the coefficient of the legal force level and the publication time are the same, the one with the larger status value of the legal text block is selected, and the corresponding compliance judgment status is taken as the final compliance status. The status value is assigned according to the preset regulatory strictness. The compliance judgment status includes the violation status and the compliance status. The status value of the violation status is greater than the status value of the compliance status. If the legal force level coefficient, publication time, and status value are all the same, a conflict flag will be output.

7. The compliance review method for large language models based on multi-source data fusion according to claim 1, characterized in that, The method for obtaining the evaluation results includes: Based on the compliance analysis reasoning chain, an entity set is extracted, and entities are extracted from the regulatory constraint set and the document to be reviewed to obtain a regulatory entity set and a document entity set, respectively. The joint overlap degree is calculated based on the entity set, the regulatory entity set, and the document entity set to obtain the entity alignment degree. The joint overlap degree is the ratio of the number of entities in the entity set that exist in both the regulatory entity set and the document entity set to the total number of entities in the entity set. The average log-likelihood is calculated based on the generation conditional probability of each term in the compliance analysis reasoning chain to obtain the generation probability score. The chain is then checked to determine whether it contains legal clause citations and to identify the citation integrity index. The entity alignment, generation probability score, and citation integrity index are weighted and fused to obtain a confidence score, which is used as the evaluation result. The confidence score is positively correlated with the entity alignment, generation probability score, and citation integrity index.

8. The compliance review method for large language models based on multi-source data fusion according to claim 1, characterized in that, The method for obtaining the compliance review results includes: Based on the compliance status, extract non-compliant text fragments from the documents pending review as non-compliant anchors, and determine the corresponding regulatory text blocks by reference identifiers according to the compliance analysis reasoning chain; Based on the violation anchor points and corresponding legal text blocks, a local sequence optimal comparison is performed using the Smith-Waltman algorithm. A semantic scoring matrix is ​​constructed based on the cosine similarity between the violation anchor points and the semantic units within the corresponding legal text blocks. The scoring function of the semantic scoring matrix is: ； in The matching score for the semantic scoring matrix, The first of the non-compliant anchor points A semantic unit, The first block of the regulatory text A semantic unit, and These are the word vectors corresponding to semantic units. To preset the semantic matching threshold, It is an indicator function; The scoring function is used as the matching score input for dynamic programming recursion, and the locally optimal aligned segment is determined through dynamic programming recursion. The dynamic programming recursion formula is as follows: ； in For the th in the dynamic programming matrix Line number The local optimal cumulative similarity of the column. This is the gap penalty value. For bandwidth, ; The system generates a character offset range for violation anchor points in the document to be reviewed based on the locally optimal alignment fragments, and sets a bidirectional traceable pointer based on the character offset range of the corresponding regulatory text blocks in the regulatory source file. A structured compliance review report is generated based on the confidence score and the bidirectional traceable pointer as the compliance review result. The structured compliance review report includes compliance status, confidence score and bidirectional traceable pointer.

9. A compliance review system for large language models based on multi-source data fusion, used to execute the compliance review method for large language models based on multi-source data fusion as described in any one of claims 1 to 8, characterized in that, The system includes: The regulatory knowledge base generation module is used to perform hierarchical structure parsing and semantic segmentation of multi-source heterogeneous regulatory documents, and bind metadata to the segments according to the level of regulatory effectiveness to obtain a regulatory knowledge base. Regulatory constraint set generation module: used to obtain the document to be reviewed, perform semantic search and keyword search on the document to be reviewed based on the legal knowledge base, and sort the search results according to the legal validity level and timeliness to obtain the regulatory constraint set; Constraint prompt word generation module: used to dynamically concatenate the document to be reviewed and the set of legal constraints to the prompt word template, and inject mandatory reference constraint instructions to generate constraint prompt words; Compliance status determination module: It is used to input the constraint prompt words into the large language model for compliance reasoning. If a regulatory conflict is detected, the conflict is resolved according to the level of regulatory effectiveness to determine the compliance status. Compliance review result output module: used to assess the confidence level of the compliance status, generate a two-way traceable audit evidence chain between the document to be reviewed and the set of regulatory constraints based on the assessment results, and obtain the compliance review results.

Images