A policy document oriented retrieval system and method

Abstract

The application discloses a kind of retrieval system and method for policy document, the system includes: implicit attention module, implicit multi semantic query and cross attention aggregation processing are carried out to the word segmentation hidden state sequence of the text to be searched, and the text embedding vector of word segmentation hidden state sequence is obtained;Similarity score conversion module determines the dense score between the text to be searched and each candidate document based on the cosine similarity between the embedding vector of each candidate document and the text embedding vector, wherein each candidate document is stored in candidate document pool in advance;Gradient boosting module carries out posterior feature extraction to the preset policy knowledge graph, candidate document pool and dense score, obtains the multi-dimensional posterior feature vector between the text to be searched and each candidate document, and carries out nonlinear combination modeling to multi-dimensional posterior feature vector, to determine the document ranking score of the preset number of candidate documents related to the text to be searched in candidate document pool.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence processing technology, and more specifically to a retrieval system and method for policy documents. Background Technology

[0002] With the widespread application of large language models in scenarios such as policy question answering and policy compliance review, the quality of semantic recall and ranking has become a core bottleneck restricting system reliability. Policy texts have highly homogeneous language expressions (such as "encourage...", "support...", "eligible enterprises..."), and there are many fine-grained differences between different regions and versions of the same policy, which places much higher demands on the semantic discrimination ability of the embedded model than in general scenarios.

[0003] However, existing technologies struggle to capture key information scattered throughout long texts (such as scope of application and subsidy amount) during policy retrieval, especially the implicit logical connections and semantic levels within the text. They fail to highlight the semantic weight of core policy clauses, making it difficult for vector representations to distinguish highly similar policy provisions across regions and versions, and consequently, to accurately obtain documents that match the retrieval target.

[0004] Therefore, existing technologies have problems in the retrieval process, such as inaccurate policy matching and low accuracy of retrieval results due to the difficulty in accurately capturing key semantics and implicit logical relationships in the text. Summary of the Invention

[0005] This invention provides a policy document retrieval system and method, aiming to solve the problems of inaccurate policy matching and low retrieval accuracy caused by the difficulty in accurately capturing key semantics and implicit logical relationships in the text during the retrieval process in existing technologies.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A policy document retrieval system, comprising:

[0008] The implicit attention module performs implicit multi-semantic query and cross-attention aggregation on the word segmentation hidden state sequence of the text to be retrieved, and obtains the text embedding vector of the word segmentation hidden state sequence.

[0009] The similarity score conversion module determines the density score between the text to be retrieved and each candidate document based on the cosine similarity between the embedding vector of each candidate document and the embedding vector of the text, wherein each candidate document is pre-stored in the candidate document pool;

[0010] The gradient boosting module performs posterior feature extraction on the preset policy knowledge graph, the candidate document pool, and the dense score to obtain a multi-dimensional posterior feature vector between the text to be retrieved and each candidate document. It then performs nonlinear combination modeling on the multi-dimensional posterior feature vector to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved.

[0011] Optionally, the policy document retrieval system further includes a backbone coding module, which is used to perform word segmentation and encoding on the text to be retrieved to obtain the word segmentation hidden state sequence.

[0012] Optionally, the policy document retrieval system further includes a retrieval matching module, an exact matching module, an implicit matching module, and a deduplication and merging module;

[0013] The retrieval and matching module is used to obtain sparse matching documents related to the text to be retrieved based on a keyword matching algorithm using word frequency statistics.

[0014] The precise matching module is used to precisely match documents related to the text to be retrieved based on the document number in the text to be retrieved.

[0015] The implicit matching module is used to perform dense vector retrieval based on the text embedding vector to obtain the implicit vector retrieval document;

[0016] The deduplication and merging module is used to deduplicatize and merge the sparse matching documents, the exact matching documents, and the implicit vector retrieval documents to obtain the candidate document pool.

[0017] Optionally, the policy document retrieval system further includes a difficult-to-bear sample construction module; the difficult-to-bear samples include geographically difficult-to-bear samples, temporally difficult-to-bear samples, and structurally difficult-to-bear samples.

[0018] The difficult-to-bear sample construction module obtains cross-regional policy document pairs with similar policy objectives but different applicable regions based on the applicable region field of the preset policy knowledge graph, and determines the cross-regional policy document pairs as the geographical dimension difficult-to-bear samples;

[0019] The unbearable sample construction module marks historical version documents of the currently effective policy as unbearable samples in the time dimension based on the failure mark field, failure date field, and preceding policy relationship field of the preset policy knowledge graph;

[0020] The difficult-to-bear sample construction module, based on the policy tools and audience semantic entity fields of the preset policy knowledge graph, pairs adjacent clauses in the same file that are semantically related but functionally different as the structural dimension difficult-to-bear samples.

[0021] The difficult-to-negative sample construction module is also used to generate a sample training set based on the difficult-to-negative samples, and the training set is used to optimize the parameters of the implicit attention module.

[0022] Optionally, the implicit attention module includes a learnable latent query matrix, a cross-attention projection matrix, and a linear projection layer;

[0023] The learnable latent query matrix is ​​used to determine multiple latent query vectors of the word segmentation hidden state sequence;

[0024] The cross-attention projection matrix is ​​used to perform multi-semantic perspective cross-aggregation on the word segmentation hidden state sequence based on the multiple potential query vectors, so as to obtain the multi-perspective semantic vector of the word segmentation hidden state sequence.

[0025] The linear projection layer is used to flatten and normalize the multi-view semantic vector to obtain the text embedding vector.

[0026] Optionally, the similarity score conversion module includes a candidate document acquisition unit and a cosine similarity calculation unit;

[0027] The candidate document acquisition unit is used to acquire candidate documents related to the text to be retrieved;

[0028] The cosine similarity calculation unit is used to determine the density score based on the cosine similarity between the text embedding vector and the embedding vector of the candidate document.

[0029] Optionally, the gradient enhancement module includes an authority unit, a timeliness unit, an applicability unit, a retrieval signal unit, a posterior feature synthesis unit, and a ranking unit;

[0030] The authority unit is used to obtain the authority field in the preset policy knowledge graph, and to perform hierarchical encoding and matrix combination cross-processing on the authority field to obtain the authority score;

[0031] The timeliness unit is used to obtain the timeliness field in the preset policy knowledge graph, and to perform timeliness quantification on the timeliness field through multi-dimensional rules to obtain a timeliness score;

[0032] The applicability unit is used to obtain the applicability field in the preset policy knowledge graph, and to perform hierarchical regional matching, applicable object matching, audience semantic matching and policy goal and query intent matching on the applicability field to obtain an applicability score;

[0033] The retrieval signal unit is used to perform multi-source signal fusion on the candidate document retrieval score and the dense score of the candidate document pool to obtain the retrieval signal score;

[0034] The posterior feature synthesis unit is used to concatenate the authority score, timeliness score, applicability score, and retrieval signal score to generate the multidimensional posterior feature vector.

[0035] The sorting unit is used to determine a preset number of candidate documents in the candidate document pool that are most relevant to the text to be retrieved based on the multidimensional posterior feature vector, and outputs a sorting score for each candidate document.

[0036] Optionally, the authoritative unit includes an administrative level coding layer, a document type validity coding layer, a two-dimensional authority matrix layer, and a joint issuance correction layer;

[0037] The administrative level coding layer is used to perform hierarchical coding on the issuing unit field in the preset policy knowledge graph to obtain the administrative level coding vector;

[0038] The document type validity coding layer is used to perform validity-level coding on the document type classification field in the preset policy knowledge graph to obtain the document type validity coding vector.

[0039] The two-dimensional authority matrix layer is used to perform matrix combination and cross-interaction of the administrative level coding vector and the document type validity coding vector to generate a two-dimensional authority matrix.

[0040] The joint publication correction layer is used to weight and correct the two-dimensional authority matrix according to the number of joint publication units to obtain the authority score.

[0041] Optionally, the timeliness unit includes an expiration marking layer, an expiration date parsing layer, and a preceding policy parsing layer;

[0042] The failure flag layer is used to set the timeliness score of documents with failure flag fields to zero.

[0043] The expiration date parsing layer is used to calculate the expiration date of non-expired documents and determine the timeliness score based on the time difference between the expiration date and the current time.

[0044] The preceding policy parsing layer is used to reduce the timeliness score of documents in the preset policy knowledge graph that have preceding policy relationship fields.

[0045] A method for retrieving policy documents, comprising:

[0046] Implicit multi-semantic query and cross-attention aggregation are performed on the word segmentation hidden state sequence of the text to be retrieved to obtain the text embedding vector of the word segmentation hidden state sequence;

[0047] The dense score between the text to be retrieved and each candidate document is determined based on the cosine similarity between the embedding vector of each candidate document and the text embedding vector, wherein each candidate document is pre-stored in a candidate document pool;

[0048] Posterior features are extracted from the preset policy knowledge graph, the candidate document pool, and the dense score to obtain a multidimensional posterior feature vector between the text to be retrieved and each candidate document. The multidimensional posterior feature vector is then modeled using a nonlinear combination to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved.

[0049] In this embodiment, an implicit attention module performs implicit multi-semantic querying and cross-attention aggregation on the word segmentation hidden state sequence of the text to be retrieved. This effectively captures the implicit logical connections and semantic levels in the text to be retrieved, thereby generating a text embedding vector rich in deep semantics. A similarity score conversion module compares the cosine similarity between the text embedding vector and the embedding vector of each candidate document. This accurately quantifies the dense scores between the text to be retrieved and each candidate document, enabling fine-grained semantic matching for each candidate document. Finally, a gradient boosting module combines a preset policy knowledge graph, a candidate document pool, and dense scores to extract posterior features, obtaining a multi-dimensional posterior feature vector between the text to be retrieved and each candidate document. The multi-dimensional posterior feature vector is then nonlinearly combined and modeled to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved. This significantly improves the accuracy and semantic understanding depth of policy document retrieval. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this embodiment, the accompanying drawings used in the description of the embodiment will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A schematic block diagram illustrating the structure of an embodiment of the policy document retrieval system provided by the present invention;

[0052] Figure 2 A data reasoning flowchart of an embodiment of the enhanced embedded model provided by the present invention;

[0053] Figure 3 This is a general architecture diagram of an embodiment of the policy document retrieval system provided by the present invention;

[0054] Figure 4This is a schematic flowchart illustrating an embodiment of constructing training data provided by the present invention.

[0055] Figure 5 This is a flowchart illustrating an embodiment of the gradient boosting module provided by the present invention;

[0056] Figure 6 This is a flowchart illustrating an embodiment of the policy document retrieval method provided by the present invention. Detailed Implementation

[0057] The technical solutions in this embodiment will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] In the following description, specific embodiments of the invention will be illustrated with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and computer execution as referred to herein includes operations by a computer processing unit representing electronic signals of data in a structured format. This operation transforms the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise alter the operation of the computer in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the invention described above are not intended to be limiting, and those skilled in the art will understand that many of the steps and operations described below can also be implemented in hardware.

[0059] The terms "module" or "unit" as used herein can be considered as software objects executing on the computing system. The different components, modules, engines, and services described herein can be considered as implementation objects on the computing system. The apparatus and methods described herein are preferably implemented in software, but can also be implemented in hardware, both of which are within the scope of this invention.

[0060] This invention provides a retrieval system and method for policy documents.

[0061] Please see Figure 1 , Figure 1 This is a schematic block diagram of an embodiment of the policy document retrieval system provided by the present invention. The policy document retrieval system 100 includes:

[0062] Implicit attention module 101 performs implicit multi-semantic query and cross-attention aggregation on the word segmentation hidden state sequence of the text to be retrieved, and obtains the text embedding vector of the word segmentation hidden state sequence.

[0063] The similarity score conversion module 102 determines the dense score between the text to be retrieved and each candidate document based on the cosine similarity between the embedding vector of each candidate document and the text embedding vector. Each candidate document is pre-stored in the candidate document pool.

[0064] The gradient boosting module 103 performs posterior feature extraction on the preset policy knowledge graph, candidate document pool, and dense scores to obtain a multi-dimensional posterior feature vector between the text to be retrieved and each candidate document. It then performs nonlinear combination modeling on the multi-dimensional posterior feature vector to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved.

[0065] The text to be retrieved refers to the policy-related query text entered by the user, such as "Does a certain technology-based SME meet the application conditions for the R&D expense deduction policy in City A?", which is the original input for semantic retrieval.

[0066] Hidden state sequence of word segmentation refers to the sequence of word vectors generated after segmenting and encoding the text to be retrieved. Each element corresponds to a dynamic representation of a word segmentation unit in the deep semantic space.

[0067] Implicit multi-semantic query refers to the automatic learning of implicit semantic representations of text from multiple semantic perspectives (such as "application conditions" and "regional restrictions") through end-to-end comparative learning. The specific number of perspectives can be customized according to actual needs and is not limited here.

[0068] Cross-attention aggregation refers to using learnable potential query vectors as the query endpoint to dynamically assign weights and aggregate context-aware information on the word segmentation hidden state sequence in order to adaptively focus on key tokens (such as clause qualifiers) in policy texts, thereby accurately capturing the logical dependencies between policy elements such as "City A", "technology-based SMEs", and "R&D expense deduction".

[0069] In one specific embodiment, latent attention refers to a mechanism that uses a learnable latent query vector matrix (Q_latent) independent of the input sequence as the query endpoint, and weights and aggregates the bidirectional hidden state sequence H output by the backbone encoder through a cross-attention mechanism. Here, the latent query vector is a learnable model parameter independent of the input, which is automatically learned through end-to-end contrastive learning to acquire multiple aggregation perspectives in the policy semantic space, remaining fixed during inference and not changing with the query text.

[0070] The text embedding vector refers to the high-dimensional dense vector generated after processing by the implicit attention module 101. It represents the overall intent and structured element distribution of the text to be retrieved in the policy semantic space. Its dimension is consistent with the hidden layer space of the underlying encoder, which can effectively represent the fine-grained semantics of the policy text.

[0071] Dense score conversion transforms the text embedding vector and the candidate document embedding vector into a quantified semantic matching score by calculating the cosine similarity. This score reflects the deep semantic relevance between the queried text and the candidate documents.

[0072] The pre-defined policy knowledge graph refers to a knowledge base constructed by extracting structured entities and performing semantic analysis on policy documents. It includes structured fields and semantic entities such as issuing unit, applicable region, document number, issuance time, expiration date, policy objective, and applicable object.

[0073] The candidate document pool is a collection of documents that are dynamically retrieved and filtered from the document library and are highly semantically related to the text to be retrieved. Its size is dynamically adjusted according to the actual retrieval needs, usually ranging from 50 to 200 documents, to ensure a balance between retrieval efficiency and accuracy.

[0074] The multidimensional posterior feature vector is a highly discriminative feature representation generated by fusing the structured constraints of the policy knowledge graph, the contextual distribution of the candidate document pool, and the semantic confidence of the dense scores. Each dimension corresponds to the posterior probability estimate of a specific policy dimension (such as regional adaptability, subject compliance, and timeliness effectiveness), supporting the interpretability attribution of the final document ranking score. That is, each candidate document has a corresponding multidimensional posterior feature vector, and the dimensions of the multidimensional posterior feature vector correspond one-to-one with the policy dimensions considered in the actual scenario, without any restrictions.

[0075] In one specific embodiment, a gradient boosting model is used to perform nonlinear combination modeling of multidimensional posterior feature vectors, learn the interaction effects and nonlinear weight allocation between policy dimensions, and output the final ranking score.

[0076] In this embodiment, an implicit attention module performs implicit multi-semantic querying and cross-attention aggregation on the word segmentation hidden state sequence of the text to be retrieved. This effectively captures the implicit logical connections and semantic levels in the text to be retrieved, thereby generating a text embedding vector rich in deep semantics. A similarity score conversion module compares the cosine similarity between the text embedding vector and the embedding vector of each candidate document. This accurately quantifies the dense scores between the text to be retrieved and each candidate document, enabling fine-grained semantic matching for each candidate document. Finally, a gradient boosting module combines a preset policy knowledge graph, a candidate document pool, and dense scores to extract posterior features, obtaining a multi-dimensional posterior feature vector between the text to be retrieved and each candidate document. The multi-dimensional posterior feature vector is then nonlinearly combined and modeled to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved. This significantly improves the accuracy and semantic understanding depth of policy document retrieval.

[0077] To improve the retrieval system's ability to understand and process policy texts, the policy document retrieval system also includes a backbone coding module. This module is used to segment and encode the text to be retrieved, resulting in a segmentation hidden state sequence.

[0078] The backbone encoding module includes an encoder-only architecture text encoding model with a bidirectional attention mechanism. Specifically, it can be a BGE series model (BGE-M3, BGE-large-zh, BGE-base-zh, BGE-v1.5), an E5 series model (multilingual-e5-large, E5-large-v2), a GTE series model (gte-large, gte-Qwen2-7B), etc., without limitation. The following examples use BGE-M3 as the backbone for specific illustration; other backbone encoders with an encoder-only bidirectional attention architecture apply the same parameter freezing strategy and implicit attention module access method, with all backbone parameters remaining frozen, and only the implicit attention module parameters participating in training.

[0079] Specifically, using BGE-M3 as the backbone encoder, the original Dense vector output head of BGE-M3 (based on CLSToken or Mean Pooling) is replaced with a Latent Attention Layer (implicit attention module 101) to construct an enhanced embedding model for the policy domain. That is, the enhanced embedding model retains all the pre-trained parameters of the BGE-M3 backbone encoder, sets a cutoff point after the hidden state sequence H output by the backbone network, and replaces the original CLS Token extraction or Mean Pooling operation after the cutoff point with the implicit attention module 101 (Latent Attention Layer). This allows the model to maintain its general multilingual encoding capabilities while gaining adaptive aggregation capabilities for policy semantics.

[0080] Here, the cutoff point refers to the position where the H sequence output by the backbone network is cut off. After this position, CLS or Mean Pooling is no longer performed. Instead, it is fed into a trainable Latent Attention Layer. Since the vector extraction method is a learnable mechanism that can be dynamically adapted according to actual needs, applying Latent Attention to the bidirectional hidden state of the Encoder has both bidirectional semantic modeling capabilities and a learnable aggregation mechanism.

[0081] Correspondingly, the gradient boosting module 103 can specifically be a GBDT ranking model, whose input features include the dense score output by the implicit attention module 101 and the posterior features extracted from the policy knowledge graph, to achieve supervised fusion ranking of candidate documents, and to realize the function of the policy document retrieval system 100.

[0082] Please see Figure 2 , Figure 2 A data reasoning flowchart for an embodiment of the enhanced embedding model provided by the present invention:

[0083] Input text → Tokenizer → BGE-M3 Encoder (all layers, parameters frozen) → H∈R^{L×d} → [Cutoff point] → Latent Attention Layer (trainable) → v_norm∈R^d

[0084] Among them, Tokenizer is a word segmenter, BGE-M3 Encoder is the BGE-M3 encoder, H is the word segmentation hidden state sequence, R represents the set of real numbers, L is the sequence length, and d is the dimension of the hidden layer of BGE-M3 (default 1024). The text embedding vector v_norm output by the LatentAttention Layer has the same dimension as the standard Dense vector of BGE-M3 and can be directly used for the indexing and retrieval of the vector library. The Sparse and Multi-vector output headers of BGE-M3 itself are not affected, and the rest of the backbone Encoder remains unchanged. It is completely transparent to the downstream recall module and does not require modification of the retrieval side infrastructure.

[0085] In addition, after the input text is segmented by the Tokenizer, it is fed into the backbone network composed of multiple stacked Transformer Encoders, and outputs a complete token-level hidden state sequence H, while retaining the original Tokenizer and all Transformer Encoder layers and their pre-trained weights of BGE-M3 unchanged.

[0086] First, Latent Attention assigns higher attention weights to core policy clause tokens and lower weights to function words, resulting in a higher semantic density in the output vector compared to equal-weighted Mean Pooling. Second, n LatentQuery queries are used in parallel to extract n semantic perspectives (such as applicable objects, geographical scope, subsidy standards, etc.), which are then fused into a single vector via Linear_{nd→d}. This single output vector carries multi-dimensional policy semantic information, improving the fine-grained differentiation capability for cross-regional and cross-version policies during retrieval. In specific policy retrieval scenarios, the quality of the output vector in the fine-grained semantic differentiation task is significantly improved. Specifically, the cosine similarity of similar policies across regions and historical policies across versions in the vector space is significantly reduced, leading to a decrease in retrieval confusion.

[0087] In the process of training the implicit attention module 101, all parameters of BGE-M3 are frozen. Under this constraint, Latent Query must extract policy domain semantics from the fixed BGE-M3 hidden state with only a limited number of trainable parameters, and cannot improve the domain adaptability of the hidden state by fine-tuning the backbone model. Therefore, this constraint needs to be addressed specifically in the initialization and training objective design of Latent Query.

[0088] A parameter freezing strategy is adopted to adapt the enhanced embedding model to the policy domain. The specific scheme is as follows:

[0089] Parameter freezing phase: Freeze all the original parameters of the BGE-M3 Encoder to retain the pre-training capability of BGE-M3 in general multilingual text encoding and avoid the catastrophic forgetting problem caused by domain fine-tuning.

[0090] Trainable parameter range: Only the parameters of the Latent Attention Layer are updated, including the latent query vector matrix Q_latent, the projection matrices of the cross-attention modules (W_Q, W_K, W_V), and the parameters of the output linear projection layer. The number of trainable parameters is extremely small compared to the total number of parameters in the model, resulting in high fine-tuning efficiency.

[0091] Training objective: Using InfoNCE contrastive learning loss as the optimization objective, the Latent Attention Layer is trained by combining policy question-answering labeled data (query-related policy provision positive examples) with policy domain-specific hard negative samples, so that the Latent Attention Layer can learn key aggregation patterns in the policy semantic space.

[0092] Domain transfer scalability: Since only the Latent Attention Layer participates in training, the same BGE-M3 backbone can independently train multiple Latent Attention Heads for different vertical domains (such as law, medical care, and finance), realizing cross-domain reuse of the backbone model.

[0093] Furthermore, the policy document retrieval system also includes a retrieval matching module, an exact matching module, an implicit matching module, and a deduplication and merging module. The retrieval matching module uses a keyword matching algorithm based on word frequency statistics to obtain sparse matching documents related to the text to be retrieved. The exact matching module uses document numbers in the text to accurately match documents related to the text to be retrieved. The implicit matching module performs dense vector retrieval based on text embedding vectors to obtain implicit vector retrieval documents. The deduplication and merging module deduplicates and merges the sparse matching documents, exact matching documents, and implicit vector retrieval documents to obtain a candidate document pool.

[0094] Please see Figure 3 , Figure 3This is an overall architecture diagram of an embodiment of the policy document retrieval system provided by the present invention. User queries first trigger three parallel retrieval paths: generating a sparse candidate set through BM25 sparse retrieval, generating a precise matching candidate set through document number precise matching recall, and generating a dense candidate set through innovation point one "enhanced embedding" (after BGE-M3 Encoder parameter freezing and bidirectional attention processing, the latent query matrix is ​​learned through a Latent Attention Layer and v_norm is obtained through cross-attention aggregation). Subsequently, the three candidate sets are merged into a unified candidate document pool after deduplication, and then enter innovation point two "supervised fusion ranking". The authoritativeness, timeliness, applicability fields and retrieval signals (from the policy-specific dense score) are extracted from the policy knowledge graph as posterior features. After learning nonlinear feature interactions through the GBDT ranking model, the Top-N ranking results are obtained and finally input into the generation layer.

[0095] In one specific embodiment, based on the user input query: "Does a certain technology-based SME meet the application conditions for the R&D expense deduction policy in City A?", the processing flow of the policy document retrieval system is as follows: The query text is encoded into a dense vector using an enhanced embedding model (BGE-M3 + Latent Attention Layer), and a BM25 sparse representation is generated simultaneously. Dense retrieval and sparse retrieval are performed respectively, and the results are merged into a candidate pool after deduplication. Subsequently, the system extracts posterior features for each candidate document from the knowledge graph: In terms of authority, documents jointly issued by multiple units have higher document type validity scores, and the number of jointly issuing units is the corresponding quantity value; in terms of timeliness, the system checks that the expiration marker field is empty, and the document with the latest issuance time has the highest score; in terms of applicability, documents with the province field "A" and the applicable object containing "organization" have the highest scores for both region and subject; clauses containing "technology-based SME" in the audience field have the highest applicability scores; in terms of retrieval signal, the source code of documents recalled by both channels is given an additional score. The above features are input into the GBDT ranking model, which will ultimately prioritize policy provisions that are highly authoritative, timely, applicable to City A, and geared towards technology-based enterprises.

[0096] In this embodiment, the backbone coding module performs word segmentation and encoding on the text to be retrieved, generating a word segmentation hidden state sequence to provide a basic representation for subsequent retrieval. Based on this, the system integrates multiple retrieval mechanisms: the retrieval matching module uses a keyword matching algorithm based on word frequency statistics to obtain relevant sparse matching documents, achieving preliminary screening based on explicit keywords; the precise matching module accurately locates relevant precise matching documents based on document number information in the text to be retrieved, ensuring accurate acquisition of key specific documents; the implicit matching module performs dense vector retrieval through text embedding vectors, mining deep semantic relationships between texts to obtain implicit vector retrieval documents, expanding the comprehensiveness of the retrieval. Finally, the deduplication and merging module performs deduplication and merging processing on sparse matching documents, precise matching documents, and implicit vector retrieval documents, effectively integrating the results of different retrieval methods to form a comprehensive and non-redundant candidate document pool, thereby significantly improving the accuracy, comprehensiveness, and efficiency of policy document retrieval, ensuring that users can quickly obtain all relevant policy document information.

[0097] Furthermore, to optimize the parameters of the implicit attention module 101 and thus improve the retrieval system's ability to distinguish policy documents across complex dimensions such as geographical differences, temporal evolution, and clause structure, the policy document retrieval system also includes a difficult-to-handle sample construction module. Difficult-to-handle samples include geographical, temporal, and structural difficult-to-handle samples. Specifically, the difficult-to-handle sample construction module obtains cross-regional policy document pairs with similar policy objectives but different applicable regions based on the applicable region field of a preset policy knowledge graph, and identifies these cross-regional policy document pairs as geographically difficult-to-handle samples. The difficult-to-handle sample construction module marks historical versions of currently effective policies as temporally difficult-to-handle samples based on the expiration marker field, expiration date field, and preceding policy relationship field of a preset policy knowledge graph. The difficult-to-handle sample construction module pairs semantically related but functionally different adjacent clauses in the same document as structurally difficult-to-handle samples based on the policy tool and audience semantic entity fields of a preset policy knowledge graph. The difficult-to-handle sample construction module is also used to generate a sample training set based on the difficult-to-handle samples, and the training set is used to optimize the parameters of the implicit attention module 101.

[0098] Among them, three types of hard-to-bear samples can be constructed automatically without manual annotation:

[0099] First, difficult-to-bear samples of similar policies across regions (geographical dimension): Utilizing the province / city / district applicable region field in the knowledge graph, the system automatically identifies policy document pairs with highly similar policy objective fields but different applicable region fields, and configures them as difficult-to-bear samples in the geographical dimension. The similarity of policy objective fields is calculated using a pre-trained model with a general backbone coding module that has not undergone policy fine-tuning. For example, for a query of "application conditions for financing subsidies for small and medium-sized enterprises in City A," the system automatically retrieves similar policies with the applicable region of "City B" as difficult-to-bear samples through the applicable region field, without manual judgment. These negative samples are highly similar in language expression and semantic structure, but have different applicable regions, which can effectively improve the model's sensitivity to geographically limited semantics.

[0100] Second, cross-version negative samples of the same policy (time dimension): Utilizing the prior policy relationship field in the knowledge graph, historical versions of currently valid policy documents are automatically identified. Any document with a "prior policy" relationship in the knowledge graph is automatically marked as a negative sample in the time dimension. This is further confirmed by combining the expiration flag and expiration date fields to ensure that the expiration status of historical versions is verifiable. Such negative samples enable the model to learn to distinguish the timeliness differences of policies, avoiding the inclusion of repealed clauses in the search results.

[0101] Third, difficult-to-bear samples of adjacent clauses in the same document (structural dimension): Adjacent clauses in the same policy document that are semantically related but functionally different (such as the "subsidy recipient" clause and the "subsidy amount" clause) are paired as difficult-to-bear samples in the structural dimension. By combining the semantic entity fields of policy tools and audience objects in the knowledge graph, the functional types of clauses can be further distinguished, and the boundaries of clauses can be automatically marked without manual processing of each clause. This type of negative sample requires the model to understand the functional boundaries of policy clauses, rather than relying solely on the semantic similarity of the overall document for matching.

[0102] In this embodiment, by setting up a difficult-to-bear sample construction module, difficult-to-bear samples can be constructed in a targeted manner along the geographical, temporal, and structural dimensions. Specifically, the geographical dimension difficult-to-bear samples obtain cross-regional policy document pairs with similar policy objectives but different applicable regions through the applicable region field of the policy knowledge graph; the temporal dimension difficult-to-bear samples mark historical versions of currently valid policies based on the expiration marker field, expiration date field, and preceding policy relationship field of the policy knowledge graph; and the structural dimension difficult-to-bear samples utilize the policy tools and audience semantic entity fields of the policy knowledge graph to pair semantically related but functionally different adjacent clauses in the same document. The sample training set generated based on these difficult-to-bear samples can be effectively used to optimize the parameters of the implicit attention module 101, thereby improving the retrieval system's ability to distinguish policy documents across complex dimensions such as geographical differences, temporal evolution, and clause structure, and enhancing the accuracy and precision of the retrieval results.

[0103] In addition to hard-to-bear samples, training data is automatically constructed using the structural relationships of knowledge graphs to address the scarcity of labeled data in the policy domain. Specifically, this includes:

[0104] Positive example expansion based on citation relationships: When query Q forms a positive example pair with policy document A, if there is a relationship in the knowledge graph where A cites policy B, then (Q, B) also constitutes a positive example pair—the cited document is the basis of the current policy and is highly relevant to the query. This mechanism can automatically expand the set of positive examples without additional annotation.

[0105] Positive example extension based on detailed rules relationships: When a query matches a general policy document, the detailed policy documents associated with it in the knowledge graph are also highly relevant to the query and can be automatically included in the positive example set. This mechanism solves the problem of missed detections and recalls where the general policy document and the implementation details are semantically related but their vector similarity may be low.

[0106] Negative examples are constructed based on prior relations: when the prior policy field indicates that a document already has a successor version, the prior (already replaced) version is automatically marked as a low-relevance document, forming a training negative example for the timeliness dimension, enabling the GBDT model to learn to sort invalid documents with reduced weight.

[0107] The aforementioned automated annotation mechanism minimizes the cost of constructing training data while ensuring annotation quality—policy relationships in the knowledge graph all derive from explicit textual evidence in policy documents, making them more reliable than weakly supervised signals from user behavior. It should be noted that the training data constructed based on graph relationships reflects the legal citation relevance between policy documents, which may differ somewhat from the relevance definition in actual user queries. Therefore, this invention uses the automated graph-constructed data as a cold-start training set. After the system goes live, active learning can be used to prioritize manual annotation of query-document pairs with high model uncertainty, gradually correcting distribution biases and continuously improving ranking accuracy.

[0108] In particular, the posterior features of this invention are entirely based on structured metadata of policy documents, and the training data comes from knowledge graph relationships, realizing a systematic transfer and innovation of the LTR paradigm in the field of no user behavior data.

[0109] Please see Figure 4 , Figure 4This is a flowchart illustrating an embodiment of the construction of training data provided by the present invention. Using a "policy knowledge graph" as the core source, it extracts multi-dimensional information such as policy objective fields, applicable region fields, prior policy relationships (B is a historical version of A), detailed policy relationships (general outline A → detailed rules B), seed positive examples, and cited policy relationships (A cites B) to generate various types of samples. Specifically, policy objective and region fields guide cross-regional difficult-to-bear samples (same topic, different regions; similarity is calculated using BGE-M3 to avoid circular dependencies) and cross-clause difficult-to-bear samples (adjacent functional clauses in the same document). Prior policy relationships generate cross-version difficult-to-bear samples (combined with secondary confirmation using failure markers). Detailed rules and citation relationships respectively output detailed rule extended positive examples (Q and B confidence 0.8), citation extended positive examples (Q and B confidence 1.0), and seed positive example pairs. These samples together constitute the "training dataset," which is then used to train the GBDT ranking model (LambdaMART training) and the Latent Attention Layer (InfoNCE contrastive learning). Finally, through active learning, manual annotations are added to uncertain samples to achieve continuous iterative optimization of the model. By accurately identifying policy relationships, effectively distinguishing complex and difficult-to-bear samples, and continuously optimizing, the accuracy and reliability of policy retrieval and matching are improved, providing efficient support for policy analysis, retrieval, and other scenarios.

[0110] It should be noted that, in order to improve the ability to capture and represent the deep semantics of policy document texts, and to enable the retrieval system to understand the content of policy documents more accurately, the implicit attention module 101 includes a learnable latent query matrix, a cross-attention projection matrix, and a linear projection layer. Among them, the learnable latent query matrix is ​​used to determine multiple latent query vectors of the word segmentation hidden state sequence; the cross-attention projection matrix is ​​used to perform multi-semantic perspective cross-aggregation on the word segmentation hidden state sequence based on multiple latent query vectors to obtain multi-perspective semantic vectors of the word segmentation hidden state sequence; the linear projection layer is used to flatten and normalize the multi-perspective semantic vectors to obtain text embedding vectors.

[0111] In one specific embodiment, the input word segmentation hidden state sequence H of the implicit attention module 101 is a complete token-level hidden state sequence, H = [h_1, h_2, ..., h_L] ∈ R^{L×d}. The hidden state H_i at each position can simultaneously perceive information from all other positions in the sequence (including the complete context to its left and right). This characteristic is particularly important for policy texts: key qualifiers in policy clauses (such as applicable objects, geographical scope, and validity period) are often scattered across different positions in the sentence. Bidirectional attention ensures that the hidden state at each position fully integrates the semantics of the entire sentence, providing the implicit attention module 101 with high-quality position-level semantic raw materials, making its weighted aggregation information foundation significantly superior to that of unidirectional decoding hidden states.

[0112] Specifically, the implicit attention module 101 completes the adaptive aggregation from token-level sequences to sentence-level vectors through the following three steps:

[0113] First, the learnable latent query matrix: Initialize a set of parameter matrices Q_latent ∈ R^{n×d}, where n is the number of latent query vectors (configurable, typically 1 to 16), and d is consistent with the dimension of the target hidden layer. Each query vector in Q_latent represents a semantic perspective that the model needs to extract from the sequence (such as "application condition related information", "regional restriction information", etc.), and its specific meaning is automatically formed through end-to-end contrastive learning training without manual specification. Q_latent, together with the cross-attention projection matrices (W_Q, W_K, W_V) and the output linear projection layer, constitutes the complete set of trainable parameters for the implicit attention module 101; among them, Q_latent is the core parameter, and its initialization strategy needs to be specifically designed for the uniform activation distribution of the bidirectional hidden state of the backbone encoder to avoid the attention distribution from degenerating into uniform weights in the initial training stage.

[0114] Where W_Q is the query projection matrix, used to map input features to the query space; W_K is the key projection matrix, used to map input features to the key space; and W_V is the value projection matrix, used to map input features to the value space.

[0115] Second, cross-attention aggregation: using Q_latent as the query and H as the key and value, perform scaled dot product attention:

[0116] A = softmax( Q_latent · H^T / √d ) · H,A ∈ R^{n×d}

[0117] Specifically, attention weights are calculated for each potential query vector at all token positions in H, and then H is summed using weighted averages. Latent Attention assigns higher weights to token positions containing key clauses in the policy text and lower weights to low-semantic-density tokens such as function words and conjunctions, achieving adaptive focusing for information extraction.

[0118] Third, output projection and normalization: After flattening the n aggregated vectors, they are mapped to the target dimension d through a linear projection layer and then normalized using L2.

[0119] v = Linear_{n·d → d}( Flatten(A) ),v_norm = v / ‖v‖2

[0120] In this model, the Linear layer has an input dimension of n·d and an output dimension of d. The Flatten operation flattens A ∈ R^{n×d} into R^{n·d} before feeding the entire result into the linear layer. This allows the aggregation results from different potential query perspectives to interact and merge during the projection process, rather than simply adding the n vectors after projecting them separately. v_norm is the final output text embedding vector with a dimension of d (consistent with the target output dimension). L2 normalization ensures that the vectors can be directly compared and retrieved using cosine similarity.

[0121] In this embodiment, multiple potential query vectors are generated through a learnable potential query matrix. Guided by this, the word segmentation hidden state sequence is cross-aggregated from multiple semantic perspectives using a cross-attention projection matrix to obtain multi-perspective semantic vectors that reflect different semantic dimensions. After flattening and normalization by a linear projection layer, the final output is a text embedding vector that takes into account multi-dimensional semantic information and has a more reasonable distribution. This effectively improves the ability to capture and represent the deep semantics of policy document texts, enabling the retrieval system to understand the content of policy documents more accurately, thereby improving the relevance and accuracy of policy document retrieval.

[0122] To improve the accuracy of text similarity measurement in policy document retrieval, the similarity score conversion module 102 includes a candidate document acquisition unit and a cosine similarity calculation unit. The candidate document acquisition unit is used to acquire candidate documents related to the text to be retrieved. The cosine similarity calculation unit is used to determine the density score based on the cosine similarity between the text embedding vector and the embedding vector of the candidate document.

[0123] In this embodiment, the candidate document acquisition unit locates candidate documents related to the text to be retrieved, and then the cosine similarity calculation unit performs cosine similarity calculation between the text embedding vector and the candidate document embedding vector. This transforms the semantic association between texts into a quantifiable dense score, achieving accurate measurement of text similarity in policy document retrieval. This provides a scientific and reliable numerical basis for the subsequent ranking and filtering of retrieval results, effectively improving the accuracy of the policy document retrieval system in judging the degree of text semantic matching.

[0124] To achieve accurate ranking of candidate documents, the gradient enhancement module 103 includes an authority unit, a timeliness unit, an applicability unit, a retrieval signal unit, a posterior feature synthesis unit, and a ranking unit. Specifically, the authority unit acquires authority fields from a pre-defined policy knowledge graph and performs hierarchical encoding and matrix combination cross-processing on these fields to obtain an authority score. The timeliness unit acquires timeliness fields from the pre-defined policy knowledge graph and performs timeliness quantification on these fields using multi-dimensional rules to obtain a timeliness score. The applicability unit acquires applicability fields from the pre-defined policy knowledge graph and performs appropriate retrieval signal processing on these fields to obtain an applicability score. The system uses the gender field to perform hierarchical regional matching, applicable object matching, audience semantic matching, and policy objective and query intent matching to obtain an applicability score; the retrieval signal unit is used to perform multi-source signal fusion on the candidate document retrieval scores and density scores of the candidate document pool to obtain a retrieval signal score; the posterior feature synthesis unit is used to concatenate the authority score, timeliness score, applicability score, and retrieval signal score to generate a multi-dimensional posterior feature vector; the ranking unit is used to determine a preset number of candidate documents in the candidate document pool that are most relevant to the text to be retrieved based on the multi-dimensional posterior feature vector, and outputs the ranking score of each candidate document.

[0125] The authority unit comprises an administrative level coding layer, a document type validity coding layer, a two-dimensional authority matrix layer, and a joint issuance correction layer. The administrative level coding layer is used to hierarchically encode the issuing unit field in a pre-defined policy knowledge graph, resulting in an administrative level coding vector. The document type validity coding layer is used to hierarchically encode the document type classification field in a pre-defined policy knowledge graph, resulting in a document type validity coding vector. The two-dimensional authority matrix layer combines the administrative level coding vector and the document type validity coding vector to generate a two-dimensional authority matrix. The joint issuance correction layer is used to weight and correct the two-dimensional authority matrix based on the number of joint issuing units, resulting in an authority score.

[0126] Among them, the administrative level code of the issuing unit: the issuing unit field is directly extracted from the knowledge graph and hierarchically coded (national level > provincial level > municipal level > district / county level), and the authority score is positively correlated with the administrative level.

[0127] Document Type Validity Hierarchy Coding: Document type classification fields extracted from a knowledge graph are used to encode policy documents according to their validity hierarchy. Different document types have significant differences in legal validity. The coding rules are as follows: First level (highest validity): Regulations, Orders; Second level: Measures, Provisions; Third level: Opinions, Plans, Measures; Fourth level: Notices, Announcements, Public Notices; Fifth level: Replies, Letters, Minutes; Sixth level: Reports and other document types.

[0128] Two-dimensional authority comprehensive score: This score combines the administrative level of the issuing unit with the level of authority of the document type to construct a two-dimensional authority scoring matrix. For example, a document of type 1 issued by a first-level unit scores higher than a document of type 2 issued by a first-level unit; similarly, a document of type 1 issued by a first-level unit scores higher than a document of type 1 issued by a second-level unit. In cases of the same level and type of document, the date of issuance is further considered for differentiation.

[0129] Number of co-issuing units: Co-issuance of documents by multiple units usually indicates higher execution and policy coordination. The number of co-issuing units is a positive characteristic of authority.

[0130] In this embodiment, an authoritative unit comprising an administrative level coding layer, a document type validity coding layer, a two-dimensional authority matrix layer, and a joint issuance correction layer is constructed to achieve a precise quantitative assessment of the authority of policy documents. Specifically, the administrative level coding layer hierarchically encodes the issuing unit field in the policy knowledge graph, generating an administrative level coding vector that reflects the foundation of the policy document's authority from an administrative level perspective. The document type validity coding layer hierarchically encodes the document type classification field in the policy knowledge graph, obtaining a document type validity coding vector that provides an authoritative framework from the perspective of the validity attribute of the document type itself. The two-dimensional authority matrix layer combines and cross-references the above two coding vectors, generating a two-dimensional authority matrix that comprehensively reflects both administrative level and document type validity factors through dimensional cross-fusion. The joint issuance correction layer further weights and corrects the two-dimensional authority matrix based on the number of jointly issuing units, ultimately obtaining an accurate authority score. This multi-level, multi-dimensional coding and correction mechanism can comprehensively and objectively assess the authority of policy documents, effectively improving the retrieval accuracy and result quality of policy document retrieval systems, ensuring that users can prioritize access to policy documents with high authority and significant reference value.

[0131] The timeliness unit includes an expiration marker layer, an expiration date parsing layer, and a preceding policy parsing layer. The expiration marker layer is used to set the timeliness score of documents with an expiration marker field to zero. The expiration date parsing layer is used to calculate the expiration date of non-expired documents and determine the timeliness score based on the time difference between the expiration date and the current time. The preceding policy parsing layer is used to reduce the timeliness score of documents that have a preceding policy relationship field in the preset policy knowledge graph.

[0132] Among them, the failure marker field is a failure marker field extracted directly from the knowledge graph. For any document whose policy title is marked with the word "failed", the timeliness score is directly set to zero, and the strongest penalty is applied in the ranking, regardless of its vector similarity score.

[0133] Precise calculation of the expiration date refers to using the expiration date field extracted from the knowledge graph and combining it with the current date at the time of the query to calculate an accurate timeliness score. The calculation logic is as follows: if an expiration mark exists, the timeliness score is set to zero; if the expiration date is earlier than the current date, the timeliness score is set to zero (expired); if the expiration date is later than the current date, the timeliness score decreases linearly with the remaining validity period; if the expiration date is empty, the timeliness score is calculated based on the publication time field using a time decay function.

[0134] Precedence Policy Relationship Timeliness Inference: Utilizing the precedence policy relationship field of the knowledge graph, if a document has a successor version (i.e., the document is a precedence policy of other documents), it is automatically inferred that it has been replaced, further reducing the timeliness score.

[0135] In this embodiment, a three-layer structure—an expiration marker layer, an expiration date parsing layer, and a preceding policy parsing layer—achieves accurate assessment of the timeliness of policy documents. The expiration marker layer directly sets the timeliness score of documents with an expiration marker field to zero, ensuring the complete exclusion of policy documents that are clearly expired. The expiration date parsing layer, for non-expired documents, scientifically determines the timeliness score by calculating the time difference between the expiration date and the current time, closely linking the timeliness assessment of policy documents to the passage of time; a smaller time difference results in a higher score, and vice versa. The preceding policy parsing layer, using a pre-defined policy knowledge graph, lowers the timeliness score of documents with preceding policy relationship fields. This is because, in the policy system, preceding policies are often replaced or improved by subsequent updated policies, and their timeliness is relatively weaker than later policies. This mechanism effectively highlights the latest and more instructive policy documents. These three layers work together to comprehensively and dynamically reflect the timeliness status of policy documents, significantly improving the retrieval accuracy and relevance of the policy document retrieval system, ensuring that users can prioritize obtaining currently valid, up-to-date, and more practically valuable policy documents.

[0136] In the applicability unit, before calculating the applicability score, the system performs query parsing on the text to be retrieved and constructs a structured semantic representation of the query. The structured semantic representation includes the target region, the type of query subject, and the query intent, which is used to drive the calculation of each sub-feature of applicability.

[0137] Hierarchical Regional Matching Score: The system uses province / city / district level regional fields extracted from a knowledge graph to perform hierarchical matching and scoring with the target region identified in the query. Matching Rules: A perfect match at the district level receives the highest score; a match at the city level is next; a match at the provincial level is next; nationwide applicability (province, city, and district fields are all empty) receives a base score; the lowest score is given when the target region and the query region do not match at all. This hierarchical design enables the system to accurately distinguish the applicability differences between "Policy of District b in City A" and "General Policy of City A" for specific regional queries.

[0138] Applicable Object Matching: The applicable object field (organization / individual / government department) extracted using the knowledge graph is matched with the query subject. Queries targeting businesses prioritize policies applicable to "organizations," while queries targeting individuals prioritize policies applicable to "individuals." Documents with mismatched applicable objects are penalized in the ranking.

[0139] Audience semantic matching: By using the audience field in the semantic entity of the knowledge graph, semantic matching and scoring are performed with the enterprise size and industry type involved in the query, which is more accurate and stable than extracting from the text.

[0140] Policy objective and query intent matching: Using the policy objective field in the semantic entity of the knowledge graph, the semantic matching score between the policy objective and the query intent is calculated as an interpretable matching feature that goes beyond pure vector similarity.

[0141] In the retrieval signal unit, the Dense vector retrieval similarity score is calculated by the enhanced embedding model (BGE-M3 + Latent Attention Layer) of this invention, which is the cosine similarity between the query and the candidate document. It reflects the deep semantic relevance. The score is fine-tuned by special hard-to-bear samples in the policy domain, and has a stronger ability to distinguish fine-grained semantic differences in policies.

[0142] Sparse vector retrieval BM25 score: Keyword matching score based on word frequency statistics, which has unique value in the policy field: When users directly enter policy document numbers, BM25's accurate matching ability is significantly better than vector retrieval.

[0143] Document number exact match score refers to extracting the document number field as an exact match feature, independent of the BM25 score. When a string in the query exactly matches the document number of a candidate document, the highest exact match score is assigned, ensuring that users can accurately retrieve the target file when searching directly by document number.

[0144] Recall loop source encoding: Label candidate documents from the Dense loop, Sparse loop, or both. Documents from both loops typically have higher relevance and are used as positive features input to the GBDT model.

[0145] In the process of constructing posterior features for candidate documents, the posterior features are not obtained by simply concatenating features from various dimensions, but are generated based on a joint mapping mechanism constructed from semantic representation and structured information, so as to realize the correlation modeling between features from different sources.

[0146] After extracting the four-dimensional posterior features, a multi-dimensional posterior feature vector is constructed based on the multi-dimensional features and used as the input to the GBDT ranking model. The training process of the GBDT ranking model is as follows:

[0147] Training data construction: Query-document relevance annotation data is automatically generated based on the knowledge graph relationship fields. Specifically, positive examples of citation relationship expansion (confidence 1.0), positive examples of rule relationship expansion (confidence 0.8), and seed positive examples are used as high-relevance document annotations; cross-version hard-to-bear samples, cross-region hard-to-bear samples, and cross-clause hard-to-bear samples are used as low-relevance document annotations. These positive and negative examples together constitute query-document-relevance annotation triples, which serve as the training set for GBDT. It should be noted that the training data for the GBDT ranking model is query-document level annotation, which differs in granularity and format from the query-clause level contrastive learning data used by the implicit attention module 101; the two are trained independently.

[0148] Training Objective: LambdaMART is used as the training objective to directly optimize ranking quality metrics such as nDCG, belonging to the listwise ranking learning paradigm. Compared with pointwise (document-by-document regression) and pairwise (document pair comparison) methods, LambdaMART can optimize ranking results at the level of the entire candidate document list, making it more suitable for application scenarios with high Top-N accuracy requirements in policy retrieval.

[0149] Implementation details: XGBoost or LightGBM frameworks can be used. LightGBM offers higher efficiency in native support for categorical features such as administrative level coding and document type validity coding, and has better training efficiency and ranking accuracy in scenarios where policy posterior features are mainly categorical and discrete.

[0150] Continuous optimization: The automated graph data is used as a cold start training set. After the system goes online, it can actively learn to manually annotate and supplement query-document pairs with high model uncertainty, gradually correcting the deviation between the training data and the distribution of real user queries, and continuously improving the ranking accuracy.

[0151] In this embodiment, policy documents are accurately ranked through multi-dimensional feature fusion. The authority unit utilizes the authority field of the policy knowledge graph, and generates an authority score through hierarchical encoding and matrix combination cross-processing to quantify the credibility of the policy issuing entity and source. The timeliness unit uses multi-dimensional rules to quantify the timeliness of the timeliness field in the knowledge graph, accurately assessing the time effectiveness of the policy. The applicability unit comprehensively measures the fit between the policy and user needs through hierarchical regional matching, applicable object matching, audience semantic matching, and policy objectives and query intent matching. The retrieval signal unit performs multi-source signal fusion on the retrieval scores and dense scores of candidate documents to enhance the reliability of the basic retrieval results. The posterior feature synthesis unit concatenates the above multi-dimensional scores to construct a multi-dimensional posterior feature vector that comprehensively reflects the quality of policy documents. The ranking unit selects and outputs a preset number of candidate documents most relevant to the text to be retrieved and their ranking scores based on this vector, effectively improving the accuracy and relevance of policy document retrieval and ensuring that users can quickly obtain authoritative, timely, and highly applicable policy information. Please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a flowchart illustrating an embodiment of the gradient boosting module provided by the present invention. Taking a "unified candidate document pool" and a "policy knowledge graph" as inputs, it extracts multi-dimensional features from documents using a "four-dimensional posterior feature system" (including authority, timeliness, applicability, and retrieval signal dimensions). After processing, the features are "feature vector concatenated" and then input into a "GBDT ranking model (XGBoost / LightGBM)" for model training and computation. Finally, it outputs "Top-N ranking results." This achieves a comprehensive characterization of policy document attributes through a multi-dimensional feature system, combined with a machine learning model for accurate ranking, ultimately providing users with "authoritative, up-to-date, applicable, and relevant" policy information, significantly improving the accuracy and efficiency of policy retrieval.

[0152] This invention provides a method for retrieving policy documents. Please refer to [link / reference]. Figure 6 , Figure 6 A flowchart illustrating an embodiment of the policy document retrieval method provided by the present invention includes:

[0153] S601: Perform implicit multi-semantic query and cross-attention aggregation on the word segmentation hidden state sequence of the text to be retrieved to obtain the text embedding vector of the word segmentation hidden state sequence;

[0154] S602: Determine the dense score between the text to be retrieved and each candidate document based on the cosine similarity between the embedding vector of each candidate document and the text embedding vector, wherein each candidate document is pre-stored in the candidate document pool;

[0155] S603: Perform posterior feature extraction on the preset policy knowledge graph, candidate document pool, and dense scores to obtain a multi-dimensional posterior feature vector between the text to be retrieved and each candidate document. Then, perform nonlinear combination modeling on the multi-dimensional posterior feature vector to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved.

[0156] In this embodiment, by performing implicit multi-semantic query and cross-attention aggregation on the hidden state sequence of the segmented text to be retrieved, the implicit logical connections and semantic levels in the text to be retrieved can be effectively captured, thereby generating a text embedding vector rich in deep semantics. Then, by comparing the cosine similarity between the text embedding vector and the embedding vector of each candidate document, the dense score between the text to be retrieved and each candidate document can be accurately quantified, realizing fine-grained semantic matching for each candidate document. Finally, by combining the preset policy knowledge graph, candidate document pool and dense score, posterior feature extraction is performed to obtain a multi-dimensional posterior feature vector between the text to be retrieved and each candidate document. The multi-dimensional posterior feature vector is then nonlinearly combined and modeled to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved, which significantly improves the accuracy and semantic understanding depth of policy document retrieval.

[0157] The above provides a detailed description of a policy document retrieval system and method provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will know that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A policy document oriented retrieval system characterized by, include: An implicit attention module performs implicit multi-semantic querying and cross-attention aggregation on the word segmentation hidden state sequence of the text to be retrieved, obtaining the text embedding vector of the word segmentation hidden state sequence. The implicit attention module includes a learnable latent query matrix, a cross-attention projection matrix, and a linear projection layer. The learnable latent query matrix is ​​used to determine multiple latent query vectors of the word segmentation hidden state sequence. The cross-attention projection matrix is ​​used to perform multi-semantic perspective cross-aggregation on the word segmentation hidden state sequence based on the multiple latent query vectors, obtaining a multi-perspective semantic vector of the word segmentation hidden state sequence. The linear projection layer is used to flatten and normalize the multi-perspective semantic vector to obtain the text embedding vector. The similarity score conversion module determines the density score between the text to be retrieved and each candidate document based on the cosine similarity between the embedding vector of each candidate document and the embedding vector of the text, wherein each candidate document is pre-stored in the candidate document pool; The gradient boosting module performs posterior feature extraction on the preset policy knowledge graph, the candidate document pool, and the dense score to obtain a multi-dimensional posterior feature vector between the text to be retrieved and each candidate document. The module then performs nonlinear combination modeling on the multi-dimensional posterior feature vector to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved. The policy document retrieval system further includes a retrieval matching module, an exact matching module, an implicit matching module, and a deduplication and merging module. The retrieval matching module is used to obtain sparse matching documents related to the text to be retrieved based on a keyword matching algorithm using word frequency statistics. The exact matching module is used to accurately match documents related to the text to be retrieved based on the document number in the text to be retrieved. The implicit matching module is used to perform dense vector retrieval based on the text embedding vector to obtain implicit vector retrieval documents. The deduplication and merging module is used to deduplicate and merge the sparse matching documents, the exact matching documents, and the implicit vector retrieval documents to obtain the candidate document pool.

2. The policy file oriented retrieval system of claim 1, wherein, The policy document retrieval system also includes a backbone coding module, which is used to perform word segmentation and encoding on the text to be retrieved to obtain the word segmentation hidden state sequence.

3. The policy document retrieval system according to claim 2, characterized in that, The policy document retrieval system also includes a difficult-to-bear sample construction module; the difficult-to-bear samples include geographical dimension difficult-to-bear samples, time dimension difficult-to-bear samples, and structural dimension difficult-to-bear samples. The difficult-to-bear sample construction module obtains cross-regional policy document pairs with similar policy objectives but different applicable regions based on the applicable region field of the preset policy knowledge graph, and determines the cross-regional policy document pairs as the geographical dimension difficult-to-bear samples; The unbearable sample construction module marks historical version documents of the currently effective policy as unbearable samples in the time dimension based on the expiration mark field, expiration date field, and preceding policy relationship field of the preset policy knowledge graph; The difficult-to-bear sample construction module, based on the policy tools and audience semantic entity fields of the preset policy knowledge graph, pairs adjacent clauses in the same file that are semantically related but functionally different as the structural dimension difficult-to-bear samples. The difficult-to-negative sample construction module is also used to generate a sample training set based on the difficult-to-negative samples, and the training set is used to optimize the parameters of the implicit attention module.

4. The policy document retrieval system according to claim 1, characterized in that, The similarity score conversion module includes a candidate document acquisition unit and a cosine similarity calculation unit; The candidate document acquisition unit is used to acquire candidate documents related to the text to be retrieved; The cosine similarity calculation unit is used to determine the density score based on the cosine similarity between the text embedding vector and the embedding vector of the candidate document.

5. The policy document retrieval system according to claim 1, characterized in that, The gradient enhancement module includes an authority unit, a timeliness unit, an applicability unit, a retrieval signal unit, a posterior feature synthesis unit, and a ranking unit; The authority unit is used to obtain the authority field in the preset policy knowledge graph, and to perform hierarchical encoding and matrix combination cross-processing on the authority field to obtain the authority score; The timeliness unit is used to obtain the timeliness field in the preset policy knowledge graph, and to perform timeliness quantification on the timeliness field through multi-dimensional rules to obtain a timeliness score; The applicability unit is used to obtain the applicability field in the preset policy knowledge graph, and to perform hierarchical regional matching, applicable object matching, audience semantic matching and policy goal and query intent matching on the applicability field to obtain an applicability score; The retrieval signal unit is used to perform multi-source signal fusion on the candidate document retrieval score and the dense score of the candidate document pool to obtain the retrieval signal score; The posterior feature synthesis unit is used to concatenate the authority score, timeliness score, applicability score, and retrieval signal score to generate the multidimensional posterior feature vector. The sorting unit is used to determine a preset number of candidate documents in the candidate document pool that are most relevant to the text to be retrieved based on the multidimensional posterior feature vector, and outputs a sorting score for each candidate document.

6. The policy document retrieval system according to claim 5, characterized in that, The authoritative unit includes an administrative level coding layer, a document type validity coding layer, a two-dimensional authority matrix layer, and a joint issuance correction layer; The administrative level coding layer is used to perform hierarchical coding on the issuing unit field in the preset policy knowledge graph to obtain the administrative level coding vector; The document type validity coding layer is used to perform validity-level coding on the document type classification field in the preset policy knowledge graph to obtain the document type validity coding vector. The two-dimensional authority matrix layer is used to perform matrix combination and cross-interaction of the administrative level coding vector and the document type validity coding vector to generate a two-dimensional authority matrix. The joint publication correction layer is used to weight and correct the two-dimensional authority matrix according to the number of joint publication units to obtain the authority score.

7. The policy document retrieval system according to claim 5, characterized in that, The timeliness unit includes an expiration marking layer, an expiration date parsing layer, and a preceding policy parsing layer; The failure flag layer is used to set the timeliness score of documents with failure flag fields to zero. The expiration date parsing layer is used to calculate the expiration date of non-expired documents and determine the timeliness score based on the time difference between the expiration date and the current time. The preceding policy parsing layer is used to reduce the timeliness score of documents in the preset policy knowledge graph that have preceding policy relationship fields.

8. A method for retrieving policy documents, characterized in that, include: Implicit multi-semantic query and cross-attention aggregation are performed on the word segmentation hidden state sequence of the text to be retrieved to obtain the text embedding vector of the word segmentation hidden state sequence; The dense score between the text to be retrieved and each candidate document is determined based on the cosine similarity between the embedding vector of each candidate document and the text embedding vector, wherein each candidate document is pre-stored in a candidate document pool; Posterior features are extracted from the preset policy knowledge graph, the candidate document pool, and the dense score to obtain a multidimensional posterior feature vector between the text to be retrieved and each candidate document. The multidimensional posterior feature vector is then modeled using a nonlinear combination to determine the document ranking score of a preset number of candidate documents in the candidate document pool that are related to the text to be retrieved. The step of performing implicit multi-semantic query and cross-attention aggregation on the word segmentation hidden state sequence of the text to be retrieved to obtain the text embedding vector of the word segmentation hidden state sequence includes: determining multiple potential query vectors of the word segmentation hidden state sequence; performing multi-semantic perspective cross-aggregation on the word segmentation hidden state sequence based on the multiple potential query vectors to obtain a multi-perspective semantic vector of the word segmentation hidden state sequence; and performing flattening and normalization processing on the multi-perspective semantic vector to obtain the text embedding vector. The steps for obtaining the candidate document pool include: obtaining sparse matching documents related to the text to be retrieved based on a keyword matching algorithm using word frequency statistics; precisely matching documents related to the text to be retrieved based on document numbers in the text to be retrieved; performing dense vector retrieval based on the text embedding vector to obtain implicit vector retrieval documents; and deduplicating and merging the sparse matching documents, the precisely matching documents, and the implicit vector retrieval documents to obtain the candidate document pool.

Images