An intelligent retrieval and hierarchical authorization system and method for archival resources

Abstract

The application discloses a kind of file resource intelligent retrieval and hierarchical authorization utilization system and method, belong to intelligent retrieval field, by receiving user natural language query and identifying its language kind, generate semantic vector in combination with multilingual natural language processing model, and then with each file semantic representation in file database is matched, to the situation involving cross-language matching, further calculate cross-language similarity offset rate and language context consistency score, and according to preset weight model, semantic generalization ability is divided into high, middle, low grade, for middle grade situation, introduce misjudgment risk assessment mechanism to judge whether additional authorization is needed;And when low grade and language are inconsistent, directly prohibit user to access high-density level file, so as to effectively prevent the information leakage risk brought by semantic generalization, realize the safe, accurate control to high sensitive file under multilingual environment.

Description

Technical Field

[0001] This invention relates to the field of intelligent retrieval technology, specifically to an intelligent retrieval and hierarchical authorization system and method for archival resources. Background Technology

[0002] Intelligent retrieval and tiered authorization for archival resources refers to a mechanism that utilizes technologies such as artificial intelligence and big data to intelligently retrieve and classify archival resources. It automatically determines the scope of archives accessible to users based on their access levels, enabling the secure, efficient, accurate acquisition and rational utilization of archival information. This model helps improve the level of intelligence in archival management while ensuring information security and compliant use.

[0003] The existing technology has the following shortcomings:

[0004] In existing technologies, when using NLP models for semantic retrieval in a multilingual environment, their strong generalization ability may lead users to match highly classified file content across languages. If the system does not have an independent permission isolation mechanism for the language dimension, there may be situations where confidential files are mistakenly identified as accessible due to semantic mismatches. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent retrieval and hierarchical authorization system and method for archival resources, in order to address the shortcomings of the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for intelligent retrieval and hierarchical authorized use of archival resources, comprising:

[0007] Receive natural language query requests from users and determine the language of the query statement;

[0008] Based on a natural language processing model, semantic analysis is performed on the query statement to generate corresponding multilingual semantic vectors;

[0009] The semantic similarity between the user query semantic vector and the semantic representation of each file in the archive database is calculated to obtain a set of matching results;

[0010] For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model;

[0011] When the semantic generalization ability is medium and the search results contain high-secret files, a further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required.

[0012] When the semantic generalization ability level is low and the user's query language is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

[0013] Preferably, the language recognition model for determining the language type of the query statement includes a language classification model based on character features, word frequency distribution, and language character set, and outputs the language confidence probability.

[0014] Preferably, generating multilingual semantic vectors includes: selecting a pre-trained semantic model that matches the query language based on the bound language tags; performing language segmentation on the query statement and constructing word vectors, positional encodings, and paragraph identifiers; inputting the embedded vectors into a Transformer-type neural network for semantic modeling; extracting the sentence vector representing the entire query statement, performing L2 normalization, and outputting it as a semantic vector.

[0015] Preferably, semantic similarity calculation includes: pre-generating semantic vectors for each file in the archive database using the same semantic model; calculating the normalized cosine similarity between the user semantic vector and the file vector to obtain a similarity score; and filtering file entries with similarity greater than a set threshold to form a preliminary matching result set.

[0016] Preferably, the method for obtaining the cross-language similarity offset rate (CLSD) is as follows: the user inputs a query statement QA in language A, and a semantic vector in language A is generated through a multilingual semantic model. ;

[0017] The machine translation module is used to translate QA into archival language B, resulting in the corresponding statement QB; the same semantic vector generation process is then performed on QB to obtain... The calculation of the vector difference between language A and language B in the semantic space for expressing the same query is called the cross-lingual similarity offset rate (CLSD), which is expressed as: In the formula, This is the formula for calculating cosine similarity.

[0018] Preferably, the method for obtaining the language context consistency score is as follows: receiving the original user query statement Qorig; translating it into the target language using a neural machine translation module to obtain Qtrans; standardizing the two texts; combining Qorig and Qtrans into a sentence pair, inputting it into the cross-lingual natural language inference XNLI model, extracting feature vectors using its multilingual sentence pair encoding structure, and inputting them into the classifier; the model outputs the probability distributions of three categories: Pe: semantic consistency; Pn: semantic ambiguity; Pc: semantic inconsistency; setting the language context consistency score LCCS as: LCCS=Pe.

[0019] Preferably, the cross-language similarity offset rate and the language context consistency score are normalized so that they are both between [0,1]. The semantic generalization ability analysis value in the cross-language matching process is calculated based on the normalized cross-language similarity offset rate and the language context consistency score.

[0020] Preferably, the semantic generalization ability analysis value obtained in the cross-language matching process is compared with the gradient standard threshold, which includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold. The semantic generalization ability analysis value in the cross-language matching process is compared with the first standard threshold and the second standard threshold respectively.

[0021] If the semantic generalization ability analysis value in the cross-language matching process is greater than the second standard threshold, it indicates poor semantic generalization ability. It will be marked as high semantic generalization ability, triggering the language dimension permission isolation mechanism or blocking access.

[0022] If the semantic generalization ability analysis value in the cross-language matching process is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is marked as medium semantic generalization ability, and a risk assessment for misjudgment is performed on high-secret archives.

[0023] If the semantic generalization ability analysis value in the cross-language matching process is less than the first standard threshold, it indicates that the semantic expression is stable. It is marked as having low semantic generalization ability, and semantic matching and result display are carried out according to the normal process.

[0024] Preferably, when the semantic generalization ability is at a medium level, that is, the semantic generalization ability analysis value generated in the cross-language matching process within a fixed time period T is greater than or equal to the first standard threshold and less than or equal to the second standard threshold; collect the semantic generalization ability analysis values ​​generated in the cross-language matching process within subsequent fixed time periods that are greater than or equal to the first standard threshold and less than or equal to the second standard threshold, establish a dataset, and calculate the mean and standard deviation of the dataset;

[0025] If the mean of the semantic generalization ability analysis value in the dataset is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, the risk level is high, and it is judged as a high-risk mismatch trend. It is mandatory to apply for manual authorization to access the high-confidential files.

[0026] If the mean of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, it is determined to be an unstable matching state. The high-secret archive content is blocked and the user is prompted to adjust the search language or intent.

[0027] If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, only the title or classification information of the high-level confidential archive will be returned, and secondary confirmation is required before it can be obtained.

[0028] If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, it is determined to be a controllable state of medium-level matching, and authorized fields are returned. Desensitization processing is performed on highly sensitive fields.

[0029] This invention also provides an intelligent retrieval and hierarchical authorization system for archival resources, including a language recognition module, a query semantic understanding module, a semantic matching and retrieval module, a semantic generalization ability evaluation module, and a language dimension permission isolation module;

[0030] Language recognition module: Receives natural language query requests input by the user and determines the language of the query statement;

[0031] Query semantic understanding module: Based on a natural language processing model, the module performs semantic analysis on query statements and generates corresponding multilingual semantic vectors.

[0032] Semantic matching and retrieval module: calculates the semantic similarity between the semantic vector of the user query and the semantic representation of each file in the archive database, and obtains a set of matching results;

[0033] Semantic generalization ability assessment module: For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model;

[0034] Misjudgment Risk Assessment Module: When the semantic generalization ability is at a medium level and the search results contain high-secret files, further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required.

[0035] Language-based access control module: When the semantic generalization ability level is low and the language of the user query is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

[0036] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0037] 1. This invention addresses the risk of mismatches in high-secret documents during semantic retrieval in multilingual environments. It proposes a semantic generalization ability assessment mechanism based on cross-lingual semantic shift and contextual consistency scoring. This mechanism accurately determines the credibility of semantic matches and classifies semantic generalization levels accordingly, enabling dynamic authorization control and access restrictions for medium- to high-risk scenarios. This method effectively avoids the problem of accidental triggering of sensitive information due to linguistic ambiguity and semantic drift, enhancing the security and intelligence of multilingual document retrieval systems.

[0038] 2. This invention integrates multiple AI technologies, including natural language processing, machine translation, semantic similarity calculation, and natural language inference (XNLI), to construct a semantic generalization ability analysis value and introduce time series statistics and semantic fluctuation analysis mechanisms. This further enhances the risk identification capability for medium-level fuzzy matching, supporting fine-grained access control and decision-making. This method improves the compliance and controllability of access to classified files while ensuring retrieval accuracy. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0040] Figure 1 This is a mind map of the method of the present invention.

[0041] Figure 2 This is a mind map of the system modules of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1, please refer to Figure 1 As shown in this embodiment, a method for intelligent retrieval and hierarchical authorization of archival resources includes:

[0044] Receive natural language query requests from users and determine the language of the query statement;

[0045] Based on a natural language processing model, semantic analysis is performed on the query statement to generate corresponding multilingual semantic vectors;

[0046] Calculate the semantic similarity between the user query semantic vector and the semantic representations of each file in the file database to obtain a set of matching results;

[0047] For cross-language matching situations involved in the semantic matching process, calculate the cross-language similarity offset rate and the language context consistency score, and divide the semantic generalization ability into three levels: high, medium, and low according to the preset weight model;

[0048] When the semantic generalization ability is at the medium level and the retrieval results include high-classified files, further evaluate the misjudgment risk of the matching results to determine whether additional authorization is required;

[0049] When the semantic generalization ability level is low and the user query language is inconsistent with the file content language, prohibit the user from accessing the matching high-classified file resources.

[0050] The ways for users to input natural language queries may include but are not limited to the following:

[0051] Text input: The user manually enters a retrieval statement through the front-end interface (such as a web page, system client, or mobile application), such as "Find overseas cooperation files in 2020".

[0052] Speech-to-text input: The user inputs through speech, and the system uses the speech recognition module to convert the speech content into text and then enters the subsequent processing process.

[0053] Interface data input (system integration scenario): If the system is integrated with other business systems, the user's retrieval request can be automatically passed in through the API or middleware and submitted in JSON or plain text format.

[0054] Regardless of the method, the system uniformly converts the user query into a standardized UTF-8 encoded text format and enters the language recognition stage.

[0055] The Language Identification Model (LID for short) is used to determine which language the text belongs to. Its core goal is to identify the language type based only on language features such as word distribution and grammatical structure without understanding the text semantics. This model generally has the following technical components:

[0056] Feature extraction module: Mainly extract linguistic features from the input text, commonly including: word frequency distribution (such as "de", "le" frequently appear in Chinese); character n-gram patterns (such as "th", "ing" frequently appear in English); specific language character sets (such as Cyrillic letters for Russian); special symbol and punctuation rules.

[0057] The main body of the model algorithm: Depending on the system's accuracy requirements and performance conditions, models of different complexities can be adopted, including: rule matching models (based on predefined character / word rule judgment); statistical models (such as Naive Bayes, TF-IDF language fingerprint matching); pre-trained language recognition models (such as the language recognizer of fastText, or CLD3 / CLD2 models); multilingual BERT embedding + classification head (for high-precision scenarios).

[0058] Excellent language recognition models support hundreds of languages ​​and maintain high accuracy in short texts. Of particular importance is their detail in recognizing similar languages ​​(such as Simplified / Traditional Chinese, English / Dutch).

[0059] Model Inference: After inputting standardized text, the system calls the language recognition model to perform inference and judgment, and outputs a set of candidate languages ​​and their probability distribution.

[0060] Language confidence score calculation: The model assigns a confidence score to each candidate language (e.g., Chinese 0.93, English 0.06, Japanese 0.01), and the confidence score can be used for subsequent error correction.

[0061] Language determination logic: If the highest confidence value exceeds the set threshold (e.g., ≥0.90), it is directly determined to be that language; if the confidence values ​​of multiple languages ​​are close, the system will mark it as "language uncertain" and prompt the user to confirm or select a language; for mixed language text (e.g., Chinese and English mixed), the system can identify the main language as the primary language and supplement it with secondary language tags.

[0062] Special processing mechanisms: For short texts (such as just a few words), a context enhancement mechanism is enabled to enhance the judgment based on user history input, interface language, or geographical location; for important users in the system (such as high-privilege users), the user's language preference can be combined to assist in decision-making; in multi-turn dialogue systems, the language of the previous turn is combined to maintain consistency and improve stability.

[0063] Accurate language identification is a core component of intelligent archive retrieval systems, primarily serving the following purposes: determining the semantic model to be used subsequently (Chinese and English BERT cannot be mixed).

[0064] Provide security strategies for cross-language semantic matching; support language-level permission isolation mechanisms to prevent semantic misjudgments from leading to the leakage of highly classified files; optimize the language of search result display to improve user experience.

[0065] Based on a natural language processing model, semantic analysis is performed on the query statement to generate corresponding multilingual semantic vectors, specifically including:

[0066] The natural language query statements entered by users are uniformly converted to UTF-8 encoding, and special symbols and illegal characters are removed to ensure that the text format is adapted to the model input requirements.

[0067] Language recognition models (such as CLD3 and fastText) are used to determine the language of the input text and bind the corresponding language tags (such as zh, en, ru, etc.) for subsequent model selection.

[0068] Perform operations such as stop word removal, punctuation normalization, and date / number standardization to improve the quality of effective input to the semantic model.

[0069] Based on the language label, select the pre-trained multilingual model corresponding to that language, for example:

[0070] Chinese: BERT, ERNIE;

[0071] English: English BERT, RoBERTa;

[0072] Multilingual: Multilingual-BERT (mBERT), XLM-RoBERTa.

[0073] Load the required model weights, vocabulary, positional encoding, and other parameters from the model library to prepare the neural network structure needed for vector generation.

[0074] Language-specific word segmentation is performed on the input text: WordPiece or BPE (Byte-Pair Encoding) is used for English; character-level segmentation or dictionary-based word segmenters (such as jieba) are used for Chinese; and a unified word encoding method is used for multilingual models.

[0075] Each word segmentation result is mapped to a fixed-dimensional word vector, including: word vector; position encoding; language / paragraph identifier; these vectors together constitute the tensor form of the model input.

[0076] The preprocessed embedded input is fed into a multi-layer Transformer structure (such as a BERT layer) for context-dependent semantic modeling.

[0077] Common methods for extracting the full sentence semantic representation vector from the model output include:

[0078] Use the output at the [CLS] position as the semantic vector for the entire query;

[0079] Alternatively, mean pooling can be applied to the vector of all tokens.

[0080] The generated semantic vectors are L2 normalized to form fixed-length, cross-language aligned vector representations, and cached to improve the efficiency of subsequent comparisons.

[0081] Output a floating-point vector with uniform dimensions (such as 768-dimensional or 1024-dimensional) to represent the position of the user query in the semantic space, which can be used to calculate the similarity with the semantic vector of the archive.

[0082] By attaching metadata such as language labels, recognition confidence, and contextual identifiers to the semantic vectors, necessary support is provided for the next stage of "cross-language semantic comparison" and "risk assessment".

[0083] The semantic similarity between the user's query semantic vector and the semantic representation of each file in the archive database is calculated to obtain a set of matching results, specifically including:

[0084] During the initial or periodic update phases of the system, each record in the database (including title, summary, and body text) is processed using the same pre-trained semantic model as the user query to generate a semantic vector.

[0085] All file semantic vectors are L2 normalized and stored in a vector retrieval engine (such as FAISS, Milvus, Weaviate), along with metadata such as file ID, language tag, and security classification information.

[0086] Let the vector dimension be d, and let the user query semantic vector be... R is the set of real numbers;

[0087] The set of semantic vectors of archives is ,in For each archive vector Calculate its relationship with the query vector cosine similarity The expression is: ;in: This represents the dot product of two vectors; the result ranges from [−1, 1], and the closer the value is to 1, the more similar the semantics.

[0088] The system iterates through all file vectors or uses a vector search engine (such as FAISS) to perform nearest neighbor search to obtain the top k neighbors. The most similar vectors and their similarity scores.

[0089] A similarity threshold θ is introduced to filter out low-relevance results. The typical value is 0.7, 0.8, etc., and it can be dynamically adjusted according to the system accuracy requirements.

[0090] All rights reserved Files with similarity ≥ θ are sorted from highest to lowest to form a preliminary matching result set. File metadata (security level, language, creation time, topic tags, etc.) is appended to the matching results for subsequent permission checks and display control.

[0091] For cross-linguistic matching involved in the semantic matching process, the cross-linguistic similarity shift rate and language context consistency score are calculated. Based on a pre-defined weight model, semantic generalization ability is divided into three levels: high, medium, and low, specifically including:

[0092] Cross-lingual similarity offset rate (CLSD) measures the degree of similarity fluctuation between a user query and the semantic vector of the target file under different language expressions, and is used to identify the stability and risk of cross-lingual semantic models.

[0093] The acquisition method is as follows: the user inputs a query statement QA in language A (such as English), and a semantic vector in language A is generated through a multilingual semantic model (such as XLM-R). ;

[0094] The machine translation module is used to translate QA into the archival language B (such as Chinese), resulting in the corresponding statement QB; the same semantic vector generation process is then performed on QB to obtain... Translation uses context-aware models, such as Transformer, or neural network translation engines (such as Google Translate API, Baidu Translate, and DeepL) to reduce semantic loss.

[0095] The cross-linguistic similarity offset rate (CLSD) is calculated by comparing the vector differences in semantic space between language A and language B for expressing the same query. The expression is as follows: In the formula, This is the formula for calculating cosine similarity.

[0096] If CLSD < 0.1, it indicates high cross-linguistic consistency, and semantic matching can be performed with confidence; if CLSD is between 0.1 and 0.3, it indicates moderate semantic shift, and semantic fuzzy matching annotation is recommended; if CLSD > 0.3, it indicates high-risk semantic drift, and language isolation mechanisms or manual review should be triggered.

[0097] The Language Contextual Consistency Score (LCCS) is used to judge the professionalism and accuracy of language use and whether it is consistent with the user's context by comparing the context, thus preventing inappropriate semantic expansion caused by ambiguity in language.

[0098] The method for obtaining the language context consistency score is as follows: receive the original user query statement Qorig (e.g., English); use a neural machine translation module to translate it into the target language (e.g., Chinese) to obtain Qtrans; and perform standardization processing on the two texts (denoising, punctuation unification, space standardization, etc.).

[0099] The Qorig and Qtrans are combined into a sentence pair and input into the cross-lingual natural language inference XNLI model. The model uses its multilingual sentence pair encoding structure (such as Transformer) to extract feature vectors, which are then fed into the classifier. The model outputs probability distributions for three categories: Pe: Entailment; Pn: Neutral; Pc: Contradiction.

[0100] The Language Context Consistency Score (LCCS) is set as: LCCS = Pe; that is, the LCCS value is equal to the probability judgment result of the model on the semantic consistency of the two statements.

[0101] If LCCS ≥ 0.85, it indicates a high degree of semantic consistency, and high-precision semantic retrieval can be initiated; if 0.60 ≤ LCCS < 0.85, it indicates contextual drift, and it is recommended to trigger fuzzy matching tagging; if LCCS < 0.60, it indicates potential misunderstanding or semantic conflict, and the language isolation mechanism should be activated.

[0102] The cross-linguistic similarity offset rate and the language context consistency score are normalized so that they are both between [0,1]. The semantic generalization ability analysis value in the cross-linguistic matching process is calculated based on the normalized cross-linguistic similarity offset rate and the language context consistency score.

[0103] For example, the present invention can use the following formula to calculate the semantic generalization ability analysis value in the cross-language matching process, the calculation expression is: In the formula, This represents the semantic generalization ability analysis value in the cross-language matching process. Cross-language similarity offset rate, Scoring based on language context consistency, The weighting coefficients for cross-language similarity offset rate and language context consistency score (which can be optimized based on experimental experience or machine learning) are: All are greater than 0.

[0104] The semantic generalization ability analysis value obtained in the cross-language matching process is compared with the gradient standard threshold, which includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold. The semantic generalization ability analysis value in the cross-language matching process is compared with the first standard threshold and the second standard threshold respectively.

[0105] If the semantic generalization ability analysis value in the cross-language matching process is greater than the second standard threshold, it indicates poor semantic generalization ability, with a large cross-language semantic offset and contextual inconsistency. It should be marked as high semantic generalization ability and the language dimension permission isolation mechanism should be triggered or access should be blocked.

[0106] If the semantic generalization ability analysis value in the cross-language matching process is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it indicates that there is a certain degree of uncertainty in semantic generalization. It is marked as medium semantic generalization ability. It is recommended to perform a misjudgment risk assessment on high-secret files or return only part of the information.

[0107] If the semantic generalization ability analysis value in the cross-language matching process is less than the first standard threshold, it indicates that the semantic expression is stable. It is marked as having low semantic generalization ability and can be semantically matched and displayed according to the normal process.

[0108] When the semantic generalization ability analysis value is in the medium-level range, that is, greater than or equal to the first standard threshold and less than or equal to the second standard threshold, if the search results contain high-level classified files (such as files tagged with "confidential" or "classified"), this invention further determines whether additional authorization is required through a misjudgment risk assessment mechanism to achieve a dynamic and secure access control policy, specifically including:

[0109] When the semantic generalization ability is at a medium level, that is, the semantic generalization ability analysis value generated in the cross-language matching process within a fixed time period T (e.g., 10 minutes, 30 searches) is greater than or equal to the first standard threshold and less than or equal to the second standard threshold; collect the semantic generalization ability analysis values ​​generated in the cross-language matching process within subsequent fixed time periods that are greater than or equal to the first standard threshold and less than or equal to the second standard threshold, establish a dataset, and calculate the mean and standard deviation of the dataset.

[0110] The current semantic generalization is compared with the corresponding reference threshold to determine its stability and mismatch risk level, specifically including:

[0111] If the mean of the semantic generalization ability analysis value in the dataset is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, the system is continuously in a stable but high semantic generalization state; the risk level is high, it is judged as a high-risk mismatch trend, and manual authorization must be applied for to access the high-confidential files.

[0112] If the mean of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, the system is in a state of high drift and large fluctuation in semantic generalization; it is determined to be an unstable matching state, and the high-secret archive content can be blocked and the user can be prompted to adjust the search language or intent;

[0113] If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, the system is generally weak in generalization, but is unstable and judged to have a risk of semantic ambiguity. The system suggests that only the title or classification information of the high-level confidential archives be returned, and secondary confirmation is required before it can be obtained.

[0114] If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, the overall semantic generalization of the system is low and stable, and it is determined to be a controllable state of medium-level matching. Authorizable fields are returned, and desensitization processing is performed on highly sensitive fields.

[0115] When the semantic generalization ability level is low and the user's query language is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

[0116] In the process of cross-language semantic matching, if the system judges the semantic generalization ability level of the current match to be low, it usually means that: the expression of the query statement in the target semantic space has a large drift (i.e., the cross-language similarity offset rate is high); at the same time, the language context consistency score is low, indicating that there is potential misunderstanding, misuse or vague expression.

[0117] If the user's query language does not match the language of the matched file content (for example, the user queries in English, but the matched file is a high-security Chinese file), there is a high risk of mismatch and exposure of high-security files due to semantic misjudgment. Therefore, to prevent security vulnerabilities caused by uncontrolled semantic generalization, a mandatory access blocking mechanism must be implemented for such matching scenarios.

[0118] No content fields (including title, summary, number, etc.) of this highly classified file may be returned; matching records will remain in the internal audit log, but will not be displayed on the user front end.

[0119] The system frontend returns a security warning, such as: "Due to the current search being a cross-language semantic high-drift match and involving highly classified archive content, access has been restricted by the system. If you need to access it, please use the official keywords to search again in the corresponding language environment, or submit an authorization request."

[0120] Record the following for the current matching behavior: query language; file language; user permission level; matched file ID; access denied reason code (e.g., "low-level semantic generalization + cross-language + high-security level").

[0121] When the system determines the semantic generalization ability level of cross-language matching to be low, and the language of the user's query statement is inconsistent with the language of the target file content, the system automatically identifies a potential risk of semantic mismatch. If the file resource involved in the match is a high-security file, the system will implement an access blocking mechanism, that is, prohibit the user from accessing all content fields of the file, and prompt the user to reconfirm the search language or submit an authorization request. At the same time, this operation process will be fully recorded in the access audit log for subsequent authorization management and security traceability.

[0122] Example 2, please refer to Figure 2 As shown in this embodiment, an intelligent retrieval and hierarchical authorization system for archival resources includes a language recognition module, a query semantic understanding module, a semantic matching and retrieval module, a semantic generalization ability evaluation module, and a language dimension permission isolation module.

[0123] Language recognition module: Receives natural language query requests input by the user and determines the language of the query statement;

[0124] Query semantic understanding module: Based on a natural language processing model, the module performs semantic analysis on query statements and generates corresponding multilingual semantic vectors.

[0125] Semantic matching and retrieval module: calculates the semantic similarity between the semantic vector of the user query and the semantic representation of each file in the archive database, and obtains a set of matching results;

[0126] Semantic generalization ability assessment module: For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model;

[0127] Misjudgment Risk Assessment Module: When the semantic generalization ability is at a medium level and the search results contain high-secret files, further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required.

[0128] Language-based access control module: When the semantic generalization ability level is low and the language of the user query is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

[0129] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0130] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for intelligent retrieval and hierarchical authorization of archival resources, characterized in that: include: Receive natural language query requests from users and determine the language of the query statement; Based on a natural language processing model, semantic analysis is performed on the query statement to generate corresponding multilingual semantic vectors; The semantic similarity between the user query semantic vector and the semantic representation of each file in the archive database is calculated to obtain a set of matching results; For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model; When the semantic generalization ability is medium and the search results contain high-secret files, a further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required. When the semantic generalization ability level is low and the user's query language is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources. The method for obtaining the cross-language similarity offset rate (CLSD) is as follows: the user inputs a query statement QA in language A, and a semantic vector in language A is generated through a multilingual semantic model. ; The machine translation module is used to translate QA into archival language B, resulting in the corresponding statement QB; the same semantic vector generation process is then performed on QB to obtain... The calculation of the vector difference between language A and language B in the semantic space for expressing the same query is called the cross-lingual similarity offset rate (CLSD), which is expressed as: In the formula, Here is the formula for calculating cosine similarity; The method for obtaining the language context consistency score is as follows: The original user query statement Qorig is received; it is translated into the target language using a neural machine translation module to obtain Qtrans; the two texts are standardized; Qorig and Qtrans are combined into a sentence pair, which is then input into the cross-lingual natural language inference (XNLI) model. Feature vectors are extracted using its multilingual sentence pair encoding structure and fed into the classifier; the model outputs probability distributions for three categories: Pe: semantically consistent; Pn: semantically ambiguous; Pc: semantically inconsistent; the language context consistency score LCCS is set as: LCCS = Pe.

2. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 1, characterized in that: The language recognition model for determining the language of a query statement includes a language classification model based on character features, word frequency distribution, and language character sets, and outputs the language confidence probability.

3. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 1, characterized in that: Generating multilingual semantic vectors includes: selecting a pre-trained semantic model that matches the query language based on the bound language tags; performing language segmentation on the query statement and constructing word vectors, positional encodings, and paragraph identifiers; inputting the embedded vectors into a Transformer-type neural network for semantic modeling; extracting the sentence vector representing the entire query statement, performing L2 normalization, and outputting it as a semantic vector.

4. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 3, characterized in that: Semantic similarity calculation includes: pre-generating semantic vectors for each file in the archive database using the same semantic model; calculating the normalized cosine similarity between the user semantic vector and the file vector to obtain a similarity score; and filtering file entries with similarity greater than a set threshold to form a preliminary matching result set.

5. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 1, characterized in that: The cross-linguistic similarity offset rate and the language context consistency score are normalized so that they are both between [0,1]. The semantic generalization ability analysis value in the cross-linguistic matching process is calculated based on the normalized cross-linguistic similarity offset rate and the language context consistency score.

6. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 5, characterized in that: The semantic generalization ability analysis value obtained in the cross-language matching process is compared with the gradient standard threshold, which includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold. The semantic generalization ability analysis value in the cross-language matching process is compared with the first standard threshold and the second standard threshold respectively. If the semantic generalization ability analysis value in the cross-language matching process is greater than the second standard threshold, it indicates poor semantic generalization ability. It will be marked as high semantic generalization ability, triggering the language dimension permission isolation mechanism or blocking access. If the semantic generalization ability analysis value in the cross-language matching process is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is marked as medium semantic generalization ability, and a risk assessment for misjudgment is performed on high-secret archives. If the semantic generalization ability analysis value in the cross-language matching process is less than the first standard threshold, it indicates that the semantic expression is stable. It is marked as having low semantic generalization ability, and semantic matching and result display are carried out according to the normal process.

7. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 6, characterized in that: When the semantic generalization ability is at the medium level, that is, the semantic generalization ability analysis value generated in the cross-language matching process within a fixed time period T is greater than or equal to the first standard threshold and less than or equal to the second standard threshold; collect the semantic generalization ability analysis values ​​generated in the cross-language matching process within subsequent fixed time periods that are greater than or equal to the first standard threshold and less than or equal to the second standard threshold, establish a dataset, and calculate the mean and standard deviation of the dataset; If the mean of the semantic generalization ability analysis value in the dataset is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, the risk level is high, and it is judged as a high-risk mismatch trend. It is mandatory to apply for manual authorization to access the high-confidential files. If the mean of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, it is determined to be an unstable matching state. The high-secret archive content is blocked and the user is prompted to adjust the search language or intent. If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, only the title or classification information of the high-level confidential archive will be returned, and secondary confirmation is required before it can be obtained. If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, it is determined to be a controllable state of medium-level matching, and authorized fields are returned. Desensitization processing is performed on highly sensitive fields.

8. An intelligent retrieval and hierarchical authorization system for archival resources, used to implement the intelligent retrieval and hierarchical authorization method for archival resources as described in any one of claims 1-7, characterized in that: It includes a language recognition module, a query semantic understanding module, a semantic matching and retrieval module, a semantic generalization ability evaluation module, and a language dimension permission isolation module; Language recognition module: Receives natural language query requests input by the user and determines the language of the query statement; Query semantic understanding module: Based on a natural language processing model, the module performs semantic analysis on query statements and generates corresponding multilingual semantic vectors. Semantic matching and retrieval module: calculates the semantic similarity between the semantic vector of the user query and the semantic representation of each file in the archive database, and obtains a set of matching results; Semantic generalization ability assessment module: For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model; Misjudgment Risk Assessment Module: When the semantic generalization ability is at a medium level and the search results contain high-secret files, further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required. Language-based access control module: When the semantic generalization ability level is low and the language of the user query is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

Images