Archive resource intelligent retrieval and hierarchical authorization utilization system and method

Abstract

The invention discloses an intelligent retrieval and hierarchical authorization utilization system and method for archive resources, and belongs to the field of intelligent retrieval. Natural language query of a user is received, the language type of the user is recognized, a semantic vector is generated in combination with a multi-language natural language processing model, and then the semantic vector is matched with each archive semantic representation in an archive database; for the situation related to cross-language matching, a cross-language similarity offset rate and a language context consistency score are further calculated, the semantic generalization ability is divided into a high level, a middle level and a low level according to a preset weight model, and for the middle level situation, a misjudgment risk assessment mechanism is introduced to judge whether additional authorization is needed or not; and when the file is low in level and the languages are inconsistent, the user is directly prohibited from accessing the high-security-level file, so that the information leakage risk caused by semantic generalization is effectively prevented, and the high-sensitivity file is safely and accurately controlled in a multi-language environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent retrieval, and particularly relates to an intelligent retrieval and hierarchical authorization utilization system and method for archive resources. BACKGROUND

[0002] The intelligent retrieval and hierarchical authorization utilization of archive resources refers to the intelligent retrieval and classification processing of archive resources by artificial intelligence and big data technology, automatic judgment of the accessible archive range of a user according to the permission level of the user, and a mechanism for safe, efficient and accurate acquisition and reasonable utilization of archive information. This mode helps to improve the intelligent level of archive management, and ensures information security and compliance use.

[0003] The prior art has the following deficiencies: In the prior art, when performing semantic retrieval by using an NLP model in a multilingual environment, the strong generalization ability of the NLP model may lead to cross-language matching of the user to high-density archive content. If the system does not set an independent permission isolation mechanism for the language dimension, the confidential archive may be misjudged as accessible after semantic mis-matching. SUMMARY

[0004] The present application provides an intelligent retrieval and hierarchical authorization utilization system and method for archive resources to solve the problems in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical solution: an intelligent retrieval and hierarchical authorization utilization method for archive resources, comprising: receiving a natural language query request input by a user and determining the language type of the query sentence; performing semantic analysis on the query sentence based on a natural language processing model to generate a corresponding multilingual semantic vector; performing semantic similarity calculation on the user query semantic vector and the semantic representation of each archive in the archive database to obtain a matching result set; calculating the cross-language similarity offset rate and the language context consistency score for the cross-language matching involved in the semantic matching process, and dividing the semantic generalization ability into three levels of high, medium and low according to a preset weight model; when the semantic generalization ability is at a medium level and the retrieval result contains high-density archive, further performing misjudgment risk assessment on the matching result to determine whether additional authorization is required; when the semantic generalization ability level is at a low level and the language of the user query is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-density archive resources.

[0006] Preferably, the language recognition model for determining the query language category comprises a language classification model based on character features, word frequency distribution and language character set, and outputs a language confidence probability.

[0007] Preferably, the generation of the multi-language semantic vector comprises: selecting a pre-trained semantic model matching the query language according to the bound language label; performing language segmentation processing on the query sentence, and constructing word vectors, position encoding and paragraph identification; inputting the embedding vector into a Transformer type neural network for semantic modeling; extracting a sentence vector representing the entire query sentence, performing L2 normalization, and outputting the semantic vector.

[0008] Preferably, the semantic similarity calculation comprises: pre-generating semantic vectors of each archive in the archive database through the same semantic model; performing normalized cosine similarity calculation on the user semantic vector and the archive vector to obtain a similarity score; screening archive entries with a similarity greater than a set threshold to form a preliminary matching result set.

[0009] Preferably, the method for obtaining the cross-language similarity deviation rate CLSD is: a user inputs a query sentence QA in language A, and generates a semantic vector in language A through a multi-language semantic model ; translating the QA into an archive language B using a machine translation module to obtain a comparison sentence QB; performing the same semantic vector generation processing on the QB to obtain ; calculating the vector difference in the semantic space of the same query expressed in language A and language B, i.e. calculating the cross-language similarity deviation rate CLSD, the expression is: ; in the formula, is a cosine similarity calculation formula.

[0010] Preferably, the method for obtaining the language context consistency score is: receiving an original user query sentence Qorig; translating it into a target language using a neural machine translation module to obtain Qtrans; performing standardization processing on the two texts; forming a sentence pair of Qorig and Qtrans, inputting it into a cross-language natural language inference XNLI model, extracting a feature vector using the multi-language sentence pair encoding structure of the model, and inputting it into a classifier; the model outputs the probability distribution of three categories: Pe: semantic consistency; Pn: semantic ambiguity; Pc: semantic inconsistency; set the language context consistency score LCCS as: LCCS=Pe.

[0011] Preferably, the cross-language similarity deviation rate and the language context consistency score are normalized to be between 0 and 1, and the semantic generalization ability analysis value in the cross-language matching process is calculated according to the normalized cross-language similarity deviation rate and the language context consistency score.

[0012] Preferably, the acquired semantic generalization capability analysis value in the cross-language matching process is compared with gradient standard threshold values, the gradient standard threshold values include a first standard threshold value and a second standard threshold value, and the first standard threshold value is smaller than the second standard threshold value; the semantic generalization capability analysis value in the cross-language matching process is compared with the first standard threshold value and the second standard threshold value respectively; If the semantic generalization capability analysis value in the cross-language matching process is greater than the second standard threshold value, it indicates that the semantic generalization capability is poor, and is marked as high semantic generalization capability, triggering the language dimension permission isolation mechanism or preventing access; If the semantic generalization capability analysis value in the cross-language matching process is greater than or equal to the first standard threshold value and less than or equal to the second standard threshold value, it is marked as medium semantic generalization capability, and a false positive risk assessment is performed on high-density archives; If the semantic generalization capability analysis value in the cross-language matching process is less than the first standard threshold value, it indicates that the semantic expression is stable, and is marked as low semantic generalization capability, and the semantic matching and result display are performed according to the normal process.

[0013] Preferably, when the semantic generalization capability is medium, that is, the semantic generalization capability analysis value in the cross-language matching process generated within a fixed time period T is greater than or equal to the first standard threshold value and less than or equal to the second standard threshold value; the semantic generalization capability analysis values greater than or equal to the first standard threshold value and less than or equal to the second standard threshold value generated in the subsequent fixed time period are collected, a data set is established, and the mean and standard deviation of the data set are calculated; If the mean of the semantic generalization capability analysis values in the data set is greater than or equal to the reference threshold value of the mean of the semantic generalization capability analysis values, and the standard deviation of the semantic generalization capability analysis values is less than the reference threshold value of the standard deviation of the semantic generalization capability analysis values, the risk level is high, and it is determined that there is a high-risk false matching trend, and the applicant can access the high-density archives only after being authorized by the artificial force; If the mean of the semantic generalization capability analysis values is greater than or equal to the reference threshold value of the mean of the semantic generalization capability analysis values, and the standard deviation of the semantic generalization capability analysis values is greater than or equal to the reference threshold value of the standard deviation of the semantic generalization capability analysis values, it is determined that the matching state is unstable, and the content of the high-density archives is shielded and the user is prompted to adjust the search language or intention; If the mean of the semantic generalization capability analysis values is less than the reference threshold value of the mean of the semantic generalization capability analysis values, and the standard deviation of the semantic generalization capability analysis values is greater than or equal to the reference threshold value of the standard deviation of the semantic generalization capability analysis values, only the title or classification information of the high-density archives is returned, and the high-density archives can be obtained only after secondary confirmation; If the mean of the semantic generalization capability analysis values is less than the reference threshold value of the mean of the semantic generalization capability analysis values, and the standard deviation of the semantic generalization capability analysis values is less than the reference threshold value of the standard deviation of the semantic generalization capability analysis values, it is determined that the matching state is controllable, and the authorized field is returned, and the desensitization processing is performed on the high-sensitivity field.

[0014] The application also provides an archive resource intelligent retrieval and hierarchical authorization utilization system, comprising a language recognition module, a query semantic understanding module, a semantic matching and retrieval module, a semantic generalization capability evaluation module, and a language dimension permission isolation module. The language recognition module receives a natural language query request input by a user and determines the language type of the query statement. The query semantic understanding module performs semantic analysis on the query statement based on a natural language processing model to generate a corresponding multilingual semantic vector. The semantic matching and retrieval module performs semantic similarity calculation on the user query semantic vector and the semantic representation of each archive in the archive database to obtain a matching result set. The semantic generalization capability evaluation module calculates the cross-language similarity offset rate and the language context consistency score for the cross-language matching involved in the semantic matching process, and divides the semantic generalization capability into three levels of high, medium and low according to a preset weight model. The misjudgment risk evaluation module further evaluates the matching results when the semantic generalization capability is at the medium level and the retrieval results contain high-density-level archives, and determines whether additional authorization is required. The language dimension permission isolation module prohibits the user from accessing the matched high-density-level archive resources when the semantic generalization capability is at the low level and the user query language is inconsistent with the archive content language.

[0015] In the above technical solution, the application provides the following technical effects and advantages: 1. The application proposes a semantic generalization capability evaluation mechanism based on cross-language semantic offset and context consistency score to accurately judge the credibility of semantic matching and divide the semantic generalization level, thereby realizing dynamic authorization control and access restriction for medium and high risk scenarios, to solve the problem of high-density-level archive mis-matching risk in semantic retrieval in a multi-language environment. This method effectively avoids the problem of sensitive information mis-triggering caused by language ambiguity and semantic drift, and enhances the security and intelligence level of the multi-language archive retrieval system.

[0016] 2. The application further improves the risk identification capability of medium-level fuzzy matching by fusing natural language processing, machine translation, semantic similarity calculation, natural language inference (XNLI) and other AI technologies, by constructing a semantic generalization capability analysis value and introducing a time series statistical and semantic fluctuation analysis mechanism, to support fine-grained permission control and access decision-making. This method improves the compliance and controllability of high-density-level archive access while ensuring retrieval accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a mind map of the method of the present invention.

[0019] Figure 2 This is a mind map of the system modules of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1, please refer to Figure 1 As shown in this embodiment, a method for intelligent retrieval and hierarchical authorization of archival resources includes: Receive natural language query requests from users and determine the language of the query statement; Based on a natural language processing model, semantic analysis is performed on the query statement to generate corresponding multilingual semantic vectors; The semantic similarity between the user query semantic vector and the semantic representation of each file in the archive database is calculated to obtain a set of matching results; For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model; When the semantic generalization ability is medium and the search results contain high-secret files, a further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required. When the semantic generalization ability level is low and the user's query language is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

[0022] Users can input natural language queries in the following ways, including but not limited to: Text input: Users manually enter search queries through a front-end interface (such as a webpage, system client, or mobile application), such as "find overseas cooperation archives in 2020".

[0023] Voice-to-text input: User inputs through voice, and the system converts voice content into text with the help of a voice recognition module, and then enters the subsequent processing flow.

[0024] Interface data input (system integration scenario): If the system is integrated with other business systems, user search requests can be automatically transmitted through API or middleware, submitted in JSON or plain text format.

[0025] Regardless of the way, the system uniformly converts user queries into standardized UTF-8 encoded text format and enters the language recognition stage.

[0026] Language identification model (LID) is used to determine which language the text belongs to. Its core goal is to identify the language category based on language features such as word distribution and syntax structure without understanding the text semantics. The model generally has the following technical composition: Feature extraction module: Mainly extracts linguistic features from input text, commonly including: word frequency distribution (such as "de" and "le" frequently appear in Chinese); character n-gram pattern (such as "th" and "ing" frequently appear in English); specific language character set (such as Cyrillic used in Russian); special symbol and punctuation rules.

[0027] Model algorithm main body: According to system accuracy requirements and performance conditions, different complexity models can be used, including: rule matching model (based on predefined character / word rules to determine); statistical model (such as Naive Bayes, TF-IDF language fingerprint matching); pre-trained language recognition model (such as fastText language recognizer or CLD3 / CLD2 model); multilingual BERT embedding + classification head (for high-precision scenarios).

[0028] Excellent language recognition models support hundreds of languages and can maintain high accuracy in short texts. It is particularly important to identify similar languages (such as Chinese simplified / complex, English / Dutch) with precision.

[0029] Model inference: After inputting the standardized text, the system calls the language recognition model for inference and judgment, outputting a set of candidate languages and their probability distribution.

[0030] Language confidence calculation: The model assigns a confidence score to each candidate language (e.g., Chinese 0.93, English 0.06, Japanese 0.01). The confidence score can be used for subsequent fault tolerance processing.

[0031] Language determination logic: If the highest confidence value exceeds a set threshold (e.g., ≥ 0.90), it is directly determined as the language; if multiple language confidences are close, the system will be marked as "language uncertain" and prompt the user to confirm or select the language; for mixed language text (e.g., Chinese and English mixed), the system can identify the main language as the main language and supplement the secondary language tag.

[0032] Special processing mechanism: For short text (e.g., only a few words), enable context enhancement mechanism, enhance the judgment through user historical input, interface language or geographical location; for important users in the system (e.g., high-privilege users), auxiliary decision can be made in combination with user language preference; in multi-round dialogue system, consistency is maintained in combination with the previous round of language, and stability is improved.

[0033] Accurate identification of language types is a core link in the intelligent retrieval system of archives, which mainly plays the following roles: determining the subsequent semantic model (Chinese and English BERT cannot be mixed); Guiding the security policy of cross-language semantic matching; supporting language dimension permission isolation mechanism to prevent semantic misjudgment from causing high-density level archive leakage; optimizing the display language of retrieval results to improve user experience.

[0034] Based on natural language processing model, the semantic analysis of query statement is carried out, and the corresponding multilingual semantic vector is generated, which includes: The natural language query statement input by the user is uniformly converted to UTF-8 encoding, and special symbols, illegal characters, etc. are removed to ensure that the text format meets the input requirements of the model.

[0035] Use language recognition model (such as CLD3, fastText) to determine the language type of the input text, and bind the corresponding language label (such as zh, en, ru, etc.) for subsequent model selection.

[0036] Perform operations such as removing stop words, punctuation normalization, date / digit standardization, etc. to improve the effective input quality of the semantic model.

[0037] According to the language label, select the pre-trained multilingual model corresponding to the language, for example: Chinese: Chinese BERT, ERNIE; English: English BERT, RoBERTa; Multilingual general: Multilingual-BERT (mBERT), XLM-RoBERTa.

[0038] Load the required model weights, vocabulary, position encoding, etc. from the model library to prepare the neural network structure required for vector generation.

[0039] Language-specific tokenization: English uses WordPiece or BPE (Byte-Pair Encoding); Chinese uses character-level segmentation or dictionary-based tokenizer (such as jieba); multi-language models use unified word encoding.

[0040] Map each tokenization result to a fixed-dimensional word vector, including: word vector; position encoding; language / paragraph identification; these vectors will collectively form the tensor form of the model input.

[0041] Input the preprocessed embedding into a multi-layer Transformer structure (such as BERT layer) for context-dependent semantic modeling.

[0042] Extract the full sentence semantic representation vector from the model output, common methods include: Use the output of the [CLS] position as the semantic vector of the entire query sentence; Or average pooling (Mean Pooling) processing is performed on all Token vectors.

[0043] L2 normalize the generated semantic vector to form a fixed-length, cross-language aligned vector representation, and cache it to improve subsequent comparison efficiency.

[0044] Output a uniform floating-point vector (such as 768 dimensions or 1024 dimensions) representing the user's query position in the semantic space, which can be used for similarity calculation with the archive semantic vector.

[0045] Attach semantic vectors with language labels, recognition confidence, context identification, and other metadata to provide necessary support for the next stage of "cross-language semantic comparison" and "risk assessment".

[0046] Calculate the semantic similarity between the user query semantic vector and the semantic representation of each archive in the archive database to obtain a matching result set, which includes: In the initial stage or regular update stage of the system, each archive record (including title, abstract, body, etc.) in the database is processed through the same pre-trained semantic model as the user query to generate a semantic vector.

[0047] L2 normalize all archive semantic vectors and store them in a vector retrieval engine (such as FAISS, Milvus, Weaviate), along with metadata such as archive ID, language label, and security level information.

[0048] Set the vector dimension to d, let: the user query semantic vector be ; R is the set of real numbers; The archive semantic vector set is , where ; for each profile vector , compute its cosine similarity with the query vector , expressed as: ; where: denotes the dot product of two vectors; the result ranges from [−1, 1], and the closer the value is to 1, the more similar the semantics.

[0049] The system iterates through all profile vectors or performs a nearest neighbor search using a vector search engine (such as FAISS) to obtain the top k most similar vectors and their similarity scores.

[0050] A similarity lower threshold θ is introduced to filter out low-relevance results, typically taking values such as 0.7, 0.8, etc., which can be dynamically adjusted based on system accuracy requirements.

[0051] All profiles with ≥θ are retained, sorted by similarity from high to low, forming a preliminary matching result set. Additional profile metadata (classification, language, creation time, topic tags, etc.) is attached to the matching results for subsequent permission judgment and display control.

[0052] For cross-language matching involved in the semantic matching process, calculate the cross-language similarity deviation rate and language context consistency score, and divide the semantic generalization ability into high, medium, and low levels according to the preset weight model, which includes: Cross-language similarity deviation rate CLSD, used to measure the degree of similarity fluctuation between user queries in different languages and target profile semantic vectors, to identify the stability and risk of cross-language semantic models.

[0053] The method is as follows: the user inputs the query statement QA in language A (such as English), and generates the semantic vector in language A through a multilingual semantic model (such as XLM-R). Use a machine translation module to translate QA into profile language B (such as Chinese) to get the comparison statement QB; perform the same semantic vector generation process on QB to get ; translation uses context-aware models such as Transformer or neural network translation engines (such as Google Translate API, Baidu Translate, DeepL) to reduce semantic loss.

[0054] Calculate the vector difference in semantic space between the same query expressed in language A and language B, i.e., calculate the cross-language similarity deviation rate CLSD, expressed as: ; in the formula, is the cosine similarity calculation formula.

[0055] ​If CLSD < 0.1, it means that the cross-language expression consistency is high, and semantic matching can be performed with confidence; if CLSD is between 0.1-0.3, it means that there is moderate semantic drift, and it is recommended to perform semantic fuzzy matching annotation; if CLSD > 0.3, it means high-risk semantic drift, and the language isolation mechanism or manual review should be triggered. A language context consistency score LCCS is used to judge whether the language use is professional, accurate and consistent with the user context through context comparison, to prevent improper semantic expansion caused by language ambiguity.

[0056] The method for obtaining the language context consistency score is as follows: receiving an original user query sentence Qorig (such as English); using a neural machine translation module to translate it into a target language (such as Chinese), obtaining Qtrans; standardizing the two sentences (denoising, punctuation unification, space specification, etc.); Qorig and Qtrans form a sentence pair, which is input into the cross-language natural language inference XNLI model, and the feature vector is extracted using the multilingual sentence pair coding structure (such as Transformer) and input into the classifier; the model outputs the probability distribution of three categories: Pe: Entailment (semantic consistency); Pn: Neutral (semantic ambiguity); Pc: Contradiction (semantic inconsistency); The language context consistency score LCCS is set as: LCCS = Pe; that is, the LCCS value is equal to the probability determination result of the model for the semantic consistency of the two sentences.

[0057] If LCCS is greater than or equal to 0.85, it means that the semantics are highly consistent, and high-precision semantic retrieval can be performed; if 0.60 ≤ LCCS < 0.85, it means that there is context drift, and it is recommended to trigger fuzzy matching marking; if LCCS is less than 0.60, it means that there is potential misunderstanding or semantic conflict, and the language isolation mechanism is started.

[0058] The cross-language similarity deviation rate and the language context consistency score are normalized to be between 0 and 1, and the semantic generalization ability analysis value in the cross-language matching process is calculated according to the normalized cross-language similarity deviation rate and the language context consistency score.

[0059] For example, the present application can use the following formula to calculate the semantic generalization ability analysis value in the cross-language matching process, and the calculation expression is: ; in the formula, is the semantic generalization ability analysis value in the cross-language matching process, is the cross-language similarity deviation rate, is the language context consistency score, is a weight coefficient of the cross-language similarity offset rate and the language context consistency score (which can be optimized according to experimental experience or machine learning), and are both greater than 0.

[0060] The semantic generalization capability analysis value obtained in the cross-language matching process is compared with gradient standard thresholds, the gradient standard thresholds include a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold. The semantic generalization capability analysis value in the cross-language matching process is compared with the first standard threshold and the second standard threshold, respectively. If the semantic generalization capability analysis value in the cross-language matching process is greater than the second standard threshold, it indicates that the semantic generalization capability is poor, there is a large cross-language semantic offset and context inconsistency, and it is marked as high semantic generalization capability, which should trigger the language dimension permission isolation mechanism or prevent access. If the semantic generalization capability analysis value in the cross-language matching process is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it indicates that the semantic generalization has certain uncertainty, and it is marked as medium semantic generalization capability, which suggests that a false positive risk assessment should be performed on high-density archives, or only part of the information is returned. If the semantic generalization capability analysis value in the cross-language matching process is less than the first standard threshold, it indicates that the semantic expression is stable, and it is marked as low semantic generalization capability, and the semantic matching and result display can be performed according to the normal process.

[0061] When the semantic generalization capability analysis value is in the medium level interval, i.e., greater than or equal to the first standard threshold and less than or equal to the second standard threshold, if the search result contains high-density archives (such as "confidential" and "classified" labeled archives), the application further determines whether additional authorization is required through a false positive risk assessment mechanism to realize a dynamic and safe access control strategy, which specifically includes: When the semantic generalization capability is in the medium level, i.e., the semantic generalization capability analysis value in the cross-language matching process generated within a fixed time period T (for example, 10 minutes, 30 times of search) is greater than or equal to the first standard threshold and less than or equal to the second standard threshold; the semantic generalization capability analysis value in the cross-language matching process greater than or equal to the first standard threshold and less than or equal to the second standard threshold generated within the subsequent fixed time period is collected, and a data set is established, and the mean and standard deviation of the data set are calculated.

[0062] It is compared with the corresponding reference threshold to judge the stability of the current semantic generalization and the false matching risk level, which specifically includes: If the average semantic generalization capability analysis value in the data set is greater than or equal to the reference threshold value of the average semantic generalization capability analysis value, and the standard deviation of the semantic generalization capability analysis value is less than the reference threshold value of the standard deviation of the semantic generalization capability analysis value, the system is in a stable but high semantic generalization state; the risk level is high, and it is determined that there is a high-risk mismatch trend, and the applicant must be authorized to access the high-density level archives; If the average semantic generalization capability analysis value is greater than or equal to the reference threshold value of the average semantic generalization capability analysis value, and the standard deviation of the semantic generalization capability analysis value is greater than or equal to the reference threshold value of the standard deviation of the semantic generalization capability analysis value, the system is in a high-drift and large-fluctuation semantic generalization state; it is determined that the matching state is unstable, and the high-density level archive content can be shielded and the user is prompted to adjust the search language or intent; If the average semantic generalization capability analysis value is less than the reference threshold value of the average semantic generalization capability analysis value, and the standard deviation of the semantic generalization capability analysis value is greater than or equal to the reference threshold value of the standard deviation of the semantic generalization capability analysis value, the system is generally weakly generalized, but unstable, and it is determined that there is a semantic ambiguity risk, and the system suggests returning only the high-density level archive title or classification information, which can be obtained after secondary confirmation; If the average semantic generalization capability analysis value is less than the reference threshold value of the average semantic generalization capability analysis value, and the standard deviation of the semantic generalization capability analysis value is less than the reference threshold value of the standard deviation of the semantic generalization capability analysis value, the system is generally low and stable in semantic generalization, and it is determined that the matching state is controllable, and the returnable authorized field is returned, and desensitization processing is performed on the high-sensitive field.

[0063] When the semantic generalization capability level is low, and the user query language is inconsistent with the archive content language, the user is prohibited from accessing the matched high-density level archive resource.

[0064] In the cross-language semantic matching process, if the system determines that the current matching semantic generalization capability level is low, it usually means that: the query statement has a large expression drift in the target semantic space (i.e., the cross-language similarity offset rate is high); and the language context consistency score is low, indicating that there is potential misunderstanding, misuse or ambiguous expression.

[0065] If the user's query language is inconsistent with the matched archive content language at this time (for example, the user uses English query, and the matched is a Chinese high-density level archive), it is extremely likely that there will be a high-density level archive mismatch exposure risk caused by semantic misjudgment. Therefore, in order to prevent security vulnerabilities caused by out-of-control semantic generalization, a forced access blocking mechanism needs to be performed on this type of matching situation.

[0066] Any content field (including title, abstract, number, etc.) of the high-density level archive is prohibited from being returned; the matching record is still retained in the internal audit log, but is not displayed to the user front end.

[0067] System front-end returns a security prompt, such as: "Due to the current search is cross-language semantic high drift matching, and involves high-density level archive content, access has been limited by the system. If you need to access, please use the official keyword to re-search in the corresponding language environment, or submit an authorization application." Records the current matching behavior: query language; archive language; user permission level; matched archive ID; access denial reason code (such as "low-level semantic generalization + cross-language + high-density level").

[0068] When the semantic generalization ability level of cross-language matching is determined by the system to be low, and the language of the user query statement is inconsistent with the language of the target archive content, the system automatically determines that there is a potential risk of semantic mis-matching. If the archive resources involved in this matching are high-density level archives, the system will execute the access blocking mechanism, that is, prohibit the user from accessing all content fields of the archive, and prompt the user to reconfirm the search language or submit an authorization application. At the same time, this operation process will be completely recorded in the access audit log for subsequent authorization management and security tracing.

[0069] Embodiment 2, please refer to Figure 2 The archive resource intelligent retrieval and hierarchical authorization system described in the embodiment includes a language recognition module, a query semantic understanding module, a semantic matching and retrieval module, a semantic generalization ability evaluation module, and a language dimension permission isolation module. Language recognition module: receives the user's natural language query request and determines the language type of the query statement; Query semantic understanding module: based on natural language processing model, performs semantic analysis on the query statement to generate corresponding multilingual semantic vector; Semantic matching and retrieval module: calculate the semantic similarity between the user query semantic vector and the semantic representation of each archive in the archive database to obtain the matching result set; Semantic generalization ability evaluation module: calculate the cross-language similarity deviation rate and language context consistency score for the cross-language matching involved in the semantic matching process, and divide the semantic generalization ability into high, medium and low levels according to the preset weight model; Misjudgment risk assessment module: when the semantic generalization ability is medium and the search result contains high-density level archives, further assess the misjudgment risk of the matching result to determine whether additional authorization is needed; Language dimension permission isolation module: when the semantic generalization ability level is low, and the user query language is inconsistent with the archive content language, prohibit the user from accessing the matched high-density level archive resources.

[0070] The above formulas are all dimensionless values calculated, the formula is obtained by collecting a large amount of data to simulate the recent real situation, and the preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0071] It should be understood that the term "and / or" in this document is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after it, but it can also represent an "and / or" relationship. The specific meaning can be understood according to the context before and after it.

[0072] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0073] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for intelligent retrieval and hierarchical authorization of archival resources, characterized in that: include: Receive natural language query requests from users and determine the language of the query statement; Based on a natural language processing model, semantic analysis is performed on the query statement to generate corresponding multilingual semantic vectors; The semantic similarity between the user query semantic vector and the semantic representation of each file in the archive database is calculated to obtain a set of matching results; For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model; When the semantic generalization ability is medium and the search results contain high-secret files, a further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required. When the semantic generalization ability level is low and the user's query language is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

2. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 1, characterized in that: The language recognition model for determining the language type of the query statement includes a language classification model based on character features, word frequency distribution, and language character sets, and outputs the language confidence probability.

3. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 1, characterized in that: Generating multilingual semantic vectors includes: selecting a pre-trained semantic model that matches the query language based on the bound language tags; performing language segmentation on the query statement and constructing word vectors, positional encodings, and paragraph identifiers; inputting the embedded vectors into a Transformer-type neural network for semantic modeling; extracting the sentence vector representing the entire query statement, performing L2 normalization, and outputting it as a semantic vector.

4. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 3, characterized in that: Semantic similarity calculation includes: pre-generating semantic vectors for each file in the archive database using the same semantic model; calculating the normalized cosine similarity between the user semantic vector and the file vector to obtain a similarity score; and filtering file entries with similarity greater than a set threshold to form a preliminary matching result set.

5. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 4, characterized in that: The method for obtaining the Cross-Language Similarity Shift (CLSD) is as follows: the user inputs a query statement QA in language A, and a semantic vector in language A is generated through a multilingual semantic model. ; The machine translation module is used to translate QA into archival language B, resulting in the corresponding statement QB. Perform the same semantic vector generation process on QB to obtain The calculation of the vector difference between language A and language B in the semantic space for expressing the same query is called the cross-lingual similarity offset rate (CLSD), which is expressed as: In the formula, This is the formula for calculating cosine similarity.

6. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 5, characterized in that: The method for obtaining the language context consistency score is as follows: receive the original user query statement Qorig; use the neural machine translation module to translate it into the target language to obtain Qtrans; and perform standardization processing on the two texts. The Qorig and Qtrans are combined into a sentence pair and input into the cross-lingual natural language inference XNLI model. The feature vector is extracted using its multilingual sentence pair encoding structure and fed into the classifier. The model outputs the probability distribution of three categories: Pe: semantic consistency. Pn: semantic ambiguity; Pc: semantic inconsistency; set the linguistic context consistency score LCCS as: LCCS=Pe.

7. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 6, characterized in that: The cross-linguistic similarity offset rate and the language context consistency score are normalized so that they are both between [0,1]. The semantic generalization ability analysis value in the cross-linguistic matching process is calculated based on the normalized cross-linguistic similarity offset rate and the language context consistency score.

8. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 7, characterized in that: The semantic generalization ability analysis value obtained in the cross-language matching process is compared with the gradient standard threshold, which includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold. The semantic generalization ability analysis value in the cross-language matching process is compared with the first standard threshold and the second standard threshold respectively. If the semantic generalization ability analysis value in the cross-language matching process is greater than the second standard threshold, it indicates poor semantic generalization ability. It will be marked as high semantic generalization ability, triggering the language dimension permission isolation mechanism or blocking access. If the semantic generalization ability analysis value in the cross-language matching process is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it is marked as medium semantic generalization ability, and a risk assessment for misjudgment is performed on high-secret archives. If the semantic generalization ability analysis value in the cross-language matching process is less than the first standard threshold, it indicates that the semantic expression is stable. It is marked as having low semantic generalization ability, and semantic matching and result display are carried out according to the normal process.

9. The method for intelligent retrieval and hierarchical authorization of archival resources according to claim 8, characterized in that: When the semantic generalization ability is at the medium level, that is, the semantic generalization ability analysis value generated in the cross-language matching process within a fixed time period T is greater than or equal to the first standard threshold and less than or equal to the second standard threshold; collect the semantic generalization ability analysis values ​​generated in the cross-language matching process within subsequent fixed time periods that are greater than or equal to the first standard threshold and less than or equal to the second standard threshold, establish a dataset, and calculate the mean and standard deviation of the dataset; If the mean of the semantic generalization ability analysis value in the dataset is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, the risk level is high, and it is judged as a high-risk mismatch trend. It is mandatory to apply for manual authorization to access the high-confidential files. If the mean of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, it is determined to be an unstable matching state. The high-secret archive content is blocked and the user is prompted to adjust the search language or intent. If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is greater than or equal to the reference threshold of the standard deviation of the semantic generalization ability analysis value, only the title or classification information of the high-level confidential archive will be returned, and secondary confirmation is required before it can be obtained. If the mean of the semantic generalization ability analysis value is less than the reference threshold of the mean of the semantic generalization ability analysis value, and the standard deviation of the semantic generalization ability analysis value is less than the reference threshold of the standard deviation of the semantic generalization ability analysis value, it is determined to be a controllable state of medium-level matching, and authorized fields are returned. Desensitization processing is performed on highly sensitive fields.

10. An intelligent retrieval and hierarchical authorization system for archival resources, used to implement the intelligent retrieval and hierarchical authorization method for archival resources as described in any one of claims 1-9, characterized in that: It includes a language recognition module, a query semantic understanding module, a semantic matching and retrieval module, a semantic generalization ability evaluation module, and a language dimension permission isolation module; Language recognition module: Receives natural language query requests input by the user and determines the language of the query statement; Query semantic understanding module: Based on a natural language processing model, the module performs semantic analysis on query statements and generates corresponding multilingual semantic vectors. Semantic matching and retrieval module: calculates the semantic similarity between the semantic vector of the user query and the semantic representation of each file in the archive database, and obtains a set of matching results; Semantic generalization ability assessment module: For cross-language matching involved in the semantic matching process, calculate the cross-language similarity offset rate and language context consistency score, and divide the semantic generalization ability into three levels: high, medium and low according to the preset weight model; Misjudgment Risk Assessment Module: When the semantic generalization ability is at a medium level and the search results contain high-secret files, further misjudgment risk assessment is performed on the matching results to determine whether additional authorization is required. Language-based access control module: When the semantic generalization capability level is low and the language of the user query is inconsistent with the language of the archive content, the user is prohibited from accessing the matched high-security archive resources.

Images