Data classification and grading method and device based on artificial intelligence large model technology

Abstract

The invention provides a data classification and grading method and device based on an artificial intelligence large model technology, and relates to the field of data security. The method comprises the following steps: constructing a data classification and grading catalog according to a preset classification rule, and storing each row of the classification and grading catalog as a document record in a target vector database; respectively carrying out first retrieval operation, second retrieval operation and third retrieval operation on the data classification and grading catalog by taking the business system name, the data table name and the field name as first query conditions so as to obtain a first matching document, a second matching document and a third matching document in the target vector database; sorting the first matching document, the second matching document and the third matching document according to a preset sorting mode, and obtaining a sorting result; and inputting a sorting result into the large language model, and performing data security early warning through the large language model. According to the method and the device, the security problems that corresponding security measures cannot be taken, sensitive data is leaked and the like when a data governance engineer identifies the metadata according to the data classification and classification catalog and judges the classification and security level of the metadata are solved.

Description

Technical Field

[0001] The present application relates to the field of data security, and in particular to a data classification and grading method and device based on artificial intelligence large model technology. Background Art

[0002] In the current context of increasingly stringent data security and compliance supervision, data classification and grading, as a basic link in data lifecycle management, has become one of the core means for enterprises to implement data security protection systems, meet legal and regulatory compliance requirements, and improve the controllability of data assets.

[0003] The existing data classification and categorization process involves data governance engineers identifying metadata based on a data classification and categorization catalog and determining its classification and security level. This approach alone relies heavily on the individual judgment of the data governance engineer. Due to varying experience levels among data governance engineers, less experienced engineers may make errors. Alternatively, dynamic business scenarios such as cloud data migration, data sharing and access, real-time system access, and security threat response require immediate identification of data classification and security level. If this traditional approach isn't handled promptly enough, appropriate security measures may not be implemented, leading to security issues such as sensitive data leaks.

[0004] Therefore, there is an urgent need for a data classification and grading method and device based on artificial intelligence large model technology. Summary of the Invention

[0005] The present application provides a data classification and grading method and device based on artificial intelligence big model technology, which solves the security problems that data governance engineers may not be able to take corresponding security measures and may cause sensitive data leakage when identifying metadata according to the data classification and grading catalog and judging the classification and security level of metadata.

[0006] In a first aspect of the present application, a data classification and grading method based on artificial intelligence big model technology is provided, the method comprising: constructing a data classification and grading directory according to preset classification rules, and saving each row of the classification and grading directory as a document record to a target vector database; using the business system name, data table name and field name as a first query condition, performing a first search operation on the data classification and grading directory, and obtaining a first matching document in the target vector database through the first search operation; using the business system name, data table name and field name as a second query condition, and embedding the second query condition into a dense vector using a multimodal embedding model, performing a second search operation on the data classification and grading directory, and obtaining a second matching document in the target vector database through the second search operation; using the business system name, data table name and field name as a third query condition, and embedding the third query condition into a sparse vector using a multimodal embedding model, performing a third search operation on the data classification and grading directory, and obtaining a third matching document in the target vector database through the third search operation; sorting the first matching document, the second matching document and the third matching document according to a preset sorting method, and obtaining a sorting result; inputting the sorting result into a big language model, and performing a data security warning through the big language model.

[0007] Optionally, a data classification and grading directory is constructed according to preset classification rules, and each row of the classification and grading directory is saved as a document record in the target vector database, specifically including: embedding each row of data classification and grading through a multimodal embedding model to calculate the corresponding dense vector; embedding each row of data classification and grading through a multimodal embedding model to calculate the corresponding sparse vector; constructing a data classification and grading directory according to preset classification rules, and saving each row of the classification and grading directory as a document record in the target vector database, the document record including subclasses, data items, security levels, texts, dense vectors and sparse vectors.

[0008] Optionally, a first matching document in the target vector database is obtained through a first retrieval operation, specifically including: calculating multiple relevance scores corresponding to the first query condition and multiple document records in the target vector database, one document record corresponding to one relevance score; sorting the multiple relevance scores, and obtaining the document record that meets the first ranking condition in the sorting result as the first matching document.

[0009] Optionally, calculating multiple relevance scores corresponding to the first query condition and multiple document records in the target vector database specifically includes: calculating the relevance scores using the following formula:

[0010]

[0011] Among them, Score(q,d) is the relevance score between document record d and the first query condition q, q iFor the i-th word in document record d, IDF(q i ) represents the uniqueness of the i-th word to the database document, represents the frequency of the i-th word in the document record d, k1 is an adjustable parameter used to control the impact of word frequency on the relevance score, and δ is a positive constant used to avoid the algorithm from penalizing the existence of too long text in the data classification and grading directory.

[0012] Optionally, a second matching document in the target vector database is obtained through a second retrieval operation, specifically including: embedding the second query condition into a dense vector using a multimodal embedding model, and calculating the cosine similarity scores corresponding to the dense vector and multiple document records in the target vector database, one document record corresponding to one cosine similarity score; sorting the multiple cosine similarity scores, and obtaining the document record that meets the second ranking condition in the sorting result as the second matching document.

[0013] Optionally, the first matching document, the second matching document and the third matching document are sorted according to a preset sorting method, and the sorting result is obtained, specifically including: sorting the first matching document, the second matching document and the third matching document according to the score for the first time, and obtaining the first target document that meets the preset matching conditions; inputting the first target document into the BERT model for a second sorting, and outputting the re-sorted second target document through the BERT model; and using the second target document as the sorting result.

[0014] Optionally, the sorting results are input into a large language model, and a data security warning is issued through the large language model, specifically including: obtaining a user prompt word input by the user, and using the user prompt word as a metadata field; using the sorting results as a sensitive data identification rule, and determining whether a data item in the sensitive data identification rule exists in the metadata field; if a data item in the sensitive data identification rule exists in the metadata field, classifying and grading the user prompt word; if a data item in the sensitive data identification rule does not exist in the metadata field, issuing a warning for the user prompt word and outputting a classification.

[0015] In a second aspect of the present application, a data classification and grading device based on artificial intelligence large model technology is provided, the device includes an acquisition module and a processing module, wherein:

[0016] The acquisition module is used to construct a data classification and grading directory according to preset classification rules, and save each line of the classification and grading directory as a document record in the target vector database.

[0017] A processing module is used to use the business system name, data table name and field name as a first query condition, perform a first search operation on the data classification and grading directory, and obtain a first matching document in the target vector database through the first search operation; use the business system name, data table name and field name as a second query condition, and embed the second query condition into a dense vector using a multimodal embedding model, perform a second search operation on the data classification and grading directory, and obtain a second matching document in the target vector database through the second search operation; use the business system name, data table name and field name as a third query condition, and embed the third query condition into a sparse vector using a multimodal embedding model, perform a third search operation on the data classification and grading directory, and obtain a third matching document in the target vector database through the third search operation; sort the first matching document, the second matching document and the third matching document according to a preset sorting method, and obtain the sorting result; input the sorting result into a large language model, and issue a data security warning through the large language model.

[0018] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0019] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to perform any of the above methods.

[0020] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0021] 1. Construct a data classification and grading directory according to the preset classification rules, and save each row of the classification and grading directory as a document record in the target vector database; use the business system name, data table name and field name as the first query condition, perform the first search operation, the second search operation and the third search operation on the data classification and grading directory respectively to obtain the first matching document, the second matching document and the third matching document in the target vector database; sort the first matching document, the second matching document and the third matching document according to the preset sorting method, and obtain the sorting results; input the sorting results into the large language model, and use the large language model to issue a data security warning. Therefore, the semantic matching accuracy between metadata and the classification and grading directory is improved through the vectorized embedding and multi-strategy semantic retrieval fusion sorting mechanism, and the large language model is used to perform semantic reasoning and structured warning on the sorting results, significantly enhancing the system's perception and response efficiency of data security risks, ensuring that sensitive data can be quickly identified and trigger corresponding security control policies in scenarios such as transmission, use and sharing, thereby effectively reducing security risks such as sensitive data leakage caused by recognition lag or recognition errors.

[0022] 2. Each row of data is classified and graded through a multimodal embedding model to calculate the corresponding dense vector; each row of data is classified and graded through a multimodal embedding model to calculate the corresponding sparse vector; a data classification and grading directory is constructed according to the preset classification rules, and each row of the classification and grading directory is saved as a document record in the target vector database, thereby achieving high-dimensional expression and multi-granularity modeling of the semantic features in the data classification and grading directory, and enhancing the semantic retrieval ability and feature distinction ability of the classification and grading documents in the target vector database.

[0023] 3. The first matching document, the second matching document, and the third matching document are sorted for the first time according to the scores, and the first target document that meets the preset matching conditions is obtained; the first target document is input into the BERT model for a second sorting, and the re-sorted second target document is output through the BERT model; the second target document is used as the sorting result. Thus, through the two sortings, the problem of insufficient sorting accuracy and weak structural semantic alignment ability caused by relying only on shallow retrieval features in the existing technology is effectively solved, thereby significantly improving the accuracy and interpretability of classification results in sensitive data identification tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a data classification and grading method based on artificial intelligence large model technology provided in an embodiment of the present application;

[0025] Figure 2 This is a flow chart of constructing a vector database provided by an embodiment of the present application;

[0026] Figure 3 This is a flowchart of a search classification and grading process provided by an embodiment of the present application;

[0027] Figure 4 This is a data classification and early warning judgment flow chart provided in an embodiment of the present application;

[0028] Figure 5 This is a module diagram of a data classification and grading device based on artificial intelligence large model technology provided in an embodiment of the present application;

[0029] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present application.

[0030] Explanation of the reference numerals: 51, acquisition module; 52, processing module; 601, processor; 602, communication bus; 603, user interface; 604, network interface; 605, memory. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0032] The terms used in the following examples of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application, the singular expressions "a", "an", "said", "above", "the", and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more of the listed items.

[0033] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0034] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0035] Please refer to Figure 1, which shows a flow chart of a data classification and grading method based on artificial intelligence big model technology provided in an embodiment of the present application, and the flow chart mainly includes the following steps: S101 to S106.

[0036] Step S101: construct a data classification and grading directory according to preset classification rules, and save each line of the classification and grading directory as a document record in the target vector database.

[0037] Specifically, a data classification and grading directory is constructed according to the preset classification rules, wherein the preset classification rules are "Data Security Technology-Data Classification and Grading Rules" and relevant industry standards, and semantic analysis and feature extraction are performed on all data items to clarify their attributes in dimensions such as internal business processes, information structure, and sensitivity characteristics of the organization. Then, based on the results of the semantic structured analysis, combined with the preset rule matching mechanism, the data items are classified layer by layer according to the first-level category, second-level category, and third-level category to which they belong, and a data classification path is generated. A classification and grading directory is constructed according to the combination of the path and the data item, wherein each row includes its complete classification path, the data item name, and a predefined security level label. In this process, the allocation of security levels is based on factors such as data sensitivity, the degree of impact on organizational operations, and legal compliance requirements, combined with expert experience evaluation or automated model scoring results, and finally a structured table directory is formed. Please refer to Table 1 for an example of a classification and grading directory in this application.

[0038] [Table 1]

[0039]

[0040]

[0041] As can be seen from Table 1, the classification and grading directory includes classification, data items, and security levels, and the classification consists of one or more levels, and the number of classification levels can be set according to actual needs.

[0042] In one possible implementation, step S101 further includes: embedding each row of data by classification and grading through a multimodal embedding model to calculate a corresponding dense vector; embedding each row of data by classification and grading through a multimodal embedding model to calculate a corresponding sparse vector; constructing a data classification and grading directory according to preset classification rules, and saving each row of the classification and grading directory as a document record in a target vector database, where the document record includes a subclass, a data item, a security level, text, a dense vector, and a sparse vector.

[0043] Specifically, in an embodiment of the present application, the multimodal embedding model is a bge-m3 model, which integrates a text semantic encoder and a domain feature encoder, and its input includes semantic information such as spliced multi-level classification text, data item name and security level label. On the one hand, the model outputs a set of dense vectors in the form of continuous numerical values through a dense embedding layer (for example, based on BERT or RoBERTa architecture) for semantic neighbor retrieval and high-dimensional feature matching; on the other hand, it outputs a sparse representation through a sparse encoder (for example, based on a sparse attention mechanism or vector quantization technology) for enhanced interpretability, semantic sparsity analysis and fast indexing. Each row of data is classified and graded by the bge-m3 model to calculate the corresponding dense vector densevector, and each row of data is classified and graded by the bge-m3 model to calculate the corresponding sparse vector sparsevector. Each row of the classification and grading directory is encapsulated as a structured document record and stored in the target vector database. Each document record clearly contains the following fields: first-level subclass, second-level subclass, third-level subclass, data item name, security level, original synthetic text content (i.e., the complete semantic description string after splicing), dense vector representation, and sparse vector representation. Among them, the original synthetic text is constructed by splicing the three-level classification and data item name and security level according to a fixed template format, such as "school-school basic information-personal identity information-principal name-security level 1". The target vector database needs to support the vector retrieval interface and be equipped with a multi-index mechanism to adapt to the coexistence of high-dimensional dense retrieval and low-dimensional sparse retrieval, and provide metadata label index support for document records for subsequent data classification verification, query optimization and visual annotation. Please refer to Figure 2 , which shows a vector database construction flow chart provided in an embodiment of the present application.

[0044] Step S102 : Using the business system name, data table name, and field name as a first query condition, a first search operation is performed on the data classification and grading directory, and a first matching document in the target vector database is obtained through the first search operation.

[0045] Specifically, the data classification and grading directory is searched three times, wherein the first search operation uses the business system name, data table name and field name as the first query condition, and obtains the first matching document in the target vector database.

[0046] In a possible implementation, step S102 further includes: calculating multiple relevance scores corresponding to the first query condition and multiple document records in the target vector database, with one document record corresponding to one relevance score; sorting the multiple relevance scores, and obtaining the document record that meets the first ranking condition in the sorting result as the first matching document.

[0047] Specifically, the business system name, data table name, and field name are concatenated to form the first query condition, recorded as the first query condition text; this text is input into a multimodal embedding model consistent with the embedding model used by the vector database, and the corresponding query vector is generated through vector encoding. On this basis, multiple data classification and hierarchical directory document records stored in the target vector database are traversed. Each document record includes a subcategory, data item, security level, text description, dense vector, and sparse vector. The similarity score is calculated between the query vector and the semantic vector in the document record. This score is calculated according to the following relevance scoring function:

[0048]

[0049] in:

[0050]

[0051] In the above formula, Score(q,d) is the relevance score between document record d and the first query condition q, document record d is any one of the multiple document records in the target vector database, and q i For the i-th word in document record d, IDF(q i ) represents the uniqueness of the i-th word to the database document, It represents the frequency of the i-th word in the document record d, which is used to avoid the algorithm from penalizing the existence of too long text in the data classification and grading directory. |d| is the text length of the document record d, avgdl is the average length of multiple document records in the target vector database, N is the total number of document records in the vector database, and n(q i ) contains the term q i , δ is a positive constant used to avoid negative penalties for long texts, k1 and b are empirically tuned parameters, set to 1.5 and 0.75 respectively, used to control the impact of word frequency on relevance scores.

[0052] After calculating the relevance scores for all documents, all records are sorted in descending order based on the scores. The top-ranked document, based on a preset ranking condition (such as a Top-1 or Top-k threshold), is selected as the first matching document. This document's structure is semantically consistent with the input business system name, data table name, and field name. Its corresponding subcategory information, security level label, and embedding vector can be directly used to construct subsequent data compliance assessment processes.

[0053] It is worth pointing out that in the claims, the "business system name, data table name, field name" can be defined as the "first search condition field set", the "data classification and grading directory" can be defined as the "classification data directory document set", the "target vector database" can be defined as the "semantic vector database", and the "first matching document" can be defined as the "first relevant data directory record", so as to ensure the consistency, clarity and searchability of the features in the structural expression.

[0054] In step S103, the business system name, data table name, and field name are used as the second query condition, and the second query condition is embedded into a dense vector using a multimodal embedding model. A second search operation is performed on the data classification and grading directory, and a second matching document in the target vector database is obtained through the second search operation.

[0055] Specifically, the "business system name, data table name, and field name" are used as the second query condition field set, and this query condition field set is input into a multimodal embedding model, preferably the bge-m3 semantic embedding model, to generate a corresponding dense vector representation, denoted as the query dense vector. This vector fully incorporates the contextual relationships and structural gap characteristics of the input fields in the semantic space, and its output dimension is consistent with the dense vector dimension of the classified data directory document record stored in the semantic vector database.

[0056] In one possible implementation, step S103 further includes: embedding the second query condition into a dense vector using a multimodal embedding model, calculating cosine similarity scores between the dense vector and multiple document records in the target vector database, with each document record corresponding to one cosine similarity score; and sorting the multiple cosine similarity scores, obtaining document records from the sorted results that meet the second ranking condition as second matching documents.

[0057] Specifically, after obtaining the query dense vector, for each classification data directory record in the semantic vector database, its pre-generated and saved dense vector representation is recorded as a document dense vector, and the cosine similarity score between the document dense vector and the query dense vector is calculated respectively. The specific calculation formula is as follows:

[0058]

[0059] where x i Represents the i-th vector component of the second query condition, y i represents the i-th component of the dense vector of the document record, and n represents the vector dimension. The theoretical range of the cosine similarity value is between 0 and 1. A larger similarity value indicates a closer vector direction, that is, a stronger semantic relevance between the query condition and the document.

[0060] After completing the vector comparison calculations, all document records are sorted in descending order by their cosine similarity scores. Based on a predefined second ranking criteria (e.g., Top-1, Top-5, or a dynamic similarity threshold), one or more document records with the highest scores are selected as the second matching document. This second matching document, based on dense vector semantics, enables robust matching of semantically similar but inconsistently expressed information, significantly enhancing the system's ability to recognize ambiguous descriptions, polysemous expressions, and industry jargon.

[0061] Step S104, using the business system name, data table name and field name as the third query condition, and embedding the third query condition into a sparse vector using a multimodal embedding model, performing a third search operation on the data classification and grading directory, and obtaining a third matching document in the target vector database through the third search operation.

[0062] Specifically, the acquisition of the third matching document is similar to step S103. For each sparse vector representation of the saved document in the target vector database, the cosine similarity score between it and the query sparse vector is calculated. The calculation formula remains consistent with that in the dense vector space and is expressed as follows:

[0063]

[0064] where x i is the i-th dimension component of the third query condition sparse vector, y i The i-th component of the sparse vector is recorded for the candidate document, where n is the vector dimension. Considering the sparsity feature, in actual calculations, an acceleration strategy can be adopted to improve computational efficiency by performing product operations only on non-zero dimensions.

[0065] After scoring the similarity between all document sparse vectors and the query sparse vector, the results are sorted in descending order and the highest-scoring document is selected as the third matching document based on a predefined third ranking condition (e.g., score threshold or Top-N). This matching document is highly consistent with the query in key structural semantic dimensions and is suitable for distinguishing redundant candidate documents that have keyword hit characteristics but lack overall semantic consistency, helping to improve retrieval accuracy in complex query scenarios.

[0066] Step S105 : sorting the first matching document, the second matching document, and the third matching document according to a preset sorting method, and obtaining a sorting result.

[0067] Specifically, the collection of the first matching document, the second matching document, and the third matching document are sorted twice respectively, and the document obtained by the second sorting is used as the sorting result.

[0068] In a possible implementation, step S105 further includes: sorting the first matching document, the second matching document, and the third matching document for the first time according to the scores, and obtaining a first target document that meets the preset matching conditions; inputting the first target document into the BERT model for a second sorting, and outputting the re-sorted second target document through the BERT model; and using the second target document as the sorting result.

[0069] Specifically, the first matching document, the second matching document, and the third matching document are firstly unioned to form the initial candidate document set, denoted as D. The ranking position of each document in the three independent search rankings is extracted for the first ranking, and the fusion ranking score Score(d∈D) is calculated. The calculation formula is:

[0070]

[0071] Where R represents the three sets of ranking functions corresponding to the three searches, r(d) represents the ranking position of document d under the ranking function r, and K is a smoothing constant, preferably set to 60, to prevent the scores of the lower-ranked documents from tending to infinity and to enhance the stability of the ranking. After completing the above calculations, the documents are sorted from high to low according to the Score(d∈D) score for the first time, and the top d1 documents are obtained as the first target documents that meet the preset matching conditions. The embodiments of this application do not limit the number of rankings that meet the preset matching conditions and can be set according to actual needs.

[0072] After that, the top d1 documents are sorted for the second time, i.e. deep semantic re-ranking operation, with the input being query condition q and the top d1 documents. First, q is compared with each d i The documents are concatenated into a unified input structure in the format of [CLS]q[SEP]di[SEP]. The concatenated structure is segmented and word embedding vectors, paragraph tag embedding vectors, and position embedding vectors are generated respectively. The three are summed at the vector level as the input to the Transformer embedding layer. The embedded input is then passed through the 12-layer Transformer encoder in the BERT model to extract the hidden state vector V corresponding to the CLS tag in the final 12th layer. CLS The hidden vector is input into a linear fully connected neural network, and a 1-dimensional logits output is generated through weight matrix mapping to represent the global semantic matching degree between the query condition and the document. Finally, the logits values of the d1 candidate documents are sorted in descending order, and the top d2 documents are obtained as the second sorting result. The second sorting has the ability to distinguish deep semantic differences, and can correct the mismatch results caused by shallow vector neighbors in the first sorting, and achieve consistency comparison of semantic granularity through the context alignment mechanism. Please refer to Figure 3, which presents a flow chart of the retrieval classification and grading process provided in an embodiment of the present application.

[0073] Step S106: input the sorting results into a large language model, and perform a data security warning through the large language model.

[0074] Specifically, after completing multiple rounds of semantic retrieval and sorting, sensitive data identification rules are constructed based on the final sorting results, and the rules are input into the large language model. Combined with the metadata prompt words provided by the user, structured sensitive information matching and data security warning judgment are completed through thought chain reasoning.

[0075] In a possible implementation, step S106 further includes: obtaining a user prompt word input by the user, and using the user prompt word as a metadata field; using the sorting result as a sensitive data identification rule, and determining whether a data item in the sensitive data identification rule exists in the metadata field; if a data item in the sensitive data identification rule exists in the metadata field, classifying and grading the user prompt word; if a data item in the sensitive data identification rule does not exist in the metadata field, issuing a warning for the user prompt word and outputting a classification.

[0076] Specifically, user input is obtained, including the business system name, data table name, and field name, as metadata field input. Each record in the sorted target classification directory obtained in step S105 is converted into a sensitive data identification rule. Each rule includes a first-level subcategory, a second-level subcategory, a third-level subcategory, a data item name, and a corresponding security level. This rule is embedded in the prompt using a structured Markdown table format to enhance the large language model's ability to identify and process semantic correspondences between structured fields. An example is shown in Table 2.

[0077] [Table 2]

[0078]

[0079]

[0080] A unified input template, including few-sample prompts and chain-of-thought reasoning logic, is constructed to guide the large language model to gradually perform logical judgments. In this chain of reasoning, based on the representational content of the metadata field, a full or near-matching match of the structural semantics or field name in the sensitive data identification rules is first searched. If a match exists, the corresponding classification hierarchy is output, including first-level subcategories, second-level subcategories, third-level subcategories, and data item name fields, and a unified formatted output is completed in combination with few-sample examples. If a match does not exist, a logical reasoning chain is constructed to explain the missing reason and a structured warning message is output, indicating that the data item is unclassified, unidentified, or has an unknown security level, prompting manual review or rule expansion.

[0081] The language model performs the following specific process when processing: the user metadata field and the sorted rule set d2 are used as input, and the model automatically executes the following logic chain: determine whether the metadata field is highly semantically related to the "data item name" in d2; if so, extract the subclass hierarchy structure corresponding to the rule and output it; if there is no relevant item, output a warning message and retain the current input as the data to be classified for subsequent training or manual annotation expansion. Please refer to Figure 4 , which presents a data classification and early warning judgment flow chart provided by an embodiment of the present application, and an example provided in the embodiment of the present application is as follows:

[0082] Assuming the input data is "Transaction Amount of Student Growth Big Data Service Platform One Card Flow Information Table", the response is as follows:

[0083] <Thinking>

[0084] The input data is the transaction amount in the one-card flow information table of the student growth big data service platform. According to the sensitive data identification rules, the transaction amount in the one-card flow information table of the student growth big data service platform belongs to the student's personal financial information on campus, so the sensitive data identification rules corresponding to the input data are

[0085]

[0086] This application adopts the above method to construct a data classification and grading directory according to preset classification rules, and saves each row of the classification and grading directory as a document record to the target vector database; uses the business system name, data table name and field name as the first query condition, and performs a first search operation, a second search operation and a third search operation on the data classification and grading directory respectively to obtain the first matching document, the second matching document and the third matching document in the target vector database; sorts the first matching document, the second matching document and the third matching document according to a preset sorting method, and obtains the sorting result; inputs the sorting result into a large language model, and performs data security warning through the large language model, thereby improving the semantic matching accuracy between metadata and the classification and grading directory through vectorized embedding and multi-strategy semantic retrieval fusion sorting mechanism, and performs semantic reasoning and structured warning on the sorting results through the large language model, significantly enhancing the system's perception and response efficiency of data security risks, ensuring that sensitive data can be quickly identified and trigger corresponding security control policies in scenarios such as transmission, use, and sharing, thereby effectively reducing security risks such as sensitive data leakage caused by recognition lag or recognition errors.

[0087] Please refer to Figure 5, which shows a module schematic diagram of a data classification and grading device based on artificial intelligence large model technology provided by an embodiment of the present application, the device includes an acquisition module 21 and a processing module 22, wherein,

[0088] The acquisition module 51 is used to construct a data classification and grading directory according to a preset classification rule, and save each line of the classification and grading directory as a document record in the target vector database.

[0089] Processing module 52 is used to use the business system name, data table name and field name as the first query condition, perform a first search operation on the data classification and grading directory, and obtain the first matching document in the target vector database through the first search operation; use the business system name, data table name and field name as the second query condition, and use the multimodal embedding model to embed the second query condition into a dense vector, perform a second search operation on the data classification and grading directory, and obtain the second matching document in the target vector database through the second search operation; use the business system name, data table name and field name as the third query condition, and use the multimodal embedding model to embed the third query condition into a sparse vector, perform a third search operation on the data classification and grading directory, and obtain the third matching document in the target vector database through the third search operation; sort the first matching document, the second matching document and the third matching document according to a preset sorting method, and obtain the sorting result; input the sorting result into the large language model, and issue a data security warning through the large language model.

[0090] In one possible implementation, the acquisition module 51 is used to construct a data classification and grading directory according to preset classification rules, and save each row of the classification and grading directory as a document record to the target vector database, specifically including: embedding each row of data classification and grading through a multimodal embedding model to calculate the corresponding dense vector; embedding each row of data classification and grading through a multimodal embedding model to calculate the corresponding sparse vector; constructing a data classification and grading directory according to preset classification rules, and saving each row of the classification and grading directory as a document record to the target vector database, where the document record includes subclasses, data items, security levels, text, dense vectors, and sparse vectors.

[0091] In one possible implementation, the processing module 52 is used to obtain a first matching document in the target vector database through a first retrieval operation, specifically including: calculating multiple relevance scores corresponding to the first query condition and multiple document records in the target vector database, with one document record corresponding to one relevance score; sorting the multiple relevance scores, and obtaining a document record in the sorting result that meets the first ranking condition as the first matching document.

[0092] In a possible implementation, the processing module 52 is configured to calculate a plurality of relevance scores corresponding to the first query condition and a plurality of document records in the target vector database, specifically including: calculating the relevance scores using the following formula:

[0093]

[0094] Among them, Score(q,d) is the relevance score between document record d and the first query condition q, q i For the i-th word in document record d, IDF(q i ) represents the uniqueness of the i-th word to the database document, represents the frequency of the i-th word in the document record d, k1 is an adjustable parameter used to control the impact of word frequency on the relevance score, and δ is a positive constant used to avoid the algorithm from penalizing the existence of too long text in the data classification and grading directory.

[0095] In one possible implementation, the processing module 52 is used to obtain a second matching document in the target vector database through a second retrieval operation, specifically including: embedding the second query condition into a dense vector using a multimodal embedding model, and calculating the cosine similarity scores corresponding to the dense vector and multiple document records in the target vector database, one document record corresponding to one cosine similarity score; sorting the multiple cosine similarity scores, and obtaining the document record that meets the second ranking condition in the sorting result as the second matching document.

[0096] In one possible implementation, the processing module 52 is used to sort the first matching document, the second matching document, and the third matching document according to a preset sorting method and obtain a sorting result, specifically including: sorting the first matching document, the second matching document, and the third matching document according to the scores for the first time, and obtaining a first target document that meets the preset matching conditions; inputting the first target document into the BERT model for a second sorting, and outputting the re-sorted second target document through the BERT model; and using the second target document as the sorting result.

[0097] In one possible implementation, the processing module 52 is used to input the sorting results into a large language model and perform a data security warning through the large language model, specifically including: obtaining a user prompt word input by the user and using the user prompt word as a metadata field; using the sorting results as a sensitive data identification rule, and determining whether a data item in the sensitive data identification rule exists in the metadata field; if a data item in the sensitive data identification rule exists in the metadata field, classifying and grading the user prompt word; if a data item in the sensitive data identification rule does not exist in the metadata field, issuing a warning for the user prompt word and outputting a classification.

[0098] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0099] This application also provides an electronic device. Figure 6 , Figure 6 6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. The electronic device may include: at least one processor 601, at least one communication bus 602, a user interface 603, at least one network interface 604, and a memory 605.

[0100] The communication bus 602 is used to implement the connection and communication between these components.

[0101] The user interface 603 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 603 may also include a standard wired interface and a wireless interface.

[0102] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0103] The processor 601 may include one or more processing cores. The processor 601 utilizes various interfaces and circuits to connect various parts of the entire server. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 605, and calling data stored in the memory 605, the processor 601 performs various server functions and processes data. Optionally, the processor 601 may be implemented using at least one hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 601 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display; and the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 601 and may be implemented separately on a single chip.

[0104] Among them, the memory 605 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 605 includes a non-transitory computer-readable storage medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may also be optionally at least one storage device located away from the aforementioned processor 601. Reference Figure 6 , as a computer storage medium, the memory 605 may include an operating system, a network communication module, a user interface module, and a data classification and grading application based on artificial intelligence large model technology.

[0105] exist Figure 6In the electronic device shown, the user interface 603 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 601 can be used to call the data classification and grading application based on the artificial intelligence large model technology stored in the memory 605. When executed by one or more processors 601, the electronic device executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the described order of actions, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0106] The present application also provides a computer-readable storage medium storing instructions, which, when executed by one or more processors, enable an electronic device to execute one or more of the methods described in the above embodiments.

[0107] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0109] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0110] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0112] The above descriptions are merely exemplary embodiments disclosed in this application and are not intended to limit the scope of this application. That is, any equivalent changes and modifications made based on the teachings disclosed in this application are still within the scope of this application.

[0113] This application is intended to cover any modifications, uses or adaptations disclosed in this application, which follow the general principles disclosed in this application and include common knowledge or customary technical means in the technical field not disclosed in this application.

Claims

1. A data classification and grading method based on artificial intelligence large model technology, characterized in that: The method comprises: Constructing a data classification and grading directory according to preset classification rules, and saving each row of the classification and grading directory as a document record in a target vector database; Using the business system name, data table name, and field name as first query conditions, a first search operation is performed on the data classification and grading directory, and a first matching document in the target vector database is obtained through the first search operation; Using the business system name, data table name, and field name as a second query condition, and embedding the second query condition into a dense vector using a multimodal embedding model, performing a second search operation on the data classification and grading directory, and obtaining a second matching document in the target vector database through the second search operation; Using the business system name, data table name, and field name as a third query condition, and embedding the third query condition into a sparse vector using the multimodal embedding model, performing a third search operation on the data classification and grading directory, and obtaining a third matching document in the target vector database through the third search operation; Sorting the first matching document, the second matching document, and the third matching document according to a preset sorting method, and obtaining a sorting result; The ranking results are input into a large language model, and a data security warning is performed through the large language model.

2. The method according to claim 1, characterized in that The step of constructing a data classification and grading directory according to preset classification rules and saving each row of the classification and grading directory as a document record in the target vector database specifically includes: Embed each row of data by classification and grading using the multimodal embedding model to calculate the corresponding dense vector; Embed each row of data by classification and grading using the multimodal embedding model to calculate the corresponding sparse vector; A data classification and grading directory is constructed according to preset classification rules, and each row of the classification and grading directory is saved as a document record in a target vector database, wherein the document record includes a subclass, a data item, a security level, a text, the dense vector, and the sparse vector.

3. The method according to claim 1, characterized in that The obtaining of the first matching document in the target vector database through the first search operation specifically includes: Calculating a plurality of relevance scores corresponding to the first query condition and a plurality of document records in the target vector database, where each document record corresponds to one relevance score; The plurality of relevance scores are sorted, and the document records satisfying the first ranking condition in the sorting results are obtained as the first matching documents.

4. The method according to claim 1, wherein The calculating of a plurality of relevance scores corresponding to the first query condition and a plurality of document records in the target vector database specifically includes: The relevance score is calculated by the following formula: Wherein, Score(q,d) is the relevance score between the document record d and the first query condition q, q i For the i-th word in the document record d, IDF(q i ) represents the uniqueness of the i-th word to the database document, represents the frequency of the i-th word in the document record d, k1 represents an adjustable parameter used to control the impact of word frequency on the relevance score, and δ is a positive constant used to avoid the algorithm from penalizing the existence of too long text in the data classification and grading directory.

5. The method according to claim 1, wherein The acquiring of the second matching document in the target vector database through the second search operation specifically includes: Embed the second query condition into a dense vector using a multimodal embedding model, and calculate cosine similarity scores between the dense vector and a plurality of document records in the target vector database, where one cosine similarity score corresponds to one document record; The multiple cosine similarity scores are sorted, and the document records that meet the second ranking condition in the sorting result are obtained as the second matching documents.

6. The method according to claim 1, characterized in that Sorting the first matching document, the second matching document, and the third matching document according to a preset sorting method and obtaining a sorting result specifically includes: Sorting the first matching document, the second matching document, and the third matching document for the first time according to the scores, and obtaining a first target document that meets a preset matching condition; Inputting the first target document into the BERT model for a second ranking, and outputting the re-ranked second target document through the BERT model; The second target document is used as the ranking result.

7. The method according to claim 1, characterized in that Inputting the ranking results into a large language model and performing data security warnings through the large language model specifically includes: Obtaining a user prompt word input by the user, and using the user prompt word as a metadata field; Using the sorting result as a sensitive data identification rule, and determining whether a data item in the sensitive data identification rule exists in the metadata field; If the metadata field contains a data item in the sensitive data identification rule, classifying and grading the user prompt words; If the data item in the sensitive data identification rule does not exist in the metadata field, an alert is issued for the user prompt word and a classification is output.

8. A data classification and grading device based on artificial intelligence large model technology, characterized in that: The device includes an acquisition module and a processing module, wherein: The acquisition module is used to construct a data classification and grading directory according to preset classification rules, and save each line of the classification and grading directory as a document record in the target vector database; The processing module is configured to perform a first search operation on the data classification and grading directory using the business system name, data table name, and field name as a first query condition, and obtain a first matching document in the target vector database through the first search operation; perform a second search operation on the data classification and grading directory using the business system name, data table name, and field name as a second query condition, and embed the second query condition into a dense vector using a multimodal embedding model, and obtain a second matching document in the target vector database through the second search operation; perform a third search operation on the data classification and grading directory using the business system name, data table name, and field name as a third query condition, and embed the third query condition into a sparse vector using the multimodal embedding model, and obtain a third matching document in the target vector database through the third search operation; sort the first matching document, the second matching document, and the third matching document according to a preset sorting method, and obtain a sorting result; input the sorting result into a large language model, and perform a data security warning through the large language model.

9. An electronic device, characterized in that: The electronic device comprises a processor, a communication bus, a user interface, a network interface and a memory, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images