Data classification method and device

Abstract

The invention discloses a data classification method and device, and relates to the technical field of data processing. The method comprises the following steps: acquiring to-be-identified data; obtaining a first data type and a first confidence coefficient through the small model cluster; converting the to-be-identified data into a to-be-identified feature vector, obtaining a similarity score through a similarity calculation mode based on a preset feature vector library, and fusing the similarity score and the to-be-identified feature vector to obtain a second confidence coefficient; when the second confidence coefficient is greater than a first threshold value, determining the target data type as a first data type; and when the second confidence degree is between the first threshold value and the second threshold value, obtaining a target data type through the large model cluster. The second confidence coefficient is obtained through small model cluster primary judgment and feature vector library verification, the target data type is directly determined through the high confidence coefficient, secondary fine judgment is conducted on the medium confidence coefficient through a large model cluster, the problems that small model generalization ability and semantic understanding depth are insufficient, and a large model cannot cover all sub-fields are solved, and the accuracy of the target data type is improved. And the accuracy of data classification is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data classification method and apparatus. Background Technology

[0002] With the deepening development of the digital economy and the information society, the amount of data generated, collected, processed, and stored by basic telecommunications enterprises in their production, operation, and management activities is growing explosively. This data is not only a core asset of these enterprises but also relates to national security, social order, and the personal privacy of a vast number of users. Based on the mandatory requirements of relevant laws and regulations, as well as the explicit statements in telecommunications industry standards, efficient and accurate processing (with classification as the core) of massive, multi-source, and heterogeneous telecommunications data has become the prerequisite and foundation for fulfilling data security protection responsibilities, achieving differentiated security management, and preventing data leakage risks. It is also a core requirement for data governance in the telecommunications industry.

[0003] In current technologies, common data classification methods rely on a single model. This can be achieved by using a small, specialized model trained for a specific data type, or by leveraging the powerful general semantic understanding capabilities of a large language model.

[0004] Therefore, current technologies that rely on single models for data classification suffer from poor classification accuracy due to the insufficient generalization ability and semantic understanding depth of small models. While large models possess generalization ability and deep semantic understanding, they cannot cover all professional sub-fields in the communications industry, resulting in poor classification accuracy as well. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a data classification method and apparatus. The method obtains a second confidence level by fusing initial judgment from a small model cluster with verification from a feature vector library. High confidence level directly determines the target data type, while medium confidence level is determined by a second fine-tuning from a large model cluster. This approach not only compensates for the shortcomings of insufficient generalization ability and semantic understanding depth of a single small model, but also solves the problem that a single large model cannot cover all professional sub-fields in the communications industry, effectively improving the accuracy of data classification.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a data classification method, including:

[0008] Obtain the data to be identified;

[0009] The data to be identified is classified and identified by a cluster of small models to obtain a first data type and a first confidence level of the data to be identified; wherein, the cluster of small models includes: multiple pre-trained small models;

[0010] The data to be identified is converted into a feature vector to be identified, and based on the feature vector to be identified and a preset feature vector library, a similarity score is obtained for the data to be identified as the first data type through a similarity calculation method.

[0011] Based on the first confidence level and the similarity score, a second confidence level of the data to be identified is obtained;

[0012] When the second confidence level is greater than the preset first threshold, the target data type of the data to be identified is determined to be the first data type;

[0013] When the second confidence level is between the preset first threshold and the preset second threshold, the target data type of the data to be identified is obtained through a large model cluster based on the first data type and the data to be identified; wherein the first threshold is greater than the second threshold.

[0014] In one possible implementation, acquiring the data to be identified includes:

[0015] Raw network traffic is collected and preprocessed to obtain a file to be identified; wherein, the file to be identified includes: data to be identified.

[0016] In one possible implementation, the step of classifying and identifying the data to be identified using a small model cluster to obtain a first data type and a first confidence level of the data to be identified includes:

[0017] Based on the file type of the file to be identified, match the target small model in the small model cluster that corresponds to the file type;

[0018] The data to be identified in the file to be identified is input into the target small model so that the target small model can output the first data type and the first confidence level of the data to be identified based on the data to be identified.

[0019] In one possible implementation, the feature vector library includes: a preset feature vector corresponding to each data type, wherein the preset feature vector is obtained based on the data type and the preset features, keywords, and data samples corresponding to the data type; the step of converting the data to be identified into a feature vector to be identified, and obtaining a similarity score of the data to be identified as the first data type based on the feature vector to be identified and the preset feature vector library through a similarity calculation method, includes:

[0020] The data to be identified is converted into a feature vector to be identified;

[0021] The similarity between the feature vector to be identified and the preset feature vector corresponding to each data type in the feature vector library is calculated to obtain the similarity score of the data to be identified for each data type, so as to obtain the similarity score of the data to be identified as the first data type.

[0022] In one possible implementation, the method further includes: determining candidate data types based on the similarity scores of the data to be identified for each data type; wherein the candidate data types are the data types corresponding to the first N similarity scores, and N is a positive integer;

[0023] When the second confidence level is between the preset first threshold and the preset second threshold, the target data type of the data to be identified is obtained through a large model cluster based on the first data type and the data to be identified, including:

[0024] When the second confidence level is between the preset first threshold and the preset second threshold, the target large model in the large model cluster corresponding to the first data type is matched based on the first data type.

[0025] Based on the target large model, the target data type of the data to be identified is obtained from the data to be identified, the first data type, and the candidate data type.

[0026] In one possible implementation, obtaining the target data type of the data to be identified based on the target large model, the first data type, and the candidate data types includes:

[0027] The data to be identified, the first data type, and the candidate data type are input together into the target large model so that the target large model can output the second data type of the data to be identified, the third confidence level corresponding to the second data type, and the judgment basis of the second data type based on the data to be identified, the first data type, and the candidate data type.

[0028] When the third confidence level is greater than the preset third threshold, the target data type of the data to be identified is determined to be the second data type.

[0029] In one possible implementation, the method further includes:

[0030] If the third confidence level is not greater than the third threshold, then the data to be identified is marked as data to be reviewed.

[0031] In one possible implementation, the method further includes:

[0032] When the third confidence level is greater than the preset fourth threshold, the data to be identified, the target data type corresponding to the data to be identified, and the judgment criteria are stored in the preset classification knowledge base; wherein, the fourth threshold is greater than or equal to the third threshold, and the classification knowledge base is used as a prompt word input to the target large model.

[0033] In one possible implementation, the method further includes:

[0034] If the second confidence level is less than the second threshold, the data to be identified is marked as data to be reviewed.

[0035] Secondly, embodiments of this application provide a data classification apparatus, including:

[0036] The data acquisition module is used to acquire the data to be identified.

[0037] The preliminary identification module is used to classify and identify the data to be identified through a cluster of small models to obtain a first data type and a first confidence level of the data to be identified; wherein, the cluster of small models includes: multiple pre-trained small models;

[0038] The vector similarity module is used to convert the data to be identified into a feature vector to be identified, and based on the feature vector to be identified and a preset feature vector library, to obtain a similarity score of the data to be identified as the first data type through a similarity calculation method.

[0039] The comprehensive scoring module is used to obtain the second confidence level of the data to be identified based on the first confidence level and the similarity score;

[0040] The target identification module is used to determine the target data type of the data to be identified as a first data type when the second confidence level is greater than a preset first threshold; and to obtain the target data type of the data to be identified based on the first data type and the data to be identified through a large model cluster when the second confidence level is between the preset first threshold and a preset second threshold; wherein the first threshold is greater than the second threshold.

[0041] Compared to existing technologies, this application offers the following advantages: It integrates the first confidence score obtained from a small model cluster with the similarity score obtained from a feature vector library to generate a second confidence score, and constructs a hierarchical decision-making mechanism, achieving an optimal balance between classification efficiency and accuracy. For highly reliable data with a second confidence score higher than the first threshold, the target data type is directly determined without subsequent redundant processing, ensuring efficient processing of massive amounts of conventional data. For moderately confident data between the two thresholds, a large model cluster is then invoked for precise judgment, accurately focusing on complex data requiring in-depth analysis. This hierarchical strategy avoids the computational consumption and inefficiency of fully invoking large models, while ensuring the classification accuracy of complex data through the large model as a safety net. It compensates for the shortcomings of a single small model in terms of generalization ability and semantic understanding depth, and solves the problem that a single large model cannot cover all professional sub-fields in the communications industry, effectively improving the accuracy of data classification. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart illustrating a data classification method provided in an embodiment of this application;

[0044] Figure 2 A flowchart illustrating the process of determining the target data type for a large model cluster provided in this application embodiment;

[0045] Figure 3 A flowchart illustrating another data classification method provided in this application embodiment;

[0046] Figure 4 This is a schematic diagram of the structure of a data classification device provided in an embodiment of this application. Detailed Implementation

[0047] As described earlier, early data classification primarily relied on manual sorting or automated processing based on keyword and regular expression rule matching. Manual sorting involved professionals meticulously identifying and classifying data one by one, while rule matching compared pre-defined data feature rules with target data to complete the classification. However, manual sorting was inefficient and costly, unsuitable for massive data processing scenarios. While keyword and regular expression-based rule matching achieved automation, it struggled to understand contextual semantics, had limited ability to recognize unstructured and semi-structured data, and had complex rule base maintenance, resulting in low accuracy and recall rates for classification and identification, and a large number of false positives and false negatives.

[0048] With the development of artificial intelligence technology, machine learning and deep learning models are widely used in the classification of data in the communications industry. Currently, common data classification methods rely on a single model. One approach is to use specialized small models trained for specific data types (such as named entity recognition models or digit recognition models) for accurate type identification. Another approach is to use general-purpose large language models (LLMs) to handle complex data semantic understanding needs and perform type identification.

[0049] For data classification methods that rely on small models, dedicated small models are trained based on specific data types or single-scenario samples. The model parameters and feature extraction logic are highly tied to the training data distribution, making them unable to cope with the complex scenarios of multi-source heterogeneous and interwoven data in the telecommunications industry. In other words, small models have poor generalization ability and insufficient semantic understanding depth, making it difficult to adapt to the complexity of telecommunications industry data. Dedicated small models focus on single-dimensional feature recognition and cannot combine context and industry rules to determine the core attributes of the data. They can only classify based on surface features, resulting in poor classification accuracy.

[0050] For data classification methods that rely on large models (large language models), these models are typically trained on general corpora. The specialized knowledge specific to the telecommunications industry (such as industry classification standards, business logic, and compliance rules) constitutes a very small proportion of the training corpus. This results in a superficial understanding of industry terminology, data boundary delineation, and sensitive attribute judgment standards. Furthermore, the telecommunications industry encompasses a wide range of data, including highly specialized sub-domains such as user identity data, user service content data, construction data, and network and system resource data. The classification logic varies significantly across these domains, and the parameters of a single general-purpose large model cannot simultaneously optimize the professional judgment logic of all sub-domains. This makes it difficult to achieve full-scenario adaptability, and the inherent illusionary flaws of large models can lead to unreasonable inferences in unfamiliar professional domains, generating inaccurate results. In short, while large language models possess generalization and deep semantic understanding capabilities, their incomplete coverage of specialized sub-domains within the telecommunications industry and insufficient professional judgment capabilities fail to meet the highly specialized classification needs of the telecommunications sector, resulting in poor classification accuracy.

[0051] Furthermore, data classification methods that rely on large language models directly process massive amounts of raw data, resulting in high consumption of computing resources and severe processing delays, which cannot meet the near real-time monitoring needs of data centers.

[0052] This application provides a data classification method, including: acquiring data to be identified; classifying and identifying the data to be identified using a small model cluster to obtain a first data type and a first confidence level; converting the data to be identified into a feature vector to be identified, and obtaining a similarity score for the data to be identified as the first data type based on the feature vector to be identified and a preset feature vector library through a similarity calculation method; obtaining a second confidence level for the data to be identified based on the first confidence level and the similarity score; determining the target data type of the data to be identified as the first data type when the second confidence level is greater than a preset first threshold; and obtaining the target data type of the data to be identified using a large model cluster based on the first data type and the data to be identified when the second confidence level is between the preset first threshold and the preset second threshold. In this embodiment, the second confidence level is obtained through a fusion of initial judgment by a small model cluster and verification by a feature vector library. High confidence levels directly determine the target data type, while medium confidence levels are refined by a large model cluster. This not only compensates for the shortcomings of insufficient generalization ability and semantic understanding depth of a single small model, but also solves the problem that a single large model cannot cover all professional sub-fields of the communications industry, effectively improving the accuracy of data classification.

[0053] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0054] Example 1:

[0055] The following is combined with Figures 1-3 This application provides a detailed description of a data classification method based on its embodiments.

[0056] like Figure 1 The data classification method provided in this application includes the following steps:

[0057] S101. Obtain the data to be identified.

[0058] The data to be identified refers to telecommunications industry data, which is the direct object of classification. Specifically, the data to be identified includes: structured data (such as ID card numbers and mobile phone numbers), semi-structured data (such as contract fragments and report frames), and unstructured data (such as text on document photocopies and plain text descriptions).

[0059] In one possible implementation, raw network traffic is collected, and the raw network traffic is preprocessed to obtain a file to be identified; wherein the file to be identified includes data to be identified.

[0060] Among them, raw network traffic refers to the dynamic and mixed set of raw data packets transmitted in the network link within the data center, including various types of data generated in the enterprise's production, operation and management activities (such as user interaction data, network operation and maintenance data, business processing data, etc.), which is the original source of the data to be identified.

[0061] The file to be identified refers to the standardized static file generated after preprocessing. It is a structured representation of the original network traffic, including the data to be identified. The file to be identified supports multiple file types / formats, such as Word files, PDF files, image files (such as igp, png), and plain text files (such as txt).

[0062] Specifically, network splitters or traffic mirroring devices are deployed at core network link nodes (such as switch and router ports) in the data center; using "network splitting" or "traffic mirroring" technology, the original network traffic in the network link is copied in real time.

[0063] Specifically, raw network traffic is collected, and a series of technical processes are performed on the collected raw network traffic to transform the dynamic and non-standardized raw network traffic into a static and standardized file to be identified. This preprocessing of the raw network traffic removes redundant information and parses the data structure to obtain a file to be identified that includes the data to be identified.

[0064] For example, deep packet inspection (DPI), protocol parsing, and file reconstruction are performed on the collected raw network traffic to parse and reconstruct various types of files to be identified from the raw network traffic.

[0065] S102. Through a cluster of small models, classify and identify the data to be identified to obtain the first data type and the first confidence level of the data to be identified.

[0066] The term "small model cluster" refers to a collection of specialized small models, each with a specific function and pre-trained on industry data, designed to address the heterogeneous characteristics and classification needs of telecommunications industry data. Each small model focuses on a specific recognition task, collectively covering multiple pre-trained small models across the telecommunications industry. The small model cluster comprises multiple pre-trained small models.

[0067] Among them, the pre-trained small model refers to a lightweight model that is trained and fine-tuned based on specific telecommunications industry data (such as user identity data, network operation and maintenance data, business processing records, etc.), and has the characteristics of strong targeting, high operating efficiency and high recognition accuracy.

[0068] For example, the small model cluster includes: a digit recognition model, a named entity recognition model, an image recognition model, a biometric recognition model, and a document recognition model. The digit recognition model focuses on extracting and classifying numerical information such as phone numbers; the named entity model focuses on recognizing entity information such as customer names, company names, and addresses, and matching them with corresponding data types; the image recognition model integrates optical character recognition (OCR) technology to recognize and classify text information in image documents; the biometric recognition model focuses on recognizing biometric data such as fingerprints and irises, and matching them with corresponding data types; and the document recognition model focuses on parsing the core data of structured / semi-structured documents such as Word documents and PDF documents, and matching them with corresponding data types.

[0069] The first data type refers to the preliminary classification result obtained after the small model cluster performs preliminary classification on the data to be identified. For example, the first data type is: natural person identity identifier.

[0070] The first confidence level refers to the quantitative score reflecting the reliability of the first data type, generated synchronously when the small model cluster obtains the first data type. Specifically, the first confidence level is usually between 0 and 100 points.

[0071] In one possible implementation, based on the file type of the file to be identified, a target small model corresponding to the file type is matched in the small model cluster; the data to be identified included in the file to be identified is input into the target small model so that the target small model can output the first data type and the first confidence level of the data to be identified based on the data to be identified.

[0072] Specifically, depending on the file type of the file to be identified, it may match one target small model or multiple target small models; similarly, it may obtain one set of first data types and corresponding first confidence scores, or it may obtain multiple sets of first data types and corresponding first confidence scores.

[0073] For example, a mapping relationship between all small models and file types in the small model cluster is pre-built. For instance, the file types corresponding to the digit recognition model include plain text files and structured table files. Based on the mapping relationship between "small models and file types", the target small model is matched according to the file type of the file to be recognized.

[0074] In this embodiment, the collaborative coverage of multi-functional small models solves the problem that a single small model can only handle specific data types. This allows for the identification of multi-source heterogeneous data, including structured, semi-structured, and unstructured data, in the communications industry, significantly improving model generalization ability. The lightweight and parallel operation of the small models avoids the high computational consumption of large models, enabling rapid preliminary classification and meeting the real-time processing needs of massive amounts of data in the communications industry. Furthermore, file type-based targeted matching adapts the small models, preventing irrelevant models from participating in calculations and reducing unnecessary computational consumption.

[0075] S103. Convert the data to be identified into a feature vector to be identified, and based on the feature vector to be identified and the preset feature vector library, obtain the similarity score of the data to be identified as the first data type through similarity calculation.

[0076] The feature vector to be identified refers to a high-dimensional numerical vector generated by mapping the data to be identified through technical means. It is a quantitative representation of the core features of the data to be identified. Specifically, the vector dimension can be dynamically adjusted according to the data type.

[0077] The preset feature vector library is a structured set of feature vectors built on communication industry standards, which stores structured feature vectors of all legally defined data types in the communication industry (such as user identity-related data, network and system resource data, etc.).

[0078] The similarity calculation method refers to the mathematical algorithm that measures the degree of similarity between the feature vector to be identified and the structured feature vectors in a pre-defined feature vector library. For example, the cosine similarity calculation method. The similarity score refers to a quantitative score (generally 0-100 points) converted from the similarity calculation result, directly reflecting the degree of similarity between the data to be identified and the structured feature vectors corresponding to the first data type. A higher similarity score indicates a stronger match.

[0079] In the embodiments of this application, the similarity score of the data to be identified as the first data type is an intuitive quantification of the degree of matching of the first data type, which supplements the deficiencies of the first confidence score output by the small model (which only reflects the reliability of the small model's own judgment).

[0080] In one possible implementation, the feature vector library includes a preset feature vector corresponding to each data type. The preset feature vector is obtained by transforming the data type and the preset features, keywords and data samples corresponding to the data type.

[0081] Specifically, the feature vector library stores a one-to-one correspondence between "data type - preset feature vector", covering all legally defined data types in the communications industry, and providing a unified benchmark for the similarity calculation of global data types.

[0082] Specifically, for the preset feature vector, for each data type, the elements such as "data type, preset features, keywords and data samples" of that data type are integrated and quantized into a high-dimensional feature vector to obtain the preset feature vector corresponding to that data type.

[0083] Furthermore, the data to be identified is converted into a feature vector to be identified; the similarity between the feature vector to be identified and the preset feature vector corresponding to each data type in the feature vector library is calculated to obtain the similarity score of the data to be identified for each data type, so as to obtain the similarity score of the data to be identified as the first data type.

[0084] Specifically, the similarity score between the feature vector to be identified and each of the preset feature vectors corresponding to all data types in the feature vector library is calculated. This yields the similarity score between the feature vector to be identified and the preset feature vector corresponding to each data type, thus providing a similarity score for the data to be identified for each data type. For example, assuming there are data types a, b, and c, the similarity score between the feature vector to be identified and the preset feature vectors corresponding to data types a, b, and c is calculated, resulting in similarity scores for the data to be identified as data type a, data type b, and data type c.

[0085] In this embodiment of the application, a full similarity comparison is performed between the feature vector to be identified and the preset feature vectors of all data types in the feature vector library. This verifies the rationality of the preliminary classification result (i.e., the first data type) obtained by the small model from the perspective of industry peer standards, and avoids misjudgment caused by the small model's insufficient range capability and limited semantic understanding.

[0086] S104. Based on the first confidence level and similarity score, obtain the second confidence level of the data to be identified.

[0087] The first confidence score is a quantitative score generated by the small model cluster based on the matching degree of its own training features when the data to be identified is of the first data type. It reflects the reliability of the small model's subjective judgment. The similarity score is the matching score between the feature vector to be identified and the preset feature vector corresponding to the first data type in the feature vector. It reflects the degree of objective matching between the data to be identified and the standard data type. The second confidence score is a comprehensive quantitative score generated by combining the first confidence score and the similarity score. It is the core indicator for measuring the reliability of the first data type. The second confidence score is generally between 0 and 100.

[0088] In one possible implementation, the first confidence score and similarity score are weighted based on preset weights to obtain the second confidence score of the data to be identified.

[0089] For example, the preset weight W1 of the first confidence level is 0.4, and the preset weight W2 of the similarity score is 0.6, where W1+W2=1; if the first confidence level is 90 points and the similarity score is 85 points, then the second confidence level = 0.4×90+0.6×85=87 points.

[0090] In this embodiment, the first confidence score only reflects the reliability of the small model's own judgment and cannot verify whether the first data type conforms to industry standards; the similarity score only reflects the degree of matching with industry standards and cannot reflect the model's accuracy in recognizing data details. The second confidence score, obtained by combining the two, can more comprehensively and accurately reflect the true reliability of the first data type compared to a single confidence score, and can reduce the misclassification rate of data classification to a certain extent.

[0091] S105. When the second confidence level is greater than the preset first threshold, the target data type of the data to be identified is determined to be the first data type.

[0092] The preset first threshold is a high-confidence threshold set for data classification scenarios.

[0093] Specifically, when the second confidence level is greater than the preset first threshold, it means that the preliminary classification result (i.e. the first data type) has both the high accuracy of small model recognition and the high adaptability to industry standards. The reliability of the first data type has reached the level that no further verification is required, and the target data type of the data to be identified can be directly determined as the first data type.

[0094] For example, if the first data type output by the small model cluster is "natural person identity", the corresponding second confidence score is 87 points, and the preset first threshold is 85 points, then if the second confidence score is greater than the preset first threshold, the target data type of the data to be identified is determined to be "natural person identity".

[0095] In this embodiment, when the second confidence level is greater than the preset first threshold, the first data type is considered to have sufficient reliability and compliance and can be directly output as the target data type (final classification result) without the need for additional verification through a large model cluster. This ensures classification accuracy and improves overall processing efficiency.

[0096] S106. When the second confidence level is between the preset first threshold and the preset second threshold, the target data type of the data to be identified is obtained through a large model cluster based on the first data type and the data to be identified.

[0097] Wherein, the preset first threshold is greater than the preset second threshold. Then, the second confidence level is between the preset first threshold and the preset second threshold, meaning the preset second threshold ≤ the second confidence level ≤ the preset first threshold.

[0098] The preset second threshold is a low-confidence threshold set for data classification scenarios.

[0099] The large model cluster comprises multiple dedicated large models, each fine-tuned using data of specific data types. Each large model within the cluster is obtained through this fine-tuning process, enabling it to understand the underlying business logic of its corresponding data type. This large model cluster ensures adaptability across all scenarios, thereby improving the accuracy of data classification.

[0100] For example, the large model cluster includes: a privacy compliance professional large model, a network technology professional large model, and a finance and legal professional large model. The privacy compliance professional large model is a large model that has been fine-tuned using a large amount of specific data such as laws and regulations, privacy policies, and user agreements; the network technology professional large model is a large model that has been fine-tuned using a large amount of specific data such as network architecture, technical specifications, and operation and maintenance logs; and the finance and legal professional big data model is a large model that has been fine-tuned using a large amount of legal scenarios and specific data such as financial data, legal documents, and industry regulations.

[0101] In this embodiment of the application, when the second confidence level is between the preset first threshold and the preset second threshold, it indicates that the first data type only has partial reliability of "small model recognition + industry matching". It cannot be directly determined and does not need to be re-identified. Then, the large model in the large model cluster is called to make up for the shortcomings of the small model's insufficient generalization ability and shallow semantic understanding, solve the problem of classification ambiguity, and achieve accurate classification of data.

[0102] Furthermore, the large model cluster is only invoked for the data to be identified whose second confidence level is between the preset first threshold and the preset second threshold, rather than for all data to be identified. This avoids the waste of computing power caused by invoking the large model for the entire dataset, and also avoids misjudgment of medium confidence data caused by using only the small model, thus achieving the optimal balance between efficiency and accuracy.

[0103] In one possible implementation, candidate data types are determined based on the similarity scores of the data to be identified for each data type.

[0104] Here, the candidate data types are the data types corresponding to the top N similarity scores, where N is a positive integer. Specifically, based on the similarity scores between the data to be identified and all data types in the feature vector library, the data types corresponding to the top N highest scores are selected to obtain the candidate data types.

[0105] Furthermore, when the second confidence level is between the preset first threshold and the preset second threshold, the target large model corresponding to the first data type in the large model cluster is matched based on the first data type; through target-to-model, the target data type of the data to be identified is obtained based on the data to be identified, the first data type, and the candidate data types.

[0106] To make it easier to understand, the following will be combined with... Figure 2 This application provides a detailed explanation of how the target data type is obtained in its embodiments.

[0107] S201. When the second confidence level is between the preset first threshold and the preset second threshold, match the target large model corresponding to the first data type in the large model cluster based on the first data type.

[0108] In one possible implementation, mapping rules are established between data types and large models based on various data types. The target large model corresponding to the first data type is then matched based on these mapping rules. For example, if the data type is "user identity" or "user service," the corresponding large model is the privacy compliance professional large model; if the data type is "construction" or "network and system resources," the corresponding large model is the network technology professional large model; assuming the first data type is "user identity," the target large model is the privacy compliance professional large model.

[0109] S202. Input the data to be identified, the first data type, and the candidate data types into the target large model so that the target large model can output the second data type of the data to be identified, the third confidence level corresponding to the second data type, and the judgment basis of the second data type based on the data to be identified, the first data type, and the candidate data types.

[0110] The second data type refers to the classification result output by the target large model after comprehensive reasoning based on the data to be identified, the first data type, and the candidate data types.

[0111] The third confidence level refers to the quantitative confidence score given by the target large model for the output second data type, which is usually 0 to 100 points, reflecting the certainty of the target large model's inference about the second data type.

[0112] The criteria for determining the target data type refer to the structured text description output by the target large model. It must clearly include elements such as the analysis of the characteristics of the data to be identified, the comparison of candidate data types, the correlation of industry standard clauses, and the reasons for excluding non-target types. It is a key support for the traceability and auditability of the classification results.

[0113] In this embodiment, the candidate data type provides a clear reasoning range for the target large model, rather than allowing the target large model to reason without boundaries. This greatly reduces the risk of the general large model generating new types that do not conform to industry standards and ensures that the results conform to the preset data type system of the communications industry.

[0114] S203. Determine whether the third confidence level is greater than the preset third threshold.

[0115] The preset third threshold refers to the critical value set for the classification accuracy requirements and large model precision performance in data classification scenarios. It is the standard for judging whether the second data type is reliable.

[0116] If the third confidence level is greater than the preset third threshold, then proceed to step S204.

[0117] If the third confidence level is not greater than the preset third threshold, then proceed to step S205.

[0118] S204. Determine the target data type of the data to be identified as the second data type.

[0119] Specifically, when the third confidence level is greater than the preset third threshold, it indicates that the target large model has high certainty in its comprehensive reasoning based on the data to be identified, the first data type, and the candidate data type. The second data type it outputs not only accurately matches the core features of the data to be identified, but also fully meets the requirements of the communications industry standards. Furthermore, the reasoning logic (judgment basis) is rigorous and reliable, and there is no uncertainty or illusion risk. It can be directly identified as the target data type of the data to be identified.

[0120] S205. Mark the data to be identified as data to be reviewed.

[0121] Specifically, if the third confidence level is not greater than the preset third threshold, it indicates that the comprehensive reasoning certainty of the target large model based on the data to be identified, the first data type, and the candidate data type is insufficient, and the second data type output does not have sufficient reliability and there is a risk of reasoning bias. Therefore, the data to be identified needs to be marked as data to be reviewed.

[0122] Furthermore, when the data to be identified is marked as data to be reviewed, it is handed over to human intervention for judgment; after human review, the correct classification result is updated to the target data type.

[0123] In this embodiment, although the large model possesses deep semantic understanding capabilities, it may encounter inference uncertainties when faced with extremely complex data from the communications industry. By using a third threshold for screening, low-confidence classification results can be avoided from being directly used as the final conclusion, reducing the risk of misjudgment. Furthermore, the closed-loop design with manual review as a fallback ensures that the automated and efficient flow of high-confidence results is not affected, while also enabling precise control over low-confidence, questionable data, thus balancing classification efficiency and quality.

[0124] In one possible implementation, when the third confidence level is greater than a preset fourth threshold, the data to be identified, the target data type corresponding to the data to be identified, and the judgment criteria are stored in a preset classification knowledge base.

[0125] Among them, the fourth threshold is greater than or equal to the third threshold. The preset fourth threshold is a critical value for high-confidence classification results in the data classification scenario. It is used to filter classification results with high reference value as the entry standard for the classification knowledge base.

[0126] The classification knowledge base refers to a structured database that stores high-confidence data classification cases. Specifically, it stores "data to be identified - target data type - judgment basis", which is used as prompt words to be input into the target model, thereby improving the accuracy and efficiency of subsequent reasoning from the target to the model.

[0127] Specifically, when the target data type of the data to be identified is obtained by using a large model cluster based on the first data type and the data to be identified, the classification knowledge base, the data to be identified, the first data type and the candidate data types are input into the target large model together so that the target large model can output the second data type.

[0128] In this embodiment, the classification knowledge base is pre-input as prompt words, providing the large model with readily available high-confidence judgment logic and industry-standard association ideas. This avoids the large model performing semantic parsing, feature sorting, and rule matching from scratch on similar data, reducing ineffective reasoning steps. Simultaneously, the classification knowledge base can supplement high-value historical cases, giving the large model examples to follow.

[0129] The above combination Figure 2 This section details how to obtain the target data type from a large model. The following section will continue with... Figure 1 This application introduces a data classification method provided by an embodiment.

[0130] In one possible implementation, when the second confidence level is less than a preset second threshold, the data to be identified is marked as data to be reviewed.

[0131] To make it easier to understand, the following will be combined with... Figure 3 An example of a data classification method provided in an embodiment of this application is given.

[0132] like Figure 3As shown, the process involves: acquiring the file to be identified to obtain the data to be identified; matching the target small model in the small model cluster based on the file type of the file to be identified; determining the first data type and first confidence level of the data to be identified through the target small model; converting the data to be identified into a feature vector to be identified, and based on the feature vector to be identified and a preset feature vector library, obtaining a similarity score and candidate data types for the data to be identified as the first data type through similarity calculation; obtaining the second confidence level of the data to be identified based on the first confidence level and the similarity score; determining the relationship between the second confidence level and the first threshold and the second threshold; and when the... If the second confidence level is greater than the first threshold, the target data type of the data to be identified is determined to be the first data type. If the second confidence level is less than the second threshold, the data to be identified is marked as data to be reviewed. If the first threshold is less than or equal to the second confidence level, the target large model in the large model cluster is matched based on the first data type. The second data type and the third confidence level are output through the target large model. It is then determined whether the third confidence level is greater than the third threshold. If the third confidence level is greater than the third threshold, the target data type of the data to be identified is determined to be the second data type. If the third confidence level is not greater than the third threshold, the data to be identified is marked as data to be reviewed.

[0133] This application provides a data classification method, including: acquiring data to be identified; classifying and identifying the data to be identified using a small model cluster to obtain a first data type and a first confidence level of the data to be identified; converting the data to be identified into a feature vector to be identified, and obtaining a similarity score of the data to be identified as the first data type based on the feature vector to be identified and a preset feature vector library through a similarity calculation method; obtaining a second confidence level of the data to be identified based on the first confidence level and the similarity score; determining the target data type of the data to be identified as the first data type when the second confidence level is greater than a preset first threshold; and obtaining the target data type of the data to be identified based on the first data type and the data to be identified using a large model cluster when the second confidence level is between the preset first threshold and the preset second threshold. In this embodiment, a second confidence level is generated by fusing the first confidence level obtained from a small model cluster with the similarity score obtained from a feature vector library, and a hierarchical decision-making mechanism is constructed to achieve an optimal balance between classification efficiency and accuracy. For highly reliable data with a second confidence level higher than the first threshold, the target data type is directly determined without subsequent redundant processing, ensuring the processing efficiency of massive amounts of conventional data. For medium-confidence data between the two thresholds, a large model cluster is then invoked for fine-tuning, accurately focusing on complex data requiring in-depth analysis. This hierarchical strategy avoids the computational consumption and inefficiency caused by fully invoking a large model, while ensuring the classification accuracy of complex data through the large model as a fallback. It compensates for the shortcomings of a single small model in terms of generalization ability and semantic understanding depth, and solves the problem that a single large model cannot cover all professional sub-fields in the communications industry, effectively improving the accuracy of data classification.

[0134] Furthermore, through an intelligent routing mechanism that matches target small models to file types using small model clusters and matches corresponding domain-specific large models to the primary data type using large model clusters, the system can accurately match the corresponding small / large models. The small model clusters, leveraging the refined recognition advantages of multiple pre-trained small models, accurately capture specific numbers, key entities, and fixed format features in the data to be identified, achieving rapid and accurate initial screening of data types. The specialized large models, focusing on sub-fields such as privacy compliance, network technology, and financial and legal affairs, rely on deep semantic understanding capabilities to control data context and core attribution, conducting in-depth analysis of complex data with medium confidence levels. This effectively avoids the illusions and generalization errors of general large models, making the classification results more aligned with professional standards in the communications industry, combining authority and accuracy.

[0135] Furthermore, the second confidence level stratified decision further filters out highly reliable data, submitting only a small amount of complex and ambiguous medium-confidence data to the large model for processing, thereby reducing the frequency of high-computation-cost large model calls from the source; at the same time, combined with the third confidence level, the large model's fine-judgment results are filtered a second time, with high-confidence results taking effect directly and low-confidence results being marked for verification, avoiding unnecessary computational consumption. While ensuring high accuracy in classification throughout the entire process, this significantly reduces overall computing resource consumption and system latency, achieving optimal performance and cost matching.

[0136] Furthermore, by constructing a classification knowledge base and a dynamic update mechanism, the system achieves self-learning and iterative optimization. Highly reliable classification results with a third confidence level higher than the fourth threshold, along with the core features of the data to be identified and complete judgment criteria, are stored in the knowledge base, forming reusable classification knowledge assets. In subsequent identification tasks, the knowledge base content serves as prompt words input to the target large model, providing readily available high-value judgment logic for precise judgment of similar data, guiding the model to reason quickly and accurately, and achieving efficient knowledge reuse. It possesses a closed-loop iterative capability of self-learning and self-optimization, continuously addressing new challenges in data security management within the communications industry.

[0137] Example 2:

[0138] The following is combined with Figure 4 This application provides a detailed description of a data classification device provided in its embodiments.

[0139] like Figure 4 As shown in the embodiment of this application, a data classification device includes the following modules:

[0140] Data acquisition module 401 is used to acquire data to be identified;

[0141] The preliminary identification module 402 is used to classify and identify the data to be identified through a cluster of small models, and obtain the first data type and the first confidence level of the data to be identified; wherein, the cluster of small models includes: multiple pre-trained small models;

[0142] The vector similarity module 403 is used to convert the data to be identified into a feature vector to be identified, and based on the feature vector to be identified and the preset feature vector library, to obtain a similarity score of the data to be identified as the first data type through a similarity calculation method.

[0143] The comprehensive scoring module 404 is used to obtain the second confidence level of the data to be identified based on the first confidence level and similarity score;

[0144] The target identification module 405 is used to determine the target data type of the data to be identified as the first data type when the second confidence level is greater than the preset first threshold; when the second confidence level is between the preset first threshold and the preset second threshold, the target data type of the data to be identified is obtained through a large model cluster based on the first data type and the data to be identified; wherein the first threshold is greater than the second threshold.

[0145] In one possible implementation, the data acquisition module 401 is specifically used to collect raw network traffic, preprocess the raw network traffic, and obtain a file to be identified; wherein, the file to be identified includes: data to be identified.

[0146] In one possible implementation, the preliminary identification module 402 is specifically used to match the target small model in the small model cluster that corresponds to the file type of the file to be identified.

[0147] The data to be identified in the file to be identified is input into the target small model so that the target small model can output the first data type and the first confidence level of the data to be identified based on the data to be identified.

[0148] In one possible implementation, the feature vector library includes: a preset feature vector corresponding to each data type, the preset feature vector being obtained based on the data type and the preset features, keywords, and data samples corresponding to the data type; a vector similarity module 403, specifically used to convert the data to be identified into a feature vector to be identified; and to calculate the similarity between the feature vector to be identified and the preset feature vector corresponding to each data type in the feature vector library, to obtain the similarity score of the data to be identified for each data type, so as to obtain the similarity score of the data to be identified as the first data type.

[0149] In one possible implementation, the device further includes: a candidate type module, used to determine candidate data types based on the similarity scores of the data to be identified for each data type; wherein, the candidate data types are the data types corresponding to the first N similarity scores, and N is a positive integer.

[0150] The target identification module 405 is specifically used to match the target large model in the large model cluster corresponding to the first data type when the second confidence level is between the preset first threshold and the preset second threshold; and to obtain the target data type of the data to be identified based on the target large model, the first data type, and the candidate data types.

[0151] In one possible implementation, the target recognition module 405 is specifically used to input the data to be recognized, the first data type, and the candidate data type together into the target large model, so that the target large model can output the second data type of the data to be recognized, the third confidence level corresponding to the second data type, and the judgment basis of the second data type based on the data to be recognized, the first data type, and the candidate data type.

[0152] If the third confidence level is greater than the preset third threshold, the target data type of the data to be identified is determined to be the second data type.

[0153] In one possible implementation, the target recognition module 405 is further configured to mark the data to be recognized as data to be reviewed when the third confidence level is not greater than the third threshold.

[0154] In one possible implementation, the target recognition module 405 is further configured to mark the data to be recognized as data to be reviewed when the second confidence level is less than the second threshold.

[0155] In one possible implementation, the device further includes: a knowledge base update module, used to store the data to be identified, the target data type corresponding to the data to be identified, and the judgment criteria into a preset classification knowledge base when the third confidence level is greater than a preset fourth threshold; wherein, the fourth threshold is greater than or equal to the third threshold, and the classification knowledge base is used as a prompt word input to the target large model.

[0156] This application provides a data classification device, including: a data acquisition module 401 for acquiring data to be identified; a preliminary identification module 402 for classifying and identifying the data to be identified using a small model cluster to obtain a first data type and a first confidence level of the data to be identified; a vector similarity module 403 for converting the data to be identified into a feature vector to be identified, and obtaining a similarity score of the data to be identified as the first data type based on the feature vector to be identified and a preset feature vector library through a similarity calculation method; a comprehensive scoring module 404 for obtaining a second confidence level of the data to be identified based on the first confidence level and the similarity score; and a target identification module 405 for determining the target data type of the data to be identified as the first data type when the second confidence level is greater than a preset first threshold; and obtaining the target data type of the data to be identified based on the first data type and the data to be identified using a large model cluster when the second confidence level is between the preset first threshold and the preset second threshold. In this embodiment, a second confidence level is generated by fusing the first confidence level obtained from a small model cluster with the similarity score obtained from a feature vector library, and a hierarchical decision-making mechanism is constructed to achieve an optimal balance between classification efficiency and accuracy. For highly reliable data with a second confidence level higher than the first threshold, the target data type is directly determined without subsequent redundant processing, ensuring the processing efficiency of massive amounts of conventional data. For medium-confidence data between the two thresholds, a large model cluster is then invoked for fine-tuning, accurately focusing on complex data requiring in-depth analysis. This hierarchical strategy avoids the computational consumption and inefficiency caused by fully invoking a large model, while ensuring the classification accuracy of complex data through the large model as a fallback. It compensates for the shortcomings of a single small model in terms of generalization ability and semantic understanding depth, and solves the problem that a single large model cannot cover all professional sub-fields in the communications industry, effectively improving the accuracy of data classification.

[0157] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0158] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data classification method, characterized in that, include: Obtain the data to be identified; The data to be identified is classified and identified by a cluster of small models to obtain a first data type and a first confidence level of the data to be identified; wherein, the cluster of small models includes: multiple pre-trained small models; The data to be identified is converted into a feature vector to be identified, and based on the feature vector to be identified and a preset feature vector library, a similarity score is obtained for the data to be identified as the first data type through a similarity calculation method. Based on the first confidence level and the similarity score, a second confidence level of the data to be identified is obtained; When the second confidence level is greater than the preset first threshold, the target data type of the data to be identified is determined to be the first data type; When the second confidence level is between the preset first threshold and the preset second threshold, the target data type of the data to be identified is obtained through a large model cluster based on the first data type and the data to be identified; wherein the first threshold is greater than the second threshold.

2. The method according to claim 1, characterized in that, The acquisition of the data to be identified includes: Raw network traffic is collected and preprocessed to obtain a file to be identified; wherein, the file to be identified includes: data to be identified.

3. The method according to claim 2, characterized in that, The step of classifying and identifying the data to be identified using a small model cluster to obtain a first data type and a first confidence level for the data to be identified includes: Based on the file type of the file to be identified, match the target small model in the small model cluster that corresponds to the file type; The data to be identified in the file to be identified is input into the target small model so that the target small model can output the first data type and the first confidence level of the data to be identified based on the data to be identified.

4. The method according to claim 1, characterized in that, The feature vector library includes: a preset feature vector corresponding to each data type, wherein the preset feature vector is obtained based on the data type and the preset features, keywords, and data samples corresponding to the data type; the step of converting the data to be identified into a feature vector to be identified, and obtaining a similarity score of the data to be identified as the first data type based on the feature vector to be identified and the preset feature vector library through a similarity calculation method, includes: The data to be identified is converted into a feature vector to be identified; The similarity between the feature vector to be identified and the preset feature vector corresponding to each data type in the feature vector library is calculated to obtain the similarity score of the data to be identified for each data type, so as to obtain the similarity score of the data to be identified as the first data type.

5. The method according to claim 4, characterized in that, The method further includes: determining candidate data types based on the similarity scores of the data to be identified for each data type; wherein, the candidate data types are the data types corresponding to the first N similarity scores, and N is a positive integer; When the second confidence level is between the preset first threshold and the preset second threshold, the target data type of the data to be identified is obtained through a large model cluster based on the first data type and the data to be identified, including: When the second confidence level is between the preset first threshold and the preset second threshold, the target large model in the large model cluster corresponding to the first data type is matched based on the first data type. Based on the target large model, the target data type of the data to be identified is obtained from the data to be identified, the first data type, and the candidate data type.

6. The method according to claim 5, characterized in that, The step of obtaining the target data type of the data to be identified based on the target large model, the first data type, and the candidate data types includes: The data to be identified, the first data type, and the candidate data type are input together into the target large model so that the target large model can output the second data type of the data to be identified, the third confidence level corresponding to the second data type, and the judgment basis of the second data type based on the data to be identified, the first data type, and the candidate data type. When the third confidence level is greater than the preset third threshold, the target data type of the data to be identified is determined to be the second data type.

7. The method according to claim 6, characterized in that, The method further includes: If the third confidence level is not greater than the third threshold, then the data to be identified is marked as data to be reviewed.

8. The method according to claim 6, characterized in that, The method further includes: When the third confidence level is greater than the preset fourth threshold, the data to be identified, the target data type corresponding to the data to be identified, and the judgment criteria are stored in the preset classification knowledge base; wherein, the fourth threshold is greater than or equal to the third threshold, and the classification knowledge base is used as a prompt word input to the target large model.

9. The method according to claim 1, characterized in that, The method further includes: If the second confidence level is less than the second threshold, the data to be identified is marked as data to be reviewed.

10. A data classification device, characterized in that, include: The data acquisition module is used to acquire the data to be identified. The preliminary identification module is used to classify and identify the data to be identified through a cluster of small models to obtain a first data type and a first confidence level of the data to be identified; wherein, the cluster of small models includes: multiple pre-trained small models; The vector similarity module is used to convert the data to be identified into a feature vector to be identified, and based on the feature vector to be identified and a preset feature vector library, to obtain a similarity score of the data to be identified as the first data type through a similarity calculation method. The comprehensive scoring module is used to obtain the second confidence level of the data to be identified based on the first confidence level and the similarity score; The target identification module is used to determine the target data type of the data to be identified as a first data type when the second confidence level is greater than a preset first threshold; and to obtain the target data type of the data to be identified based on the first data type and the data to be identified through a large model cluster when the second confidence level is between the preset first threshold and a preset second threshold; wherein the first threshold is greater than the second threshold.

Images