Data classification method, program product, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data classification method, a program product, electronic equipment and a storage medium. The method comprises the steps of obtaining a first entry and a second entry of to-be-classified data; the first entry and the second entry are at least partially identical; determining the target category of the to-be-classified data by using the first language model: determining the category of the first entry based on the first language feature of the first entry and the second language feature of each classification category by using the first language model, and determining the target category based on the category; if the classification of the to-be-classified data fails, performing language processing on the second item by using a second language model to obtain to-be-classified data carrying a language processing result, and returning to determine the target category by using the first language model; the language processing result comprises a semantic summary and / or a translation result. Therefore, automatic classification is achieved, the first language model can comprehensively understand semantics of the to-be-classified data after language processing of the second language model is introduced, and classification and grading accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security, in particular to a data classification method, a program product, an electronic device and a storage medium. BACKGROUND

[0002] Data security is the cornerstone of information security, covering data confidentiality, integrity and availability. As enterprises and organizations increasingly rely on data-driven decision-making and business operations, the security protection of data becomes particularly important. Among them, data classification is the core work of data security management. In related technologies, data classification is based on manual rule setting, which is time-consuming and laborious, and is difficult to cope with large-scale data. At the same time, the traditional method has limited understanding of context and semantics, which is easy to cause misclassification and cannot quickly adapt to changes in data content or business requirements. SUMMARY

[0003] The purpose of the embodiments of the present application is to provide a data classification method, a program product, an electronic device and a storage medium, so as to improve the accuracy, efficiency and adaptability of data classification.

[0004] The first aspect of the embodiments of the present application provides a data classification method, which comprises: obtaining a first entry and a second entry of to-be-classified data; the first entry and the second entry are at least partially the same; determining a target category of the to-be-classified data by using a trained first language model, comprising: determining the category of the first entry based on the first language feature of the first entry and the second language feature of each classification category by using the first language model, and determining the target category based on the category of all first entries; In the case that the to-be-classified data fails to be classified, performing language processing on the second entry by using a trained second language model to obtain to-be-classified data carrying a language processing result, and returning to the step of determining the target category of the to-be-classified data by using the trained first language model; wherein the language processing result includes semantic summary and / or translation result.

[0005] In the above implementation process, the automatic processing of large-scale data is realized, the manual intervention is reduced, and the efficiency can be greatly improved. At the same time, by introducing the language processing of the second language model, the first language model can comprehensively understand the semantics and content of the to-be-classified data, and the accuracy of classification and grading is significantly improved.

[0006] Further, before the obtaining a first entry and a second entry of to-be-classified data, the method further comprises: In response to the classification task creation instruction, a target classification strategy is obtained; wherein the target classification strategy is used to indicate a plurality of classification categories, a category to which the first entry belongs, and the target category is one or more of the plurality of classification categories; Based on the configuration instruction of the language model, the first entry and the second entry of the data to be classified are determined.

[0007] In the above implementation process, the target classification strategy is selected by the classification task creation instruction, a plurality of classification categories are determined, and the configuration of the first entry and the second entry is completed by the configuration instruction of the language model, so that the entire data classification process is automated, manual intervention is reduced, and the classification efficiency is greatly improved.

[0008] Further, the target classification strategy further includes a classification rule; the classification rule includes a preset keyword and / or a classification regular expression; the first entry and the second entry of the data to be classified include: matching the data to be classified with the classification rule; if the matching is successful, determining the target category of the data to be classified according to the matching result; if the matching fails, obtaining the first entry and the second entry of the data to be classified.

[0009] In the above implementation process, the classification rule is first used to classify the data to be classified, and in the case of classification failure, the language model is used for classification, thereby reducing the computational power consumption of data classification and grading.

[0010] Further, the language processing result includes a translated result and a recommended category of the data to be classified; the method further includes: updating the classification rule related to the recommended category in the target classification strategy based on the translated result.

[0011] In the above implementation process, the powerful data processing capability of the second language model is used to generate the recommended category of the data to be classified, and then the translated result of the second entry is used to update the classification rule related to the recommended category in the target classification strategy, thereby optimizing the classification rule. Through the dynamic optimization and feedback mechanism, it can quickly adapt to changes in data content and business requirements, effectively supplement the data that cannot be classified, and meet the needs of different scenarios.

[0012] Further, the method further includes: if the data to be classified carrying the language processing result fails to be classified, determining the recommended category as the target category of the data to be classified.

[0013] In the implementation process, when the first language model cannot classify the to-be-classified data, the recommended category output by the second language model is used as the target category of the to-be-classified data, which makes up for the deficiency of the first language model and improves the automation and accuracy of classification.

[0014] Further, before determining the belonging category, the method further comprises: extracting first language features of the first entry by using the first language model; wherein the first language features are used to represent one or more of sentence structure, sentiment orientation, text theme and text semantics of the first entry.

[0015] In the implementation process, the first language model extracts the first language features by comprehending the whole first entry, and performs similarity matching between the dimensions of the sentence structure, sentiment orientation, text theme and text semantics of the first entry and the second language features of each classification category, thereby improving the classification accuracy.

[0016] Further, the extracting the first language features of the first entry by using the first language model comprises: performing one or more of word segmentation, part-of-speech tagging, entity recognition and semantic role labeling processing on the first entry; performing syntax analysis, sentiment recognition, theme extraction and semantic extraction on the processed first entry to obtain the first language features.

[0017] In the implementation process, the first language model performs word segmentation, part-of-speech tagging, entity recognition and semantic role labeling processing on the first entry, and then performs syntax analysis, sentiment recognition, theme extraction and semantic extraction on the processed first entry to finally obtain the first language features, so that the first language model can comprehensively understand the whole content of the first entry in multiple dimensions, thereby improving the classification accuracy.

[0018] Further, the first entry includes multiple items; and the determining the target category based on the belonging categories of all the first entries comprises: if the number of target categories is set to multiple, determining the belonging category of all the first entries as the target category; if the number of target categories is set to one, counting the number of each belonging category, and determining the belonging category with the largest number as the target category.

[0019] In the implementation process, the number of target categories is set to determine the target category of the to-be-classified data from the belonging categories of all the first entries, and the classification process refers to the classification results of the first entries included in the to-be-classified data, thereby effectively improving the classification accuracy.

[0020] Further, the data to be classified fails in classification includes: All first entries of the data to be classified fail in classification, or the number of first entries belonging to the category is less than a preset number threshold.

[0021] In the implementation process, the number of first entries corresponding to each category is counted, and the number is compared with the preset number threshold, so as to improve the credibility of the category and the target category, thereby improving the classification accuracy.

[0022] The second aspect of the embodiment of the application provides a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor to implement the method of any one of the first aspect.

[0023] The third aspect of the embodiment of the application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor-executable instructions; The processor calls the executable instructions to implement the operations of the method of any one of the first aspect.

[0024] The fourth aspect of the embodiment of the application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are executed by a processor to implement the steps of the method of any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments of the application. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0026] Figure 1 A flowchart of a data classification method provided by the embodiment of the application; Figure 2 A flowchart of another data classification method provided by the embodiment of the application; Figure 3 A hardware structure diagram of an electronic device provided by the embodiment of the application. DETAILED DESCRIPTION

[0027] The technical solutions of the embodiments of the application will be described below with reference to the drawings in the embodiments of the application.

[0028] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0029] Data classification and grading is of great significance to data security. It not only helps enterprises and organizations to clarify data protection priorities, optimize resource allocation, and reduce security risks, but also supports data governance, compliance audit, and data value mining. Through scientific data classification and grading, enterprises and organizations can more efficiently protect data assets and achieve a balance between security and development in the digital era.

[0030] Specifically, the significance of data classification and grading lies in the following aspects: first, it can help identify high-value and high-risk data, so as to concentrate limited resources on protecting the most critical data and avoid wasting resources or inadequate protection caused by "one-size-fits-all" security policies. Second, data classification and grading is a necessary step to meet legal and regulatory compliance requirements, which can help enterprises and organizations avoid legal risks due to data leakage or misuse. In addition, data classification and grading provides a clear basis for data governance, supporting compliance audit and risk assessment, while optimizing data storage, access, and sharing processes by clarifying the nature, purpose, and sensitivity of data, providing strong support for data-driven business decisions and innovation. In the context of data sharing and openness, classification and grading can help clarify which data can be disclosed, which data needs to be desensitized or encrypted, so as to protect data security while promoting the legal use and circulation of data. In summary, data classification and grading is not only the basis of data security management, but also the key means for enterprises and organizations to achieve security and development in the digital era. Through scientific data classification and grading, enterprises and organizations can more efficiently protect data assets, reduce security risks, and maximize the value of data, providing strong impetus for business innovation and sustainable development.

[0031] In order to improve the accuracy, efficiency and adaptability of data classification and grading, the present application provides a data classification method for realizing data classification or data classification and grading. Figure 1 The flowchart of the data classification method is shown, which includes steps 110-130.

[0032] Step 110: obtaining a first item and a second item of the data to be classified; the first item and the second item are at least partially the same.

[0033] The to-be-classified data is stored in the classification hierarchical database. When classifying the data, the to-be-classified data is first read from the classification hierarchical database, and then the data classification process is sequentially or in parallel performed for each piece of to-be-classified data to obtain the target category of each piece of to-be-classified data.

[0034] The to-be-classified data is structured data. The structured to-be-classified data includes a plurality of entries. Since the structured data has a fixed format, uniform fields, and clear logical relationships, the first entry and the second entry can be directly read from the structured to-be-classified data. The first entry is one or more of the plurality of entries, and the second entry is one or more of the plurality of entries. The first entry and the second entry are at least partially the same. In addition, for unstructured data that has classification hierarchical requirements, the unstructured data can be first structured and converted into structured to-be-classified data.

[0035] In step 120, the target category of the to-be-classified data is determined by using the trained first language model, including: determining the category to which the first entry belongs based on the first language feature of the first entry and the second language feature of each classification category by using the first language model, and determining the target category based on the categories to which all first entries belong.

[0036] The first language model is used for classifying or classifying the to-be-classified data. The classification means that a plurality of classification categories are preset, and it is determined which category or categories the to-be-classified data belongs to. For example, the plurality of preset classification categories can include a personal information category, an operation data category, and a control data category. In addition, each category corresponds to a preset level. For example, the preset levels can include an important level, a core level, a sensitive level, and a general level. One category corresponds to one level, for example, the personal information category corresponds to the sensitive level, and the operation data category corresponds to the important level. Therefore, the classification hierarchical means determining which category the to-be-classified data belongs to and the level corresponding to the category, or which category or categories and the level corresponding to each category.

[0037] Specifically, the to-be-classified data includes one or more first entries, each first entry corresponds to a first language feature, and each classification category corresponds to a second language feature, and the first language feature and the second language feature are represented in the form of a vector. Each classification category corresponds to a description text for describing and defining the classification category, and the second language feature of each classification category is extracted from the corresponding description text and is used to represent the content and meaning of the classification category. For each first entry of the to-be-classified data, the first language model determines the category to which the first entry belongs based on the first language feature of the first entry and the second language feature of each classification category.

[0038] Exemplarily, for each first entry, the first language model calculates the similarity between the first language feature of the first entry and the second language feature of each classification category, respectively. As an example, the classification category with the highest similarity is determined as the category to which the first entry belongs. As another example, the classification category with a similarity higher than a preset similarity threshold is determined as the category to which the first entry belongs. At this time, if there are multiple similarities higher than the similarity threshold, the categories to which the first entry belongs include multiple categories; if all similarities are lower than the similarity threshold, the classification of the first entry fails.

[0039] After the classification of all first entries of the data to be classified is completed by using the first language model, the target category of the data to be classified is determined based on the categories to which all first entries of the data to be classified belong. Optionally, after the target category of the data to be classified is determined, the target classification corresponding to the target category can be obtained, and the target category and the target classification corresponding to the data to be classified are determined, so as to complete the classification and grading of the data to be classified.

[0040] In addition, if the target category of the data to be classified is successfully determined in step 120, the current data classification process is ended. Subsequently, the next data to be classified can be obtained for classification or classification and grading. If the target category of the data to be classified cannot be determined in step 120, step 130 is entered. The reason for the classification failure will be described below.

[0041] Step 130: In the case of classification failure of the data to be classified, the second entry is processed by using the trained second language model to obtain the data to be classified carrying the language processing result, and the step of determining the target category of the data to be classified by using the trained first language model is returned; wherein the language processing result includes semantic summary and / or translation result.

[0042] The second language model is used for language processing of the data to be classified, specifically for language processing of the second entry in the data to be classified, to obtain a language processing result. The second entry includes one or more entries. If the second entry includes multiple entries, the second language model can process each second entry one by one to output the language processing result corresponding to each second entry. Alternatively, the second language model can uniformly process all second entries.

[0043] Language processing includes semantic summarization and / or translation. The results of language processing include semantic summarization and / or translation results. Semantic summarization refers to summarizing the semantics expressed by the second item. For example, summarizing the semantics of each second item separately, or summarizing the semantics of all second items in the data to be classified as a whole. Furthermore, the data to be classified often uses English, English abbreviations, or Chinese pinyin in its naming, which can hinder the first language model from classifying the data. Therefore, if the second item contains non-Chinese characters, the second language model can be used to translate the second item to obtain the translation result.

[0044] After the second language model completes the language processing of the second item, the obtained language processing result can be carried in the data to be classified, and the process returns to step 120. When step 120 is executed a second time for the same data to be classified, the first language features can be re-extracted for each first item of the data to be classified carrying the language processing result, and the target category of the data to be classified carrying the language processing result can be determined using the first language model. Specifically, if the language processing result is a translation result, since the first item and the second item are at least partially the same, that is, at least part of the first item has been translated in the second language model, the first language features can be re-extracted for the translated first item, and step 120 can be executed. If the language processing result is a semantic summary, the re-extracted first language features carry the semantic information corresponding to the semantic summary.

[0045] The data classification method provided in this application first uses a first language model to classify each first entry of the data to be classified, determining the target category of the data based on the categories of all first entries. If classification fails, a second language model is then used to semantically summarize or translate the second entries of the data. The language processing results are then incorporated into the data to be classified, and the first language model is used again to re-determine the target category. This entire process automates the processing of large-scale data, reduces manual intervention, and significantly improves efficiency. Furthermore, by introducing language processing from the second language model, the first language model can fully understand the semantics and content of the data to be classified, significantly improving the accuracy of classification and grading.

[0046] The following provides a detailed description of steps 110-140.

[0047] According to some embodiments of this application, before performing step 110, the method further includes the steps of: in response to a classification task creation instruction, obtaining a target classification strategy, and determining the first entry and the second entry of the data to be classified based on a language model configuration instruction. The target classification strategy is used to indicate multiple classification categories, and the category to which the first entry belongs and the target category are one or more of the multiple classification categories.

[0048] The classification task creation command is used to create a classification task for the data to be classified. The data system can have multiple preset classification strategies, each recording multiple classification categories, with different strategies recording different categories. The classification task creation command carries indication information of the target classification strategy; therefore, in response to the command, the classification strategy corresponding to this indication information can be determined from among the multiple strategies as the target classification strategy. Based on the target classification strategy, it can be determined in which categories the first item and the data to be classified will be classified. That is, the category of the first item and the target category of the data to be classified both belong to one or more of the multiple classification categories indicated by the target classification strategy.

[0049] The language model configuration instructions are used to configure the first and second language models, including configuration instructions for the first language model and the second language model. The first language model configuration instructions configure the first entry and the similarity threshold for each first entry. Different first entries may have the same or different similarity thresholds. A higher similarity threshold results in higher classification accuracy but lower classification success rate. Therefore, to ensure accuracy, the similarity threshold can be set relatively low, for example, 80%. The second language model configuration instructions configure the second entries. Thus, based on the language model configuration instructions, it is possible to determine which entries in the data to be classified are the first and second entries.

[0050] As an example, the instructions for creating a classification task and configuring a language model can be generated by the data system. For instance, when the triggering conditions for a classification task are met, the data system automatically generates the instructions for creating the classification task and configuring the language model, thereby triggering the execution of the classification task.

[0051] As another example, the instructions for creating classification tasks and configuring language models can be input by the user. For instance, users can create classification tasks and configure the first and second language models through the human-computer interaction interface. When creating a classification task, users can enter the task name, select the target classification strategy, and select the data to be classified. Simultaneously, in the first language model configuration module, users can select the first entry of the data to be classified, including the data column name, data column description, file name, parameter name, and parameter description. Users can also set corresponding similarity thresholds for each first entry. In the second language model configuration module, users can select the second entry of the data to be classified, including the data column name, data column description, file name, file content, parameter name, and parameter description. In this way, users can complete the creation of classification tasks and the configuration of language models through the human-computer interaction interface.

[0052] As can be seen in this embodiment, by selecting the target classification strategy and determining multiple classification categories through the classification task creation instruction, and by completing the configuration of the first and second entries through the language model configuration instruction, the entire data classification process is automated, reducing manual intervention and significantly improving classification efficiency.

[0053] Furthermore, in some embodiments, the target classification strategy includes classification rules in addition to indicating multiple classification categories. These classification rules include preset keywords and / or classification regular expressions. Specifically, the classification rules record the correspondence between keywords and classification categories, and / or the classification regular expression corresponding to each category. When the data to be classified matches one or more keywords recorded in the classification rules, the category corresponding to the matched keywords can be determined as the target category of the data. Alternatively, when the data to be classified matches a classification regular expression recorded in the classification rules, the category corresponding to that classification regular expression is determined as the target category of the data.

[0054] Based on this, step 110 above obtains the first and second entries of the data to be classified, specifically including steps 111-113.

[0055] Step 111: Match the data to be classified with the classification rules; Step 112: If the match is successful, determine the target category of the data to be classified based on the matching result; Step 113: If the matching fails, obtain the first and second entries of the data to be classified.

[0056] After determining the target classification strategy, the data to be classified is first matched against the classification rules in the target classification strategy. Specifically, if the classification rule includes preset keywords, the data to be classified is matched against the keywords included in the classification rule, that is, it is checked whether the keyword exists in the data to be classified. If it exists, the match is successful, and the classification category corresponding to the matched keyword recorded in the obtained matching results can be used as the target category of the data to be classified. If the classification rule includes a classification regular expression, the data to be classified is matched against the classification regular expression, that is, it is determined whether the data to be classified matches the classification regular expression. If it matches, the match is successful, and the classification category corresponding to the matched classification regular expression recorded in the obtained matching results can be used as the target category of the data to be classified.

[0057] If the data to be classified successfully matches the classification rule, the current data classification process ends. Then, the next piece of data to be classified can be obtained for classification or categorization. If the data to be classified fails to match the classification rule, it means that the target category of the data cannot be determined using the classification rule. In this case, a first language model, or a combination of a first language model and a second language model, can be used to classify the data. Therefore, proceed to step 110 to obtain the first and second entries of the data to be classified. In other words, the failure to classify the data using the classification rule is the trigger condition for the failure to classify the data using the language model.

[0058] Understandably, classification rules-based classification consumes less computational power compared to language model-based classification. Therefore, this embodiment first uses classification rules to classify the data to be classified, and then uses a language model to classify it if classification fails, thereby reducing the computational power consumption of data classification and grading.

[0059] Furthermore, in some embodiments, the first language model and the second language model are two different language models. The data processing capability of the second language model is higher than that of the first language model, which is reflected in the fact that the model parameter scale of the second language model is larger than that of the first language model. For example, the first language model can be a language model based on HMM (Hidden Markov Model) or a language model based on RNN (Recurrent Neural Network), etc. The second language model can be a large language model (LLM).

[0060] Based on this, the language processing results output by the aforementioned second language model include translation results and recommended categories for the data to be classified. It is understood that, since the data processing capabilities of second language models (such as LLM models) are higher than those of first language models, the second language model can also generate recommended categories for the data to be classified based on the language processing results during the language processing of the second item of the data to be classified. Therefore, the above method may also include the following steps: The classification rules related to the recommended categories in the target classification strategy are updated based on the translation results.

[0061] As described above, the classification strategy includes classification rules, which record the correspondence between keywords and classification categories, and / or the classification regular expression corresponding to each classification category. To improve the classification strategy, this embodiment updates the classification rules in the target classification strategy using the translation result of the second entry in the data to be classified. Specifically, this includes: if the classification rule records the correspondence between keywords and classification categories, then adding the translation result of the second entry to the keywords corresponding to the above-mentioned recommended categories; if the classification rule records the classification regular expression corresponding to each classification category, then updating the classification regular expression corresponding to the recommended categories based on the translation result of the second entry.

[0062] As can be seen in this embodiment, the powerful data processing capabilities of the second language model are utilized to generate recommended categories for the data to be classified. Then, the translation results of the second item are used to update the classification rules related to the recommended categories in the target classification strategy, thereby optimizing the classification rules. Through dynamic optimization and feedback mechanisms, it can quickly adapt to changes in data content and business needs, effectively supplementing unclassifiable data to meet the needs of different scenarios.

[0063] In addition, in some embodiments, the above method may further include the following steps: If the classification of the data to be classified, which carries the results of language processing, fails, then the recommended category is determined as the target category for the data to be classified.

[0064] The term "classification failure of data carrying language processing results" refers to the following: after the data fails to be classified the first time using a first language model, it is processed again using a second language model, and then the first language model is used a second time to classify the data carrying the language processing results, which also fails. After two failures using the first language model, the recommended category output by the second language model can be directly determined as the target category of the data.

[0065] Understandably, if the data to be classified is from new business or new scenarios, the first language model may fail to classify it because it has not encountered this new data before. However, the second language model has a higher data processing capability than the first language model. Therefore, the second language model can comprehensively understand the data to be classified through its powerful semantic summarization capabilities, thus completing the classification of the new data. In this case, the recommended category obtained is the target category of the data to be classified.

[0066] As can be seen, in this embodiment, when the first language model is unable to classify the data to be classified, the recommended category output by the second language model is used as the target category of the data to be classified, which makes up for the shortcomings of the first language model and improves the automation and accuracy of classification.

[0067] Furthermore, based on any of the above embodiments, in the process of determining the target category of the data to be classified using a first language model, before determining the category to which the first entry belongs, the method further includes the step of: extracting first language features of the first entry using a first language model. The first language features are used to characterize one or more of the following: sentence structure, sentiment tendency, text topic, and text semantics of the first entry.

[0068] In other words, in the process of determining the target category of the data to be classified using a first language model, the first language features of the first item are first extracted using the first language model. Then, the first language model is used to determine the category to which the first item belongs based on the first language features of the first item and the second language features of each category. Finally, the target category of the data to be classified is determined based on the categories to which all the first items belong.

[0069] First-language features are used to characterize one or more of the following aspects of a first-entry text: sentence structure, sentiment, theme, and semantics. Sentence structure refers to the grammatical relationships between words in the first-entry text. Sentiment includes positive, negative, and neutral sentiment. Semantics refers to contextual relationships and semantic coherence.

[0070] In this way, the first language model extracts first language features by understanding the first item as a whole, and performs similarity matching between the second language features of each classification category and the sentence structure, sentiment, text topic and semantic dimensions of the first item, thereby improving classification accuracy.

[0071] In some embodiments, the process of extracting first language features using a first language model includes the following steps: The first entry is processed by one or more of the following methods: word segmentation, part-of-speech tagging, entity recognition, and semantic role recognition; the processed first entry is then subjected to grammatical analysis, sentiment recognition, topic extraction, and semantic extraction to obtain first language features.

[0072] Among them, the word segmentation of the first item refers to splitting the text of the first item into words or phrases. For example, splitting "I love China" into "I / love / China". Optionally, after word segmentation, meaningless words or phrases can be removed, such as "de", "shi", "zai", etc. The part-of-speech tagging of the first item refers to tagging the part of speech for each word or phrase, such as noun, verb, and adjective, etc. Optionally, if the first item is a non-Chinese language, such as English, Japanese, the words in the first item, such as verbs, can also be restored to the basic form. For example, restoring "running" to "run". The entity recognition of the first item refers to identifying specific entities in the text of the first item, such as person names, place names, and organization names, etc. The semantic role recognition of the first item refers to identifying the semantic roles of verbs in the text of the first item, such as agent and patient.

[0073] Among them, the processing results of each of the above-mentioned processes can be represented in the first item in the form of annotations. That is, the processed first item carries the identifiers corresponding to the processing results of each process. After completing the above-mentioned processes on the first item, grammatical analysis, sentiment recognition, topic extraction, and semantic extraction are performed on the processed first item, so as to obtain the first language feature.

[0074] Among them, grammatical analysis includes analyzing the grammatical relationships between words in the first item and understanding the sentence structure. Sentiment recognition includes judging the sentiment tendency of the text of the first item. Topic extraction includes extracting the topic of the text of the first item. For example, the LDA (Latent Dirichlet Allocation) model can be called for topic extraction. Semantic extraction includes understanding the context relationship of the text of the first item and capturing the semantic coherence in long texts or conversations. For example, semantic features can include summary generation and dialogue generation.

[0075] It can be seen that in this embodiment, the first item of the待分类数据 (to-be-classified data) is processed by the first language model for word segmentation, part-of-speech tagging, entity recognition, and semantic role recognition, and then grammatical analysis, sentiment recognition, topic extraction, and semantic extraction are performed on the processed first item. Finally, the first language feature is obtained, enabling the first language model to comprehensively understand the overall content of the first item in multiple dimensions and improving the classification accuracy.

[0076] According to some embodiments of the present application, based on any of the above-mentioned embodiments, the first item of the to-be-classified data includes multiple items. Based on this, in step 120, determining the target category of the to-be-classified data based on the categories to which all the first items belong specifically includes the following two situations.

[0077] Situation 1: If the number of target categories is set to multiple, determine the categories to which all the first items belong as the target categories.

[0078] For example, the number of target categories can be set as needed when creating a classification task. This can be achieved by specifying the number of target categories in the classification task creation instruction. Setting the number of target categories to multiple allows the data to be classified into multiple categories.

[0079] As an example, it can be determined that the category to which the first successfully classified item in the data to be classified belongs is the target category of the data to be classified.

[0080] As another example, when there are multiple target categories and a specified number is set, if the number of categories to which the first entry belongs is greater than the specified number, then the number of the first entry corresponding to each category is counted, and the specified number of categories is determined as the target category of the data to be classified in descending order of the count.

[0081] Case 2: If the number of target categories is set to one, count the number of each category and determine the category with the largest number as the target category.

[0082] If the number of target categories is set to one, meaning that only one category is allowed for the data to be classified, then if all successfully classified first entries correspond to a single category, that category is determined as the target category for the data to be classified. If the number of categories for the first entry is greater than one, then the number of first entries corresponding to each category is counted, and the category with the largest number of entries is determined as the target category for the data to be classified.

[0083] As can be seen, in this embodiment, by setting the number of target categories, the target category of the data to be classified is determined from the categories to which all first entries belong. The entire classification process refers to the classification results of the first entries included in the data to be classified, which can effectively improve the classification accuracy.

[0084] Furthermore, based on any of the above embodiments, the failure to classify the data to be classified may include the following reasons: Reason 1: All first entries of the data to be classified failed to be classified.

[0085] Understandably, the target category of the data to be classified is determined based on the category to which all first entries belong. If all first entries of the data to be classified fail to be classified, meaning the category to which the first entry belongs cannot be obtained, then the target category of the data to be classified cannot be determined based on the category to which the first entry belongs. In this case, the data to be classified is deemed to have failed to be classified. The reasons for the failure to classify the first entries include: the similarity between the first language feature and each second language feature is less than a preset similarity threshold.

[0086] Reason 2: The number of first entries belonging to the category is less than the preset threshold.

[0087] To improve the reliability of the target classification, after obtaining the category of each first entry, the number of first entries belonging to each category can be counted. It can be understood that the more first entries belonging to a category, the higher the reliability of the target category determined based on that category. Therefore, a threshold can be preset. When the number of first entries belonging to a category is greater than or equal to this threshold, the target category determined based on that category is considered reliable. Conversely, if the number of first entries belonging to a category is less than the threshold, the target category determined based on that category is considered reliable, meaning that category should not be identified as the target category for the data to be classified. If the number of first entries corresponding to all categories is less than the threshold, then all categories are unreliable, and the data to be classified is considered to have failed to be classified.

[0088] As can be seen in this embodiment, by counting the number of first entries corresponding to each category and comparing the number with a preset number threshold, the credibility of the category and the target category is improved, thereby improving the classification accuracy.

[0089] In addition, this application also provides a data classification and grading method, such as Figure 2 As shown, users can first create a classification and grading task through the human-computer interaction interface, including selecting a target classification strategy and a grading database storing the structured data to be classified. Then, task parameters are configured, including configuring the first language model and the second language model. The second language model is a large language model. The configuration of the first language model includes selecting the first item of the structured data and configuring the similarity threshold for each first item. The configuration of the second language model includes selecting the second item of the structured data. The first and second items are at least partially identical. After completing the task parameter configuration, the classification and grading task is started.

[0090] After initiating the classification and grading task, the task reads the structured data and the target classification strategy. The target classification strategy indicates that there are multiple classification categories and the corresponding grading for each category. In addition, the target classification strategy also includes classification rules.

[0091] Next, the structured data is first matched against the classification rules in the target classification strategy. If a match is successful, the target category of the structured data is determined based on the matching result, and the structured data carrying the target category identifier is stored in the hierarchical classification database.

[0092] If a match fails, it means the structured data cannot be classified using the classification rules. In this case, the first entry of the structured data can be obtained. The target category of the structured data is determined using a first-language model, including: segmenting the text of the first entry into words or phrases and removing meaningless words. Then, each word is labeled with its part of speech, and the words are restored to their basic forms. Specific entities in the text are then identified. The grammatical relationships between words in the first entry are identified to understand the sentence structure. The semantic roles of verbs in the sentences are identified. The sentiment of the text is determined, and the theme is extracted. The contextual relationships of the text are understood, and semantic coherence in long texts or dialogues is captured, thereby extracting the first-language features of the first entry.

[0093] Subsequently, for each first entry, the extracted primary language features are compared with the secondary language features of each classification category using similarity calculations such as cosine similarity and Euclidean distance. If the similarity is greater than the similarity threshold configured for that first entry, the classification category is determined to be the category to which the first entry belongs. If the similarity is less than the similarity threshold, the classification of the first entry is determined to be unsuccessful.

[0094] After obtaining the categories of all first entries in the structured data, the target category is determined based on these categories. Specifically, if there are multiple target categories, the category of each first entry is determined as the target category. If there is only one target category, the number of each category is counted, and the category with the largest number of entries is determined as the target category.

[0095] If all first entries fail to classify, or if the number of first entries belonging to a given category is less than a preset threshold, then the structured data classification fails. In this case, the second entries of the structured data are obtained, and a second language model is used to process them, including semantic summarization and translation. It's understandable that structured data is typically named using English, English abbreviations, or Chinese pinyin when creating database tables. The training data for the first language model is compiled based on national data security classification and grading standards, various industry data security classification and grading standards, and local data security classification and grading standards. The training data for the first language model is often written in Chinese, which may prevent it from understanding foreign language entries in the structured data. The second language model, however, possesses strong semantic understanding and contextual modeling capabilities, enabling it to perform well in complex scenarios, adapt to different data naming methods, and unify complex data into recognizable forms. Therefore, when the first language model fails to classify the structured data, the second language model is used to process the second entries, resulting in structured data carrying the language processing results. The first language model is then used to perform a second classification of this structured data. If the first language model successfully classifies the structured data, the structured data carrying the target category identifier will be stored in a hierarchical classification database.

[0096] Optionally, if the second language model outputs the translation result of the second item and the recommended category of the structured data, the classification rules related to that recommended category in the target classification strategy are updated based on the translation result. If the update fails, the classification information is manually verified.

[0097] Optionally, if the first language model fails to classify the structured data a second time, the recommended classification output by the second language model is determined to be the target category of the structured data. If the second language model does not output a recommended category, the automatic classification of the structured data is determined to have failed, and the structured data can be manually classified by a human.

[0098] Taking 1000 structured data entries as an example, if the classification rules include 30,000 pre-defined classification regular expressions and keywords, and the similarity threshold for the first entry is set to 70%, and the second language model enables matching of data column names and descriptions, then the classification and grading task takes 14 minutes. It classifies 3715 structured data entries using classification rule matching, 4399 using the first language model, and 1606 using the second language model. The overall classification success rate is as high as 97.2%. According to the experimental results, this application, by combining classification rules, ordinary language models, and large language model technologies, can comprehensively understand data content and significantly improve the accuracy of classification and grading. It automates the processing of large-scale data, reduces manual intervention, and greatly improves efficiency. The second language model can effectively correlate the translation results with the classification strategy, providing more support for the classification rules corresponding to the classification strategy, further improving the classification success rate and accuracy. Through dynamic optimization and feedback mechanisms, it can quickly adapt to changes in data content and business needs, effectively supplementing unclassifiable data and meeting the needs of different scenarios.

[0099] Based on the data classification method described in any of the above embodiments, this application also provides a computer program product, which includes one or more computer programs or instructions. The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. When the computer program is executed by a processor, it implements the data classification method described in any of the above embodiments.

[0100] Based on the data classification method described in any of the above embodiments, this application also provides, as well as... Figure 3 The diagram shows the structure of an electronic device. Figure 3 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to implement a data classification method as described in any of the above embodiments.

[0101] This application also provides a computer storage medium storing a computer program, which, when executed by a processor, can be used to perform a data classification method as described in any of the above embodiments.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0103] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0104] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0106] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0107] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A data classification method, characterized in that, The method includes: Obtain a first entry and a second entry of the data to be classified; the first entry and the second entry are at least partially identical. Determining the target category of the data to be classified using a trained first language model includes: using the first language model to determine the category to which the first entry belongs based on the first language features of the first entry and the second language features of each classification category, and determining the target category based on the categories to which all first entries belong; If the classification of the data to be classified fails, the second entry is processed using a trained second language model to obtain the data to be classified carrying the language processing results, and the process returns to the step of determining the target category of the data to be classified using the trained first language model; wherein the language processing results include semantic summarization and / or translation results.

2. The method according to claim 1, characterized in that, Before obtaining the first and second entries of the data to be classified, the method further includes: In response to a classification task creation instruction, a target classification strategy is obtained; wherein the target classification strategy is used to indicate a plurality of classification categories, and the category to which the first entry belongs and the target category are one or more of the plurality of classification categories; Based on the configuration instructions of the language model, the first entry and the second entry of the data to be classified are determined.

3. The method according to claim 2, characterized in that, The target classification strategy further includes classification rules; the classification rules include preset keywords and / or classification regular expressions; obtaining the first and second entries of the data to be classified includes: Match the data to be classified with the classification rules; If a match is successful, the target category of the data to be classified is determined based on the matching result; If the match fails, retrieve the first and second entries of the data to be classified.

4. The method according to claim 3, characterized in that, The language processing results include the translated results and the recommended categories of the data to be classified. The method further includes: The classification rules related to the recommended categories in the target classification strategy are updated based on the translation results.

5. The method according to claim 4, characterized in that, The method further includes: If the classification of the data to be classified, which carries the language processing results, fails, then the recommended category is determined as the target category of the data to be classified.

6. The method according to any one of claims 1-5, characterized in that, Before determining the category, the method further includes: The first language features of the first entry are extracted using the first language model; wherein the first language features are used to characterize one or more of the sentence structure, sentiment tendency, text topic and text semantics of the first entry.

7. The method according to claim 6, characterized in that, The step of extracting the first language features of the first entry using the first language model includes: The first entry is processed by one or more of the following methods: word segmentation, part-of-speech tagging, entity recognition, and semantic role recognition. The first processed entry is subjected to grammatical analysis, sentiment recognition, topic extraction, and semantic extraction to obtain the first language feature.

8. The method according to any one of claims 1-5, characterized in that, The first entry includes multiple entries; determining the target category based on the categories to which all the first entries belong includes: If the number of target categories is set to multiple, the category to which all the first entries belong is determined as the target category; If the number of target categories is set to one, count the number of each category and determine the category with the largest number as the target category.

9. The method according to any one of claims 1-5, characterized in that, The failure to classify the data to be classified includes: All first entries of the data to be classified fail to be classified, or the number of first entries belonging to the category is less than a preset threshold.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

11. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; Wherein, when the processor invokes the executable instructions, it implements the operation of any one of the methods described in claims 1-9.

12. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-9.

Images