A data classification method, device, equipment and storage medium
By constructing government data sets and extracting prominent features in the hierarchy, combined with the multi-task learning framework, the problem of high consumption of existing multi-label classification calculations is solved, and efficient data classification and hierarchical correlation capture is achieved.
Patent Information
- Application Number
- CN202411118331.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-15
AI Technical Summary
The existing hierarchical multi-label classification method consumes too much calculation and fails to effectively capture the dependencies between levels.
By obtaining government data and classification label system, building data sets and performing word segmentation, transforming them into word vectors, extracting significant features based on the hierarchy, and combining with a multi-task learning framework, the significant features of each level are spliced with the features of the previous level to achieve classification .
It reduces calculation losses, improves classification efficiency, accurately captures the correlation between levels, and improves the accuracy of data classification models.
Smart Images

Figure CN118643158B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data classification, and in particular to a data classification method, device, equipment and storage medium. Background Art
[0002] Classification is a classic problem in machine learning. However, in many real-world problems, there is a hierarchy between labels. The lower labels in the hierarchy are constrained by the higher-level labels. While the hierarchy brings about a hierarchical relationship between class labels, it also brings more challenging features such as computational complexity.
[0003] There are two main types of known hierarchical multi-label classification algorithms. The first method is to flatten the labels, convert the hierarchical multi-label classification problem into a multi-label classification problem, and use a single classifier for prediction. However, this method ignores the dependencies between categories in the hierarchical classification. Based on the previous method, the second method is proposed to train a classifier for each category or level separately. Each classifier is responsible for predicting the corresponding category or the corresponding category level. When predicting, the final subclass is predicted based on the prediction result of the parent class. However, because there are too many classifiers involved, the number of model parameters will be huge, so this method has the defect of huge computational consumption.
[0004] Therefore, in the prior art, there is a problem that the hierarchical multi-label classification method consumes too much computation.
[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention
[0006] The main purpose of the present invention is to provide a data classification method, device, equipment and storage medium, aiming to solve the technical problem of excessive computational consumption of hierarchical multi-label classification methods.
[0007] To achieve the above object, the present invention provides a data classification method, comprising:
[0008] Acquire government data and a classification label system corresponding to the government data, and construct a government data set based on the government data and the classification label system;
[0009] Segmenting each data in the government data set, and converting each segmented word into a word vector;
[0010] Based on the hierarchical structure in the classification label system, feature extraction is performed on each of the word vectors to obtain significant features of each level;
[0011] The salient features of each level are respectively concatenated with part of the salient features of the previous level to obtain the target features of each level, and the target features of each level are used as the classification results of each level.
[0012] Optionally, the step of constructing a government affairs data set according to the government affairs data and the classification label system includes:
[0013] Identify preset category information in the government data according to the preset category field in the classification label system;
[0014] Classifying the government data based on the preset category information to determine the data category of each piece of government data;
[0015] Performing data enhancement on each of the government affairs data according to the data category corresponding to each of the government affairs data to obtain government affairs enhanced data;
[0016] A government affairs data set is constructed based on the government affairs data and the government affairs enhanced data.
[0017] Optionally, the step of performing data enhancement on each of the government affairs data according to the data category corresponding to each of the government affairs data to obtain the government affairs enhanced data includes:
[0018] Conducting data volume statistics on the government data of each data category to determine the data volume proportion of each data category;
[0019] Determine whether the data volume ratio of each data category is balanced;
[0020] If balanced, all data categories are taken as categories to be enhanced;
[0021] If it is unbalanced, the data category with a relatively small proportion of data in each data category will be taken as the category to be enhanced;
[0022] Data enhancement is performed on each of the government data according to the category to be enhanced to obtain government data enhancement data, wherein in the data collection of the government data enhancement data and the government data, the proportion of the data volume of each data corresponding to the data category is balanced.
[0023] Optionally, the step of classifying the government data based on the preset category information to determine the data category of each piece of government data includes:
[0024] Determine a first category corresponding to the preset category information;
[0025] Acquire the hierarchical relationship of the preset category information according to the classification label system, and determine whether the preset category information has a second category at a lower level according to the hierarchical relationship;
[0026] If so, the first category and the second category are used as data categories of the government data;
[0027] If it does not exist, the first category will be used as the data category of the government data.
[0028] Optionally, the step of extracting features from each word vector based on the hierarchical structure in the classification label system to obtain significant features at each level includes:
[0029] Input each of the word vectors into a preset pre-trained language model to obtain semantic information corresponding to each word vector;
[0030] Acquire a hierarchical structure existing in the classification label system, and determine a plurality of levels existing in the hierarchical structure;
[0031] The features of each semantic information are extracted through a convolutional neural network to obtain the significant features of each level.
[0032] Optionally, the step of extracting features from each of the semantic information by using a convolutional neural network to obtain significant features of each level includes:
[0033] Inputting each of the semantic information into convolutional neural networks at different levels, performing feature extraction at different levels on the semantic information, and obtaining semantic features corresponding to each level;
[0034] The semantic features of each level are pooled, and the salient features of each level are extracted from the semantic features.
[0035] Optionally, the convolution kernel widths of convolution neural networks at different levels are different, wherein the lower the level of the convolution neural network, the wider the convolution kernel of the convolution neural network, and the coarser the granularity of the semantic features extracted at the corresponding level.
[0036] In addition, to achieve the above purpose, the present application also provides a data classification device, the data classification device comprising:
[0037] A data module, used to obtain government data and a classification label system corresponding to the government data, and to construct a government data set based on the government data and the classification label system;
[0038] A vector module, used for segmenting each data in the government data set and converting each segmented word into a word vector;
[0039] A feature module, used to extract features from each word vector based on the hierarchical structure in the classification label system to obtain significant features at each level;
[0040] The classification module is used to concatenate the salient features of each level with part of the salient features of the previous level to obtain target features of each level, and use the target features of each level as the classification results of each level.
[0041] In addition, to achieve the above-mentioned purpose, the present application also provides a data classification device, which includes: a memory, a processor, and a data classification program stored in the memory and executable on the processor, and when the data classification program is executed by the processor, the data classification method as described above is implemented.
[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a data classification program is stored on the computer-readable storage medium. When the data classification program is executed by the processor, the steps of the data classification method as described above are implemented.
[0043] In this embodiment, by obtaining government data and the classification label system corresponding to the government data, and constructing a government data set according to the government data and the classification label system, the government data is enhanced in combination with the generative model after domain fine-tuning, and the classification categories of the government data are balanced, so that the enhanced government data set is more in line with the original characteristics of the government data, and as the input of the classification model, the accuracy of the data classification model can be improved; each data in the government data set is segmented, and each segmentation is converted into a word vector; based on the hierarchical structure in the classification label system, each word vector is feature extracted to obtain significant features of each level, and the classification of different levels is regarded as an independent classification task, so as to realize parallel learning of multiple tasks; the significant features of each level are spliced with part of the significant features of the previous level, respectively, to obtain the target features of each level, and the target features of each level are used as the classification results of each level, so that the high-level classification can obtain the important information of the previous level, capture the correlation between levels, and realize hierarchical multi-label classification. This application introduces a multi-task learning framework for data classification, realizes parallel learning of multiple tasks, can improve the efficiency of classification, and reduce computational loss. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a flow chart of a first embodiment of a data classification method of the present invention;
[0045] Figure 2 This is a flow chart of a second embodiment of the data classification method of the present invention;
[0046] Figure 3 This is a flow chart of a third embodiment of the data classification method of the present invention;
[0047] Figure 4It is a flow chart of the present invention from "building a government data set" to "BERT structure output";
[0048] Figure 5 It is a flow chart of the present invention from "BERT structure output" to "output level classification result";
[0049] Figure 6 A schematic diagram of the structure of the multi-device of data classification level of the present invention;
[0050] Figure 7 It is a structural diagram of a data classification device in a hardware operating environment involved in an embodiment of the present invention.
[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0053] Classification is a classic problem in machine learning. However, in many real-world problems, there is a hierarchy between labels. The lower labels in the hierarchy are constrained by the higher-level labels. While the hierarchy brings about a hierarchical relationship between class labels, it also brings more challenging features such as computational complexity.
[0054] There are two main types of known hierarchical multi-label classification algorithms. The first method is to flatten the labels, convert the hierarchical multi-label classification problem into a multi-label classification problem, and use a single classifier for prediction. However, this method ignores the dependencies between categories in the hierarchical classification. Based on the previous method, the second method is proposed to train a classifier for each category or level separately. Each classifier is responsible for predicting the corresponding category or the corresponding category level. When predicting, the final subclass is predicted based on the prediction result of the parent class. However, because there are too many classifiers involved, the number of model parameters will be huge, so this method has the defect of huge computational consumption.
[0055] Therefore, in the prior art, there is a problem that the hierarchical multi-label classification method consumes too much computation.
[0056] Based on this, the embodiment of the present invention provides a data classification method, referring to Figure 1 , Figure 1 The figure is a flow chart of a first embodiment of a data classification method of the present invention.
[0057] In this embodiment, the data classification method includes:
[0058] Step S10, obtaining government data and a classification label system corresponding to the government data, and constructing a government data set based on the government data and the classification label system.
[0059] Government data is a type of text data in a specific field. It contains specific fields or identifiers to mark or identify key information. The fields are specialized, terminological, and standardized. Secondly, government data has a small sample size, high annotation costs, and imbalanced categories.
[0060] The classification label system is a hierarchical multi-label structure that exists in the classification of government data. As an example, a certain type of government data exists in three levels. The first level includes: information type A, information type B, information type C. In the second level: information type A contains information 1, information 2, information 3; information type B contains information 4, information 5, information 6; information type C contains information 7, information 8; in the third level: information 1 contains specific data a and specific data b; information 2 contains specific data c and specific data d; information 3 contains specific data e... Among them, the hierarchical structure is the corresponding level in the classification label system. In the hierarchical multi-label problem, the levels are usually multiple; the hierarchical relationship is the implicit and intrinsic relationship between the level and the category. For example, in the large category of information type A, information type A is the parent class of information 1, information 2 and information 3, and information 1, information 2 and information 3 are subclasses of information type A. Specific data a, specific data b, specific data c, specific data d and specific data e are grandchildren of information type A. The connection between the parent class, subclass and grandchildren in the above example is the hierarchical relationship in the hierarchical multi-label.
[0061] In another example, there is currently a piece of government data with the structure of "Monitoring Query_Monitoring and Early Warning Information Type A_Information 1 Specific Data a", where "Monitoring Query_Monitoring and Early Warning Information Type A" is the table name, and "Information 1 Specific Data a" is the field name, which records the specific data content. The hierarchical relationship of the field can be "parent class: Type A - child class: Information 1 - grandchild class: Specific Data a". It should be noted that based on the composition structure of the government data in the above example "Monitoring Query_Monitoring and Early Warning Information Type A_Information 1 Specific Data a", in the actual government data, "Monitoring and Early Warning Information Type A" can be a detailed table that specifically records the data of Information Type A of this piece of government data, and "Information 1 Specific Data a" can be the real data in Information Type A that records Information 1 and Specific Data a.
[0062] Step S20, segmenting each data in the government data set, and converting each segmented word into a word vector.
[0063] The government data set includes government data and government enhanced data, and each word in each piece of data is represented by a vector. For example, "monitoring query_monitoring and early warning information type A_information 1 specific data a" is segmented to obtain "monitoring", "query", "monitoring", "early warning", "information type A", "information 1 specific data a", etc., and there is no restriction on the segmentation standard.
[0064] Step S30, based on the hierarchical structure in the classification label system, feature extraction is performed on each of the word vectors to obtain significant features of each level.
[0065] There are multiple levels in the classification label system, and each level is combined as the hierarchical structure of the data. In this application, the classification of different levels is carried out separately, and the significant features under the standards of different levels can be obtained. It should be noted that the granularity of the significant features obtained at different levels is different.
[0066] Step S40, concatenating the salient features of each level with part of the salient features of the previous level to obtain target features of each level, and using the target features of each level as the classification results of each level.
[0067] It should be noted that in the commonly used hierarchical multi-label classification method, the hierarchical multi-label classification problem can be converted into a solution for the multi-label classification problem by flattening the labels. However, although this method is simple and efficient, it does not take into account the implicit relationship between the levels. In the hierarchical multi-label label system, there is a hierarchical correlation between the parent class, child class, and grandchild class, and there is a natural, given "parent-child" relationship between the labels. Therefore, when dealing with hierarchical, multi-label classification problems, it is very important to take the implicit relationship between the levels into account. After obtaining the salient features corresponding to each level, since there is a hierarchical relationship between the levels and the granularity of the levels is different, the coarse-grained salient features can be spliced with the fine-grained salient features, so that the data can be more accurately classified into parent class, child class and grandchild class labels. At the same time, the number of classifiers can be reduced as much as possible when dealing with hierarchical multi-label classification problems.
[0068] It should be noted that Hierarchical Multi-label Classification (HMC) can be regarded as a special case of multi-label classification. Its main features are: there is a hierarchical structure (tree structure) between labels, and different labels are not independent of each other. If an object belongs to a child label, then it naturally belongs to the corresponding parent label; each object can belong to multiple classes, so each object can be associated with multiple different paths of the class hierarchy (i.e., tree structure). Multi-label classification means that each sample may have multiple categories (labels). For example, there may be multiple labels to judge a book. For example, some books may be labeled "realism, folk tales", and some books may be labeled "psychology, spiritual growth, family affection". Each sample may belong to one category or multiple categories.
[0069] Specifically, some of the significant features of the first level are spliced with all the significant features of the second level to obtain the final features of the second level. Similarly, some of the final features of the second level are spliced with all the significant features of the third level to obtain the final features of the third level. Among them, some of the significant features can be a significant feature in the level, and when there are multiple significant features in the level, one of the significant features can be spliced with all the significant features of the next level, so as to obtain multiple classification results of the next level. The purpose of this is to enable deep-level classification to obtain important information of the previous level, capture the correlation between levels, and take the classification relationship between levels into account in the classification model.
[0070] For example, the salient features obtained at the first level are information type A, information type B, etc., and the salient features obtained at the second level are information 1, information 2, information 3, information 4, etc. Then, the classification results at the second level can be information type A, information 1, information type A, information 2, or information type A, information 3. When the salient feature of the first level is information type B, the classification results at the second level can be information 4, information 5.
[0071] In this embodiment, by obtaining government data and the classification label system corresponding to the government data, and constructing a government data set according to the government data and the classification label system, the government data is enhanced in combination with the generative model after domain fine-tuning, and the classification categories of the government data are balanced, so that the enhanced government data set is more in line with the original characteristics of the government data, and as the input of the classification model, the accuracy of the data classification model can be improved; each data in the government data set is segmented, and each segmentation is converted into a word vector; based on the hierarchical structure in the classification label system, each word vector is feature extracted to obtain significant features of each level, and the classification of different levels is regarded as an independent classification task, so as to realize parallel learning of multiple tasks; the significant features of each level are spliced with part of the significant features of the previous level, respectively, to obtain the target features of each level, and the target features of each level are used as the classification results of each level, so that the high-level classification can obtain the important information of the previous level, capture the correlation between levels, and realize hierarchical multi-label classification. This application introduces a multi-task learning framework for data classification, realizes parallel learning of multiple tasks, can improve the efficiency of classification, and reduce computational loss.
[0072] Further, see Figure 2 The second embodiment of the present invention provides a data classification method based on the above Figure 2 In the embodiment shown, the step of constructing a government data set based on the government data and the classification label system includes:
[0073] Step A10, identifying preset category information in the government data according to the preset category field in the classification label system;
[0074] Step A20, classifying the government data based on the preset category information to determine the data category of each piece of government data;
[0075] Step A30, performing data enhancement on each of the government data according to the data category corresponding to each of the government data to obtain government enhanced data;
[0076] Step A40, constructing a government affairs data set according to the government affairs data and the government affairs enhanced data.
[0077] In this embodiment, it should be noted that the government data contains preset category fields or identifiers, which are used to mark and identify preset category information, and have the characteristics of professional, terminological and standardized field identifiers. The preset category fields can be various categories of information existing in the classification label system, and can specifically be unique identification codes corresponding to a certain category, personal accounts, etc. In addition, government data has the characteristics of small sample size, high annotation costs, and imbalanced categories. Therefore, it is considered to use a generative model that has been fine-tuned by domain to enhance the government data, so that the enhanced government data set is more in line with the original characteristics of the government data.
[0078] Exemplarily, steps A10 to A40 include:
[0079] Obtain preset category fields in the classification label system, and identify preset category information corresponding to the preset category fields from the government data based on the preset category fields; classify the government data based on the preset category information in the government data, so as to determine the data category of each government data, wherein each government data may be of multiple data categories; enhance the government data based on the data category to obtain government enhanced data; and construct a government data set based on the government data and the government enhanced data.
[0080] This embodiment can identify data categories with small and unbalanced sample data from government data, and enhance the data categories, thereby expanding the data volume in the government data set and balancing the data categories of each data in the government data set, ensuring that the enhanced results are more in line with the non-publicity, confidentiality, domain characteristics of the government data, etc.
[0081] Furthermore, the step of classifying the government data based on the preset category information and determining the data category of each piece of government data includes:
[0082] Step A21, determining a first category corresponding to the preset category information;
[0083] Step A22, obtaining the hierarchical relationship of the preset category information according to the classification label system, and determining whether the preset category information has a second category at a lower level according to the hierarchical relationship;
[0084] Step A23: if it exists, taking the first category and the second category as data categories of the government data;
[0085] Step A24: if it does not exist, the first category is used as the data category of the government data.
[0086] In this embodiment, it should be noted that the first category is the data category directly belonging to the preset category information, and the second category is the data category of the lower level corresponding to the preset category information. For example, the preset category information identified in the "Dog Number in the Information Record Table of Dog Keepers in a Certain Community" is "Dog Number", and the first category corresponding to the preset category information is "Dog Information", but in fact, in the hierarchical relationship corresponding to "Dog Information", the parent category is "Community Management", the child category is "Dog Management", and the grandchild category is "Dog Information", and the above parent category and child category are low-level data categories of the first category "Dog Information", and "Community Management" and "Dog Management" are used as the second category of the preset category information, and in the case of the existence of the second category, both the first category and the second category are used as data categories of the government data.
[0087] In the prior art, EDA is generally used for data enhancement. EDA is a simple but very effective text data enhancement method, including synonym replacement, random insertion, random exchange, and random deletion. However, the disadvantage of EDA is that EDA may reduce the performance of the model. The reason is that EDA may change the meaning of the sentence during the enhancement process, but it still retains the original category label, thereby generating a sentence with an incorrect label. At the same time, in the problem of the application for classifying government data, since the data has a strong domain (for example, the correlation between government data and entertainment data is not strong), the general data enhancement method also has the disadvantage that the enhanced data may not have a semantic relevance to the original data, that is, it is possible to enhance the semantic deviation or even the opposite sentence, which is not conducive to the subsequent classification of the data. Therefore, considering the non-publicity, confidentiality, and domain characteristics of government data, the existing and general data enhancement methods are no longer applicable, because the data enhanced by the general data enhancement method may have a semantic difference from the original data. In this application, the LLaMA2 model is selected for data enhancement. The LLaMA large model is a large language model developed by Meta, which is free and commercially available. The performance of LLaMA-13B is better than GPT-3 (175B), while the size is more than 10 times smaller. Compared with LLaMA, the number of tokens in the LLaMA2 model has increased by 40% to 2T, and the context length has doubled, which can better understand and generate long texts. The advantage of using a generative model to enhance government data is that the data used to fine-tune the LLaMA2 large model is internal, non-public, and a large amount of government data, which can ensure that the enhanced results are more in line with the original characteristics of the government data. As the input of the classification model, it can improve the accuracy of the data classification model.
[0088] Exemplarily, steps A21 to A24 include:
[0089] First, the first category of the government data is determined based on the identified preset category information, and by obtaining the hierarchical relationship in the classification label system, it is determined based on the hierarchical relationship whether the government data has a second category that is lower than the first category; if so, both the first category and the second category are used as data categories of the government data; if not, only the first category is used as the data category of the government data.
[0090] This embodiment identifies preset category information of government data and determines whether it is preset category information or sub-preset category information under a parent category, so as to accurately classify the government data, thereby being able to determine whether the data of this category needs to be protected and expanded in the future.
[0091] Furthermore, the step of performing data enhancement on each of the government data according to the data category corresponding to each of the government data to obtain the government enhanced data includes:
[0092] Step A31, performing data volume statistics on the government data of each data category to determine the data volume ratio of each data category;
[0093] Step A32, determining whether the data volume ratio of each data category is balanced;
[0094] Step A33: If balanced, all data categories are taken as categories to be enhanced;
[0095] Step A34: If there is an imbalance, the data category with a relatively small data volume ratio in each data category is taken as the category to be enhanced;
[0096] Step A35, performing data enhancement on each of the government data according to the category to be enhanced to obtain government data enhancement, wherein in the data collection of the government data enhancement data and the government data, the proportion of the data volume of each data corresponding to the data category is balanced.
[0097] In this embodiment, it should be noted that a common problem in the government data classification scenario is the imbalance of data categories, that is, some classification categories occupy a large amount of data, while some categories occupy a very small amount of data. Therefore, by introducing data enhancement methods in the government data classification problem, data enhancement can generate more data samples based on existing data samples, thereby improving the model effect at a low cost.
[0098] In order to perform fine-tuning, each data example needs to be converted into an instruction, where "instruction" is a different sentence structure that issues instructions for data enhancement, "input" is the data to be enhanced, and "output" is the expected result after enhancement. This application performs data enhancement on categories with small data volumes, so that the data volume occupied by each category tends to be balanced, which is conducive to subsequent classification.
[0099] It should be noted that the original LLaMA2 model cannot give a perfect answer to the input instruction. Therefore, in this application, LLaMA2 needs to be fine-tuned. The LoRA method is used to fine-tune LLaMA2. LoRA is an efficient model fine-tuning method that adjusts the model output by adding a learnable linear layer on top of the large model. This linear layer is used to weight the features of the pre-trained model to achieve adaptability to the input. LoRA uses the data corresponding to the downstream task and only trains the newly added parameters to adapt to the downstream task. After the new parameters are trained, the new parameters are merged with the old model parameters, so that the effect of fine-tuning the entire model can be achieved on the new task. For example, before LLaMA2 is fine-tuned, the user issues an instruction: data enhancement for the following text: "Monitor query monitor a community to manage dog and puppy information". The model reply may explain in detail each word segmentation of "monitor", "query", "monitor", "a community", "manage dog", and "puppy information" in the text, rather than optimizing it on the original text. Therefore, it is necessary to fine-tune the model to better adapt to the task. After fine-tuning LLaMA2, the user issued a command: to enhance the following text: "Monitoring query monitoring a community management dog puppy information". The model replied: Monitoring query monitoring community management management dog dog information. It can be seen that the fine-tuned model gave a response that met the requirements, and the semantics were consistent with the original dataset.
[0100] Exemplarily, steps A31 to A35 include:
[0101] According to the categories of each government data determined above, data volume statistics are performed on each data category of government data to determine the data volume ratio corresponding to each data category, wherein the data volume ratio is calculated for the multi-level data categories to which the same government data belongs, that is, a government data may belong to multiple data categories at the same time; it is judged whether the data volume ratio of each data category counted is balanced, that is, it is determined whether there are some data categories in the government data that account for a very small amount of data or even no data of some data categories. If the data volume ratios are balanced, the data volume of the current government enhanced data tends to be balanced, and there is no need to balance the data volume through data enhancement; if the data volume ratios are unbalanced, it means that the data volume of some data categories is very small, which may affect the result of data classification. Therefore, the data category with a smaller data volume ratio in each data category is taken as the category to be enhanced, and the government data is enhanced based on the category to be enhanced, so as to expand the data in the data type, and at the same time, it can be ensured that the expanded data meets the characteristics of government data; at the same time, after data enhancement, the data volume ratio of each data corresponding to the data category in the data collection of the original government data and the enhanced government enhanced data is balanced.
[0102] This embodiment uses a generative model to enhance the government data. While compensating for the imbalance in data categories of the original government data, it ensures that the enhanced data conforms to the characteristics of the government data. As the input of the classification model, it can improve the accuracy of the data classification model.
[0103] Further, see Figure 3 The third embodiment of the present invention provides a data classification method based on the above Figure 3 In the embodiment shown, the step of extracting features from each word vector based on the hierarchical structure in the classification label system to obtain significant features at each level includes:
[0104] Step S31, inputting each of the word vectors into a preset pre-trained language model to obtain semantic information corresponding to each word vector;
[0105] Step S32, obtaining a hierarchical structure existing in the classification label system, and determining a plurality of levels existing in the hierarchical structure;
[0106] Step S33, extracting features from each of the semantic information through a convolutional neural network to obtain significant features at each level.
[0107] In this embodiment, it should be noted that the preset pre-trained language model is usually a BERT structure. BERT is composed of multiple Transformer Encoders stacked up layer by layer, and finally a BERT structure output is obtained. This structure can capture the contextual information and semantics of the input text so as to analyze the text from different angles. The BERT structure uses a transformer network to establish a bidirectional pre-training model, which captures semantic information by learning contextual information in two directions. At the same time, this application introduces a multi-task learning method, in which multiple tasks are learned in parallel, but the results affect each other.
[0108] Exemplarily, steps S31 to S33 include:
[0109] The word vectors of each word segment in the same government data are input into the BERT structure for text analysis, so as to obtain the semantic information corresponding to the government data; at the same time, the hierarchical structure in the classification label system is determined, and the number of levels in the hierarchy is determined from the hierarchy; the semantic information is extracted at multiple levels through CNN (convolutional neural network) to obtain the significant features of each level. For example, if there are three levels in the current hierarchy, the semantic information needs to be extracted at three levels respectively, and the significant features obtained at each level are different and the depth is different.
[0110] Furthermore, the step of extracting features from each of the semantic information by using a convolutional neural network to obtain significant features at each level includes:
[0111] Step S331, inputting each of the semantic information into convolutional neural networks at different levels, performing feature extraction at different levels on the semantic information, and obtaining semantic features corresponding to each level;
[0112] Step S332: pooling the semantic features of each level, and extracting the salient features of each level from the semantic features.
[0113] Exemplarily, step S331 to step S332 include:
[0114] The semantic information is input into the convolutional neural networks corresponding to different levels, and the features of the semantic information are extracted at different levels, so as to obtain the semantic information corresponding to each level. That is, the classifications at different levels are regarded as independent classification tasks, and the features are extracted separately; the semantic features corresponding to each level are obtained; the semantic features are pooled to obtain the significant features of each level.
[0115] Among them, the convolution kernel widths of convolution neural networks at different levels are different, wherein the lower the level of the convolution neural network, the wider the convolution kernel of the convolution neural network, and the coarser the granularity of the semantic features extracted at the corresponding level. Specifically, the convolution kernel of the low level is wider, and the granularity of the semantic features corresponding to the low level is coarser, while the convolution kernel of the high level is narrower, and the granularity of the semantic features of the high level is finer; a wider convolution kernel is used to extract features for the first level, and a narrower convolution kernel is used to extract features for the deep level. The reason is that the classification categories of the first level are usually macroscopic, and the use of a wider convolution kernel can roughly scan the feature matrix to obtain a coarse-grained feature representation, while the classification of the deep level is usually microscopic, and the use of a narrower convolution kernel can carefully scan the feature matrix to obtain a more refined feature representation.
[0116] In this embodiment, it should be noted that in the commonly used hierarchical multi-label classification method, a classifier can be trained separately for each category or each level, and each classifier is responsible for predicting the corresponding category or the corresponding category level, and predicting the final subclass based on the prediction result of the parent class. Because there are too many classifiers involved, the number of parameters of the model will be huge, so this method has the defect of huge computational consumption. In addition, if the category prediction of the parent class is wrong in this method, it will directly lead to downstream classification initialization errors, and there is a risk of propagation. However, the present application can directly reduce the number of classifiers and thus reduce computational consumption by setting different convolution kernel widths for the classifiers, thereby controlling the feature refinement of the outputs of different levels.
[0117] This embodiment classifies different levels separately. Compared with the method of training classifiers for each category or each level separately, it can reduce the data of the classifier and the parameters of the model as much as possible, and further reduce the computing consumption.
[0118] This invention adopts the method of combining "generative model / large model for domain data enhancement + multi-task learning framework" to achieve the classification of government data. The technical solution is divided into two major processes for point-by-point explanation. The two major processes correspond to Figure 4 and Figure 5 The flowcharts in the figure are: from "building government data set" to "BERT structure output", "BERT structure output" to "output hierarchical classification results".
[0119] In addition, the present invention also provides a data classification device, please refer to Figure 6 , Figure 6 This is a module diagram of a data classification device involved in an embodiment of the present invention, wherein the device comprises:
[0120] The data module 10 is used to obtain government data and a classification label system corresponding to the government data, and to construct a government data set according to the government data and the classification label system;
[0121] The vector module 20 is used to segment each data in the government data set and convert each segmented word into a word vector;
[0122] A feature module 30, configured to extract features from each of the word vectors based on the hierarchical structure in the classification label system to obtain significant features at each level;
[0123] The classification module 40 is used to combine the significant features of each level with part of the significant features of the previous level to obtain target features of each level, and use the target features of each level as the classification results of each level.
[0124] In one embodiment, the data module 10 is further used for:
[0125] Identify preset category information in the government data according to the preset category field in the classification label system;
[0126] Classifying the government data based on the preset category information to determine the data category of each piece of government data;
[0127] Performing data enhancement on each of the government affairs data according to the data category corresponding to each of the government affairs data to obtain government affairs enhanced data;
[0128] A government affairs data set is constructed based on the government affairs data and the government affairs enhanced data.
[0129] In one embodiment, the data module 10 is further used for:
[0130] Conducting data volume statistics on the government data of each data category to determine the data volume proportion of each data category;
[0131] Determine whether the data volume ratio of each data category is balanced;
[0132] If balanced, all data categories are taken as categories to be enhanced;
[0133] If it is unbalanced, the data category with a relatively small proportion of data in each data category will be taken as the category to be enhanced;
[0134] Data enhancement is performed on each of the government data according to the category to be enhanced to obtain government data enhancement data, wherein in the data collection of the government data enhancement data and the government data, the proportion of the data volume of each data corresponding to the data category is balanced.
[0135] In one embodiment, the data module 10 is further used for:
[0136] Determine a first category corresponding to the preset category information;
[0137] Acquire the hierarchical relationship of the preset category information according to the classification label system, and determine whether the preset category information has a second category at a lower level according to the hierarchical relationship;
[0138] If so, the first category and the second category are used as data categories of the government data;
[0139] If it does not exist, the first category will be used as the data category of the government data.
[0140] In one embodiment, the feature module 30 is further configured to:
[0141] Input each of the word vectors into a preset pre-trained language model to obtain semantic information corresponding to each word vector;
[0142] Acquire a hierarchical structure existing in the classification label system, and determine a plurality of levels existing in the hierarchical structure;
[0143] The features of each semantic information are extracted through a convolutional neural network to obtain the significant features of each level.
[0144] In one embodiment, the feature module 30 is further configured to:
[0145] Inputting each of the semantic information into convolutional neural networks at different levels, performing feature extraction at different levels on the semantic information, and obtaining semantic features corresponding to each level;
[0146] The semantic features of each level are pooled, and the salient features of each level are extracted from the semantic features.
[0147] In addition, the present invention also provides a data classification device, which includes a memory, a processor, and a data classification program stored in the memory and executable on the processor, wherein the data classification program implements the steps of the above-mentioned data classification method when executed by the processor.
[0148] Please refer to Figure 7 , Figure 7 It is a structural diagram of a data classification device in a hardware operating environment involved in an embodiment of the present invention.
[0149] like Figure 7 As shown, the data classification device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0150] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the data classification device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0151] like Figure 7 As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module and a data classification program.
[0152] exist Figure 7In the data classification device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the data classification device of the present invention can be set in the data classification device, and the data classification device calls the data classification program stored in the memory 1005 through the processor 1001, and executes the data classification method provided by the embodiment of the present invention.
[0153] In addition, an embodiment of the present application also proposes a storage medium, which is a computer-readable storage medium. The computer-readable storage medium can be a non-volatile computer-readable storage medium. A data classification program is stored on the computer-readable storage medium. When the data classification program is executed by a processor, the data classification method of the present application as described above is implemented.
[0154] The various embodiments of the electronic device and computer-readable storage medium of the present application can all refer to the various embodiments of the data classification method of the present application, which will not be repeated here.
[0155] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or system including the element.
[0156] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0158] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A data classification method, characterized in that: The data classification method comprises the following steps: Acquire government data and a classification label system corresponding to the government data, and construct a government data set based on the government data and the classification label system; Segmenting each data in the government data set, and converting each segmented word into a word vector; Based on the hierarchical structure in the classification label system, feature extraction is performed on each of the word vectors to obtain significant features of each level; The salient features of each level are respectively concatenated with some salient features of the previous level to obtain target features of each level, and the target features of each level are used as the classification results of each level; The step of extracting features from each word vector based on the hierarchical structure in the classification label system to obtain significant features at each level includes: Input each of the word vectors into a preset pre-trained language model to obtain semantic information corresponding to each word vector; Acquire a hierarchical structure existing in the classification label system, and determine a plurality of levels existing in the hierarchical structure; Extracting features of the semantic information through a convolutional neural network to obtain significant features of each level; The step of extracting features from each of the semantic information by a convolutional neural network to obtain significant features at each level includes: Inputting each of the semantic information into convolutional neural networks of different levels, performing feature extraction of different levels on the semantic information, and obtaining semantic features corresponding to each level, wherein the convolution kernel widths of the convolutional neural networks of different levels are different, wherein the lower the level of the convolutional neural network, the wider the convolution kernel of the convolutional neural network, and the coarser the granularity of the semantic features extracted at the corresponding level; The semantic features of each level are pooled, and the salient features of each level are extracted from the semantic features.
2. The data classification method according to claim 1, characterized in that: The step of constructing a government affairs data set according to the government affairs data and the classification label system comprises: Identify preset category information in the government data according to the preset category field in the classification label system; Classifying the government data based on the preset category information to determine the data category of each piece of government data; Performing data enhancement on each of the government affairs data according to the data category corresponding to each of the government affairs data to obtain government affairs enhanced data; A government affairs data set is constructed based on the government affairs data and the government affairs enhanced data.
3. The data classification method according to claim 2, characterized in that: The step of performing data enhancement on each of the government affairs data according to the data category corresponding to each of the government affairs data to obtain enhanced government affairs data comprises: Conducting data volume statistics on the government data of each data category to determine the data volume proportion of each data category; Determine whether the data volume ratio of each data category is balanced; If balanced, all data categories are taken as categories to be enhanced; If it is unbalanced, the data category with a relatively small proportion of data in each data category will be taken as the category to be enhanced; Data enhancement is performed on each of the government data according to the category to be enhanced to obtain government data enhancement data, wherein in the data collection of the government data enhancement data and the government data, the proportion of the data volume of each data corresponding to the data category is balanced.
4. The data classification method according to claim 2, characterized in that: The step of classifying the government data based on the preset category information and determining the data category of each piece of government data comprises: Determine a first category corresponding to the preset category information; Acquire the hierarchical relationship of the preset category information according to the classification label system, and determine whether the preset category information has a second category at a lower level according to the hierarchical relationship; If so, the first category and the second category are used as data categories of the government data; If it does not exist, the first category will be used as the data category of the government data.
5. A data classification device, characterized in that: The data classification device comprises: A data module, used to obtain government data and a classification label system corresponding to the government data, and to construct a government data set based on the government data and the classification label system; A vector module, used for segmenting each data in the government data set and converting each segmented word into a word vector; A feature module, used to extract features from each word vector based on the hierarchical structure in the classification label system to obtain significant features at each level; A classification module is used to combine the significant features of each level with some significant features of the previous level to obtain target features of each level, and use the target features of each level as the classification results of each level; Wherein, the feature module is specifically used for: Input each of the word vectors into a preset pre-trained language model to obtain semantic information corresponding to each word vector; Acquire a hierarchical structure existing in the classification label system, and determine a plurality of levels existing in the hierarchical structure; Inputting each of the semantic information into convolutional neural networks of different levels, performing feature extraction of different levels on the semantic information, and obtaining semantic features corresponding to each level, wherein the convolution kernel widths of the convolutional neural networks of different levels are different, wherein the lower the level of the convolutional neural network, the wider the convolution kernel of the convolutional neural network, and the coarser the granularity of the semantic features extracted at the corresponding level; The semantic features of each level are pooled, and the salient features of each level are extracted from the semantic features.
6. A data classification device, characterized in that: The device comprises: a memory, a processor, and a data classification program stored in the memory and executable on the processor, wherein the data classification program is configured to implement the steps of the data classification method according to any one of claims 1 to 4.
7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a data classification program is stored. When the data classification program is executed by a processor, the steps of the data classification method according to any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Multi-level Chinese text classification method and system
CN112966109A
Construction method of government purchase item hierarchical classification model
CN113946678A
Method and equipment for improving multi-level patent text classification effect and storage medium
CN115238076A
Text classification method and device, text classification model training method and device and storage medium
CN115599921A
Text detection model training method and apparatus, text region determination method and apparatus, and text content determination method and apparatus
WO2020221298A1
