A data classification method, device, electronic equipment and readable storage medium

Abstract

The application discloses a data classification method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: obtaining structured data to be classified; extracting a plurality of statistical features corresponding to a plurality of types of data in the structured data to be classified; inputting the statistical features into a classification model to obtain the data types corresponding to the plurality of types of data, wherein the classification model comprises a feature extraction network and a classification network, the feature extraction network is used for feature extraction of the input data to obtain neural network features, and the classification network is used for classification of the neural network features to obtain the data types. The data types classified by the method are more accurate, the range of the structured data to be classified is wider, and the classification efficiency is higher.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data classification method, a data classification device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the advent of the big data era, the global data volume is accelerating, doubling every two years, and will exceed 40 ZB (equivalent to 4 trillion GB) by 2021. This massive increase in data has led to numerous data breaches, endangering the security of individuals, businesses, governments, and nations. Therefore, data governance and control are imperative. Data classification and grading, as the core of data governance and data security, can comprehensively and clearly clarify data assets, achieve standardized management of data assets, and facilitate data maintenance and expansion, offering significant practical value to enterprises. Currently, data classification and grading mainly rely on manual methods or classification using regular expressions and keywords. However, due to the large volume of data, numerous data types, and the continuous updating and iteration of data, current data classification methods have poor accuracy, limited applicability to various data types, and low classification efficiency. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a data classification method, a data classification device, an electronic device, and a computer-readable storage medium to improve the accuracy of data classification.

[0004] To address the aforementioned technical problems, this application provides a data classification method, including:

[0005] Obtain the structured data to be classified;

[0006] Extract several statistical features corresponding to several classes of data in the structured data to be classified;

[0007] The statistical features are used to construct input data, which is then input into a classification model to obtain the data types corresponding to the various data categories.

[0008] The classification model includes a feature extraction network and a classification network. The feature extraction network is used to extract features from the input data to obtain neural network features. The classification network is used to classify the data type using the neural network features.

[0009] Optionally, the process of generating the classification model includes:

[0010] Obtain training structured data;

[0011] Extract several training statistical features corresponding to several classes of training data from the training structured data;

[0012] The training statistical features are used to construct training input data, which is then input into the initial model to obtain the training output result.

[0013] The loss value is calculated using the training output and the label data corresponding to the training structured data, and the parameters of the initial model are adjusted using the loss value.

[0014] If the training completion condition is met, the classification model is obtained using the initial model.

[0015] Optionally, calculating the loss value using the training output and the label data corresponding to the training structured data includes:

[0016] Obtain the interval value, and use the interval value, the training output result, and the training structured data to obtain the intermediate interval data;

[0017] The loss value is generated using the intermediate interval data.

[0018] Optionally, the initial model includes the feature extraction network, and obtaining the classification model using the initial model includes:

[0019] The initial model is combined with the classification network to obtain the classification model.

[0020] Optionally, the step of extracting several statistical features corresponding to several classes of data in the structured data to be classified includes:

[0021] Invalid data is filtered out from the aforementioned data categories to obtain several initial data categories.

[0022] Statistical features are extracted from the several types of initial data respectively to obtain the statistical features.

[0023] Optionally, the step of extracting statistical features from the several types of initial data to obtain the statistical features includes:

[0024] Data sampling is performed on the aforementioned initial data to obtain several types of sampled data;

[0025] Obtain the statistical features corresponding to the several types of sampled data respectively.

[0026] Optionally, the feature extraction network includes a first processing module, a second processing module, and a third processing module. The first processing module and the second processing module include a fully connected layer, an activation function layer, a random dropout layer, and a batch normalization layer. The third processing module includes the fully connected layer, the activation function layer, and the random dropout layer.

[0027] The statistical features are processed by a batch normalization layer and then input into the first processing module to obtain the first intermediate features. The first intermediate features are then input into the second processing module to obtain the second intermediate features. The second intermediate features are then input into the third processing module to obtain the neural network features.

[0028] This application also provides a data classification apparatus, including:

[0029] The acquisition module is used to acquire structured data to be classified.

[0030] The feature extraction module is used to extract several statistical features corresponding to several classes of data in the structured data to be classified.

[0031] The classification module is used to construct input data using the statistical features, and input the input data into the classification model to obtain the data types corresponding to the several classes of data.

[0032] The classification model includes a feature extraction network and a classification network. The feature extraction network is used to extract features from the input data to obtain neural network features. The classification network is used to classify the data type using the neural network features.

[0033] This application also provides an electronic device, including a memory and a processor, wherein:

[0034] The memory is used to store computer programs;

[0035] The processor is used to execute the computer program to implement the data classification method described above.

[0036] This application also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described data classification method.

[0037] The data classification method provided in this application obtains structured data to be classified; extracts several statistical features corresponding to several classes of data in the structured data to be classified; uses the statistical features to construct input data, and inputs the input data into a classification model to obtain the data types corresponding to the several classes of data; wherein, the classification model includes a feature extraction network and a classification network, the feature extraction network is used to extract features from the input data to obtain neural network features; the classification network is used to classify using the neural network features to obtain the data types.

[0038] As can be seen, this method extracts the statistical features corresponding to each type of data in the structural data to be classified as input data. Statistical features are one of the basic concepts in statistics. Features are indicators to be examined. Statistical features are divided into quantitative features and attribute features. Quantitative features are further divided into measurement features and count features. Quantitative features can be directly represented by numerical values, while attribute features cannot. By extracting statistical features, the characteristics of the data are reflected from multiple perspectives. After training, the classification model can use a feature extraction network to further extract features from the input data, obtaining neural network features that better represent the characteristics of the data and are easier to classify. After obtaining the neural network features, the classification network is used to process them to obtain the data types corresponding to each type of data in the structural data to be classified. The data types classified using this method are more accurate, the applicable range of structural data to be classified is wider, and the classification efficiency is higher.

[0039] In addition, this application also provides a data classification device, an electronic device, and a computer-readable storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0041] Figure 1 A flowchart of a data classification method provided in an embodiment of this application;

[0042] Figure 2 A flowchart illustrating a specific classification model generation process provided in this application embodiment;

[0043] Figure 3 A schematic diagram of the structure of a specific feature extraction network provided in an embodiment of this application;

[0044] Figure 4 A flowchart illustrating a specific data classification process provided in this application embodiment;

[0045] Figure 5 This is a schematic diagram of the structure of a data classification device provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0048] Please refer to Figure 1 , Figure 1 A flowchart illustrating a data classification method provided in this application embodiment. The method includes:

[0049] S101: Obtain the structured data to be classified.

[0050] Structured data refers to data logically expressed and implemented using a two-dimensional table structure. Each data item strictly adheres to data format and length specifications and is primarily stored and managed through relational databases, such as Excel spreadsheets. Structured data contains at least one data type, and typically multiple data types, each with a different data representation format, such as numbers, text, symbols, or a combination of these. Each data type has unique characteristics related to its category, such as length characteristics, format characteristics, and legal range characteristics. By accurately identifying the characteristics of a particular data type, its specific data type can be determined.

[0051] Structured data to be classified refers to structured data that has data content but lacks data type identifiers. It needs to be processed by a classification model to identify the characteristics of each data type and accurately assign data type labels to them. The structured data to be classified includes at least one type of data; for example, the data can be divided into types based on data columns. That is, data in the same column are identified as belonging to the same data type. It is important to note that although it is possible to determine which data in the structured data to be classified belong to the same type, the specific data type of these data cannot be determined before they are processed by the classification model.

[0052] The content and domain of the structured data to be classified can be selected from various options. For example, it can be education data, financial data, health data, or urban construction and housing data. It should be noted that the content and domain of the structured data to be classified should correspond to the classification model. That is, the classification model is obtained using training data with the same type of content and domain. Only under this condition can the accuracy of data classification be guaranteed.

[0053] S102: Extract several statistical features corresponding to several classes of data in the structured data to be classified.

[0054] Statistical characteristics are one of the fundamental concepts in statistics. When using mathematical statistics to study a population, the focus is not actually on the individual components of the population, but rather on examining one (or more) characteristics associated with them. The study of the distribution of a characteristic among the individuals in the population is called the statistical characteristic of the population. Statistical characteristics are divided into quantitative characteristics and attributive characteristics. Quantitative characteristics are further divided into measurement characteristics and count characteristics. Quantitative characteristics can be directly expressed numerically; for example, the size of a component or the height of a wheat plant are measurement characteristics. The number of summer rainstorms or the number of defects in a square meter of fabric are count characteristics. Attributive characteristics cannot be directly expressed numerically, such as whether a product is qualified or not, or the gender of an individual. These characteristics are the indicators to be examined.

[0055] It is understandable that different types of data may exhibit different characteristics in a given statistical feature. By summarizing and learning the patterns among these differences, we can accurately distinguish the data types of various categories. This embodiment does not limit the number and content of statistical features. For example, statistical features may include the percentage of numeric characters, the percentage of uppercase letters, the percentage of lowercase letters, the percentage of Chinese characters, the percentage of special characters (e.g., underscores, curly braces, periods, semicolons, etc.), the percentage of other characters, the percentage of cells with lengths of 0, 1, 2, 3-4, 5-9, 10-24, and 24 or longer, the percentage of cells containing special characters, the percentage of cells with lengths of 1, 2, 3, ..., 20, and 20 or longer containing only numeric characters, and the percentage of cells containing fixed Chinese characters (e.g., province, city, high efficiency, engineering, etc.). Each cell can be considered to contain one piece of data.

[0056] By statistically analyzing several statistical features corresponding to each category of data in the structured data to be classified, we can understand each category of data from several different perspectives, and then classify each category of data according to its characteristics.

[0057] S103: Use statistical features to construct input data, and input the input data into the classification model to obtain the data types corresponding to several classes of data.

[0058] The classification model includes a feature extraction network and a classification network. The feature extraction network is used to extract features from the input data to obtain neural network features; the classification network is used to classify the data using the neural network features to obtain the data type.

[0059] After obtaining statistical features, these can be used as input data for that data type. However, the input data still cannot fully and accurately reflect the characteristics of that data type. The classification model incorporates a feature extraction network, which extracts features from the input data, further extracting higher-level semantic features that aid in classification, discarding parts that might interfere with the classification, resulting in neural network features. These neural network features are input into the classification network of the classification model, which then outputs the data type corresponding to those input features. If the structured data to be classified contains multiple data types, the output data types will also be multiple, corresponding one-to-one with each data type.

[0060] The data classification method provided in this application extracts statistical features corresponding to each type of data in the structural data to be classified as input data. Statistical features are one of the basic concepts in statistics. Features are indicators to be examined. Statistical features are divided into quantitative features and attribute features. Quantitative features are further divided into measurement features and count features. Quantitative features can be directly represented by numerical values, while attribute features cannot. By extracting statistical features, the characteristics of the data are reflected from multiple perspectives. After training, the classification model can further extract features from the input data using a feature extraction network to obtain neural network features that better represent the characteristics of the data and are easier to classify. After obtaining the neural network features, the classification network is used to process them to obtain the data types corresponding to each type of data in the structural data to be classified. The data types classified using this method are more accurate, the applicable range of structural data to be classified is wider, and the classification efficiency is higher.

[0061] Based on the above embodiments, this embodiment will specifically describe several steps in the above embodiments. It is understood that the classification network needs to be generated before classification can be performed using the classification network.

[0062] In one implementation, the process of generating the classification network includes:

[0063] Step 11: Obtain training structured data.

[0064] Step 12: Extract several training statistical features corresponding to several classes of training data in the training structured data.

[0065] Step 13: Use training statistical features to construct training input data, and input the training input data into the initial model to obtain the training output results.

[0066] Step 14: Calculate the loss value using the training output and the label data corresponding to the training structured data, and use the loss value to adjust the parameters of the initial model.

[0067] Step 15: If the training completion condition is met, the classification model is obtained using the initial model.

[0068] Here, training structured data refers to structured data with corresponding labels used for model training, which contains several categories of training data. By extracting several training statistical features corresponding to each category of training data, the input data used for model training, i.e., the training input data, is formed. It should be noted that the training statistical features should be extracted using the same method as the statistical features described above.

[0069] An initial model refers to a model that has not yet completed training. Once the initial model or its training process meets the completion conditions, a classification model can be generated using the initial model. The model training process is the process of adjusting the model parameters of the initial model. Specifically, after the training input data is input into the initial model, the initial model processes the training input data using its current model parameters, and obtains the training output result after processing. It should be noted that the form of the training output result can vary depending on the specific structure of the initial model. For example, the initial model may include a feature extraction network and a classification network; in this case, the training output result is the training data type. In another implementation, the initial model may only include a feature extraction network; in this case, the training output result is the training neural network features.

[0070] After obtaining the training output, the loss value is calculated using the output and the corresponding labeled data. This loss value is then used to adjust the parameters of the initial model. It's understood that the labeled data and the training data have the same format.

[0071] Training completion conditions refer to the conditions under which a classification model can be generated using the initial model. Specifically, these conditions can be detection conditions for the initial model itself, or detection conditions for monitoring the training process of the initial model. For example, they could be a lower limit condition for the initial model's recognition accuracy, or a lower limit condition for the number of training epochs of the initial model.

[0072] Please refer to Figure 2 , Figure 2This document provides a flowchart of a specific classification model generation process according to an embodiment of this application. The training structured data is the labeled data, i.e., structured data with labels. Through data processing and statistical feature extraction, several statistical features are obtained, and these features are used to form vector-based input data, which is then fed into the initial model. After obtaining the training output of the initial model, a loss value is calculated using this output and the label data (e.g., label data can represent name, gender, mobile phone number, ID card number, etc.). This loss value is then used to adjust the parameters of the initial model.

[0073] In one specific implementation, the initial model only has a feature extraction network. In this case, the process of generating a classification model using it can include:

[0074] Step 21: Combine the initial model with the classification network to obtain the classification model.

[0075] After the initial model has been trained, it can be combined with a classification network to obtain a classification model. The specific structure of the classification network is not limited; for example, it can be a classification network composed of fully connected layers. In one implementation, to extract higher-level semantic features for more accurate identification of the training data type, feature extraction can be performed multiple times on the training input data. Specifically, the feature extraction network includes a first processing module, a second processing module, and a third processing module. The first and second processing modules include fully connected layers, activation function layers, random dropout layers, and batch normalization layers, while the third processing module includes fully connected layers, activation function layers, and random dropout layers.

[0076] After the statistical features are processed by the batch normalization layer, they are input into the first processing module to obtain the first intermediate features. The first intermediate features are input into the second processing module to obtain the second intermediate features. The second intermediate features are input into the third processing module to obtain the neural network features.

[0077] Please refer to Figure 3 , Figure 3This is a schematic diagram of a specific feature extraction network provided in an embodiment of this application. In this embodiment, there are 81 statistical features, which constitute 81-dimensional input data. After batch normalization, the data is input to the first processing module, processed through a fully connected layer to become 128-dimensional, and then processed successively through an activation function layer (i.e., a ReLU activation layer), a dropout layer, and a batch normalization layer to obtain the first intermediate feature. The first intermediate feature is input to the second processing module, processed through a fully connected layer to become 256-dimensional, and then processed successively through an activation function layer, a dropout layer, and a batch normalization layer to obtain the second intermediate feature. The second intermediate feature is input to the third processing module, processed through a fully connected layer to become 512-dimensional, and then processed successively through an activation function layer and a dropout layer before being output, resulting in 512-dimensional output data.

[0078] The network structure described above allows for more accurate feature extraction, representing data categories and resulting in a more accurate classification model. Please refer to Tables 1, 2, and 3. Tests were conducted using three types of data: education, finance and taxation, and urban construction and housing. The education data has 87 labels and 231 columns; the finance and taxation data has 74 labels and 2557 columns; and the urban construction and housing data has 68 labels and 1804 columns. Comparative tests of input statistical features and neural network features were performed on these three test databases, yielding the following results:

[0079]

[0080] Table 1. Results of Educational Data Testing

[0081]

[0082] Table 2 Results of the Financial and Tax Data Test

[0083]

[0084] Table 3. Test Results of Urban Construction and Housing Data

[0085] Therefore, the 512-dimensional neural network features obtained through high-level semantic feature extraction demonstrate higher accuracy in classification. AMI, FMI, ARI, NMI, MI, V-measure, and RI are used as evaluation metrics; higher values ​​indicate better performance. Specifically, AMI stands for Adjusted Mutual Information; FMI for Fowlkes-Mallows index; ARI for Adjusted Rand index; NMI for Normalized Mutual Information; MI for Mutual information; V-measure is the harmonic mean of h and c values, where h measures homogeneity and c measures completeness; and RI is the Rand index.

[0086] The method for calculating the loss value can be selected as needed. In one implementation, the softmax loss function can be chosen to calculate the loss value. The loss value calculated by the softmax loss function is:

[0087]

[0088] Where s represents the scaling factor, w represents the classifier, x is the initial training output of the network, and y is the true label (i.e., label data) corresponding to sample x. Therefore, w y Let w represent the feature vector corresponding to the y-th class of labeled data. If the initial network with the above structure is used to generate the training output, then w y Given a 512-dimensional vector, y takes values ​​corresponding to the number of data categories, K. Furthermore, θ... wy，x Indicates w y The angle between x and θ. For example, if the feature vectors corresponding to the label data are w_1 and w_2, and the true label corresponding to x belongs to the first class w_1, then the training objective is to make w_1x > w_2x, that is, to obtain x that is closer to w_1. w_1x is then cos(θ). w1，x ).

[0089] Furthermore, to make this objective more valid, when calculating the loss value, an intermediate value f(m, theta_1) is found, where w_1x > f(m, theta_1) > w_2x. If all intermediate values ​​satisfy the condition f(m, theta_1) > w_2x, then the original objective w_1x > w_2x is more likely to be satisfied, thus increasing the margin between different categories and making the extracted features after training more distinguishable. In other words, the loss value calculation process can include:

[0090] Step 31: Obtain the interval value, and use the interval value, training output results, and training structured data to obtain the intermediate interval data.

[0091] Step 32: Generate loss values ​​using intermediate interval data.

[0092] By introducing an interval value, the distinction between different data types can be increased, leading to better classification and improved classification accuracy. Specifically, continuing with the softmax loss function as an example, the calculated loss value is:

[0093]

[0094] Where, f(m, θ) wy，x =w_1x-m represents the intermediate interval data, where m is a pre-set interval value. The specific size of m is not limited; for example, m can be set to 0.35.

[0095] based on Figure 2 The flowchart illustrating the classification model generation process shows that data processing is required when generating input data. Therefore, similar data processing of the structured data to be classified is necessary during classification. Please refer to [link / reference]. Figure 4 , Figure 4 This application provides a flowchart of a specific data classification process. After obtaining structured data to be classified whose data type is not specified, several statistical features corresponding to several categories of data in the structured data to be classified are extracted, including:

[0096] Step 41: Filter out invalid data from several types of data to obtain several types of initial data.

[0097] Step 42: Extract statistical features from several types of initial data to obtain statistical features.

[0098] Invalid data refers to data that fails to reflect the characteristics of the data category, such as data marked 'none,' 'unknown,' or '--'. The content of invalid data can be pre-defined. After obtaining the structured data to be classified, invalid data in each category is filtered out to obtain the initial data for that category. By extracting statistical features from the initial data, the interference of invalid data on the accuracy of statistical features can be avoided, resulting in more accurate statistical features.

[0099] Furthermore, in a possible implementation, if the number of data items in one (or more) categories is too large, in order to improve the speed of statistical feature extraction, statistical feature extraction is performed on several categories of initial data separately. The process of obtaining statistical features may include:

[0100] Step 51: Sample data from several types of initial data to obtain several types of sampled data.

[0101] Step 52: Obtain the statistical features corresponding to several types of sampled data.

[0102] Data sampling can be used to select a portion of the initial data as sample data. This sampling method reduces the amount of data processed during statistical feature extraction, thus increasing the speed of feature extraction.

[0103] The data classification apparatus provided in the embodiments of this application is described below. The data classification apparatus described below and the data classification method described above can be referred to each other.

[0104] Please refer to Figure 5 , Figure 5 A schematic diagram of a data classification device provided in this application embodiment includes:

[0105] Module 110 is used to acquire structured data to be classified.

[0106] The feature extraction module 120 is used to extract several statistical features corresponding to several classes of data in the structured data to be classified.

[0107] The classification module 130 is used to construct input data using statistical features and input the input data into the classification model to obtain the data types corresponding to several classes of data.

[0108] The classification model includes a feature extraction network and a classification network. The feature extraction network is used to extract features from the input data to obtain neural network features; the classification network is used to classify the data using the neural network features to obtain the data type.

[0109] Optionally, it includes:

[0110] The training data acquisition module is used to acquire structured training data;

[0111] The training feature extraction module is used to extract several training statistical features corresponding to several classes of training data in the training structured data.

[0112] The training processing module is used to construct training input data using training statistical features, and then input the training input data into the initial model to obtain the training output results.

[0113] The parameter tuning module is used to calculate the loss value using the training output and the label data corresponding to the training structured data, and to tune the parameters of the initial model using the loss value.

[0114] The generation module is used to obtain a classification model from the initial model if the training completion condition is detected.

[0115] Optionally, the parameter adjustment module includes:

[0116] The intermediate interval calculation unit is used to obtain the interval value and use the interval value, training output results and training structured data to obtain the intermediate interval data;

[0117] The loss generation unit is used to generate loss values ​​using intermediate interval data.

[0118] Optionally, the generation module includes:

[0119] The combination generation unit is used to combine the initial model with the classification network to obtain a classification model.

[0120] Optionally, the feature extraction module 120 includes:

[0121] The invalid filtering unit is used to filter invalid data from several types of data to obtain several types of initial data.

[0122] The extraction unit is used to extract statistical features from several types of initial data to obtain statistical features.

[0123] Optionally, the extraction unit includes:

[0124] The sampling subunit is used to sample several types of initial data to obtain several types of sampled data.

[0125] Extract sub-units to obtain the statistical features corresponding to several types of sampled data.

[0126] Optionally, the feature extraction network includes a first processing module, a second processing module, and a third processing module. The first and second processing modules include fully connected layers, activation function layers, random dropout layers, and batch normalization layers. The third processing module includes fully connected layers, activation function layers, and random dropout layers.

[0127] After the statistical features are processed by the batch normalization layer, they are input into the first processing module to obtain the first intermediate features. The first intermediate features are input into the second processing module to obtain the second intermediate features. The second intermediate features are input into the third processing module to obtain the neural network features.

[0128] The electronic device provided in the embodiments of this application is described below. The electronic device described below can be referred to in correspondence with the data classification method described above.

[0129] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 101 and a memory 102, and may further include one or more of a multimedia component 103, an information input / output (I / O) interface 104, and a communication component 105.

[0130] The processor 101 controls the overall operation of the electronic device 100 to complete all or part of the steps in the data classification method described above. The memory 102 stores various types of data to support the operation of the electronic device 100. This data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data. The memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as one or more of Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0131] Multimedia component 103 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 102 or transmitted via communication component 105. The audio component also includes at least one speaker for outputting audio signals. I / O interface 104 provides an interface between processor 101 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 105 is used for wired or wireless communication between electronic device 100 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 105 may include a Wi-Fi component, a Bluetooth component, or an NFC component.

[0132] The electronic device 100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the data classification method given in the above embodiments.

[0133] The following describes the computer-readable storage medium provided in the embodiments of this application. The computer-readable storage medium described below can be referred to in correspondence with the data classification method described above.

[0134] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data classification method.

[0135] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0137] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0139] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0140] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data classification method, characterized in that, include: Obtain the structured data to be classified; Extract several statistical features corresponding to several classes of data in the structured data to be classified; the statistical features are the statistical features of the overall structured data to be classified, and the statistical features are used to study the distribution of relevant features among the individuals in the overall data; The statistical features are used to construct input data, which is then input into a classification model to obtain the data types corresponding to the various data categories. The classification model includes a feature extraction network and a classification network. The feature extraction network is used to extract features from the input data to obtain neural network features. The classification network is used to classify the data type using the neural network features. The feature extraction network extracts higher-level semantic features from the input data that are helpful for classification and discards parts that are likely to interfere with classification, so as to obtain neural network features.

2. The data classification method according to claim 1, characterized in that, The process of generating the classification model includes: Obtain training structured data; Extract several training statistical features corresponding to several classes of training data from the training structured data; The training statistical features are used to construct training input data, which is then input into the initial model to obtain the training output result. The loss value is calculated using the training output and the label data corresponding to the training structured data, and the parameters of the initial model are adjusted using the loss value. If the training completion condition is met, the classification model is obtained using the initial model.

3. The data classification method according to claim 2, characterized in that, The step of calculating the loss value using the training output and the label data corresponding to the training structured data includes: Obtain the interval value, and use the interval value, the training output result, and the training structured data to obtain the intermediate interval data; The loss value is generated using the intermediate interval data.

4. The data classification method according to claim 2, characterized in that, The initial model includes the feature extraction network, and obtaining the classification model using the initial model includes: The initial model is combined with the classification network to obtain the classification model.

5. The data classification method according to claim 1, characterized in that, The step of extracting several statistical features corresponding to several classes of data in the structured data to be classified includes: Invalid data is filtered out from the aforementioned data categories to obtain several initial data categories. Statistical features are extracted from the several types of initial data respectively to obtain the statistical features.

6. The data classification method according to claim 5, characterized in that, The step of extracting statistical features from the several types of initial data to obtain the statistical features includes: Data sampling is performed on the aforementioned initial data to obtain several types of sampled data; Obtain the statistical features corresponding to the several types of sampled data respectively.

7. The data classification method according to claim 1, characterized in that, The feature extraction network includes a first processing module, a second processing module, and a third processing module. The first and second processing modules include a fully connected layer, an activation function layer, a random dropout layer, and a batch normalization layer. The third processing module includes the fully connected layer, the activation function layer, and the random dropout layer. The statistical features are processed by a batch normalization layer and then input into the first processing module to obtain the first intermediate features. The first intermediate features are then input into the second processing module to obtain the second intermediate features. The second intermediate features are then input into the third processing module to obtain the neural network features.

8. A data classification device, characterized in that, include: The acquisition module is used to acquire structured data to be classified. The feature extraction module is used to extract several statistical features corresponding to several classes of data in the structured data to be classified; the statistical features are the statistical features of the overall structured data to be classified, and the statistical features are used to study the distribution of relevant features among the individuals in the overall data. The classification module is used to construct input data using the statistical features, and input the input data into the classification model to obtain the data types corresponding to the several classes of data. The classification model includes a feature extraction network and a classification network. The feature extraction network is used to extract features from the input data to obtain neural network features. The classification network is used to classify the data type using the neural network features. The feature extraction network extracts higher-level semantic features from the input data that are helpful for classification and discards parts that are likely to interfere with classification, so as to obtain neural network features.

9. An electronic device, characterized in that, Includes memory and processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program to implement the data classification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the data classification method as described in any one of claims 1 to 7.

Images