Data automatic classification grading marking method and device based on unsupervised deep learning and reinforcement learning

Abstract

The embodiment of the application provides a data automatic classification and grading marking method and device based on unsupervised deep learning and reinforcement learning, constructs a preliminary clustering result and a clustering center set through a variational autoencoder coding mapping and a contour coefficient adaptive cluster number determination, combines a deep reinforcement learning agent to execute policy evaluation and action instruction driven clustering center and classification threshold optimization according to a compound reward function, and obtains a classification and grading model through source domain pre-training parameter migration fine-tuning, associates and binds a class label and a grading label obtained after coding inference of to-be-marked data to form a marked data set, effectively solves the deficiencies of traditional technologies in unsupervised deep feature extraction, adaptive clustering modeling, reinforcement learning strategy optimization, cross-domain classification and grading model construction, and full-process automatic marking execution, and provides technical support for intelligent automatic classification and grading marking of enterprise data assets and fine data management.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a method and apparatus for automatic data classification, grading and labeling based on unsupervised deep learning and reinforcement learning. Background Technology

[0002] Existing methods for automatic data classification, grading, and labeling have significant shortcomings. Traditional systems perform poorly in raw data preprocessing and unsupervised latent feature extraction. They typically rely on manual rules or simple statistical methods for data cleaning and feature processing. They lack the end-to-end unsupervised deep feature extraction and adaptive clustering modeling capabilities to perform feature standardization and principal component dimensionality reduction on the cleaned data to obtain a preprocessed dataset, input it into a variational autoencoder to perform encoding mapping to obtain latent feature vectors, and automatically determine the target number of clusters based on the silhouette coefficient to perform clustering operations and obtain preliminary clustering results and a set of cluster centers. This results in insufficient quality of feature representation for data classification and the clustering number being determined manually based on experience, making it difficult to adapt to the needs of building an automated classification foundation in diverse data distribution scenarios.

[0003] Furthermore, existing technologies face bottlenecks in deep reinforcement learning-driven clustering strategy optimization and cross-domain classification and grading model construction. Most systems lack the ability to construct a state vector from the initial clustering results, latent feature vectors, and cluster center set as input to the deep reinforcement learning agent; to perform policy evaluation based on a composite reward function that integrates contour coefficients and validation accuracy, and output action commands to fine-tune the cluster center coordinates and classification threshold boundaries to obtain an optimized classification strategy; and to load the source domain pre-trained model parameters into the target domain classifier and perform parameter fine-tuning based on the optimized classification strategy to obtain a classification and grading model. This results in difficulties in achieving dynamic self-optimization of classification and grading strategies under unlabeled conditions, and low model reuse efficiency in cross-domain scenarios, affecting the generalization performance and construction efficiency of classification and grading models.

[0004] Existing systems have technical shortcomings in the classification inference of data to be labeled, sensitivity grading determination, and generation of labeled datasets. They lack a complete closed-loop automated labeling execution mechanism that involves encoding the data to be labeled using a variational autoencoder, inputting it into a classification and grading model to perform classification inference to obtain category labels, performing sensitivity determination based on the grading threshold boundaries in the optimized classification strategy to obtain grading labels, and associating and binding the category labels and grading labels into the data to be labeled to form a labeled dataset. This affects the labeling efficiency and label consistency assurance capabilities in the classification and grading management of large-scale data assets. Solving these problems is of great significance for improving the intelligence level of automated classification, grading, and labeling of enterprise data assets and enhancing their compliant data governance capabilities. Summary of the Invention

[0005] To address the problems in existing technologies, this application provides a method and apparatus for automatic data classification, grading, and labeling based on unsupervised deep learning and reinforcement learning. This method effectively solves the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning strategy optimization and cross-domain classification and grading model construction, as well as fully automated labeling execution. It provides technical support for the intelligent automatic classification, grading, and labeling of enterprise data assets and refined data governance.

[0006] To solve at least one of the above problems, this application provides the following technical solution: Firstly, this application provides a method for automatic data classification, grading, and labeling based on unsupervised deep learning and reinforcement learning, including: Data cleaning and missing value imputation are performed on the original data to obtain cleaned data. Feature standardization and principal component dimensionality reduction are performed on the cleaned data to obtain a preprocessed dataset. The preprocessed dataset is input into a variational autoencoder to perform encoding mapping to obtain a latent feature vector. The latent feature vector is used to traverse the candidate cluster number range based on the silhouette coefficient to determine the target cluster number and perform clustering operation to obtain preliminary clustering results and a set of cluster centers. The preliminary clustering results, the latent feature vectors, and the cluster center set are constructed as state vectors and input into a deep reinforcement learning agent. The deep reinforcement learning agent performs policy evaluation and outputs action instructions based on a composite reward function that integrates contour coefficients and validation accuracy. The action instructions are used to fine-tune the position coordinates of the cluster center set and the classification threshold boundary to obtain an optimized classification policy. The parameters of the source domain pre-trained model are loaded into the target domain classifier, and the parameters are fine-tuned according to the optimized classification policy to obtain a classification and grading model. The data to be labeled is encoded by the variational autoencoder and then input into the classification and grading model to perform classification inference to obtain category labels. The category labels are then used to determine the sensitivity based on the grading threshold boundary in the optimized classification strategy to obtain grading labels. The category labels and grading labels are then associated and bound and written into the data to be labeled to form a labeled dataset.

[0007] Furthermore, it also includes: performing duplicate record detection on the original data and deleting redundant records to obtain deduplicated data; performing missing value filling on the missing fields in the deduplicated data according to the field type by selecting the mean filling or mode filling strategy to obtain clean data; and performing truncation processing on the outliers in the clean data according to the preset deviation threshold condition to obtain normalized data. The numerical fields in the normalized data are standardized by performing a standardization transformation based on the mean and standard deviation to obtain a standardized feature matrix. The standardized feature matrix is ​​then input into the principal component analysis module to determine the number of dimensions to retain based on the cumulative variance contribution rate threshold condition, and dimensionality reduction projection is performed to obtain a preprocessed dataset.

[0008] Furthermore, it also includes: inputting the preprocessed dataset into the encoder network of the variational autoencoder to perform nonlinear mapping to obtain a mean vector and a variance vector; performing reparameterization sampling based on the mean vector and the variance vector to obtain a latent feature vector; inputting the latent feature vector into the decoder network to perform reconstruction and calculating a loss value based on the weighted sum of the reconstruction error and the divergence constraint term; and performing backpropagation based on the loss value to update the parameters of the encoder network and the decoder network until convergence. Clustering operations are performed on the potential feature vectors one by one within the range of candidate cluster numbers, and the corresponding silhouette coefficient values ​​are calculated. The cluster number corresponding to the maximum value among the silhouette coefficient values ​​is selected as the target cluster number. Clustering operations are performed on the potential feature vectors based on the target cluster number to obtain preliminary clustering results and a set of cluster centers.

[0009] Furthermore, it also includes: concatenating and encoding the category distribution features, statistical features of potential feature vectors, and position coordinates of cluster center sets in the preliminary clustering results to obtain a state vector, and inputting the state vector into the deep neural network of the deep reinforcement learning agent to perform forward propagation to obtain the value estimate corresponding to each candidate action; Based on the current clustering results, the silhouette coefficient is calculated and the classification accuracy is calculated based on the validation set. The silhouette coefficient and the classification accuracy are weighted and summed according to the preset weight coefficient to obtain the composite reward value. The composite reward value and the value estimate are updated according to the time difference rule. The candidate action corresponding to the maximum value is selected from the value estimate as the action instruction output.

[0010] Furthermore, it also includes: parsing the action instructions to obtain the cluster center offset and threshold adjustment amount, superimposing the cluster center offset onto the position coordinates of the cluster center set to obtain the updated cluster center, superimposing the threshold adjustment amount onto the classification threshold boundary to obtain the updated classification threshold, and encapsulating the updated cluster center and the updated classification threshold to form an optimized classification strategy; The encoder parameters and classifier parameters are extracted from the source domain pre-trained model and loaded into the corresponding network layer of the target domain classifier. The bottom encoder parameters of the target domain classifier are frozen and the top classifier parameters are opened. The top classifier parameters are updated according to the updated cluster centers and updated classification thresholds in the optimized classification strategy until convergence is obtained to obtain the classification and grading model.

[0011] Furthermore, it also includes: performing data cleaning and feature standardization on the data to be labeled to obtain standardized data to be labeled, and inputting the standardized data to be labeled into the encoder network of the variational autoencoder to perform nonlinear mapping to obtain the latent feature vector to be labeled; The potential feature vector to be labeled is input into the classification and grading model to calculate its distance value with each updated cluster center. The category to which the cluster center with the minimum value belongs is selected from the distance values ​​as the category label.

[0012] Furthermore, it also includes: reading the updated classification threshold from the optimized classification strategy and dividing it into multiple threshold intervals according to the sensitivity level; calculating the distance value between the potential feature vector to be labeled and the updated cluster center to which the corresponding category label belongs; and performing interval matching judgment between the distance value and the threshold interval to obtain the hierarchical label. The category label and the hierarchical label are constructed into a label pair, and an index association is established with the data to be labeled to obtain an associated binding record. The associated binding record is written into the label field of the data to be labeled to form a label dataset.

[0013] Secondly, this application provides a data automatic classification, grading, and labeling device based on unsupervised deep learning and reinforcement learning, comprising: The data clustering module is used to perform data cleaning and missing value imputation on the original data to obtain cleaned data, perform feature standardization and principal component dimensionality reduction on the cleaned data to obtain a preprocessed dataset, input the preprocessed dataset into a variational autoencoder to perform encoding mapping to obtain a latent feature vector, and traverse the candidate cluster number range based on the silhouette coefficient to determine the target cluster number and perform clustering operation to obtain preliminary clustering results and a set of cluster centers. The classification and grading module is used to construct the preliminary clustering results, the latent feature vectors, and the cluster center set into a state vector input to the deep reinforcement learning agent. The deep reinforcement learning agent performs policy evaluation and outputs action instructions based on a composite reward function that integrates contour coefficients and validation accuracy. The action instructions are used to fine-tune the position coordinates of the cluster center set and the classification threshold boundary to obtain an optimized classification policy. The parameters of the source domain pre-trained model are loaded into the target domain classifier, and the parameters are fine-tuned according to the optimized classification policy to obtain a classification and grading model. The data labeling module is used to encode the data to be labeled by the variational autoencoder and input it into the classification and grading model to perform classification inference to obtain category labels. The category labels are then used to perform sensitivity determination based on the grading threshold boundary in the optimized classification strategy to obtain grading labels. Finally, the category labels and grading labels are associated and bound and written into the data to be labeled to form a labeled dataset.

[0014] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the aforementioned automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the described automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning.

[0016] Fifthly, this application provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the aforementioned automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning.

[0017] As can be seen from the above technical solution, this application provides a method and apparatus for automatic data classification, grading and labeling based on unsupervised deep learning and reinforcement learning. It constructs preliminary clustering results and a set of cluster centers by using variational autoencoder encoding mapping and adaptive clustering number determination of silhouette coefficients. Combined with a deep reinforcement learning agent, it performs policy evaluation and action command-driven optimization of cluster centers and classification thresholds based on a composite reward function. Furthermore, it obtains a classification and grading model through source domain pre-training parameter transfer fine-tuning. The data to be labeled is encoded and inferred to obtain category and grading labels, which are then associated and bound to form a labeled dataset. This effectively solves the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning strategy optimization and cross-domain classification and grading model construction, and fully automated labeling execution. It provides technical support for intelligent automatic classification, grading and labeling of enterprise data assets and refined data governance. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning in the embodiments of this application. Figure 2 This is a structural diagram of the automatic data classification, grading, and labeling device based on unsupervised deep learning and reinforcement learning in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0022] In view of the problems existing in the prior art, this application provides a method and apparatus for automatic data classification, grading and labeling based on unsupervised deep learning and reinforcement learning. It constructs preliminary clustering results and a set of cluster centers by using variational autoencoder encoding mapping and adaptive clustering number determination of silhouette coefficients. Combined with a deep reinforcement learning agent, it performs policy evaluation and action command-driven optimization of cluster centers and classification thresholds based on a composite reward function. A classification and grading model is obtained through source domain pre-training parameter transfer fine-tuning. The data to be labeled is encoded and inferred to obtain category and grading labels, which are then associated and bound to form a labeled dataset. This effectively solves the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning policy optimization and cross-domain classification and grading model construction, and fully automated labeling execution. It provides technical support for intelligent automatic classification, grading and labeling of enterprise data assets and refined data governance.

[0023] To effectively address the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning strategy optimization and cross-domain classification and grading model construction, and fully automated labeling execution, and to provide technical support for intelligent automatic classification, grading, and labeling of enterprise data assets and refined data governance, this application provides an embodiment of an automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning. See [link to embodiment]. Figure 1 The automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning specifically includes the following: Step S101: Perform data cleaning and missing value imputation on the original data to obtain cleaned data. Perform feature standardization and principal component dimensionality reduction on the cleaned data to obtain a preprocessed dataset. Input the preprocessed dataset into a variational autoencoder to perform encoding mapping to obtain a latent feature vector. Use the latent feature vector to traverse the candidate cluster number range based on the silhouette coefficient to determine the target cluster number and perform clustering operation to obtain preliminary clustering results and cluster center set. This embodiment performs data cleaning and missing value imputation on the original data to obtain cleaned data. The original data comes from a multimodal record set exported from the business system, containing text fields, numeric fields, and category fields. The data cleaning process first performs duplicate record detection on the original data, identifying redundant records based on the hash values ​​of all fields and deleting them to obtain deduplicated data. Missing fields in the deduplicated data are then filled using a strategy selected according to their field type: numeric fields are filled using the mean of the same column, and category fields are filled using the mode. After filling, the cleaned data is obtained.

[0024] The cleaned data is used for feature standardization and principal component analysis (PCA) dimensionality reduction. Outliers in the cleaned data are truncated according to a preset deviation threshold, specifically three standard deviations as the boundary. Values ​​exceeding the boundary are truncated to the boundary value to obtain normalized data. The mean and standard deviation of each numerical field in the normalized data are calculated, and each field is mapped to a distribution with zero mean and unit variance using a standardization transformation to obtain a standardized feature matrix. The standardized feature matrix is ​​input into the PCA module, and the number of dimensions to retain is determined based on the cumulative variance contribution rate reaching a preset threshold. Dimensionality reduction projection is then performed to obtain a preprocessed dataset.

[0025] The preprocessed dataset serves as input to the variational autoencoder (VAC) for encoding mapping. The VAC comprises two substructures: an encoder network and a decoder network. The encoder network receives sample vectors from the preprocessed dataset and performs a nonlinear mapping to obtain a mean vector and a variance vector. Reparameterized sampling is then performed based on the mean and variance vectors, sampling noise vectors from a standard normal distribution and multiplying them by the variance vector before superimposing the mean vector to obtain a latent feature vector. This latent feature vector is then input into the decoder network for reconstruction to obtain a reconstructed vector. A loss value is calculated based on a weighted sum of the reconstruction error and divergence constraints. Backpropagation is then performed based on the loss value to update the parameters of both the encoder and decoder networks, iterating until the loss value converges.

[0026] The latent feature vector is used to determine the target number of clusters by traversing the candidate cluster number range based on the silhouette coefficient. The candidate cluster number range is preset to an integer range of two to fifteen. In this embodiment, the number of clusters is set one by one within the candidate cluster number range, and clustering operations are performed on the latent feature vector. After each clustering operation, the corresponding silhouette coefficient value is calculated. The silhouette coefficient value reflects the comprehensive quality of intra-cluster compactness and inter-cluster separation. For example, when the number of clusters is set to five, the clustering operation is performed to obtain the division result of five clusters. The average intra-cluster distance and the average nearest neighbor cluster distance of each sample are calculated. The silhouette coefficient of the sample is obtained based on the ratio of the difference between the two and the maximum value. The average of the silhouette coefficients of all samples is taken to obtain the silhouette coefficient value corresponding to the number of clusters.

[0027] Accordingly, the number of clusters corresponding to the maximum value among the silhouette coefficients is selected as the target number of clusters. Clustering operations are performed on the latent feature vectors based on the target number of clusters, assigning each sample to the cluster of the nearest cluster center to obtain preliminary clustering results. Simultaneously, the centroid coordinates of each cluster are recorded to obtain a set of cluster centers. The preliminary clustering results include the cluster label assignment records for each sample, and the set of cluster centers includes the position coordinates of the centroids of each cluster in the latent space. The preliminary clustering results, the latent feature vectors, and the set of cluster centers are read in subsequent step S102 to construct a state vector and input into the deep reinforcement learning agent for policy evaluation.

[0028] Step S102: The preliminary clustering results, the latent feature vectors, and the cluster center set are constructed into a state vector and input into the deep reinforcement learning agent. The deep reinforcement learning agent performs policy evaluation and outputs action instructions based on the composite reward function that fuses the contour coefficients and the validation accuracy. The action instructions are used to fine-tune the position coordinates of the cluster center set and the classification threshold boundary to obtain an optimized classification policy. The parameters of the source domain pre-trained model are loaded into the target domain classifier, and the parameters are fine-tuned according to the optimized classification policy to obtain a classification and grading model. In this embodiment, the preliminary clustering results, the latent feature vector, and the cluster center set are constructed into a state vector. The preliminary clustering results are output in step S101 and stored in the clustering result cache. In this embodiment, the cluster label allocation records of each sample are read from the cache, and the sample quantity distribution of each cluster is statistically analyzed to obtain the category distribution features. The mean and variance of the latent feature vector are calculated dimension by dimension to obtain statistical features. The position coordinates of the centroids of each cluster in the cluster center set are unfolded sequentially to obtain the center coordinate sequence. The category distribution features, the statistical features, and the center coordinate sequence are concatenated and encoded to form a one-dimensional numerical sequence to obtain the state vector.

[0029] The state vector serves as the input to the deep reinforcement learning agent for forward propagation. The deep reinforcement learning agent employs a deep neural network as a value function approximator, which includes an input layer, multiple hidden layers, and an output layer. The state vector is input to the input layer of the deep neural network, and after nonlinear transformation by the hidden layers, the value estimate corresponding to each candidate action is obtained at the output layer. The candidate action set is predefined and includes four basic actions and their combinations: positive and negative shifts of cluster centers along each coordinate axis, and adjustments to classification thresholds (up and down).

[0030] Based on this, a strategy evaluation is performed using a composite reward function that integrates the silhouette coefficient and the validation accuracy. In this embodiment, the silhouette coefficient is recalculated based on the current clustering results, and the calculation method is consistent with that described in step S101. Simultaneously, a validation set is partitioned from the preprocessed dataset, and the validation set samples are assigned cluster labels based on the current cluster centers and compared with reserved reference labels to obtain the classification accuracy. The silhouette coefficient and the classification accuracy are then weighted and summed according to preset weight coefficients to obtain a composite reward value. The weight coefficients are specified by configuration parameters, and the sum of the two is one.

[0031] The composite reward value and the value estimate are used to perform a value function update. In this embodiment, the target value is calculated based on the temporal difference rule. The target value is obtained by adding the current composite reward value to the next-state maximum value estimate weighted by the discount factor. The difference between the target value and the current value estimate is used as the temporal difference error, and the parameters of the deep neural network are updated based on the temporal difference error. The candidate action corresponding to the maximum value from the value estimates is selected as the action instruction output and written to the action instruction register.

[0032] The action command is used to fine-tune the position coordinates of the cluster center set and the classification threshold boundary. In this embodiment, the action command is parsed to obtain the cluster center offset and the threshold adjustment. The cluster center offset is the displacement increment of each cluster centroid along each coordinate axis, and the threshold adjustment is the numerical increment of the classification boundary. The cluster center offset is element-wise superimposed onto the position coordinates of the cluster center set to obtain the updated cluster centers, and the threshold adjustment is superimposed onto the classification threshold boundary to obtain the updated classification threshold. The updated cluster centers and the updated classification threshold are encapsulated to form an optimized classification strategy and written to the strategy storage area.

[0033] The optimized classification strategy is used to guide the fine-tuning of the parameters of the target domain classifier. In this embodiment, encoder parameters and classifier parameters are extracted from a source domain pre-trained model, which has been trained on large-scale data in a similar domain and possesses general feature extraction capabilities. The encoder parameters are loaded into the bottom-level encoding network of the target domain classifier, and the classifier parameters are loaded into the top-level classification network of the target domain classifier. The parameters of the bottom-level encoding network are frozen to remain unchanged during subsequent fine-tuning, while the parameters of the top-level classification network are made available for gradient updates.

[0034] After the aforementioned parameters are loaded, parameter fine-tuning is performed to obtain the classification and grading model. In this embodiment, a classification target is constructed based on the updated cluster centers in the optimized classification strategy, and the distance relationship between the latent feature vectors of the training samples and the updated cluster centers is converted into a classification supervision signal. Boundary conditions for each level are defined as grading constraints based on the updated classification threshold. Gradient updates are performed on the parameters of the top-level classification network, iteratively executed until the classification loss converges to obtain the classification and grading model. The classification and grading model and the optimized classification strategy are read in subsequent step S103 for performing classification inference and grading label determination on the data to be labeled.

[0035] Step S103: After encoding the data to be labeled by the variational autoencoder, input it into the classification and grading model to perform classification inference to obtain category labels. Perform sensitivity determination on the category labels according to the grading threshold boundary in the optimized classification strategy to obtain grading labels. Associate and bind the category labels with the grading labels and write them into the data to be labeled to form a labeling dataset.

[0036] This embodiment performs data cleaning and feature standardization on the data to be labeled to obtain standardized data to be labeled. The data to be labeled comes from the newly added set of records to be classified in the business system, and its data structure is consistent with the original data described in step S101. Duplicate record detection and missing value imputation are performed on the data to be labeled, and the processing method is the same as the cleaning process of the original data in step S101. The numerical fields in the cleaned data to be labeled are standardized according to the mean and standard deviation recorded in step S101 to obtain standardized data to be labeled.

[0037] The standardized data to be labeled is used as input to the variational autoencoder for encoding mapping. In this embodiment, each sample vector in the standardized data to be labeled is input into the encoder network of the variational autoencoder trained in step S101, and a nonlinear mapping is performed to obtain a mean vector and a variance vector. Based on the mean vector and the variance vector, reparameterization sampling is performed to obtain a latent feature vector to be labeled. The latent feature vector to be labeled and the latent feature vector in step S101 are in the same latent space.

[0038] The classification and grading model obtained in step S102, where the latent feature vector to be labeled is input, is used to perform classification inference. In this embodiment, the updated cluster centers in the optimized classification strategy are read from the strategy storage area, and the Euclidean distance between the latent feature vector to be labeled and each updated cluster center is calculated to obtain a set of distance values. Each distance value in the set is compared, and the cluster center corresponding to the minimum value is selected, with the category to which that cluster center belongs as the category label. For example, when the updated cluster centers include three categories: core data class, important data class, and general data class, if the distance value between a sample to be labeled and the important data class cluster center is the smallest, then the category label of that sample is determined to be the important data class.

[0039] The category labels are used to determine sensitivity based on the hierarchical threshold boundaries in the optimized classification strategy. In this embodiment, the updated classification threshold is read from the strategy storage area and divided into multiple threshold intervals according to the sensitivity level, with each threshold interval corresponding to a different hierarchical label. The distance between the latent feature vector to be labeled and the updated cluster center to which the corresponding category label belongs is calculated, and interval matching is performed between the distance value and the threshold interval. When the distance value falls within a certain threshold interval, the sensitivity level corresponding to that interval is used as the hierarchical label. The smaller the distance value, the closer the sample is to the cluster center, and the higher the corresponding sensitivity level.

[0040] The category labels and grading labels are used to perform association binding and write to the data to be labeled. In this embodiment, the category labels and grading labels are constructed as label pairs, each label pair containing two components: a category field and a grading field. An association binding record is obtained by establishing an index association between the label pair and each sample in the data to be labeled, the index association being matched based on the sample's unique identifier. The association binding record is written to the label field of the data to be labeled; the label field is predefined in the data structure and storage space is reserved. After writing is completed, a labeled dataset is formed, which includes the original data fields and the newly added category label field and grading label field, which can be used for subsequent data security management and differentiated protection measures.

[0041] As described above, the automatic data classification and labeling method based on unsupervised deep learning and reinforcement learning provided in this application can construct preliminary clustering results and a set of cluster centers by using variational autoencoder encoding mapping and adaptive clustering number determination of silhouette coefficients. Combined with deep reinforcement learning agents, the method optimizes cluster centers and classification thresholds by executing strategy evaluation and action instructions based on composite reward functions, and obtains a classification and grading model through source domain pre-training parameter transfer fine-tuning. After encoding and inferring the category label and grading label of the data to be labeled, the method is associated and bound to form a labeled dataset. This effectively solves the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning strategy optimization and cross-domain classification and grading model construction, and fully automated labeling execution. It provides technical support for intelligent automatic classification and grading labeling and refined data governance of enterprise data assets.

[0042] In one embodiment of the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning in this application, the method may further include the following: Step S201: Perform duplicate record detection on the original data and delete redundant records to obtain deduplicated data. Select the mean or mode filling strategy according to the field type to fill missing fields in the deduplicated data to obtain clean data. Perform truncation processing on the outliers in the clean data according to the preset deviation threshold to obtain normalized data. Step S202: Perform standardization transformation on the numerical fields in the normalized data according to the mean and standard deviation to obtain a standardized feature matrix. Input the standardized feature matrix into the principal component analysis module to determine the number of dimensions to retain according to the cumulative variance contribution rate threshold condition and perform dimensionality reduction projection to obtain a preprocessed dataset.

[0043] This embodiment performs duplicate record detection on the original data to identify and delete redundant records. The original data comes from a multimodal record set exported from the business system, containing text fields, numeric fields, and category fields. Each record is stored row by row in the data cache. During the detection process, all field values ​​of each record are concatenated and a hash value is calculated. The hash value is then written to a hash index table. If an entry with the same hash value already exists in the hash index table, the current record is determined to be redundant and deleted. If no entry with the same hash value exists, the current record is retained and its hash value is written to the hash index table. After detection and deletion, deduplicated data is obtained, and the field combinations of each record in the deduplicated data are unique.

[0044] After the deduplicated data is generated, this embodiment performs completion processing on the missing fields. Some records in the deduplicated data have empty values ​​in certain fields, requiring the selection of a corresponding completion strategy based on the field type. For numeric fields, all non-empty values ​​of the field are extracted from the deduplicated data, and the arithmetic mean is calculated and written into the empty value position of the field. For categorical fields, the frequency of each value in the field is counted, and the value with the highest frequency is selected as the mode, which is written into the empty value position of the field. After completing the completion for each field, clean data is obtained, in which each field has complete values ​​and there are no empty records.

[0045] Accordingly, this embodiment performs outlier truncation on the cleaned data. Outlier determination is based on a preset deviation threshold, specifically using three times the standard deviation of each numerical field as the deviation boundary. For each numerical field, the mean and standard deviation of all values ​​in the field are calculated. The mean minus three times the standard deviation is used as the lower truncation boundary, and the mean plus three times the standard deviation is used as the upper truncation boundary. Each value in the field is checked one by one. If a value is less than the lower truncation boundary, it is replaced with the lower truncation boundary value; if a value is greater than the upper truncation boundary, it is replaced with the upper truncation boundary value. After the truncation process is completed, normalized data is obtained, in which the values ​​of each numerical field fall within the preset boundary range.

[0046] After the normalized data is generated, this embodiment performs a standardization transformation on the numerical fields. The numerical fields in the normalized data have different dimensions and numerical ranges; directly using them in subsequent calculations will result in large numerical fields dominating the distance metric results. For each numerical field, the mean and standard deviation of all values ​​for that field are calculated and recorded in the parameter storage area. Each value of the field is subtracted from the mean and then divided by the standard deviation; after transformation, each value follows a distribution with zero mean and unit variance. After the transformation is completed field by field, the numerical fields are reorganized in their original column order to obtain a standardized feature matrix. The number of rows in the standardized feature matrix equals the number of samples, and the number of columns equals the number of numerical fields.

[0047] Based on the aforementioned standardization results, this embodiment inputs the standardized feature matrix into the principal component analysis module for dimensionality reduction projection. The principal component analysis module first calculates the covariance matrix of the standardized feature matrix, and performs eigenvalue decomposition on the covariance matrix to obtain an eigenvalue sequence and an eigenvector sequence. The eigenvalue sequence is arranged in descending order, and the proportion of each eigenvalue to the sum of all eigenvalues ​​is accumulated to form a cumulative variance contribution rate sequence. The cumulative variance contribution rate threshold is preset to 95%. The position index that first reaches or exceeds this threshold is found in the cumulative variance contribution rate sequence, and the number of eigenvectors corresponding to the position index is used as the number of retained dimensions. The first few eigenvectors are extracted from the eigenvector sequence to form a projection matrix. Matrix multiplication is performed between the standardized feature matrix and the projection matrix to obtain a preprocessed dataset. The number of columns in the preprocessed dataset is equal to the number of retained dimensions. The preprocessed dataset and the projection matrix are read in subsequent step S301 for encoding mapping and feature extraction by the variational autoencoder.

[0048] In one embodiment of the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning in this application, the method may further include the following: Step S301: Input the preprocessed dataset into the encoder network of the variational autoencoder and perform nonlinear mapping to obtain the mean vector and variance vector. Based on the mean vector and variance vector, perform reparameterization sampling to obtain the latent feature vector. Input the latent feature vector into the decoder network to perform reconstruction and calculate the loss value based on the weighted sum of the reconstruction error and divergence constraint term. Based on the loss value, perform backpropagation to update the parameters of the encoder network and decoder network until convergence. Step S302: Perform clustering operations on the potential feature vectors one by one within the range of candidate cluster numbers and calculate the corresponding silhouette coefficient values. Select the cluster number corresponding to the maximum value from the silhouette coefficient values ​​as the target cluster number. Perform clustering operations on the potential feature vectors according to the target cluster number to obtain preliminary clustering results and a set of cluster centers.

[0049] In this embodiment, a preprocessed dataset is input into the encoder network of a variational autoencoder to perform nonlinear mapping. The preprocessed dataset is output in step S202 and stored in a data buffer. In this embodiment, sample vectors are read from the buffer in batches as input to the encoder network. The encoder network contains multiple fully connected layers and activation function layers. The weight matrix and bias vector of each fully connected layer are initialized with initial values ​​according to a random initialization strategy before training. After the input sample vector undergoes a linear transformation in the first fully connected layer, nonlinear characteristics are introduced through the activation function layer, and then it undergoes transformation and activation operations in subsequent layers. The output layer of the encoder network is divided into two parallel branches, which output a mean vector and a variance vector, respectively. The mean vector and the variance vector have the same dimension and are both smaller than the dimension of the input sample vector.

[0050] After the mean vector and variance vector are generated, this embodiment performs a sampling operation based on the reparameterization technique. The reparameterization technique transforms the random sampling process into a combination of deterministic computation and noise injection, allowing the sampling operation to be penetrated by gradient backpropagation. Specifically, a noise vector with the same dimension as the mean vector is sampled from a standard normal distribution. The noise vector is then multiplied element-wise by the square root of the variance vector and added to the mean vector to obtain a latent feature vector. This latent feature vector resides in a low-dimensional latent space, retains the core semantic information of the input samples, and follows an approximately normal distribution.

[0051] Accordingly, in this embodiment, the latent feature vector is input into the decoder network to perform a reconstruction operation. The decoder network has a mirror-symmetric structure to the encoder network, containing multiple fully connected layers and activation function layers, with the dimensions of each layer increasing sequentially until the output layer dimension is the same as the original input dimension. After the latent feature vector is transformed layer by layer by the decoder network, a reconstructed vector is output, which is the estimated restoration result of the original input sample. The element-wise difference between the reconstructed vector and the original input sample is calculated, and the sum of squares is obtained to obtain the reconstruction error, which measures the accuracy of the decoder network in restoring the input information.

[0052] After the reconstruction error is calculated, this embodiment further calculates the divergence constraint term and sums it with the reconstruction error using weighted averages to obtain the loss value. The divergence constraint term uses divergence to measure the difference between the distribution defined by the mean vector and the variance vector and the standard normal distribution. Its calculation is based on the analytical formula, performing element-wise operations on the mean vector and the variance vector and then summing the results. The reconstruction error and the divergence constraint term are weighted and summed according to preset weight coefficients to obtain the loss value. These weight coefficients control the balance between reconstruction accuracy and distribution regularization. Backpropagation is performed on all parameters of the encoder network and decoder network based on the loss value. The gradient values ​​of each parameter are calculated, and the parameter values ​​are adjusted according to the optimizer update rules. The forward and backward propagation processes are repeated until the loss value converges to below a preset threshold. The converged encoder network and decoder network parameters are then stored.

[0053] Based on the encoder network trained as described above, this embodiment performs clustering operations on the latent feature vectors of all samples one by one within the candidate clustering range. The candidate clustering range is preset according to the business scenario, with a lower bound of two and an upper bound of ten. For each candidate clustering number within the range, this number is used as the cluster number parameter of the clustering algorithm to perform clustering operations on the latent feature vectors. The clustering operation adopts an iterative optimization method. First, several latent feature vectors are randomly selected as initial cluster centers, and then the sample allocation and center update steps are alternately executed until the cluster center positions are stable. After each clustering operation, the silhouette coefficient value is calculated based on the average distance between a sample and samples in the same cluster and the average distance between a sample and the nearest sample in a different cluster. The silhouette coefficient value ranges from -1 to +1, with a larger value indicating a better clustering effect.

[0054] After calculating all silhouette coefficient values ​​within the candidate clustering range, this embodiment selects the clustering number corresponding to the maximum value as the target clustering number. Each candidate clustering number and its corresponding silhouette coefficient value are paired and written into an evaluation result table. The evaluation result table is traversed to select the entry with the largest silhouette coefficient value, and the candidate clustering number of that entry is extracted as the target clustering number. Clustering operations are then performed again on the latent feature vector based on the target clustering number, following the same process as described above. After convergence, the cluster label assignment for each sample is output as the preliminary clustering result, and the centroid coordinates of each cluster are output as the cluster center set. The preliminary clustering result, the cluster center set, and the latent feature vector are then read in subsequent step S401 to construct the state vector of the deep reinforcement learning agent.

[0055] In one embodiment of the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning in this application, the method may further include the following: Step S401: Concatenate and encode the category distribution features, statistical features of potential feature vectors, and location coordinates of cluster center sets in the preliminary clustering results to obtain a state vector. Input the state vector into the deep neural network of the deep reinforcement learning agent to perform forward propagation to obtain the value estimate corresponding to each candidate action. Step S402: Calculate the silhouette coefficient based on the current clustering result and calculate the classification accuracy based on the validation set. Perform a weighted summation of the silhouette coefficient and the classification accuracy based on a preset weight coefficient to obtain a composite reward value. Update the composite reward value and the value estimate based on the time-series difference rule. Select the candidate action corresponding to the maximum value from the value estimate as the action instruction output.

[0056] This embodiment extracts and encodes the category distribution features from the preliminary clustering results into a vector form. The preliminary clustering results are output in step S302 and include cluster label assignment information for each sample. The extraction process of the category distribution features involves counting the number of samples contained in each cluster, dividing the number of samples in each cluster by the total number of samples to obtain the sample proportion of each cluster, and arranging the sample proportions in order of cluster number to form a category distribution vector. The dimension of the category distribution vector is equal to the number of target clusters, and the value of each element ranges from zero to one, with the sum of all elements being one.

[0057] After the category distribution features are extracted, this embodiment further extracts the statistical features of the latent feature vectors. The latent feature vectors are output in step S301, with each sample corresponding to a low-dimensional vector representation. The mean and standard deviation of the latent feature vectors for all samples are calculated column-wise along each dimension. The means of each dimension are arranged sequentially to form a mean statistical vector, and the standard deviations of each dimension are arranged sequentially to form a standard deviation statistical vector. The mean statistical vector and the standard deviation statistical vector are concatenated end-to-end to obtain the latent spatial statistical feature vector, which characterizes the central tendency and dispersion of the latent feature vectors across each dimension.

[0058] Accordingly, in this embodiment, the position coordinates of the cluster center set are flattened into vector form and concatenated with the aforementioned features. The cluster center set, output by step S302, includes the coordinate positions of each cluster centroid in the latent space. The coordinate vectors of each cluster center are concatenated sequentially according to the cluster number to obtain the cluster center position vector, the dimension of which is equal to the product of the target number of clusters and the dimension of the latent space. The category distribution vector, the latent space statistical feature vector, and the cluster center position vector are concatenated sequentially to obtain the state vector, which completely encodes all observable information of the current cluster state.

[0059] After the state vector is constructed, this embodiment performs forward propagation on the deep neural network input to the deep reinforcement learning agent. The deep neural network includes an input layer, multiple hidden layers, and an output layer. The dimension of the input layer is the same as the dimension of the state vector, and the dimension of the output layer is the same as the number of candidate actions. The candidate action set is predefined and includes operation types such as adjusting cluster center positions, merging adjacent clusters, splitting discrete clusters, and adjusting grading thresholds. After being received by the input layer, the state vector is sequentially passed through linear transformations and nonlinear activations of each hidden layer, and finally, the output layer outputs the value estimate corresponding to each candidate action. The value estimate represents the expected cumulative reward that can be obtained by performing the corresponding candidate action in the current state.

[0060] Based on the aforementioned value estimates, this embodiment calculates a composite reward value to evaluate the quality of the current clustering results. The silhouette coefficient is calculated based on the average distance between each sample in the current clustering result and its own cluster, as well as the average distance to the nearest dissimilar cluster. The average silhouette coefficient of all samples is then used to obtain the overall silhouette coefficient. The classification accuracy is calculated using a validation set, which includes samples with manually labeled categories. The cluster labels of the corresponding samples in the current clustering result are compared with the manually labeled labels, and the proportion of matching samples to the total number of samples in the validation set is used to obtain the classification accuracy. The silhouette coefficient and the classification accuracy are then weighted and summed according to preset weighting coefficients to obtain the composite reward value. These preset weighting coefficients are set according to business requirements, with a default value of 0.4 for the silhouette coefficient and 0.6 for the classification accuracy.

[0061] After the composite reward value is calculated, this embodiment updates the value function and outputs action instructions according to the temporal difference rule. The temporal difference rule adjusts the value estimate of the current state towards the sum of the composite reward value and the value estimate of the next state, with the adjustment magnitude controlled by the learning rate parameter. Specifically, the target value is calculated by adding a discount factor to the composite reward value and multiplying it by the maximum value estimate of the next state. The difference between the target value and the current value estimate is used as the temporal difference error, and the parameters of the deep neural network are updated based on the temporal difference error. The candidate action corresponding to the maximum value among the candidate action value estimates of the current state is selected as the action instruction and output. The action instruction is read in the subsequent step S501 for performing clustering adjustment and hierarchical strategy optimization operations.

[0062] In one embodiment of the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning in this application, the method may further include the following: Step S501: Parse the action command to obtain the cluster center offset and threshold adjustment amount, superimpose the cluster center offset onto the position coordinates of the cluster center set to obtain the updated cluster center, superimpose the threshold adjustment amount onto the classification threshold boundary to obtain the updated classification threshold, and encapsulate the updated cluster center and the updated classification threshold to form the optimized classification strategy. Step S502: Extract encoder parameters and classifier parameters from the source domain pre-trained model and load them into the corresponding network layer of the target domain classifier. Freeze the bottom encoder parameters of the target domain classifier and open the top classifier parameters. Perform gradient update on the top classifier parameters according to the updated cluster centers and updated classification thresholds in the optimized classification strategy until convergence to obtain the classification and grading model.

[0063] This embodiment parses action commands to extract cluster center offsets and threshold adjustment amounts. The action commands are output in step S402 and stored in the command buffer as encoded vectors. The parsing process follows predefined action encoding rules, which divide the action command vector into multiple semantic segments, each corresponding to a different adjustment parameter type. Cluster center offsets are extracted from the first segment of the action command vector. These offsets are vectors with the same dimension as the cluster center position vector, and each element represents the displacement of the corresponding cluster center along each coordinate axis. Threshold adjustment amounts are extracted from the last segment of the action command vector. These threshold adjustment amounts are vectors with the same number of classification threshold boundaries, and each element represents the increase or decrease in the corresponding threshold boundary value.

[0064] After the cluster center offsets are extracted, this embodiment superimposes them onto the position coordinates of the cluster center set. The cluster center set is output in step S302 and read in step S401 to participate in the state vector construction. In this embodiment, the position coordinates of the current cluster centers are read from the parameter storage area. The cluster center offsets are split into offset sub-vectors for each cluster according to the correspondence between cluster number and coordinate dimension. Each offset sub-vector is element-wise added to the corresponding cluster center position coordinates to obtain the updated cluster centers. The number of updated cluster centers is the same as the original cluster center set, and the coordinate positions of each cluster center are more accurately located in the sample-dense region of the corresponding cluster after offset adjustment.

[0065] Accordingly, in this embodiment, the threshold adjustment amount is superimposed on the classification threshold boundary to obtain the updated classification threshold. The classification threshold boundary is preset according to business rules during the system initialization phase and is used to divide the distance intervals corresponding to different sensitivity levels. The current classification threshold boundary sequence is read from the parameter storage area, and the threshold adjustment amount is added element-wise to the classification threshold boundary sequence to obtain the updated classification threshold. The values ​​of each element of the updated classification threshold are adjusted to better match the current data distribution characteristics, so that the division boundary of the sensitivity level matches the actual sample aggregation.

[0066] After the updated cluster centers and the updated classification thresholds are generated, this embodiment encapsulates them to form an optimized classification strategy. The encapsulation process serializes the updated cluster centers into cluster center parameter blocks and the updated classification thresholds into threshold parameter blocks. The two parameter blocks are then assembled into an optimized classification strategy object according to the strategy data structure specification. The optimized classification strategy object contains four fields: strategy version identifier, generation timestamp, cluster center parameter block, and threshold parameter block, and is written to the strategy storage area for subsequent steps.

[0067] Based on the aforementioned optimized classification strategy, this embodiment extracts encoder parameters and classifier parameters from the source domain pre-trained model. The source domain pre-trained model is trained on a large-scale general dataset; its encoder network possesses general feature extraction capabilities, and its classifier network possesses basic class determination capabilities. The weight matrices and bias vectors of each layer of the encoder network and the weight matrices and bias vectors of each layer of the classifier network are read from the parameter file of the source domain pre-trained model. The encoder parameters and classifier parameters are then loaded into the corresponding network layers of the target domain classifier. The network structure of the target domain classifier is consistent with that of the source domain pre-trained model to ensure parameter dimension matching.

[0068] After parameter loading is complete, this embodiment performs parameter freezing and unlocking operations on the target domain classifier. The gradient calculation flag of the bottom-level encoder parameters of the target domain classifier is set to the off state, ensuring that the bottom-level encoder parameters remain unchanged during subsequent gradient updates. The gradient calculation flag of the top-level classifier parameters of the target domain classifier is set to the on state, allowing the top-level classifier parameters to receive gradient feedback and perform parameter adjustments. The freezing operation of the bottom-level encoder parameters preserves the general feature extraction capability learned by the source domain pre-trained model, while the unlocking operation of the top-level classifier parameters allows the model to be adapted and optimized for the specific distribution of the target domain data.

[0069] Accordingly, this embodiment performs gradient updates on the parameters of the top-level classifier based on the optimized classification strategy. The updated cluster centers and updated classification thresholds from the strategy storage area are read, and the updated cluster centers are used as the target category prototype vectors of the classifier output layer. The latent feature vectors are input into the target domain classifier to perform forward propagation, calculating the distance loss between the classifier output and the updated cluster centers, and simultaneously calculating the hierarchical boundary constraint loss based on the updated classification thresholds. The distance loss and the hierarchical boundary constraint loss are summed to obtain the total loss value, and the parameters of the top-level classifier are updated using backpropagation and gradient descent based on the total loss value. The forward and backpropagation processes are repeated until the total loss value converges to below a preset threshold; the converged target domain classifier is the classification and hierarchical model. The classification and hierarchical model and the optimized classification strategy are read in subsequent step S601 for performing classification inference and hierarchical determination on the data to be labeled.

[0070] In one embodiment of the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning in this application, the method may further include the following: Step S601: Perform data cleaning and feature standardization on the data to be labeled to obtain standardized data to be labeled. Input the standardized data to be labeled into the encoder network of the variational autoencoder to perform nonlinear mapping to obtain the latent feature vector to be labeled. Step S602: Input the potential feature vector to be labeled into the classification and grading model to calculate its distance value with each updated cluster center, and select the category to which the cluster center with the minimum value belongs as the category label.

[0071] This embodiment performs data cleaning on the data to be labeled to remove redundant and abnormal records. The data to be labeled comes from a set of records to be classified newly added to the business system. Its data structure is consistent with the original data described in step S201, including text fields, numeric fields, and category fields. Duplicate record detection is performed on the data to be labeled, identifying and deleting redundant records based on a comparison of all field hash values. Missing value completion is performed on the data after deleting redundancy. Numeric fields are filled with the mean of the records in step S201, and category fields are filled with the mode of the records in step S201. Outlier truncation is performed on the completed data, replacing out-of-range values ​​with boundary values ​​based on the upper and lower truncation boundaries of each field recorded in step S201.

[0072] After data cleaning, this embodiment performs feature standardization transformation on the cleaned data to be labeled. The mean and standard deviation of each numerical field recorded in step S202 are read from the parameter storage area. These mean and standard deviation are statistics calculated and saved during the original data preprocessing stage. For each numerical field in the cleaned data to be labeled, each value is subtracted from the mean of the corresponding field and then divided by the standard deviation of the corresponding field. The transformation process uses the same statistical parameters as the training stage to ensure that the data to be labeled and the training data are in the same feature scale space. After the transformation is completed field by field, the numerical fields are reorganized in their original column order to obtain the standardized data to be labeled.

[0073] Accordingly, in this embodiment, the standardized data to be labeled is input into the principal component analysis module for dimensionality reduction projection. The projection matrix saved in step S202 is read from the parameter storage area. The projection matrix is ​​generated by the principal component analysis operation in the original data preprocessing stage. The standardized data to be labeled is multiplied by the projection matrix to obtain dimensionality-reduced data to be labeled with the same dimensions as the preprocessed dataset. The number of columns in the dimensionality-reduced data to be labeled is equal to the number of retained dimensions determined in step S202, and each sample is comparable to the training samples in the dimensionality-reduced feature space.

[0074] After dimensionality reduction, this embodiment inputs the dimensionality-reduced data to be labeled into the encoder network of a variational autoencoder to perform nonlinear mapping. The encoder network parameters trained and stored in step S301 are read from the model storage area and loaded into the weight matrices and bias vectors of each layer of the encoder network. Each sample vector from the dimensionality-reduced data to be labeled is input into the encoder network in batches. After multiple nonlinear transformations, the mean vector and variance vector are output by two parallel branches of the output layer, respectively. Based on the reparameterization technique described in step S301, noise vectors are sampled from the standard normal distribution and combined with the mean vector and variance vector to obtain the latent feature vector to be labeled. The latent feature vector to be labeled and the latent feature vector generated in step S301 are in the same low-dimensional latent space.

[0075] Based on the aforementioned potential feature vectors to be labeled, this embodiment inputs them into a classification and grading model to perform distance calculations. The optimized classification strategy encapsulated in step S501 is read from the strategy storage area, and the updated cluster center sequence is extracted from it. For each potential feature vector to be labeled, its Euclidean distance to each updated cluster center is calculated to obtain a set of distance values. The Euclidean distance is calculated based on the square root of the sum of the squares of the differences between corresponding elements of the two vectors, and the number of elements in the distance value set is equal to the target number of clusters.

[0076] After the distance value set is calculated, this embodiment selects the category to which the cluster center corresponding to the minimum value belongs as the category label. The distance values ​​in the set are compared, and the minimum distance value is located and its corresponding cluster center number is recorded. Each cluster center is assigned a category semantic identifier during generation in step S302. The category semantic identifier is predefined according to business rules and includes category names such as core data class, important data class, and general data class. The category semantic identifier of the cluster center corresponding to the minimum distance value is extracted as the category label for the current sample. After class label determination is completed for each sample, the category label for each sample is written to the label cache. The category label is read in subsequent step S701 for sensitivity determination based on the hierarchical threshold boundary.

[0077] In one embodiment of the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning in this application, the method may further include the following: Step S701: Read the updated classification threshold from the optimized classification strategy and divide it into multiple threshold intervals according to the sensitivity level. Calculate the distance between the potential feature vector to be labeled and the updated cluster center of the corresponding category label. Perform interval matching judgment between the distance value and the threshold interval to obtain the hierarchical label. Step S702: Construct a label pair by combining the category label and the hierarchical label, and establish an index association with the data to be labeled to obtain an associated binding record. Write the associated binding record into the label field of the data to be labeled to form a label dataset.

[0078] This embodiment reads the updated classification threshold from the optimized classification strategy and parses it into a threshold interval sequence. The optimized classification strategy is encapsulated in step S501 and stored in the strategy storage area. This embodiment reads the threshold parameter block from this storage area and performs a deserialization operation to obtain the updated classification threshold sequence. The updated classification threshold sequence contains multiple threshold boundary points, each dividing the distance value axis into several continuous intervals. The sensitivity level is predefined according to business rules and includes names such as high sensitivity, medium sensitivity, and low sensitivity. Each level corresponds to the threshold interval based on the distance value. The smaller the distance value, the closer the sample is to the cluster center, and the higher the corresponding sensitivity level.

[0079] After the threshold interval is defined, this embodiment calculates the distance between the potential feature vector to be labeled and the updated cluster center to which the corresponding category label belongs. The potential feature vector to be labeled is generated in step S601 and stored in the feature cache, and the category label is determined in step S602 and stored in the label cache. For each sample to be labeled, the coordinates of the corresponding cluster center are located from the updated cluster center sequence based on its category label. The Euclidean distance between the potential feature vector to be labeled of the sample and the coordinates of the cluster center is calculated to obtain the sample distance value, which measures the degree to which the sample deviates from the core region of its category.

[0080] Accordingly, this embodiment performs interval matching determination between the sample distance value and the threshold interval. The sample distance value is sequentially compared with each threshold boundary point to determine the specific threshold interval it falls into. When the sample distance value is less than the first threshold boundary point, the sample is determined to fall into the first threshold interval, and the high sensitivity level corresponding to this interval is used as the classification label. When the sample distance value is between the first and second threshold boundary points, the sample is determined to fall into the second threshold interval, and the medium sensitivity level corresponding to this interval is used as the classification label. When the sample distance value is greater than the last threshold boundary point, the sample is determined to fall into the last threshold interval, and the low sensitivity level corresponding to this interval is used as the classification label. After completing the classification label determination for each sample, the classification label of each sample is written into the label cache and stored alongside the category label.

[0081] After both the category label and the level label are determined, this embodiment constructs them into a label pair. For each sample to be labeled, its category label and level label are read from the label cache. The category label is used as the first element of the label pair, and the level label is used as the second element. The label pair is encapsulated in key-value pairs, with the key names being the category field and the level field, and the key values ​​being the corresponding category label value and level label value, respectively.

[0082] Accordingly, this embodiment establishes an index association between the label pairs and the data to be labeled to obtain an associated binding record. Each sample in the data to be labeled has a unique identifier field, which is generated based on a timestamp and sequence number during the data collection phase. The label pairs are associated with the unique identifiers of the corresponding samples, using the unique identifiers as the primary key and the label pairs as attribute values ​​to construct the associated binding record. The associated binding record supports quick retrieval of the category and classification labels of the corresponding samples using the unique identifiers.

[0083] After the associated binding record is generated, this embodiment writes it into the label field of the data to be labeled to form a labeled dataset. The data structure of the data to be labeled predefines the label field and reserves storage space. The label field consists of two components: a category subfield and a level subfield. Based on the unique identifier in the associated binding record, the corresponding sample record in the data to be labeled is located. The category label value from the label pair is written into the category subfield of that record, and the level label value from the label pair is written into the level subfield of that record. After label writing is completed for each sample, a labeled dataset is formed. The labeled dataset includes the original data fields and the newly added category label field and level label field. The labeled dataset can be called by the subsequent data security management module to implement differentiated access control and protection measures based on the category label and level label.

[0084] To effectively address the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning strategy optimization and cross-domain classification and grading model construction, and fully automated labeling execution, and to provide technical support for intelligent automatic classification, grading, and labeling of enterprise data assets and refined data governance, this application provides an embodiment of an automatic data classification, grading, and labeling device based on unsupervised deep learning and reinforcement learning for implementing all or part of the aforementioned automatic data classification, grading, and labeling method. See [link to embodiment]. Figure 2 The automatic data classification, grading, and labeling device based on unsupervised deep learning and reinforcement learning specifically includes the following components: Data clustering module 10 is used to perform data cleaning and missing value imputation on the original data to obtain cleaned data, perform feature standardization and principal component dimensionality reduction on the cleaned data to obtain a preprocessed dataset, input the preprocessed dataset into a variational autoencoder to perform encoding mapping to obtain a latent feature vector, and determine the target number of clusters by traversing the candidate cluster number range based on the silhouette coefficient of the latent feature vector and perform clustering operation to obtain preliminary clustering results and a set of cluster centers. The classification and grading module 20 is used to construct the preliminary clustering results, the latent feature vectors, and the cluster center set into a state vector and input it into the deep reinforcement learning agent. The deep reinforcement learning agent performs policy evaluation and outputs action instructions based on a composite reward function that integrates contour coefficients and validation accuracy. The action instructions are used to fine-tune the position coordinates of the cluster center set and the classification threshold boundary to obtain an optimized classification policy. The source domain pre-trained model parameters are loaded into the target domain classifier, and the parameters are fine-tuned according to the optimized classification policy to obtain a classification and grading model. The data labeling module 30 is used to encode the data to be labeled by the variational autoencoder and input it into the classification and grading model to perform classification inference to obtain category labels, perform sensitivity determination on the category labels according to the grading threshold boundary in the optimized classification strategy to obtain grading labels, and associate and bind the category labels and grading labels and write them into the data to be labeled to form a labeling dataset.

[0085] As described above, the automatic data classification and labeling device based on unsupervised deep learning and reinforcement learning provided in this application can construct preliminary clustering results and a set of cluster centers by using variational autoencoder encoding mapping and adaptive clustering number determination of silhouette coefficients. Combined with deep reinforcement learning agents, it performs policy evaluation and action command-driven optimization of cluster centers and classification thresholds based on composite reward functions, and obtains a classification and grading model through source domain pre-training parameter transfer fine-tuning. After encoding and inferring the category label and grading label of the data to be labeled, it is associated and bound to form a labeled dataset. This effectively solves the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning strategy optimization and cross-domain classification and grading model construction, and fully automated labeling execution, providing technical support for intelligent automatic classification and grading labeling and refined data governance of enterprise data assets.

[0086] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning.

[0087] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning.

[0088] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning.

[0089] In this embodiment of the invention, a preliminary clustering result and a set of cluster centers are constructed by using variational autoencoder encoding mapping and silhouette coefficient adaptive clustering number determination. A deep reinforcement learning agent then performs strategy evaluation and action command-driven optimization of cluster centers and classification thresholds based on a composite reward function. A classification and grading model is obtained through source domain pre-training parameter transfer fine-tuning. The data to be labeled is encoded and inferred to obtain category and grading labels, which are then associated and bound to form a labeled dataset. This effectively solves the shortcomings of traditional technologies in unsupervised deep feature extraction and adaptive clustering modeling, reinforcement learning strategy optimization and cross-domain classification and grading model construction, and fully automated labeling execution. It provides technical support for intelligent automatic classification, grading, and labeling of enterprise data assets and refined data governance.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatic data classification, grading, and labeling based on unsupervised deep learning and reinforcement learning, characterized in that, The method includes: Data cleaning and missing value imputation are performed on the original data to obtain cleaned data. Feature standardization and principal component dimensionality reduction are performed on the cleaned data to obtain a preprocessed dataset. The preprocessed dataset is input into a variational autoencoder to perform encoding mapping to obtain a latent feature vector. The latent feature vector is used to traverse the candidate cluster number range based on the silhouette coefficient to determine the target cluster number and perform clustering operation to obtain preliminary clustering results and a set of cluster centers. The preliminary clustering results, the latent feature vectors, and the cluster center set are constructed as state vectors and input into a deep reinforcement learning agent. The deep reinforcement learning agent performs policy evaluation and outputs action instructions based on a composite reward function that integrates contour coefficients and validation accuracy. The action instructions are used to fine-tune the position coordinates of the cluster center set and the classification threshold boundary to obtain an optimized classification policy. The parameters of the source domain pre-trained model are loaded into the target domain classifier, and the parameters are fine-tuned according to the optimized classification policy to obtain a classification and grading model. The data to be labeled is encoded by the variational autoencoder and then input into the classification and grading model to perform classification inference to obtain category labels. The category labels are then used to determine the sensitivity based on the grading threshold boundary in the optimized classification strategy to obtain grading labels. The category labels and grading labels are then associated and bound and written into the data to be labeled to form a labeled dataset.

2. The automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning according to claim 1, characterized in that, The process of performing data cleaning and missing value imputation on the original data to obtain cleaned data, and then performing feature standardization and principal component dimensionality reduction on the cleaned data to obtain a preprocessed dataset, includes: The original data is subjected to duplicate record detection and redundant records are deleted to obtain deduplicated data. Missing fields in the deduplicated data are filled with missing values ​​by selecting the mean or mode filling strategy according to the field type to obtain clean data. Outliers in the clean data are truncated according to the preset deviation threshold to obtain normalized data. The numerical fields in the normalized data are standardized by performing a standardization transformation based on the mean and standard deviation to obtain a standardized feature matrix. The standardized feature matrix is ​​then input into the principal component analysis module to determine the number of dimensions to retain based on the cumulative variance contribution rate threshold condition, and dimensionality reduction projection is performed to obtain a preprocessed dataset.

3. The automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning according to claim 1, characterized in that, The process of inputting the preprocessed dataset into a variational autoencoder to perform encoding mapping to obtain a latent feature vector, and then using the latent feature vector to determine the target number of clusters by traversing the candidate cluster number range based on the silhouette coefficient and performing clustering operations to obtain preliminary clustering results and a set of cluster centers includes: The preprocessed dataset is input into the encoder network of the variational autoencoder to perform nonlinear mapping to obtain the mean vector and variance vector. Based on the mean vector and variance vector, reparameterization sampling is performed to obtain the latent feature vector. The latent feature vector is input into the decoder network to perform reconstruction and calculate the loss value based on the weighted sum of the reconstruction error and divergence constraint term. Based on the loss value, backpropagation is performed to update the parameters of the encoder network and decoder network until convergence. Clustering operations are performed on the potential feature vectors one by one within the range of candidate cluster numbers, and the corresponding silhouette coefficient values ​​are calculated. The cluster number corresponding to the maximum value among the silhouette coefficient values ​​is selected as the target cluster number. Clustering operations are performed on the potential feature vectors based on the target cluster number to obtain preliminary clustering results and a set of cluster centers.

4. The automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning according to claim 1, characterized in that, The process of constructing a state vector from the preliminary clustering results, the latent feature vectors, and the cluster center set is input into a deep reinforcement learning agent. The deep reinforcement learning agent then performs policy evaluation and outputs action instructions based on a composite reward function that fuses contour coefficients and validation accuracy. This includes: The category distribution features, statistical features of potential feature vectors, and location coordinates of cluster center sets in the preliminary clustering results are concatenated and encoded to obtain a state vector. The state vector is then input into the deep neural network of the deep reinforcement learning agent to perform forward propagation and obtain the value estimate corresponding to each candidate action. Based on the current clustering results, the silhouette coefficient is calculated and the classification accuracy is calculated based on the validation set. The silhouette coefficient and the classification accuracy are weighted and summed according to the preset weight coefficient to obtain the composite reward value. The composite reward value and the value estimate are updated according to the time difference rule. The candidate action corresponding to the maximum value is selected from the value estimate as the action instruction output.

5. The automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning according to claim 1, characterized in that, The step of using the action command to fine-tune the position coordinates of the cluster center set and the classification threshold boundary to obtain an optimized classification strategy, loading the source domain pre-trained model parameters into the target domain classifier, and performing parameter fine-tuning according to the optimized classification strategy to obtain a classification and grading model includes: The cluster center offset and threshold adjustment are obtained by parsing the action instructions. The cluster center offset is superimposed on the position coordinates of the cluster center set to obtain the updated cluster center. The threshold adjustment is superimposed on the classification threshold boundary to obtain the updated classification threshold. The updated cluster center and the updated classification threshold are encapsulated to form the optimized classification strategy. The encoder parameters and classifier parameters are extracted from the source domain pre-trained model and loaded into the corresponding network layer of the target domain classifier. The bottom encoder parameters of the target domain classifier are frozen and the top classifier parameters are opened. The top classifier parameters are updated according to the updated cluster centers and updated classification thresholds in the optimized classification strategy until convergence is obtained to obtain the classification and grading model.

6. The automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning according to claim 1, characterized in that, The step of encoding the data to be labeled using the variational autoencoder and then inputting it into the classification and grading model to perform classification inference and obtain category labels includes: Data cleaning and feature standardization are performed on the data to be labeled to obtain standardized data to be labeled. The standardized data to be labeled is then input into the encoder network of the variational autoencoder to perform nonlinear mapping to obtain the latent feature vector to be labeled. The potential feature vector to be labeled is input into the classification and grading model to calculate its distance value with each updated cluster center. The category to which the cluster center with the minimum distance value belongs is selected as the category label.

7. The automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning according to claim 1, characterized in that, The step of performing sensitivity determination on the category labels based on the hierarchical threshold boundary in the optimized classification strategy to obtain hierarchical labels, associating and binding the category labels with the hierarchical labels, and writing them into the data to be labeled to form a labeled dataset includes: Read the updated classification threshold from the optimized classification strategy and divide it into multiple threshold intervals according to the sensitivity level. Calculate the distance between the potential feature vector to be labeled and the updated cluster center of the corresponding category label. Perform interval matching judgment between the distance value and the threshold interval to obtain the hierarchical label. The category label and the hierarchical label are constructed into a label pair, and an index association is established with the data to be labeled to obtain an associated binding record. The associated binding record is written into the label field of the data to be labeled to form a label dataset.

8. A data automatic classification, grading, and labeling device based on unsupervised deep learning and reinforcement learning, characterized in that, The device includes: The data clustering module is used to perform data cleaning and missing value imputation on the original data to obtain cleaned data, perform feature standardization and principal component dimensionality reduction on the cleaned data to obtain a preprocessed dataset, input the preprocessed dataset into a variational autoencoder to perform encoding mapping to obtain a latent feature vector, and traverse the candidate cluster number range based on the silhouette coefficient to determine the target cluster number and perform clustering operation to obtain preliminary clustering results and a set of cluster centers. The classification and grading module is used to construct the preliminary clustering results, the latent feature vectors, and the cluster center set into a state vector input to the deep reinforcement learning agent. The deep reinforcement learning agent performs policy evaluation and outputs action instructions based on a composite reward function that integrates contour coefficients and validation accuracy. The action instructions are used to fine-tune the position coordinates of the cluster center set and the classification threshold boundary to obtain an optimized classification policy. The parameters of the source domain pre-trained model are loaded into the target domain classifier, and the parameters are fine-tuned according to the optimized classification policy to obtain a classification and grading model. The data labeling module is used to encode the data to be labeled by the variational autoencoder and input it into the classification and grading model to perform classification inference to obtain category labels. The category labels are then used to perform sensitivity determination based on the grading threshold boundary in the optimized classification strategy to obtain grading labels. Finally, the category labels and grading labels are associated and bound and written into the data to be labeled to form a labeled dataset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the automatic data classification, grading and labeling method based on unsupervised deep learning and reinforcement learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the automatic data classification, grading, and labeling method based on unsupervised deep learning and reinforcement learning as described in any one of claims 1 to 7.

Images