Data automatic level-to-level management method based on artificial intelligence

Abstract

The invention relates to the technical field of data management, in particular to an artificial intelligence-based automatic data grading management method, which comprises the following steps of: acquiring to-be-graded data through a multi-source data acquisition channel, and performing data preprocessing on the to-be-graded data to generate a training data set; constructing a feature extraction network based on an improved Transform architecture, extracting data features in the training data set in combination with a cross-modal mechanism, and generating a dynamic feature mapping matrix; and constructing a time-space domain fusion model based on a graph neural network by using the dynamic feature mapping matrix, calculating a data sensitivity index, and performing adaptive classification. According to the method, complex cross-modal data can be processed, time-space domain features are fused, and efficient hierarchical management is realized through an improved deep learning model, so that the problems of data diversity, real-time performance and accuracy of an existing data hierarchical method are effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to an automatic hierarchical data management method based on artificial intelligence. Background Art

[0002] With the advent of the big data era, the types, scale and complexity of data are growing exponentially. How to effectively classify, grade and manage these massive amounts of data has become a major challenge faced by all industries, especially finance, healthcare, the Internet and other fields.

[0003] Data classification management technology classifies data based on its characteristics, risks, value and other dimensions, thereby achieving targeted data protection, access control and resource allocation. Traditional data classification methods usually rely on manual rule settings or static models. Although these methods can meet data management needs to a certain extent, with the diversification of data types and the continuous changes in the data environment, these traditional methods have shown many limitations. For example, rule-based classification methods usually ignore the timeliness, spatial correlation and dynamic content changes of data, and are difficult to adapt to complex and changing data environments.

[0004] Existing technologies mainly rely on static data analysis methods and manual rule setting in data classification management, resulting in poor accuracy and real-time performance of classification effects. In particular, in a multi-source heterogeneous data environment, the accuracy and efficiency of the classification algorithm cannot meet actual needs. In addition, traditional methods lack flexibility in dynamic data processing and feature extraction, especially when cross-modal data is involved. They cannot effectively integrate multiple different types of information for comprehensive analysis. In the fusion of spatiotemporal domain features, existing technologies find it difficult to achieve accurate and efficient feature expression, resulting in significant limitations on the accuracy and adaptability of data classification.

[0005] We have invented and proposed an automatic data grading management method based on artificial intelligence. In response to the shortcomings of the existing technology, through the combination of an improved Transformer architecture and a graph neural network, we can not only extract more accurate features from multi-source data, but also dynamically adjust the grading results in real time, thereby improving the adaptability and accuracy of data grading. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract of the specification and the title of the invention of this application to avoid blurring the purpose of this section, the abstract of the specification and the title of the invention, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the above existing problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by the present invention is: the problems of existing data classification methods in data diversity, real-time and accuracy.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions: collecting data to be classified through a multi-source data collection channel, and performing data preprocessing on the data to be classified to generate a training data set; Building a feature extraction network based on an improved Transformer architecture, extracting data features in the training data set in combination with a cross-modal mechanism, and generating a dynamic feature mapping matrix; The dynamic feature mapping matrix is ​​used to construct a spatiotemporal domain fusion model based on a graph neural network, calculate the data sensitivity index, and perform adaptive grading.

[0010] As a preferred solution of the method for automatic data classification management based on artificial intelligence described in the present invention, a multi-source data collection channel is established to collect the data to be classified, including: Collect formatted data through database connectors, including data tables, field attributes, and data values; Collect semi-formatted data through the file system interface, including XML documents, JSON files, and log files; Collect unstructured data through API interfaces, including text documents, image data, and audio and video files; At the same time, the metadata information of the data to be classified is recorded, including data source, creation time, access frequency and usage scenarios.

[0011] As a preferred solution of the method for automatic data classification management based on artificial intelligence of the present invention, data preprocessing is performed on the data to be classified, including: Construct a data quality assessment model and score based on four quality indicators: data integrity score, data accuracy score, data timeliness score, and data format standardization score: The data quality score is calculated as follows: in, is the weight coefficient of the quality index, satisfying , They are data completeness score, data accuracy score, data timeliness score and data format standardization score; Then perform data cleaning operations based on the data quality score Q.

[0012] As a preferred solution of the method for automatic hierarchical management of data based on artificial intelligence of the present invention, performing data cleaning operation according to the data quality score includes: When Q < 0.75, deep cleaning was performed, including withdrawn value filling, outlier modification, and format standardization; When 0.75 ≤ Q<0.90, standard cleaning is performed, including basic data normalization and standard exception handling; When Q ≥ 0.90, only light cleaning is performed, including basic format checking.

[0013] As a preferred solution of the method for automatic hierarchical management of data based on artificial intelligence described in the present invention, the generation of a dynamic feature mapping matrix includes: Build an improved Transformer encoder network, including an input embedding layer, a position encoding layer, and a multi-head self-attention layer; Using the Encoder Map different types of data features into a unified feature space and calculate the attention scores between modalities; Then, the self-attention features and cross-modal features are weighted and fused to generate the final fusion feature matrix F; At the same time, the attention mechanism is introduced to capture the temporal pattern of data features; Finally, the dynamic feature mapping matrix M is generated: M = Dropout(LayerNorm(F + TemporalAttn(F))) Among them, the dimension of the dynamic feature mapping matrix M is n×d, n is the number of samples, and d is the feature dimension.

[0014] As a preferred solution of the artificial intelligence-based automatic hierarchical data management method described in the present invention, the data features include content features, structural features, similarity features and shape features.

[0015] As a preferred solution of the method for automatic hierarchical management of data based on artificial intelligence described in the present invention, a spatiotemporal association graph is constructed based on the dynamic feature mapping matrix, including: G=(V,E) Among them, E is the edge set, which represents the relationship between samples, and V is the vertex set, which represents the data sample; For each pair of samples and , calculate its spatiotemporal correlation ; when When it is greater than the threshold τ, at the node and The edges are connected between them, indicating that there is a significant spatiotemporal correlation between them.

[0016] As a preferred solution of the method for automatic data classification management based on artificial intelligence described in the present invention, the calculation of data sensitivity index and adaptive classification include: A spatiotemporal sensitivity assessment model was constructed, and the comprehensive sensitivity index S was calculated by weighted average. The formula is: in, , and They are spatial sensitivity, temporal sensitivity and content sensitivity. , and is the corresponding weight coefficient; Classification threshold It is dynamically adjusted according to the comprehensive sensitivity index S and risk assessment results, and the adjustment formula is: in, is the threshold at time t, is the learning rate, controlling the adjustment amplitude, is the change of the comprehensive sensitivity index, is the time variation, is the risk change, It is the risk weight factor, which measures the impact of risk on the adjustment of the grading threshold.

[0017] As a preferred solution of the method for automatic hierarchical management of data based on artificial intelligence described in the present invention, it specifically includes: when When the data is classified as ordinary level, i.e. D1 level, basic security strategy is implemented; when When the data is classified as sensitive, i.e. D2, enhanced security policies are implemented; when When the data is classified as confidential, i.e. D3, strict security policies are implemented; when When the data is collected, it is classified as top secret, i.e. D4, and a special security policy is implemented.

[0018] Beneficial effects of the present invention: 1. It reduces the uncertainty caused by data differences, avoids the performance degradation caused by poor data quality during the model training phase, provides more accurate original input for subsequent feature extraction and sensitivity calculation, and improves the stability and reliability of automatic classification; 2. The completeness of feature representation is improved through multi-head self-attention and cross-modal fusion, avoiding information loss or modal deviation. The dynamic feature mapping matrix can be updated or fine-tuned in real time according to new data, providing a flexible and accurate feature basis for subsequent sensitivity evaluation. This process lays a foundation for high-quality data representation for adaptive hierarchical decision-making, allowing the system to maintain high discrimination accuracy for various scenarios and different types of data; 3. It overcomes the problem of insufficient perception of multi-dimensional features in the original method. Especially in data classification management, the sensitivity index is affected by multiple factors such as spatiotemporal laws and content value. By introducing GNN, it is possible to focus on local key nodes while retaining global structural information, thereby improving the accuracy of sensitivity assessment. Adaptive classification can adjust data security or management strategies in time according to real-time calculation results, which not only avoids the waste of resources caused by excessive protection, but also prevents the security risks caused by insufficient protection, and improves the classification accuracy and efficiency of the system when processing large-scale, multimodal, and complex related data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them: Figure 1 It is a flow chart of the method for automatic hierarchical management of data based on artificial intelligence shown in the present invention; Figure 2 It is a schematic diagram for comparing the multimodal data processing efficiency shown in the present invention; Figure 3 This is a schematic diagram of feature robustness comparison shown in the present invention; Figure 4 It is a schematic diagram for comparing the dynamic classification efficiency shown in the present invention. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0021] Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without making any creative work should fall within the scope of protection of the present invention.

[0022] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0023] According to an embodiment of the present invention, Figure 1 The flowchart shown is a method for automatic hierarchical management of data based on artificial intelligence, which specifically includes the following steps: S1. Collect the data to be classified through the multi-source data collection channel, and perform data preprocessing on the data to be classified to generate a training data set. Among them, what needs to be explained in this step is: Establish a multi-source data collection channel to collect three types of data to be classified, including: Collect formatted data through database connectors, including data tables, field attributes, and data values; Collect semi-formatted data through the file system interface, including XML documents, JSON files, and log files; Collect unstructured data through API interfaces, including text documents, image data, and audio and video files; At the same time, the metadata information of the data to be classified is recorded, including the data source (such as database, file system, API interface), creation time (such as recording the generation time of the data), access frequency (such as tracking the reading frequency of data) and usage scenarios (such as the scope of application or functional field of the data); Furthermore, the quality of the collected data to be classified is evaluated, including: Build a data quality assessment model and score based on the following four quality indicators: C: Data integrity score, measuring whether the data is missing or corrupted; A: Data accuracy score, which measures whether the data is authentic and reliable; T: Data timeliness score, which measures the update time of the data and its relevance to the current situation; F: Data format standardization score, which measures whether the data meets the predetermined format requirements; Exemplarily, the calculation formula of the data quality score is: in, is the weight coefficient of the quality index, satisfying , is the score of each quality dimension; Specifically, the following operations are performed according to the data quality score Q: When Q<0.75, deep cleaning is performed, such as withdrawal value filling (based on XGBoost intelligent prediction), outlier modification (based on local anomaly factor detection) and format standardization (unified data specification template); When 0.75 ≤ Q<0.90, standard cleaning is performed, such as basic data normalization and standard exception handling; When Q ≥ 0.90, only light cleaning, such as basic format checking, is performed; At the same time, an improved data enhancement algorithm is used to generate training samples, including: Format the data and generate synthetic samples using the SMOTE-NC algorithm (which handles unbalanced datasets containing both categorical and numerical variables); For text data, synonym replacement and back translation techniques are used for enhancement; Enhance image data by rotating, scaling and adding noise; It should be noted that the number of samples after enhancement is three times that of the original data to maintain the distribution balance of the distribution data; Then perform standard transformation on the enhanced data to generate a training data set in a unified format: The Z-score standardization method was used for scoring characteristics; One-hot encoding of categorical features; The text features are converted into a vector representation of fixed dimensions using the Word2Vec model (a model for generating word vectors); Extract feature vectors from image features through a preset ResNet model (residual network); All feature vectors are concatenated to form the final training data matrix with a matrix dimension of n×m, where n is the number of samples and m is the feature dimension.

[0024] It should be noted that this step converts the training data from different sources and formats into standardized data sets to provide a high-quality data foundation for subsequent feature extraction and model training. This method ensures the accuracy, precision and standardization of the training data set by introducing data quality assessment and hierarchical carrying mechanism, combined with standardized data enhancement and standardization methods.

[0025] S2. Build a feature extraction network based on the improved Transformer architecture, extract data features from the training data set in combination with the cross-modal mechanism, and generate a dynamic feature mapping matrix. The following points need to be explained in this step: Construct an improved Transformer encoder network, including an input embedding layer, a position encoding layer, and a multi-head self-attention layer; where: Input embedding layer: maps input data from different sources (training data generated in step S1) to a unified feature space; Position encoding layer: adds position encoding to each input feature to ensure that the model can process sequential information; Multi-head self-attention layer: Use the multi-head self-attention mechanism. The specific calculation formula is: in, are query matrix, key matrix and value matrix respectively, is the output transformation matrix, The calculation result for each head; Exemplarily, the calculation for each head is: in, are query, key, and value matrices respectively, is the parameter matrix for each head, is the dimension of the key, controlling the scale of the attention calculation; Furthermore, a cross-modal feature mapping mechanism is established to capture the correlation between different types of data features, including: Using the Encoder Map different types of data features into a unified feature space and calculate the attention score between modalities: in, are the characteristic matrices of two different modes, and V is the value matrix; Then the self-attention features and cross-modal features are weighted and fused to generate the final fusion feature matrix F: in, is the adaptive weight coefficient; At the same time, the attention mechanism (GRU gated recurrent unit) is introduced to capture the temporal pattern of data features: Among them, F is the fusion feature matrix, PE is the position encoding; As an example, the calculation formula of GRU is: in, is the hidden state at the current moment, is the hidden state at the previous moment, To update the gate, To reset the gate, is the current candidate hidden state, is the weight matrix, is bias; Going further, generate the dynamic feature mapping matrix M: M = Dropout(LayerNorm(F + TemporalAttn(F))) Among them, the dimension of the dynamic feature mapping matrix M is n×d, n is the number of samples, and d is the feature dimension; The features of each sample include content features (such as the theme feature density, keyword feature number, and attribute feature number of feature samples), structural features (such as hierarchical relationships, logical feature structures, and dependencies), similarity features (such as content similarity, structural similarity, temporal similarity, and spatial similarity), and shape features (such as dimensional features, feature formats, difference features, and scale features). Each element in the matrix is ​​normalized and has a value range of [-1,1]. At the same time, in order to ensure the dynamic update capability of the features, the feature mapping matrix is ​​updated every Δt, where Δt is adaptively adjusted according to the update frequency.

[0026] It should be noted that through the implementation of this step, the feature extraction of multimodal data and the generation of dynamic feature mapping matrix are realized. This method effectively obtains the characteristics of multimodal data and the correlation between modalities through the improved Transformer architecture and cross-modal attention mechanism, providing high-quality feature representation for subsequent spatiotemporal domain fusion and sensitivity calculation.

[0027] S3. Use the dynamic feature mapping matrix to build a spatiotemporal fusion model based on the graph neural network, calculate the data sensitivity index, and perform adaptive classification. Among them, what needs to be explained in this step is: Construct a spatiotemporal correlation graph G=(V,E) based on the dynamic feature mapping matrix; Among them, E is the edge set, which represents the relationship between samples, and V is the vertex set, which represents the data sample, for example: Where N is the total number of data samples; For each pair of samples and , calculate its spatiotemporal correlation , the formula is: in, and The samples are and Spatial location parameters (such as geographic coordinates), and The samples are and The time parameters, and are the weight coefficients of space and time respectively, and are the measurement parameters for space and time, respectively, used to adjust the sensitivity of spatial and temporal differences; when When it is greater than the threshold τ, at the node and The edges are connected between them, indicating that there is a significant spatiotemporal correlation between them; Furthermore, the graph convolution operation is used to extract multi-layer features from the graph G. The calculation formula for each layer is: in, For the The node feature matrix of the layer, For the The weight matrix of the layer, is the activation function, is the adjacency matrix after adding the self-loop, A is the original adjacency matrix, is the identity matrix, ensuring that each node's own characteristics are also considered. is the degree matrix, , represents the degree of each node (i.e. the number of edges connected to it); In order to enhance the model's attention to key features, this embodiment introduces a self-attention mechanism, and the attention coefficient between each pair of nodes is Calculated by the following formula: in, and Node and The feature vector of , W is the weight matrix, a is the attention weight vector, It represents the feature concatenation operation, which aims to comprehensively consider the relationship between nodes; Specifically, a spatiotemporal sensitivity assessment model is constructed, and the comprehensive sensitivity index S is calculated by weighted average. The formula is: in, , and They are spatial sensitivity, temporal sensitivity and content sensitivity. , and is the corresponding weight coefficient; As an example, spatial sensitivity The calculation formula is: in, is the propagation range factor, which indicates the spatial range of information propagation. is the regional importance factor, indicating the relative importance of the spatial region. is the spatial dependency factor, which indicates the strength of the spatial relationship between nodes. , , is the corresponding weight coefficient; As an example, time sensitivity The calculation formula is: in, is the timeliness factor, indicating the timeliness of the data. is the life cycle factor, which indicates the length of the data life cycle. is the interdependence factor, which indicates the time dependency of data with other nodes. , , is the corresponding weight coefficient; As an example, content sensitivity The calculation formula is: in, is the information value factor, indicating the importance of data content, is the data factor, which indicates the volume or information amount of the data. is the correlation factor, which indicates the degree of correlation between the data and other nodes. , , is the corresponding weight coefficient; Furthermore, this embodiment adaptively optimizes the classification strategy based on the deep Q network (DQN). The state space S contains the current data features and historical classification results, the action space A contains possible hierarchical decisions, and the reward function R is: in, is the classification accuracy, For processing efficiency, For resource consumption, , , is the balance factor; Use a deep Q network to estimate each state-action pair, and the update formula of the Q value is: in, is the learning rate, As a protective factor, are the next state and action respectively; It should be noted that the classification threshold It is dynamically adjusted according to the comprehensive sensitivity index S and risk assessment results. The adjustment formula is: in, is the threshold at time t, is the learning rate, controlling the adjustment amplitude, is the change of the comprehensive sensitivity index, is the time variation, is the risk change, is the risk weight factor, which measures the impact of risk on the adjustment of the grading threshold; According to the changes in sensitivity and risk in actual application scenarios, the classification threshold will be adjusted to ensure that data of different levels are protected accordingly. For example, when the sensitivity index of data increases or the risk level increases, the system will automatically increase the security level of the data and implement stricter security measures. In an optional implementation, according to the comprehensive sensitivity index S and the output of the classification strategy, the data samples will be divided into different security levels and the corresponding security strategies will be implemented; the specific rules are as follows: (1) D1: General level when When the data is classified as ordinary level (D1 level), basic security policies are implemented, including: Access control: The role-based access control model (RBAC) is used. It means that the calculation formula is: Among them, R is the level (0~1), indicating the importance of the data, T is the time factor (for example, working hours is 1, non-working hours is 0.5), L is the location factor (for example, intranet is 1, extranet is 0.6), , , is the weight coefficient; Encryption protection: Use standard encryption scheme, which is based on basic encryption strength It means that the calculation formula is: in, is the basic encryption strength (e.g. 128-bit encryption), F is the access frequency, is the adjustment factor (e.g., 0.1); Audit strategy: The audit frequency is =24 hours (once a day), the audit content only records key operations, and the audit time is =1; (2) D2: sensitive level when When the data is classified as sensitive (D2), enhanced security policies are implemented, including: Access Control: The access control model is based on the permission matrix It means that the calculation formula is: Among them, S is the security level factor (0~1), which reflects the sensitivity of the data, that is, the sensitivity index, and H is the historical behavior score (0~1), which reflects the historical record of data access. , , , , is the weight coefficient; Encryption protection: Use a dynamic key encryption mechanism that enhances encryption strength It means that the calculation formula is: Among them, V is the data score (0~1), indicating the importance of the data. and are the adjustment coefficients for access frequency and data score, respectively (e.g., 0.2 and 0.3); Audit strategy: The audit frequency is = 12 hours (twice a day), record all operations and operating environment, audit time is =2; (3) D3: Confidential when When the data is classified as confidential (D3), strict security policies are implemented, including: Access control: A multi-role authentication mechanism is used, which is scored by access rights. It means that the calculation formula is: Among them, A is the authentication strength factor (for example, 0.8 for two-factor authentication and 1 for three-factor authentication), B is the matching degree of behavioral characteristics (0~1), , , , , , , is the weight coefficient; Encryption protection: A high-strength encryption scheme is used, which enhances the encryption strength It means that the calculation formula is: Among them, R is the real-time risk score (0~1), is the adjustment coefficient; Audit strategy: The audit frequency is = 4 hours (six times a day), recording all operations and environmental characteristics, the audit time is =3; (4) D4: Top Secret when When the data is classified as top secret (D4), a special security policy is implemented, including: Access control: Establish the highest level of access control mechanism, which is scored by permissions It means that the calculation formula is: Among them, C is the scenario compliance (0~1), which indicates the compliance of data access, and D is the dynamic behavior score (0~1), which reflects the abnormal behavior in the data access process. , , , , , , , , is the weight coefficient; Encryption protection: The highest level of encryption protection is adopted, which is protected by enhancing the encryption strength. It means that the calculation formula is: Among them, T is the threat level (0~1), which is scored according to the threats that the data may suffer. , , , is the adjustment coefficient, which is the optimal coefficient after adjustment by safety assessment; Audit strategy: The audit frequency is = 1 hour (24 times a day), real-time recording of all access operations, data modifications, data transmission and all abnormal behaviors. Each audit will record the identity of the data accessor, operation type, operation environment (including device information, IP address), access time and access location information. The audit time is =5.

[0028] It should be noted that through this step, the spatiotemporal domain feature fusion based on graph neural network is realized, and the adaptive optimization of the adaptive strategy is realized through deep reinforcement learning. At the same time, the temporal intensity, spatial correlation and content characteristics of the data are taken into account, providing an accurate quantitative basis for data classification.

[0029] Preferably, this embodiment effectively constructs high-quality training data through multi-source data collection and preprocessing, and then uses the improved Transformer architecture to perform cross-modal mining and dynamic mapping of data features to provide multi-dimensional feature support for subsequent algorithms, and finally combines graph neural networks to perform feature fusion and adaptive grading in the spatiotemporal domain to form an efficient data sensitivity assessment and automatic grading management system; this enables the present invention to overcome the shortcomings of traditional methods in multimodal data processing, feature robustness and dynamic grading efficiency, and achieve more accurate and flexible grading management of massive and complex data.

[0030] It should also be noted that the above-mentioned adaptive classification method based on multi-source data collection and improved Transformer architecture, combined with cross-modal mechanism and graph neural network (GNN) spatiotemporal fusion, has a beneficial effect superior to traditional methods in practical applications. Traditional methods are usually based on single-modal or weak cross-modal data fusion strategies, such as: Single modality feature extraction: In industrial sensor or log data processing, simple statistical features or classic machine learning algorithms (such as support vector machine (SVM) and K-nearest neighbor (KNN)) are used to complete classification or grading. Although this method has a certain effect in the early stage, it cannot deeply mine the correlation information between multi-modal data (including video streams, text logs, and multi-type sensor time series data); Limited feature fusion dimensions: Traditional methods often only perform feature fusion between homogeneous data (for example, limited to text logs or limited to structured table data), without systematically establishing contextual connections between multiple modalities. Even if certain deep learning methods are used, attention is only allocated to single-modal or weakly coupled multimodal data, lacking the ability to dynamically adjust cross-modal weights; Lack of efficient modeling of spatiotemporal information: Traditional methods lack the introduction of graph neural networks or other deep spatiotemporal models in time series data processing, and are insufficient in describing the correlation and time series changes between nodes. This makes it impossible to finely grade data sensitivity when the data volume is large and the correlation is complex. In summary, the traditional method easily leads to insufficient utilization of cross-modal information, missing or confused features, and untimely response to outliers and real-time data fluctuations. In order to compare the above defects and prove the superiority of the method of the present invention in feature extraction accuracy, multimodal information fusion capability and sensitivity adaptive classification, this embodiment is further verified by the following comparative experiment involving multi-source data collection and processing:

Experimental subjects

Experimental conditions

The implementation process of traditional methods

[0031] Reference Figure 2 , Figure 3 and Figure 4 , respectively, compares the performance of the traditional method and the method of the present invention in terms of multimodal data processing efficiency, feature robustness and dynamic classification efficiency, and shows the differences between the two methods when the data scale or noise level changes under different indicators.

[0032] Reference Figure 2It can be seen intuitively that as the data scale increases, the processing time of the two methods increases. However, by comparison, it is found that the overall processing time of the method of the present invention (solid line) is lower. For example, when the data scale reaches 5GB, the processing time of the method of the present invention is 260ms, while the traditional method (dashed line) requires 380ms. The main reason for this difference is that the method of the present invention performs multi-threaded concurrent processing in the acquisition and preprocessing stages, and the improved Transformer can better utilize GPU parallelism during feature extraction. The traditional method is relatively simple in data preprocessing, and is prone to synchronization lag or repeated calculations at the same data scale, thereby reducing the overall processing efficiency.

[0033] Reference Figure 3 It can be seen intuitively that as the noise level increases (meaning that the data is more deeply disturbed), the accuracy of the two methods is affected to a certain extent. The recognition accuracy of the traditional method (dashed line) decreases from 80% to 88% when the noise increases from 10dB to 50dB, while the method of the present invention (solid line) always remains in a higher range (from 88% to 95%); this result shows that the method of the present invention has stronger anti-interference ability in the process of multimodal fusion and feature extraction, especially the cross-modal attention mechanism can help compensate for the information loss when noise occurs in a single modality, thereby improving the robustness of the overall recognition. The traditional method has limited noise suppression due to the lack of deep fusion and dynamic weight adjustment between multiple modalities.

[0034] Reference Figure 4 , the classification efficiency of the method of the present invention (solid line) slowly decreases from 800 items / second to 730 items / second in the data input range of 5,000 to 25,000 items, indicating that the system can still maintain a high online processing efficiency while the amount of data increases. In contrast, the efficiency of the traditional method (dashed line) decreases more significantly, and can only reach 500 items / second when the amount of data is 25,000. This is mainly due to the fact that the method of the present invention adopts a spatiotemporal fusion strategy in the graph neural network after feature extraction, and can adjust the classification threshold in real time according to the feedback mechanism of the sensitivity index. The traditional method is difficult to maintain a high throughput in the scenario of continuous data influx due to the lack of flexible cross-modal management and efficient model structure for large amounts of data.

[0035] In summary, the method of the present invention realizes deep fusion and cross-modal attention strategy of multimodal data through the improved Transformer, and combines the spatiotemporal information modeling of graph neural network, which significantly improves the data processing efficiency, feature robustness and dynamic classification efficiency, and is more superior than traditional methods in practical applications; traditional methods still have obvious shortcomings in single or weak cross-modal processing, suppression of noise interference and dynamic adjustment, while the method of the present invention can adaptively perform fast preprocessing and fusion for multiple types of data sources, maintain high accuracy for different noise interferences, and maintain good classification efficiency when large-scale data is input in real time; therefore, through the above experiments and result analysis, the feasibility and superiority of the scheme of the present invention in the application of adaptive classification of multi-source data sensitivity can be fully verified.

[0036] The aforementioned methods for extracting content features, structural features, similarity features, and shape features can be performed using methods and means in the prior art, and will not be described in detail in this example.

[0037] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An automatic hierarchical data management method based on artificial intelligence, characterized in that: include: Collecting the data to be classified through a multi-source data collection channel, and performing data preprocessing on the data to be classified to generate a training data set; Building a feature extraction network based on an improved Transformer architecture, extracting data features in the training data set in combination with a cross-modal mechanism, and generating a dynamic feature mapping matrix; The dynamic feature mapping matrix is ​​used to construct a spatiotemporal domain fusion model based on a graph neural network, calculate the data sensitivity index, and perform adaptive grading.

2. The method for automatic hierarchical management of data based on artificial intelligence according to claim 1, characterized in that: Establish a multi-source data collection channel to collect the data to be classified, including: Collect formatted data through database connectors, including data tables, field attributes, and data values; Collect semi-formatted data through the file system interface, including XML documents, JSON files, and log files; Collect unstructured data through API interfaces, including text documents, image data, and audio and video files; At the same time, the metadata information of the data to be classified is recorded, including data source, creation time, access frequency and usage scenarios.

3. The method for automatic hierarchical management of data based on artificial intelligence according to claim 1 or 2, characterized in that: The data to be classified is preprocessed, including: Construct a data quality assessment model and score based on four quality indicators: data integrity score, data accuracy score, data timeliness score, and data format standardization score: The data quality score is calculated as follows: ； in, is the weight coefficient of the quality index, satisfying , They are data completeness score, data accuracy score, data timeliness score and data format standardization score; Then perform data cleaning operations based on the data quality score Q.

4. The method for automatic hierarchical management of data based on artificial intelligence according to claim 3 is characterized in that: Performing data cleaning operations according to the data quality score includes: When Q < 0.75, deep cleaning was performed, including withdrawal value filling, outlier modification, and format standardization; When 0.75 ≤ Q < 0.90, standard cleaning is performed, including basic data normalization and standard exception handling; When Q ≥ 0.90, only light cleaning is performed, including basic format checking.

5. The method for automatic hierarchical management of data based on artificial intelligence according to claim 1, characterized in that: The generating of the dynamic feature mapping matrix comprises: Build an improved Transformer encoder network, including an input embedding layer, a position encoding layer, and a multi-head self-attention layer; Using the Encoder Map different types of data features into a unified feature space and calculate the attention scores between modalities; Then, the self-attention features and cross-modal features are weighted and fused to generate the final fusion feature matrix F; At the same time, the attention mechanism is introduced to capture the temporal pattern of data features; Finally, the dynamic feature mapping matrix M is generated: M = Dropout(LayerNorm(F + TemporalAttn(F))); Among them, the dimension of the dynamic feature mapping matrix M is n×d, n is the number of samples, and d is the feature dimension.

6. The method for automatic hierarchical management of data based on artificial intelligence according to claim 5 is characterized in that: The data features include content features, structural features, similarity features and shape features.

7. The method for automatic hierarchical management of data based on artificial intelligence according to claim 1 or 5, characterized in that: Constructing a spatiotemporal association graph based on the dynamic feature mapping matrix includes: G=(V,E) Among them, E is the edge set, which represents the relationship between samples, and V is the vertex set, which represents the data sample; For each pair of samples and , calculate its spatiotemporal correlation ; when When it is greater than the threshold τ, at the node and The edges are connected between them, indicating that there is a significant spatiotemporal correlation between them.

8. The method for automatic data classification management based on artificial intelligence according to claim 1 is characterized in that: The calculation of the data sensitivity index and the adaptive classification include: A spatiotemporal sensitivity assessment model was constructed, and the comprehensive sensitivity index S was calculated by weighted average. The formula is: ； in, , and They are spatial sensitivity, temporal sensitivity and content sensitivity. , and is the corresponding weight coefficient; Classification threshold It is dynamically adjusted according to the comprehensive sensitivity index S and risk assessment results, and the adjustment formula is: ； in, is the threshold at time t, is the learning rate, controlling the adjustment amplitude, is the change of the comprehensive sensitivity index, is the time variation, is the risk change, It is the risk weight factor, which measures the impact of risk on the adjustment of the grading threshold.

9. The method for automatic hierarchical management of data based on artificial intelligence according to claim 8, characterized in that: Specifically include: when When the data is classified as ordinary level, i.e. D1 level, basic security strategy is implemented; when When the data is classified as sensitive, i.e. D2, enhanced security policies are implemented; when When the data is classified as confidential, i.e. D3, strict security policies are implemented; when When the data is collected, it is classified as top secret, i.e. D4, and a special security policy is implemented.

Images