Cloud edge collaboration and AI fusion data classification and grading method and system and storage medium

Abstract

The invention relates to the technical field of big data and data security, in particular to a cloud-edge collaboration and AI fusion data classification and grading method and system and a storage medium, cloud computing and edge computing are combined, the AI technology is used for data classification and grading, and rapid processing and efficient analysis of data are achieved; for structured data and unstructured data, different recognition and processing methods are adopted, including regular expression, keyword matching, dictionary matching, neural network, natural language processing and other algorithm models; particularly, a neural network and a natural language processing technology are combined with an unsupervised learning method and a semi-supervised learning method to realize an automatic classification and grading method of unstructured data in stages, rapid data processing and response can be carried out on an edge layer through a cloud edge collaboration mechanism, and the efficiency is improved. And deep analysis and model training are carried out by using the powerful computing power of the cloud layer.

Description

Technical Field

[0001] The present invention relates to the technical field of big data and data security, and specifically to a data classification and grading method, system and storage medium integrating cloud-edge collaboration and AI. Background Art

[0002] Recently, data classification and grading, as an important technology in the field of data processing, has received extensive attention and research. Especially in the context of the integration of cloud computing, edge computing and Internet of Things technologies, a large number of sensors and other smart devices continue to generate massive amounts of data, which need to be processed in a timely manner to provide real-time feedback and services. How to effectively classify and grade data from endpoint devices and take different processing and protection measures according to different levels of data has become an important research topic.

[0003] Chinese patent CN106934525A proposes uploading a large amount of data to the cloud and using the powerful computing power of cloud computing to classify and analyze it. However, this method has problems such as transmission delay and high data transmission cost when processing data with high real-time requirements or particularly large data volumes; the paper "In-depth Understanding of Edge Computing: Working Principles and Source Code Analysis of Cloud, Edge, and End" (Cui Guangzhang, Zhijiang Laboratory) uses edge computing to push data processing to edge devices closer to the data source, such as sensors and embedded systems. These devices can collect data in real time and perform preliminary processing, thereby greatly reducing data transmission delays and bandwidth usage. However, when processing complex data and advanced algorithms, edge devices are often limited by their computing power and storage capacity, and it is difficult to independently complete all tasks of data classification and grading; Chinese patent CN114997308B proposes the combination of cloud computing and edge computing, and builds a pre-trained model through multi-attribute label deconstruction and multi-input multi-output (MIMO) structure to achieve efficient classification of industrial data, improve industrial data processing efficiency, and ensure the security and availability of industrial data at different levels. However, this invention mainly focuses on the construction and use of industrial data classification models, and does not conduct in-depth discussion and optimization on the classification and grading requirements of different industry scenarios.

[0004] Existing technologies have made some progress, but there are still some obvious problems and deficiencies in the field of data classification and grading. First, cloud computing centers have latency issues when processing data with high real-time requirements, while the edge is limited by computing power and storage resources. Secondly, the specific implementation of the cloud-edge collaborative framework in the field of data classification and grading still lacks mature technical solutions, such as how to effectively allocate and schedule resources between the cloud, edge, and end to achieve accuracy and real-time data classification and grading; how to design reasonable algorithms and models to meet the needs of data classification and grading; and how to ensure the privacy and security of data during transmission and processing. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a data classification and grading method, system and storage medium that integrates cloud-edge collaboration and AI, and to achieve refined management of industry data through reasonable data classification and grading methods and models; to use the advantages of edge computing to improve the speed and efficiency of data processing and reduce the burden on cloud servers; to use neural networks and natural language processing technology combined with unsupervised learning and semi-supervised learning methods to realize automatic classification and grading of unstructured data in stages; and to promote the interconnection and interoperability of data in all walks of life by building a data classification and grading system that is both safe, reliable, efficient and convenient.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a data classification and grading method integrating cloud-edge collaboration and AI, comprising:

[0007] Data scanning: extracting core data items related to business operations from data assets based on preset scanning rules;

[0008] Data classification and grading: extract data features based on the scanned core data items. The data features include at least: data type, data sensitivity, and data usage scenario. Use a preset pattern matching model to identify sensitive data in business data and further implement data classification and grading. The preset pattern matching model includes: a structured data matching model and an unstructured data matching model. The structured data matching model uses at least regular expressions, keywords, and dictionary algorithms to match sensitive data and classify and grade data based on preset rules. The unstructured data matching model uses neural networks and natural language processing technology to build a data classification and grading model to achieve classification and grading of unstructured data.

[0009] Adjustment of classification and grading results. For situations where automatic classification and grading results are not applicable, manual dynamic adjustments are made. At the same time, user messages and social media comment information are collected to extract sentiment tendencies, provide feedback on classification and grading results, and adjust the preset pattern matching model.

[0010] In particular, the data scanning rules perform data scanning according to a preset scanning method, determine industry core data items based on prior knowledge of industry experts according to national laws and regulations, business needs and process management requirements.

[0011] In particular, the unstructured data matching model also combines unsupervised learning and semi-supervised learning methods to perform data analysis in stages, including:

[0012] In the data preparation phase, unlabeled production data is collected and preprocessed. The preprocessing includes at least noise removal, word segmentation, and stop word removal.

[0013] In the initial stage, an unsupervised learning method was used to introduce the Deep Adaptive Clustering (DAC) algorithm to cluster unlabeled production data, obtain preliminary classification results, and put them into production application;

[0014] In the later stage, some data are selected from the preliminary classification results for manual labeling to obtain a small amount of labeled data. Then, a semi-supervised learning method is adopted, and the semi-supervised support vector machine (S3VM) algorithm is introduced to further optimize the model using a small amount of labeled data and a large amount of unlabeled data. For specific classification tasks, the Transformer-based model is combined for fine-tuning to ensure that the model can capture the complex semantic information in the text data.

[0015] In the optimization phase, the data set is divided into multiple subsets using cross-validation statistical methods, and training and testing are performed alternately to evaluate model performance and adjust model parameters and configurations. Grid search is used to traverse given parameter combinations to find the parameter configuration that optimizes model performance.

[0016] In particular, the results of data classification and grading include at least three categories: public data, internal data, and sensitive data, and four levels in each category, namely level one, level two, level three, and level four, among which level one has the lowest importance and potential impact.

[0017] Furthermore, the present invention provides a data classification and grading system integrating cloud-edge collaboration and AI, the system comprising:

[0018] The device layer collects real-time data generated by data generating units, performs preliminary data processing and transmits it to the edge layer, wherein the data generating units include at least IoT devices, sensors, and mobile devices; the preliminary data processing performs simple data processing tasks through embedded AI algorithms;

[0019] The edge layer deploys segmented and tailored pattern matching models, receives data from the device layer, and performs rapid data processing and response;

[0020] The cloud layer, which consists of a big data platform, a cloud computing platform, and an artificial intelligence platform, receives data from the edge layer and performs data processing functions, including at least: global data analysis, intelligent classification and grading, model training and optimization, and data management and security;

[0021] The cloud-edge collaborative AI layer realizes cloud-edge collaborative AI through incremental learning, federated learning, and joint reasoning.

[0022] Furthermore, the system also includes:

[0023] The data source management module completes the data scanning, obtains real-time data from multiple data sources by parsing different data formats and protocols, and finally obtains features for classification and grading through data preprocessing and data feature extraction;

[0024] Policy management module, which manages the policies, rules and algorithms for data classification and grading;

[0025] Data classification module, which completes data classification, including: classification of structured data based on regular expressions, keywords, and dictionaries, and classification of unstructured data based on machine learning algorithms. The classification results include at least: public data, internal data, and sensitive data;

[0026] Data classification module completes data classification and classifies data according to the degree of data impact. The classification results include at least level 1, level 2, level 3, and level 4, among which level 1 has the lowest importance and potential impact.

[0027] Data visualization and reporting module, which performs data visualization and report generation. The displayed data and reports include at least: data classification and grading, data distribution and trend;

[0028] The cloud-edge collaborative processing module provides data interaction links for different types and levels of data transmission and computing needs through the integration of distributed streaming computing engines, time series databases, and lightweight message middleware, thus achieving collaborative work between the cloud and the edge.

[0029] Data security and privacy protection module, which uses encryption algorithms and security protocols to ensure the security and privacy of data during transmission and processing;

[0030] Data access and sharing module, which provides data query interface and sharing mechanism, and users can access data through different terminals and devices;

[0031] User rights management module, based on role-based access control mechanism, limits the access rights of different users to data.

[0032] Furthermore, the present invention provides a storage medium, which is a computer-readable storage medium for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the data classification and grading method of cloud-edge collaboration and AI fusion.

[0033] Furthermore, the present invention provides a computer device, comprising: one or more processors and a memory, wherein the memory stores one or more computer programs and is configured to be executed by the one or more processors, and the one or more computer programs are used to execute the steps of a data classification and grading method that integrates cloud-edge collaboration and AI.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The present invention targets structured data and unstructured data, and implements classification and grading of structured data by utilizing regular expressions, keyword matching, and dictionary matching. It also implements automatic classification and grading of unstructured data in stages by utilizing neural networks and natural language processing technology in combination with unsupervised learning and semi-supervised learning methods, and can more accurately identify sensitive data.

[0036] 2. Through the cloud-edge collaborative mechanism, the present invention can perform rapid data processing and response at the edge layer, while utilizing the powerful computing power of the cloud layer for in-depth analysis and model training, thereby realizing real-time classification and grading and dynamic adjustment of data.

[0037] 2. The present invention supports flexible configuration and dynamic updating of strategies, rules and algorithms for data classification and grading, and can adapt to changing needs of different industries and scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the data classification method of the present invention;

[0039] Figure 2 A flowchart of semi-structured data processing based on a neural network and a natural language processing algorithm of the present invention;

[0040] Figure 3 This is a framework diagram of the data classification system for cloud-edge collaboration and AI fusion of the present invention;

[0041] Figure 4 This is the edge-cloud collaborative AI framework diagram of the present invention;

[0042] Figure 5 This is a module diagram of the data classification system integrating cloud-edge collaboration and AI of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] First, as Figure 1-Figure 5 As shown, the present invention provides a data classification and grading method, including five core parts: formulating structured data, unstructured data scanning rules, formulating data identification rules, formulating industry classification and grading specifications, implementing data classification and grading, and adjusting classification and grading results.

[0045] Specifically, the steps include:

[0046] 1) Based on national laws and regulations, business needs and processes, use scanning rules to extract data closely related to business operations from data assets;

[0047] 2) Develop a data identification rule base based on data characteristics;

[0048] 3) Based on the established rule base, use regular matching, keyword matching, dictionary matching, neural network and natural language processing algorithm models to identify sensitive data from the extracted core data items;

[0049] 4) Formulate scientific and reasonable industry classification and grading standards;

[0050] 5) Based on the established industry classification and grading specifications, automatically classify and grade the identified sensitive data to determine the classification and level of sensitive data;

[0051] 6) In case the automatic classification and grading results are not applicable, manual dynamic adjustment of the classification and grading results is supported.

[0052] Preferably, the above step 1) comprises the following steps:

[0053] 1.1) Analyze national laws and regulations, business needs and process management requirements, and determine the core data items involved in each industry.

[0054] 1.2) The scanning rules used mainly include: scanning method selection, scanning asset type selection, database scanning method selection, file scanning method selection, sample data volume setting, and sample matching rate setting.

[0055] Scan mode selection: full scan, incremental scan;

[0056] Scan asset type selection: database (big data except HDFS and Kafka), file (Linux, Unix, Windows, HDFS, Kafka, etc.);

[0057] Database scanning mode selection: Quick discovery (when identifying content, scan the first 1000 rows, and then scan every 50 rows), precise discovery (custom number of rows);

[0058] File scanning mode selection: Quick discovery (only scan the first 1000 lines of each file, and then scan every 50 lines), precise discovery (scan all the contents of each file);

[0059] Sample data volume setting: Customize the configuration range;

[0060] Sample matching rate setting: configuration range 0-100.

[0061] 1.3) Core data items may include user information, transaction records, geographic location and other sensitive information. Different industries will have their own unique core data items.

[0062] For example, in the medical industry: user information, health records, biometric information, etc.; in the industrial industry: production data, equipment information, operation and maintenance information, etc.

[0063] Preferably, the above step 2) comprises the following steps:

[0064] 2.1) Extract data features based on core data, including: data type, data sensitivity, data usage scenarios, etc.

[0065] 2.2) Based on the data characteristics of the core data, formulate a corresponding identification rule base, including the definition, format, and association relationship of the core data.

[0066] Preferably, the above step 3) comprises the following steps:

[0067] 3.1) Based on the definition, format and association of core data in the rule base, use the preset pattern matching model to identify sensitive data in business data.

[0068] 3.2) Structured data is matched using data recognition rules formulated using algorithms such as regular expressions, keywords, and dictionaries to achieve automatic classification and grading;

[0069] For example, in the medical industry, regular expressions can be used to match patient information, such as social security number, disease name, drug name, etc.; keywords / dictionaries can be used to match disease keywords, drug keywords, etc.; the regular expression for social security number is: \b\d{18}\b; the regular expression for disease name is: \b(disease|symptom|symptom|syndrome|infection|inflammation|tumor|cancer|[A-Za-z]+osis|[A-Za-z]+itis|[A-Za-z]+oma)\b; the regular expression for drug name is: \b([AZ][az]+[0-9]{0,2})\b|\b(hydrochlorothiazide|aspirin|ibuprofen|paracetamol|vitamin[ABCD])\b.

[0070] 3.3) Unstructured data uses neural network and natural language processing technology combined with unsupervised learning and semi-supervised learning methods to achieve automatic classification and grading in stages; Figure 2 As shown, including:

[0071] In the data preparation stage, a large amount of unlabeled production data is collected and preprocessed, including noise removal, word segmentation, and stop word removal;

[0072] In the initial stage, an unsupervised learning method was used to introduce the Deep Adaptive Clustering (DAC) algorithm to cluster a large amount of unlabeled production data, obtain preliminary classification results, and put them into production application;

[0073] In the later stage, some data is selected from the preliminary classification results for manual labeling to obtain a small amount of labeled data. Then, a semi-supervised learning method is adopted, and the semi-supervised support vector machine (S3VM) algorithm is introduced to further optimize the model using a small amount of labeled data and a large amount of unlabeled data; for specific classification tasks, the Transformer-based model is combined for fine-tuning to ensure that the model can capture the complex semantic information in the text data;

[0074] In the optimization phase, the data set is divided into multiple subsets using cross-validation statistical methods, and the model performance is evaluated by training and testing in turn. Based on the evaluation results, the model parameters and configuration are adjusted to ensure good generalization performance; grid search is used to traverse the given parameter combinations to find the parameter configuration that optimizes the model performance.

[0075] For example, in the medical industry, neural networks and natural language processing technologies can be used to classify medical records into different categories, such as diagnostic reports, treatment records, discharge summaries, etc.; identify abnormal areas in medical images, such as lung nodules, fractures, etc.; and extract sentiment tendencies from patient messages and social media comments to help medical institutions understand patient satisfaction.

[0076] Preferably, the above step 4) comprises the following steps:

[0077] 4.1) According to data attributes and confidentiality levels, formulate classification and grading of core data items. In this invention, data is divided into different categories such as public data, internal data, sensitive data, etc. according to possible data leakage channels, whether it involves user privacy, national or regional security and other influencing factors. Data under each category can be further divided into levels of level 1 (lowest level), level 2, level 3, and level 4 (highest level) according to its importance and potential impact.

[0078] Preferably, the above step 5) comprises the following steps:

[0079] 5.1) Based on the classification and grading standards, the identified sensitive data is automatically classified.

[0080] For example, the name, ID number, contact information, etc. of patients in the medical industry are marked as personal information, which is sensitive data and level four; disease diagnosis information, treatment records, medical images, and laboratory test results are marked as health information, which is sensitive data and level three; drug name, dosage, frequency of use, drug allergy history, etc. are marked as drug information, which is sensitive data and level three.

[0081] Second, as Figure 3 As shown, this embodiment provides a data classification system framework that integrates cloud-edge collaboration and AI, including a device layer, an edge layer, a cloud layer, and cloud-edge collaborative AI.

[0082] Specifically, the steps include:

[0083] 1) Processing and transmitting real-time data from various IoT devices, sensors, mobile devices, etc. to the edge layer;

[0084] 2) Receive data from the device layer and perform rapid data processing and response;

[0085] 3) Undertake advanced functions such as in-depth data analysis, model training, and decision support;

[0086] 4) Realize cloud-edge collaborative AI through incremental learning, federated learning, and joint reasoning.

[0087] Preferably, the above step 1) comprises the following steps:

[0088] 1.1) The device layer is responsible for collecting real-time data generated by various IoT devices, sensors, mobile devices, etc., and performing preliminary data processing.

[0089] 1.2) The types of devices are rich, including but not limited to: smart cameras, temperature sensors, industrial controllers, drones, etc.

[0090] 1.3) The preliminary processing mentioned above means that some devices have limited computing power and storage capacity, and can perform simple data processing tasks such as data cleaning and format conversion through embedded AI algorithms to reduce the amount of data transmission and improve data quality.

[0091] Preferably, the above step 2) comprises the following steps:

[0092] 2.1) The edge layer is used to receive data from the device layer and perform rapid data processing and response, mainly including model segmentation, model pruning, model reasoning, and model fine-tuning.

[0093] 2.2) The model segmentation divides the machine learning model so that tasks with large computational workload are processed at the edge, while tasks with small computational workload are processed on the terminal device to reduce latency.

[0094] 2.3) The model pruning reduces the model size and adapts to the computing and storage limitations of edge devices by adopting techniques such as network pruning, knowledge distillation, parameter quantization and structural optimization.

[0095] 2.4) The model reasoning deploys the trained model to the edge layer so that real-time reasoning can be performed close to the data source to reduce data transmission.

[0096] 2.5) The model fine-tuning is to fine-tune the model sent from the cloud through local data to adapt to specific regional or scenario requirements and realize the personalization and optimization of the model.

[0097] Preferably, the above step 3) comprises the following steps:

[0098] 3.1) The cloud layer is composed of a big data platform, a cloud computing platform, and an artificial intelligence platform. It can receive data from the edge layer and perform tasks such as global data analysis, intelligent classification and grading, model training and optimization, data management and security. The cloud layer is responsible for functions such as in-depth data analysis, model training, and decision support.

[0099] 3.2) Global data analysis: using the powerful computing and storage capabilities of the cloud layer to deeply explore the data transmitted by the edge layer, including data mining, machine learning model training, etc., to extract valuable information and knowledge.

[0100] 3.3) Intelligent classification and grading: With the help of AI technologies such as deep learning and machine learning, data can be accurately classified and graded to provide strong support for data applications.

[0101] 3.4) Model training and optimization, including model selection, parameter adjustment, performance evaluation and other steps, aiming to improve the accuracy and generalization ability of the model.

[0102] 3.5) Model storage and distribution: Store the trained model in the cloud and use it as a central node to push the model to the edge layer.

[0103] 3.6) Data management and security services, including data encryption, access control, data backup and other measures to ensure data security and integrity.

[0104] Preferably, the above step 4) comprises the following steps:

[0105] Improve model effects through incremental learning technology, protect data privacy through federated learning technology, and optimize reasoning performance through joint reasoning technology, thereby realizing cloud-edge collaborative AI. Figure 4 As shown;

[0106] 4.1) Incremental learning technology refers to the real-time monitoring of new data generated by the device layer at the edge layer for training, evaluation, and deployment, without manual intervention, to achieve automated continuous improvement of model effects and improve the efficiency and timeliness of model updates;

[0107] 4.2) The federated learning technology is to obtain a model through joint training of edge layer data and upload it to the cloud for aggregation, which not only ensures data privacy and efficient transmission, but also provides a faster convergence speed, higher accuracy and richer scenario aggregation algorithm to complete the joint training of the model;

[0108] 4.3) The joint reasoning technology is to detect the more difficult reasoning samples on the edge and send them to the cloud for reasoning. Reasoning simpler samples on the edge guarantees latency and throughput, while reasoning more complex samples on the cloud improves overall accuracy.

[0109] In the third aspect, the data classification and grading system of cloud-edge collaboration and AI fusion provided by the present invention includes: a data source management module, a policy management module, a data classification module, a data grading module, a data grading visualization and reporting module, a cloud-edge collaborative processing module, a data security and privacy protection module, a data access and sharing module, and a user authority management module;

[0110] The data source management module is subdivided into: a data acquisition module, a data preprocessing module and a feature extraction module;

[0111] The data acquisition module supports real-time or batch acquisition of raw data from multiple data sources through a variety of data formats and protocols;

[0112] Preferably, the multiple data formats include: HTTP / HTTPS, MQTT, CoAP, etc.;

[0113] Preferably, the multi-party data sources include: sensors, databases, Internet of Things devices, etc.;

[0114] The data preprocessing module performs preprocessing operations on the collected raw data to improve the quality and usability of the data;

[0115] Preferably, the pre-processing operations include: cleaning, deduplication, formatting, etc.;

[0116] The feature extraction module extracts features for classification and grading from the data;

[0117] Preferably, these characteristics may include the type, format, source, timestamp, content attributes, etc. of the data;

[0118] The policy management module supports flexible configuration and dynamic updating of policies, rules and algorithms for data classification and grading;

[0119] Preferably, the strategies and rules include data identification rules, data dictionary rules, data type rules, and data label rules;

[0120] The data classification module classifies the data based on regular expressions, keywords, dictionaries, machine learning and other methods;

[0121] Preferably, the regular expressions, keywords, and dictionaries are used to process classification of structured data;

[0122] Preferably, the machine learning is used for unstructured data, using neural networks and natural language processing (NLP) combined with unsupervised learning and semi-supervised learning methods to achieve classification in stages;

[0123] The data classification module classifies the data according to factors such as the sensitivity, importance or impact range of the data;

[0124] The data visualization and reporting module provides data visualization and reporting functions, allowing intuitive understanding of data classification and grading, as well as data distribution and trends;

[0125] The cloud-edge collaborative processing module provides diversified data interaction links for different types and levels of data transmission and computing needs through the integration and innovation of distributed streaming computing engines, time series databases, and lightweight message middleware, thus realizing collaborative work between the cloud and the edge.

[0126] The data security and privacy protection module adopts advanced encryption algorithms and security protocols to ensure the security and privacy of data during transmission and processing;

[0127] Further, the encryption algorithms include: AES, RSA;

[0128] The data access and sharing module provides a data query interface and sharing mechanism, supporting users to access data through different terminals and devices;

[0129] The user rights management module uses a role-based access control (RBAC) mechanism to limit the access rights of different users to data.

[0130] In a fourth aspect, the present invention provides a computer-readable storage medium storing one or more programs;

[0131] The one or more programs include instructions;

[0132] Preferably, when the instructions are executed by a computing device, the computing device is used to execute the steps of implementing the above-mentioned data classification and grading method.

[0133] In a fifth aspect, the present invention provides a computer device, characterized by comprising: one or more processors and a memory;

[0134] The memory stores one or more computer programs and is configured to be executed by the one or more processors;

[0135] Preferably, the one or more computer programs are used to execute the steps of implementing the above-mentioned data classification and grading method.

[0136] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data classification and grading method integrating cloud-edge collaboration and AI, characterized by: The classification and grading method comprises: Data scanning: extracting core data items related to business operations from data assets based on preset scanning rules; Data classification and grading: extract data features based on the scanned core data items. The data features include at least: data type, data sensitivity, and data usage scenario. Use a preset pattern matching model to identify sensitive data in business data and further implement data classification and grading. The preset pattern matching model includes: a structured data matching model and an unstructured data matching model. The structured data matching model uses at least regular expressions, keywords, and dictionary algorithms to match sensitive data and classify and grade data based on preset rules. The unstructured data matching model uses neural networks and natural language processing technology to build a data classification and grading model to achieve classification and grading of unstructured data. Adjustment of classification and grading results. For situations where automatic classification and grading results are not applicable, manual dynamic adjustments are made. At the same time, user messages and social media comment information are collected to extract sentiment tendencies, provide feedback on classification and grading results, and adjust the preset pattern matching model.

2. The data classification and grading method for cloud-edge collaboration and AI fusion according to claim 1 is characterized by: The data scanning rules perform data scanning according to a preset scanning method, determine the core data items of the industry based on the prior knowledge of industry experts according to national laws and regulations, business needs and process management requirements.

3. The data classification and grading method for cloud-edge collaboration and AI fusion according to claim 1 is characterized by: The unstructured data matching model also combines unsupervised learning and semi-supervised learning methods to perform data analysis in stages, including: In the data preparation phase, unlabeled production data is collected and preprocessed. The preprocessing includes at least noise removal, word segmentation, and stop word removal. In the initial stage, an unsupervised learning method was used to introduce a deep adaptive clustering algorithm to cluster unlabeled production data, obtain preliminary classification results, and put them into production application; In the later stage, some data are selected from the preliminary classification results for manual labeling to obtain a small amount of labeled data. Then, a semi-supervised learning method is adopted, and a semi-supervised support vector machine algorithm is introduced to further optimize the model using a small amount of labeled data and a large amount of unlabeled data. For specific classification tasks, the Transformer-based model is combined for fine-tuning to ensure that the model can capture the complex semantic information in the text data. In the optimization phase, the data set is divided into multiple subsets using cross-validation statistical methods, and training and testing are performed alternately to evaluate model performance and adjust model parameters and configurations. Grid search is used to traverse given parameter combinations to find the parameter configuration that optimizes model performance.

4. The data classification and grading method for cloud-edge collaboration and AI fusion according to claim 1 is characterized by: The results of data classification and grading include at least three categories: public data, internal data, and sensitive data, and four levels in each category: level one, level two, level three, and level four. The level one has the lowest importance and potential impact.

5. A system using the data classification and grading method of cloud-edge collaboration and AI fusion according to any one of claims 1 to 4, characterized in that: The system includes: The device layer collects real-time data generated by data generating units, performs preliminary data processing and transmits it to the edge layer, wherein the data generating units include at least IoT devices, sensors, and mobile devices; the preliminary data processing performs simple data processing tasks through embedded AI algorithms; The edge layer deploys segmented and tailored pattern matching models, receives data from the device layer, and performs rapid data processing and response; The cloud layer, which consists of a big data platform, a cloud computing platform, and an artificial intelligence platform, receives data from the edge layer and performs data processing functions, including at least: global data analysis, intelligent classification and grading, model training and optimization, and data management and security; The cloud-edge collaborative AI layer realizes cloud-edge collaborative AI through incremental learning, federated learning, and joint reasoning.

6. The system of the data classification and grading method of cloud-edge collaboration and AI fusion according to claim 5 is characterized by: The system also includes: The data source management module completes the data scanning, obtains real-time data from multiple data sources by parsing different data formats and protocols, and finally obtains features for classification and grading through data preprocessing and data feature extraction; Policy management module, which manages the policies, rules and algorithms for data classification and grading; Data classification module, which completes data classification, including: classification of structured data based on regular expressions, keywords, and dictionaries, and classification of unstructured data based on machine learning algorithms. The classification results include at least: public data, internal data, and sensitive data; Data classification module completes data classification and classifies data according to the degree of data impact. The classification results include at least level 1, level 2, level 3, and level 4, among which level 1 has the lowest importance and potential impact. Data visualization and reporting module, which performs data visualization and report generation. The displayed data and reports include at least: data classification and grading, data distribution and trend; The cloud-edge collaborative processing module provides data interaction links for different types and levels of data transmission and computing needs through the integration of distributed streaming computing engines, time series databases, and lightweight message middleware, thus achieving collaborative work between the cloud and the edge. Data security and privacy protection module, which uses encryption algorithms and security protocols to ensure the security and privacy of data during transmission and processing; Data access and sharing module, which provides data query interface and sharing mechanism, and users can access data through different terminals and devices; User rights management module, based on role-based access control mechanism, limits the access rights of different users to data.

7. Storage medium, the storage medium is a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the data classification and grading method for cloud-edge collaboration and AI fusion as described in any one of claims 1-4.

Images