Multi-source heterogeneous data automatic classification system and method
By designing an automatic classification system for multi-source heterogeneous data, using a combination of corpus loading word segmentation, feature extraction, classification decision-making and rule engines, the problems of low efficiency and poor accuracy in multi-source heterogeneous data classification are solved, and intelligent data classification is realized. ization and self-optimization.
Patent Information
- Application Number
- CN202411577151.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2025-01-14
AI Technical Summary
The prior art has problems of low efficiency and poor accuracy in the classification of multi-source heterogeneous data, and traditional methods require a lot of manual intervention and resource consumption.
An automatic classification system for multi-source heterogeneous data is designed, including a data aggregation module, a data classification module and a data processing module. The system realizes automatic classification and verification of multi-source heterogeneous data through a combination of corpus loading, word segmentation, feature extraction, classification decision-making and rule engines.
It improves the automation level and classification accuracy of data classification tasks, reduces manual intervention and resource consumption, realizes intelligence and self-optimization of data classification, and provides strong data support for the decision-making process.
Smart Images
Figure CN119311877A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of big data analysis, and in particular to a system and method for automatically classifying multi-source heterogeneous data. Background Art
[0002] Data is the cornerstone of the underlying architecture of an enterprise. As the sources, types and quantities of data accumulated by an enterprise continue to increase, the costs incurred by the enterprise on data access, conversion, cleaning and integration are also increasing. The upper-level applications of the system are all heavily dependent on the quantity and quality of the underlying data. Therefore, enterprises have a great demand for high-quality automated data processing.
[0003] For data from different sources and types, the corresponding data processing logic is different due to their different internal structures and contents. In this context, automatic data recognition and classification at the front end of the high-quality automated data processing pipeline is particularly important.
[0004] At present, data classification of multi-source heterogeneous data is mostly carried out in a fixed level or one-to-one manner. The former is to preset fixed data types in the information system, and the information personnel manually determine the data attribution and then forward it to the corresponding category; the latter is to directly set up a completely fixed data processing channel for each type of data one-to-one, similar to an exclusive channel. The fixed level method is essentially a manual judgment operation, while the one-to-one method requires several times the resources to build and maintain, and it has high requirements for collaborative data sources and data purity. However, in real business, multiple data sources are often gathered at the same time, so the need for data classification cannot be avoided. Summary of the invention
[0005] In order to solve the above technical problems, the present invention provides a multi-source heterogeneous data automatic classification system; on the other hand, it also provides a multi-source heterogeneous data automatic classification method.
[0006] The technical problem solved by the present invention can be achieved by adopting the following technical solutions:
[0007] The first aspect of the present invention is to provide a multi-source heterogeneous data automatic classification system, comprising:
[0008] A data aggregation module is used to aggregate data from different data sources in a multi-source heterogeneous environment to form multi-source heterogeneous data;
[0009] A data classification module, connected to the data aggregation module, for classifying the multi-source heterogeneous data to obtain a first classification result, and verifying the first classification result to obtain a second classification result;
[0010] A data processing module is connected to the data classification module, and is used to process the multi-source heterogeneous data according to the second classification result, and normalize the processed multi-source heterogeneous data and store them in a database.
[0011] Preferably, the data aggregation module includes:
[0012] A database access unit, used to configure and connect different types of data sources to be accessed, so as to obtain the original data under each of the data sources as the multi-source heterogeneous data;
[0013] The file access unit is used to access files of different formats and save the original data of the accessed files as the multi-source heterogeneous data.
[0014] Preferably, the data classification module comprises:
[0015] A corpus loading and word segmentation unit is used to load the corpus of the multi-source heterogeneous data and perform word segmentation processing;
[0016] A feature extraction unit, connected to the corpus loading word segmentation unit, is used to convert the words after word segmentation processing into a high-dimensional feature vector;
[0017] a classification decision unit, connected to the feature extraction unit, and configured to classify the feature vector based on a pre-established classification model to obtain the first classification result, wherein the first classification result includes the data category to which the feature vector belongs;
[0018] A rule engine unit is connected to the classification decision unit, and the rule engine unit is pre-installed with a verification rule set established based on industry data characteristics and data categories corresponding to the verification rules, and is used to select verification rules from the verification rule set according to the first classification result to combine into a target verification rule set, verify the feature vector according to the target verification rule set, and when the verification result matches the first classification result, output the first classification result as the second classification result.
[0019] Preferably, the corpus loading word segmentation unit includes:
[0020] The first word segmentation subunit is used to segment the Chinese text using a word segmentation tool, decompose continuous sentences into independent words, and obtain a first word segmentation result;
[0021] The second word segmentation subunit is used to segment the English text to obtain a second word segmentation result;
[0022] A filtering subunit is connected to the first word segmentation subunit and the second word segmentation subunit respectively, and is used to filter the words in the first word segmentation result and the second word segmentation result based on a preset stop word library.
[0023] Preferably, the classification model is a supervised learning classification model or a deep learning classification model.
[0024] Preferably, for the same data category, the corresponding verification rules include one or more;
[0025] One or more of the verification rules are constructed respectively according to the structural features, field features and file features corresponding to the data in the different data sources.
[0026] Preferably, the rule engine unit is also used to determine that the verification result matches the first classification result when the feature vector satisfies any verification rule of the target verification rule set; and to determine that the verification result does not match the first classification result when the feature vector does not satisfy all verification rules of the target verification rule set.
[0027] Preferably, the rule engine unit is further configured to feed back the verification result and the first classification result to relevant personnel for processing when the verification result does not match the first classification result, and output the second classification result according to the processing result of the relevant personnel.
[0028] Preferably, the data processing module includes:
[0029] A logic processing unit, configured to call a corresponding processing program according to the first classification result, process the multi-source heterogeneous data, and normalize the processed multi-source heterogeneous data and store them in a database;
[0030] The visualization statistics unit is used to visualize and count the multi-source heterogeneous data that have been stored in the database.
[0031] A second aspect of the present invention is to provide a method for automatic classification of multi-source heterogeneous data, which is applied to the multi-source heterogeneous data automatic classification system as described above, and the method comprises:
[0032] Step S1, aggregating data from different data sources in a multi-source heterogeneous environment to form multi-source heterogeneous data;
[0033] Step S2, classifying the multi-source heterogeneous data to obtain a first classification result, and verifying the first classification result to obtain a second classification result;
[0034] Step S3: Process the multi-source heterogeneous data according to the second classification result, and normalize the processed multi-source heterogeneous data and store them in a database.
[0035] The advantages or beneficial effects of the technical solution of the present invention are:
[0036] The present invention improves the automation level and classification accuracy of classification tasks through a verification mechanism for classification results of multi-source heterogeneous data, solves the problems of low efficiency and poor accuracy in traditional data classification, and has a high degree of automation, strong flexibility, and is easy to expand and customize, thereby realizing intelligent and self-optimization of data classification and providing strong data support for the decision-making process. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 It is a structural block diagram of a multi-source heterogeneous data automatic classification system in a preferred embodiment of the present invention;
[0038] Figure 2 This is a structural diagram of a corpus loading word segmentation unit in a preferred embodiment of the present invention;
[0039] Figure 3 The figure is a flow chart of a method for automatic classification of multi-source heterogeneous data in a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0042] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.
[0043] In a preferred embodiment of the present invention, based on the above problems existing in the prior art, a multi-source heterogeneous data automatic classification system is provided. Figure 1 As shown, including:
[0044] A data aggregation module 1 is used to aggregate data from different data sources in a multi-source heterogeneous environment to form multi-source heterogeneous data;
[0045] The data classification module 2 is connected to the data aggregation module 1 and is used to classify the multi-source heterogeneous data to obtain a first classification result, and verify the first classification result to obtain a second classification result;
[0046] The data processing module 3 is connected to the data classification module 2 and is used to process the multi-source heterogeneous data according to the second classification result, and normalize the processed multi-source heterogeneous data and store them in the database.
[0047] Specifically, in response to the problems of manual classification and high-cost exclusive channels in existing data processing pipelines, in this embodiment, data is collected and integrated from multi-source heterogeneous environments through the data aggregation module 1, which can access multiple data sources including but not limited to databases, file systems, etc.; the data classification module 2 intelligently classifies the aggregated data based on predefined classification rules combined with machine learning models. This process is divided into three sub-steps: feature extraction, classification decision, and rule engine; the data processing module 3 further processes and utilizes the classified data, including but not limited to data conversion, aggregation, analysis, and visualization.
[0048] Through the close collaboration of the three modules, data aggregation module 1, data classification module 2 and data processing module 3, the automation level and classification accuracy of the classification task are effectively improved, and the problems of low efficiency and poor accuracy in traditional data classification are solved. It has a high degree of automation, strong flexibility, and is easy to expand and customize. Relying on intelligent processing, it realizes the intelligence and self-optimization of data classification and enhances decision support.
[0049] As a preferred implementation, the data aggregation module 1 includes:
[0050] The database access unit 11 is used to configure and connect different types of data sources to be accessed, so as to obtain the original data under each data source as multi-source heterogeneous data;
[0051] The file access unit 12 is used to access files of different formats and save the original data of the accessed files as multi-source heterogeneous data.
[0052] Specifically, considering that enterprise data comes from different business systems and databases, the sources, types and quantities of data accumulated by enterprises are constantly increasing. In this embodiment, through the built-in database adapter, it supports the exploration of metadata fields of different database types and the extraction of sample data, and realizes access to the data source to obtain the database, schema information, and field information of the library table under the data source.
[0053] The data aggregation module 1 also supports the input of large batches of file data to adapt to the continuous expansion of the enterprise data scale. The file access unit 12 can access files in various formats, such as comma-separated values (CSV) file format, EXCEL table format, etc. After accessing the file, the original file data of the accessed file data is saved to deal with possible exceptions during data processing, ensuring that even if a data processing error occurs, it can be recovered from the original data to avoid unnecessary losses caused by data loss.
[0054] As a preferred embodiment, the data classification module 2 includes:
[0055] The corpus loading and word segmentation unit 21 is used to load the corpus of multi-source heterogeneous data and perform word segmentation processing;
[0056] The feature extraction unit 22 is connected to the corpus loading word segmentation unit 21, and is used to convert the words after word segmentation processing into a high-dimensional feature vector;
[0057] A classification decision unit 23, connected to the feature extraction unit 22, is used to classify the feature vector based on a pre-established classification model to obtain a first classification result, wherein the first classification result includes the data category to which the feature vector belongs;
[0058] The rule engine unit 24 is connected to the classification decision unit 23. The rule engine unit is pre-installed with a verification rule set based on industry data characteristics and data categories corresponding to the verification rules. The rule engine unit is used to select verification rules from the verification rule set according to the first classification result to combine into a target verification rule set, verify the feature vector according to the target verification rule set, and output the first classification result as the second classification result when the verification result matches the first classification result.
[0059] Specifically, first, the data is effectively imported into the corpus loading and word segmentation unit 21 for processing, and a high-performance data reading technology is used to load the corpus of multi-source heterogeneous data to ensure data integrity and efficiency, and then perform word segmentation processing on it.
[0060] The feature extraction unit 22 uses word embedding technology, such as Word2Vec, to capture the semantic relationship and contextual dependency between words, and maps the words into a high-dimensional vector space, providing rich and meaningful input for the training of the machine learning model.
[0061] The words captured in the feature extraction unit belong to the field words of professional data tables in various industries. For example, for the amount and amount direction fields in the transaction table field set, there is an intrinsic connection between the fields. These feature vectors not only reflect the meaning of the words themselves, but also contain the association information between the words, which helps to improve the accuracy of classification.
[0062] A classification model is pre-established in the classification decision unit 23. The classification model adopts a supervised learning algorithm, such as a distributed gradient boosting (XGBoost) framework, or a deep learning algorithm, such as a text convolutional neural network (textCNN). The classification model can determine the data category to which the feature vector to be classified belongs based on the information of the feature vector to be classified, such as whether it belongs to amount data or information data.
[0063] The rule engine unit 24 uses engineering software methods to further verify the classification results output by the classification model based on the characteristics of industry data. For example, the characteristics of industry data mainly include the characteristics of fund amount data, such as amount type, balance type, transaction direction type, etc. Combining them can define a transaction record as valid. The rule engine unit 24 verifies the first classification result output by the classification model through one or more combinations of verification rules for the corresponding data categories. When the verification result matches the first classification result, the first classification result is output as the second classification result, that is, the finally determined data category, ensuring the accuracy and reliability of the classification result.
[0064] As a preferred embodiment, where Figure 2 As shown, the corpus loading and word segmentation unit 21 includes:
[0065] The first word segmentation subunit 211 is used to cut the Chinese text using a word segmentation tool, decompose continuous sentences into independent words, and obtain the first word segmentation result;
[0066] The second word segmentation subunit 212 is used to segment the English text to obtain the second word segmentation result;
[0067] The filtering subunit 213 is respectively connected to the first word segmentation subunit 211 and the second word segmentation subunit 212, and is used to filter the words in the first word segmentation result and the second word segmentation result based on a preset stop word library.
[0068] Specifically, the first word segmentation subunit 211 uses a mature word segmentation tool, such as jieba word segmentation, for fine-grained cutting of the Chinese text, and decomposes continuous sentences into unit words with independent meanings.
[0069] At the same time, the second word segmentation subunit 212 uses, such as NLTK word segmentation, for simple segmentation of the English text.
[0070] To ensure the accuracy of the word segmentation results, a stop word filtering strategy is introduced after word segmentation. Using a predefined or custom stop word library, common but less significant words in the text are removed, such as Chinese stop words like "and", "of", etc. and common conjunctions and prepositions in English, thereby reducing noise interference in subsequent processing and enhancing the pertinence and effectiveness of feature extraction.
[0071] As a preferred embodiment, the classification model is a supervised learning classification model or a deep learning classification model.
[0072] Specifically, the classification decision unit 23 also supports a continuous learning mechanism and can continuously optimize the classification model according to the feedback of new data to adapt to the dynamic changes of data characteristics.
[0073] As a preferred implementation, for the same data category, the corresponding verification rules include one or more;
[0074] One or more validation rules are constructed respectively according to structural features, field features and file features corresponding to data in different data sources.
[0075] Specifically, for the same data category, such as amount data or information data, there may be different ways of expression due to different data sources. These differences are mainly reflected in the structural characteristics, field characteristics and file characteristics of the data.
[0076] In this embodiment, when formulating the verification rules, these characteristics are fully considered and a comprehensive strategy is adopted to formulate the verification rules. Specifically, by comprehensively and carefully summarizing the structural characteristics, field characteristics and file characteristics corresponding to each data category in different data sources to cover various possible data expressions, verification rules applicable to each data source are formulated;
[0077] In the rule engine unit 24, based on the characteristics of industry data, the data processed by the classification decision unit 23 is further verified and fine-tuned to improve the accuracy of data classification. For example, for amount data, comprehensive verification is performed based on the structural characteristics and field characteristics of the data table to ensure that key information such as the amount, balance, transaction direction, etc. of the data is accurate. For information data, verification is performed based on the file characteristics of the data table, such as the file format, encoding, etc.
[0078] Furthermore, the rule engine unit 24 may perform verification on all data categories, or re-verify only some data categories, such as data categories that are difficult to accurately classify through machine learning.
[0079] Specifically, when the classification model determines that the data category belongs to the preset verification type, such as amount data and information data, it is input into the rule engine unit 24 for verification, so as to improve the accuracy of classification with the help of manually formulated rules.
[0080] As a preferred embodiment, the rule engine unit 24 is also used to determine that the verification result matches the first classification result when the feature vector satisfies any verification rule of the target verification rule set; and to determine that the verification result does not match the first classification result when the feature vector does not satisfy all verification rules of the target verification rule set.
[0081] Specifically, in this embodiment, for each data category in the first classification result, a target verification rule set is formed respectively. The target verification rule set may include multiple verification rules, and the multiple verification rules are different expression modes for the same data category in different data sources.
[0082] When the feature vector to be classified satisfies any one of the verification rules in the target verification rule set, it means that the verification is passed, that is, the verification result matches the first classification result.
[0083] When the feature vector to be classified does not satisfy all the verification rules in the target verification rule set, it means that the verification fails, that is, the verification result does not match the first classification result.
[0084] As a preferred embodiment, the rule engine unit 24 is also used to feed back the verification result and the first classification result to relevant personnel for processing when the verification result does not match the first classification result, and output the second classification result according to the processing result of the relevant personnel.
[0085] Specifically, when the feature vector to be classified does not satisfy all the verification rules of the target verification rule set, the rule engine unit 24 determines that the verification result does not match the first classification result, then the verification result output by the rule engine unit 24 and the first classification result output by the classification decision unit 23 are displayed to the top level and submitted to relevant personnel for manual judgment and selection, to ensure that more accurate and reliable judgments can be made when faced with complex or ambiguous data.
[0086] The rule engine unit 24 may output a second classification result according to the manually selected data category.
[0087] As a preferred embodiment, the data processing module 3 includes:
[0088] The logic processing unit 31 is used to call the corresponding processing program according to the first classification result, process the multi-source heterogeneous data, and normalize the processed multi-source heterogeneous data and store them in the database;
[0089] The visualization statistics unit 32 is used to count the stored and stored multi-source heterogeneous data in a visualized form.
[0090] In this embodiment, a plurality of processing programs are pre-installed in the logic processing unit 31, and different processing logics are provided for different data categories and formats, supporting functions including but not limited to data conversion, aggregation and analysis.
[0091] Specifically, in this embodiment, the logic processing unit 31 calls the corresponding processing program to process the data according to the data category finally obtained by the data classification module 2. After being processed by the logic processing unit 31, the multi-source heterogeneous data will be converted into a unified and standardized form for subsequent storage and analysis. The processed data will be sent to the database for storage to provide support for subsequent data applications.
[0092] The visualization statistics unit 32 is used to count the stored data in a visualized form, such as a bar chart, a line chart, a pie chart, etc., to intuitively display the distribution, trend, and association of the data, thereby improving the readability of the data.
[0093] The present invention also provides a multi-source heterogeneous data automatic classification method, which is applied to the multi-source heterogeneous data automatic classification system as described above. Figure 3 As shown, the method includes:
[0094] Step S1, aggregating data from different data sources in a multi-source heterogeneous environment to form multi-source heterogeneous data;
[0095] Step S2, classifying the multi-source heterogeneous data to obtain a first classification result, and verifying the first classification result to obtain a second classification result;
[0096] Step S3, processing the multi-source heterogeneous data according to the second classification result, and normalizing the processed multi-source heterogeneous data and storing them in a database.
[0097] Specifically, in order to solve the problems of manual classification and high-cost exclusive channels in existing data processing pipelines, an embodiment of the present invention provides a method for automatic classification of multi-source heterogeneous data, which provides strong support and assistance for enterprises' high-quality data processing pipelines from aspects such as classification models and verification rules, combined with task scheduling of processing programs.
[0098] The method of the embodiment of the present invention first aggregates data from different data sources in a multi-source heterogeneous environment. These data sources can be adapted databases or batch file aggregations. The system receives or directly connects the data by transferring these data sources, thereby forming multi-source heterogeneous data.
[0099] Next, a dedicated high-performance reading module is used to read and load the aggregated multi-source heterogeneous data. After successful loading, the data is segmented in Chinese and English to extract key information. After the segmentation is completed, the mainstream word embedding model is used to perform word vectorization to convert the text data into a numerical vector that can be recognized by a computer.
[0100] Then, the vectorized feature vector data is input into classification models such as textCNN or xgboost for multiple rounds of training. These models continuously optimize the classification performance through iterative learning until the requirements are met. It is worth noting that the model is not finalized once, but can be iterated and dynamically updated in multiple rounds according to actual conditions.
[0101] After the classification is completed, the first classification result is obtained. In order to further improve the accuracy of the classification, the rule engine is used to verify the first classification result. If the verification result is consistent with the first classification result, the final conclusion is drawn; if it is inconsistent, it is directly passed to the top layer for user identification, so as to facilitate manual intervention and adjustment.
[0102] Finally, according to the second classification result (i.e. the verified classification result), the corresponding processing program is called to process the multi-source heterogeneous data. After the processing is completed, the standardized data is stored in the database for subsequent data analysis and application.
[0103] In addition, in order to intuitively display the effect of data processing, the method of the present invention also provides a visualization statistical function. After the data is successfully processed, the visualization statistical results of the current round of data processing, such as the distribution and trend of the data, are displayed so that users can better understand the data situation.
[0104] The advantages or beneficial effects of adopting the above technical solution are: the present invention improves the automation level and classification accuracy of classification tasks through the verification mechanism of the classification results of multi-source heterogeneous data, solves the problems of low efficiency and poor accuracy in traditional data classification, and has a high degree of automation, strong flexibility, easy expansion and customization, realizes intelligent and self-optimization of data classification, and provides strong data support for the decision-making process.
[0105] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of this specification and illustrations should be included in the protection scope of the present invention.
Claims
1. A multi-source heterogeneous data automatic classification system, characterized by: include: A data aggregation module is used to aggregate data from different data sources in a multi-source heterogeneous environment to form multi-source heterogeneous data; A data classification module, connected to the data aggregation module, is used to classify the multi-source heterogeneous data to obtain a first classification result, and verify the first classification result to obtain a second classification result; A data processing module is connected to the data classification module, and is used to process the multi-source heterogeneous data according to the second classification result, and normalize the processed multi-source heterogeneous data and store them in a database.
2. The multi-source heterogeneous data automatic classification system according to claim 1 is characterized in that: The data aggregation module includes: A database access unit, used to configure and connect different types of data sources to be accessed, so as to obtain the original data under each of the data sources as the multi-source heterogeneous data; The file access unit is used to access files of different formats and save the original data of the accessed files as the multi-source heterogeneous data.
3. The multi-source heterogeneous data automatic classification system according to claim 1 is characterized in that: The data classification module comprises: A corpus loading and word segmentation unit is used to load the corpus of the multi-source heterogeneous data and perform word segmentation processing; A feature extraction unit, connected to the corpus loading word segmentation unit, is used to convert the words after word segmentation processing into a high-dimensional feature vector; a classification decision unit, connected to the feature extraction unit, and configured to classify the feature vector based on a pre-established classification model to obtain the first classification result, wherein the first classification result includes the data category to which the feature vector belongs; A rule engine unit is connected to the classification decision unit, and the rule engine unit is pre-installed with a verification rule set established based on industry data characteristics and data categories corresponding to the verification rules, and is used to select verification rules from the verification rule set according to the first classification result to combine into a target verification rule set, verify the feature vector according to the target verification rule set, and when the verification result matches the first classification result, output the first classification result as the second classification result.
4. The multi-source heterogeneous data automatic classification system according to claim 3 is characterized in that: The corpus loading word segmentation unit comprises: The first word segmentation subunit is used to segment the Chinese text using a word segmentation tool, decompose continuous sentences into independent words, and obtain a first word segmentation result; The second word segmentation subunit is used to segment the English text to obtain a second word segmentation result; A filtering subunit is connected to the first word segmentation subunit and the second word segmentation subunit respectively, and is used to filter the words in the first word segmentation result and the second word segmentation result based on a preset stop word library.
5. The multi-source heterogeneous data automatic classification system according to claim 3 is characterized in that: The classification model is a supervised learning classification model or a deep learning classification model.
6. The multi-source heterogeneous data automatic classification system according to claim 3 is characterized in that: For the same data category, the corresponding verification rules include one or more; One or more of the verification rules are constructed respectively according to the structural features, field features and file features corresponding to the data in the different data sources.
7. The multi-source heterogeneous data automatic classification system according to claim 3 is characterized in that: The rule engine unit is also used to determine that the verification result matches the first classification result when the feature vector satisfies any verification rule of the target verification rule set; and to determine that the verification result does not match the first classification result when the feature vector does not satisfy all verification rules of the target verification rule set.
8. The multi-source heterogeneous data automatic classification system according to claim 3 is characterized in that: The rule engine unit is also used to feed back the verification result and the first classification result to relevant personnel for processing when the verification result does not match the first classification result, and output the second classification result according to the processing result of the relevant personnel.
9. The multi-source heterogeneous data automatic classification system according to claim 1 is characterized in that: The data processing module comprises: A logic processing unit, configured to call a corresponding processing program according to the first classification result, process the multi-source heterogeneous data, and normalize the processed multi-source heterogeneous data and store them in a database; The visualization statistics unit is used to visualize and count the multi-source heterogeneous data that have been stored in the database.
10. A method for automatic classification of multi-source heterogeneous data, applied to the automatic classification system for multi-source heterogeneous data as claimed in any one of claims 1 to 9, the method comprising: Step S1, aggregating data from different data sources in a multi-source heterogeneous environment to form multi-source heterogeneous data; Step S2, classifying the multi-source heterogeneous data to obtain a first classification result, and verifying the first classification result to obtain a second classification result; Step S3: Process the multi-source heterogeneous data according to the second classification result, and normalize the processed multi-source heterogeneous data and store them in a database.