Intelligent data lake management and optimization system and method based on AI drive
Through the intelligent data lake management system based on AI, the problems of low efficiency and data isolation in the unstructured data lake are solved, and the automated analysis and classification of unstructured data are realized, which improves data processing efficiency and utilization. It reduces labor costs and promotes business innovation and decision-making optimization of enterprises.
Patent Information
- Application Number
- CN202411484874.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2025-01-17
AI Technical Summary
Existing data lake management solutions lack automated processing capabilities, especially inefficient and costly problems in unstructured data processing, and data isolation leads to the inability to effectively utilize the potential value of data.
Adopt an intelligent data lake management system based on AI, including data ingestion module, natural language processing module, knowledge graph construction module and cross-origin data association module, through automated data collection, in-depth analysis, entity recognition, knowledge graph construction and cross-origin data association , realize the automatic analysis and classification of unstructured data, and break the phenomenon of data isolation.
It realizes efficient and automated analysis and classification of unstructured data, reduces the need for manual intervention, improves data processing efficiency and utilization, significantly reduces labor costs, and helps enterprises to mine higher business value from massive data.
Smart Images

Figure CN119322874A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent data management and big data processing technology, and in particular to an AI-driven intelligent data lake management and optimization system and method. Background Art
[0002] With the advent of the big data era, the amount of data that enterprises and organizations need to manage and process has exploded, especially unstructured data, such as text, images, videos, audio, and social media data, which has accounted for a large proportion of the total data. In order to effectively store and manage these massive amounts of data, data lake technology has emerged. Data lake is a centralized repository that can store structured, semi-structured, and unstructured data, with advantages such as strong scalability and low storage cost.
[0003] However, the widespread use of data lakes has brought new challenges, mainly reflected in: 1. Data is disordered: Most of the data stored in data lakes are raw data, lacking effective management and classification, which makes it difficult for users to quickly extract useful information from them, resulting in the "data swamp" phenomenon. 2. Unstructured data is difficult to process: Existing technologies still have technical bottlenecks for processing unstructured data such as text, images, and audio, especially in terms of efficient information extraction and data model construction. 3. Poor data relevance: Although data lakes can store various types of data, these data are often isolated from each other and lack structured associations, making it impossible to achieve deep mining and analysis across data sources.
[0004] In the face of the above problems, intelligent data management methods based on artificial intelligence (AI), natural language processing (NLP) and knowledge graphs provide new ideas for solving these problems. Through intelligent data labeling, classification, association and automated information extraction, the management efficiency of the data lake can be greatly improved, and the commercial value of unstructured data can be fully explored.
[0005] Although existing data lake management solutions have alleviated the data storage and management problems to a certain extent, they generally have the following shortcomings: 1. Lack of automated data processing: Most solutions rely on manual configuration and management, especially when processing unstructured data, which requires a lot of manpower for data labeling, classification and archiving, which is inefficient and costly. Existing technologies cannot automatically and intelligently understand and associate data in depth. 2. Imperfect processing of unstructured data: Although some big data platforms have the ability to process unstructured data, they are mainly focused on the data storage and retrieval level, and cannot perform semantic analysis, automatic labeling or knowledge extraction on these data, resulting in the difficulty of effectively using unstructured data in the data lake together with structured data. 3. Data isolation and lack of relevance: Existing data lake management solutions can usually only process data from a single data source, and it is difficult to automatically extract, integrate and associate information from multiple data sources. This data isolation phenomenon greatly limits the potential value mining of data in the data lake. 4. The "data swamp" phenomenon is serious: Due to the lack of effective management and classification mechanisms, a large amount of unprocessed raw data is easily accumulated in the data lake, gradually forming a "data swamp". Although the data in the data lake is stored together, it is difficult for users to quickly retrieve and use it because it cannot be structured, classified or labeled, which weakens the role of the data lake. Summary of the invention
[0006] In response to the above technical problems, the present invention provides an AI-driven intelligent data lake management and optimization system and method.
[0007] The present invention is implemented by adopting the following technical solution: an AI-driven intelligent data lake management and optimization system, comprising: A data ingestion module, used to automatically collect data from multiple data sources, including one or more of a database, a file system, an IOT, and an API interface; Data preprocessing module, used to preprocess the raw data, including cleaning, deduplication, and standardization operations; The natural language processing module is used to conduct in-depth analysis of unstructured text data and extract key business information and entities; The knowledge graph construction module is used to structure entities and relationships in various types of data, helping users quickly establish associations in large-scale data and gain business insights; Data classification and annotation module, which is used to automatically classify and annotate the data entering the data lake to ensure the organization and searchability of the data; The cross-source data association module is used to realize the intelligent association of cross-source data and ensure that data from different sources can be automatically connected together.
[0008] Furthermore, the data ingestion module includes a structured data ingestion submodule and an unstructured data ingestion submodule, wherein: The structured data ingestion submodule is used to automatically ingest structured data by connecting different data sources, and regularly update the content in the data lake using scheduled tasks or streaming processing; The unstructured data ingestion submodule is used to extract unstructured data from the Internet or corporate intranet through web crawler technology, and use content parsing tools to perform basic classification of unstructured data, identify different types of files and import them into the data lake.
[0009] Furthermore, the data preprocessing module includes a structured data preprocessing submodule and an unstructured data preprocessing submodule, wherein: The structured data preprocessing submodule is used to perform data cleaning operations, remove duplicate or redundant data entries, and repair or annotate missing data items. Through data normalization, heterogeneous data is unified into the same format; The unstructured data preprocessing submodule is used to perform word segmentation, stop word filtering, and extract key information through natural language processing technology. Image and audio data are preliminarily classified through deep learning models and multimodal large models to extract basic visual features.
[0010] Furthermore, the natural language processing module includes an entity recognition submodule, a speech detection submodule and a sentiment classification submodule, wherein: The entity recognition submodule is used to perform entity recognition on text data, extract information about names of people, places, and organizations, and determine important relationships in the text through syntactic analysis; The speech detection submodule is used to perform language detection on data from multiple languages and unify them into the target language through a translation model to ensure unified analysis of cross-language data; The sentiment classification submodule is used to classify the sentiment of user comments and feedback text data through the sentiment analysis model, and provide trend analysis and sentiment prediction for the business department.
[0011] Furthermore, the knowledge graph construction module includes a graph construction submodule, a mapping submodule and an interface submodule, wherein: A graph construction submodule is used to construct a knowledge graph based on the extracted entities and relationships after processing the text. The knowledge graph is stored in RDF format and can be queried by SPARQL. The mapping submodule is used to map entities from different data sources to the same vector space using graph embedding technology to ensure accurate association of cross-source data. The interface submodule is used to provide an API interface to support users to query and expand the knowledge graph, quickly find business-related entities and their relationships, and support intelligent search and reasoning functions.
[0012] Furthermore, the data classification and labeling module includes an automatic classification submodule, a labeling submodule and an information generation submodule, wherein: Automatic classification submodule, which uses supervised learning algorithms to classify different types of text, images, and videos based on pre-trained classification models; The annotation submodule is used to identify objects in images and videos through deep learning models and multimodal large models, and to label them specifically. For dynamic video content, the inter-frame difference method is combined with target detection technology to achieve automatic annotation of video content. The information generation submodule is used to generate rich label information for structured data through metadata analysis to ensure the searchability and operability of data in the data lake.
[0013] Furthermore, the cross-source data association module includes a fast query submodule, a data link submodule and an optimization submodule, wherein: The fast query submodule is used to automatically identify the same entity from different sources through the entity matching algorithm in the knowledge graph to ensure the relevance between data; The data linking submodule uses natural language processing technology to analyze entity relationships in text data, find potential connection points in data sources, and link isolated data from different sources together to provide users with more business insights; The optimization submodule is used to support users to define custom rules to associate data from different data sources, or to automatically learn potential association rules between data sources through machine learning models to continuously optimize the association effect.
[0014] An AI-driven intelligent data lake management and optimization method is implemented based on the above-mentioned AI-driven intelligent data lake management and optimization system, and includes the following steps: S1: Intelligent analysis and processing of unstructured data, specifically: for image data, intelligent processing and classification are performed through multimodal large models and deep learning models; for audio data, text conversion, sentiment analysis and semantic understanding are performed through high-precision multilingual speech recognition models; and natural speech models are used to perform preliminary cleaning and preprocessing of unstructured data to form structured data; S2: Intelligent labeling and classification of data lakes; S3: Automatic association of data across sources.
[0015] Furthermore, step S2 includes the following sub-steps: S21: Data feature extraction, using deep learning models to extract features of unstructured data and form a structured representation of the data; S22: Classification model training: training the classification model through supervised learning methods, automatically classifying data according to business needs, and labeling based on specific business rules; S23: Labeling and classification index generation: Generate classification index to facilitate quick retrieval and query.
[0016] Furthermore, step S3 includes the following sub-steps: S31: Entity and relationship extraction, using natural language processing technology to identify entities and their relationships from unstructured data; S32: Knowledge graph construction: construct a knowledge graph based on entities and relationships, map cross-source data into a unified knowledge network, and break the phenomenon of data isolation; S33: Data query and semantic enhancement: Data query is performed through knowledge graphs to support data association and intelligent retrieval in complex business scenarios.
[0017] The beneficial effects of the present invention are: By introducing natural language processing (NLP), multimodal large models and deep learning technology, the present invention realizes the automatic analysis, classification and labeling of unstructured data, greatly reduces manual intervention and improves the efficiency of data processing. The system can automatically extract key information from various types of unstructured data such as text, images, and videos to ensure the speed and accuracy of data analysis.
[0018] The present invention uses automatic labeling, classification and indexing technology to intelligently manage data and avoid disorderly accumulation of data. It also uses knowledge graph technology to automatically associate cross-source data, helping users quickly find required information from massive data, thereby improving data operability and utilization efficiency.
[0019] This invention uses knowledge graph technology to associate data from different sources, build a cross-source knowledge network, and help users mine potential relevance and business insights from data. This method breaks the data isolation phenomenon, enables the data resources in the data lake to be fully utilized, and provides richer information support for corporate decision-making.
[0020] The present invention realizes the automation of data processing through intelligent means, greatly reduces the need for manual intervention, and significantly reduces the labor costs of enterprises in data management and processing. Enterprises can use this technology to achieve efficient management of large-scale data with fewer resources.
[0021] Through intelligent labeling, classification and knowledge graph technology, the present invention can quickly extract key information from the data lake, greatly improve the utilization rate of data, help enterprises to mine higher business value from existing data, and thus promote the business innovation and decision optimization of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0023] Figure 1 is a system block diagram of the present invention; Figure 2 It is a flow chart of the present invention. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0025] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0026] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0027] See also Figure 1 , an AI-driven intelligent data lake management and optimization system, including: data ingestion module, data preprocessing module, natural language processing module, knowledge graph construction module, data classification and annotation module and cross-source data association module.
[0028] The data ingestion module is responsible for automatically collecting data from multiple data sources (such as databases, file systems, IOT, API interfaces, etc.). It supports processing structured, semi-structured, and unstructured data types (such as text, images, audio, etc.). The data ingestion module includes a structured data ingestion submodule and an unstructured data ingestion submodule. The structured data ingestion submodule realizes automatic ingestion of structured data by connecting to different data sources (such as MySQL database, HDFS file system, or NoSQL database), and regularly updates the content in the data lake using scheduled tasks or streaming processing. The unstructured data ingestion submodule extracts unstructured data such as text, HTML page content, or media files from the Internet or corporate intranet through Web crawler technology. Use content parsing tools to perform basic classification of unstructured data, identify different types of files, and import them into the data lake.
[0029] The data preprocessing module is responsible for cleaning, deduplication, and standardization operations, and performs preliminary formatting and block processing on unstructured data (such as text, images, and audio). The data preprocessing module includes a structured data preprocessing submodule and an unstructured data preprocessing submodule. The structured data preprocessing submodule performs data cleaning operations on structured data, removes duplicate or redundant data entries, and repairs or annotates missing data items. Through data normalization, heterogeneous data is unified into the same format. For the unstructured data preprocessing submodule, for unstructured data, text data is segmented and stop words are filtered using natural language processing (NLP) technology, and key information is extracted; image and audio data are preliminarily classified through deep learning models and multimodal large models to extract basic visual features.
[0030] The natural language processing module is used to perform in-depth analysis on unstructured text data and extract key business information and entities. The natural language processing module includes an entity recognition submodule, a speech detection submodule and a sentiment classification submodule. The entity recognition submodule is used to perform entity recognition on text data and extract information such as names of people, places, and organizations; it determines important relationships in the text through syntactic analysis; the speech detection submodule is used to perform language detection on data from multiple languages and unify them into the target language through a translation model to ensure unified analysis of cross-language data; the sentiment classification submodule is used to perform sentiment classification on user comments and feedback text data through a sentiment analysis model, and provide trend analysis and sentiment prediction for the business department.
[0031] The knowledge graph construction module is used to structure the entities and relationships in various types of data, helping users to quickly establish associations in large-scale data and obtain business insights. The knowledge graph construction module includes a graph construction submodule, a mapping submodule and an interface submodule. Among them, the graph construction submodule is used to construct a knowledge graph based on the extracted entities and relationships after processing the text. The knowledge graph is stored in the RDF format and can be queried by SPARQL; the mapping submodule is used to use graph embedding technology to uniformly map entities from different data sources to the same vector space for the association of cross-source data, ensuring accurate association of cross-source data; the interface submodule is used to provide an API interface to support users to query and expand the knowledge graph, quickly find business-related entities and their relationships, and support intelligent search and reasoning functions.
[0032] The data classification and labeling module automatically classifies and labels the data entering the data lake to ensure the organization and searchability of the data. The system automatically analyzes the data type through AI algorithms and generates corresponding labels and classification information. The data classification and labeling module includes automatic classification submodule, labeling submodule and information generation submodule. Among them, the automatic classification submodule is used to classify different types of text, images and videos according to the pre-trained classification model using the supervised learning algorithm; the labeling submodule is used to identify objects in pictures and video content through deep learning models and multimodal large models, and to label them specifically; for dynamic video content, the inter-frame difference method is combined with target detection technology to achieve automatic labeling of video content; the information generation submodule is used to generate rich label information for structured data through metadata analysis to ensure the searchability and operability of data in the data lake.
[0033] The cross-source data association module uses knowledge graph technology to achieve intelligent association of cross-source data, ensuring that data from different sources can be automatically connected together. The cross-source data association module includes a quick query submodule, a data link submodule and an optimization submodule. Among them, the quick query submodule is used to automatically identify the same entity from different sources through the entity matching algorithm in the knowledge graph to ensure the correlation between data; the data link submodule is used to use natural language processing technology to analyze the entity relationship in text data, find potential association points in the data source, and link isolated data from different sources together to provide users with more business insights; the optimization submodule is used to support users to define custom rules to associate data from different data sources, or automatically learn potential association rules between data sources through machine learning models to continuously optimize the association effect.
[0034] See also Figure 2 , an AI-driven intelligent data lake management and optimization method, comprising the following steps: S1: Intelligent analysis and processing of unstructured data; S2: Intelligent labeling and classification of data lakes; S3: Automatic association of data across sources.
[0035] Step S1 specifically includes: S11: For image data, corresponding intelligent processing and classification are performed through a multimodal large model (qwen-vl) and a deep learning model (YOLO); S12: For audio data, text conversion, sentiment analysis, and semantic understanding are performed through a high-precision multilingual speech recognition model (SenseVoice); S13: Use NLP models to perform preliminary cleaning and preprocessing on unstructured data (such as text data), such as word segmentation, noise removal, grammatical parsing, etc., to form structured data.
[0036] Step S2 includes the following sub-steps: S21: Data feature extraction: Use deep learning models (CNN, RNN) to extract features of unstructured data and form a structured representation of the data; S22: Classification model training: The classification model is trained through supervised learning methods to automatically classify data according to business needs and annotate based on specific business rules; S23: Labeling and classification index generation: Generate classification index to facilitate subsequent rapid retrieval and query.
[0037] Step S3 includes the following sub-steps: S31: Entity and Relationship Extraction: Use natural language processing technology to identify entities (such as people, products, events) and their relationships (such as affiliation, dependency, etc.) from unstructured data; S32: Knowledge graph construction: Build a knowledge graph based on entities and relationships, map cross-source data into a unified knowledge network, and break the phenomenon of data isolation; S33: Data query and semantic enhancement: Data query is performed through knowledge graphs to support data association and intelligent retrieval in complex business scenarios.
[0038] The present invention introduces natural language processing (NLP) technology to perform automatic semantic analysis, knowledge extraction, entity recognition and other operations on unstructured data, thereby realizing automatic classification, labeling and structural processing of data. This method can greatly improve data processing efficiency, reduce labor costs, and extract useful information from the data for subsequent business analysis.
[0039] The present invention uses machine learning and deep learning technologies to automatically classify and label data, and establish an intelligent classification label system and indexing mechanism based on business needs. This method can effectively organize and manage data, help users quickly retrieve and use the required information, avoid the formation of "data swamps", and optimize the overall structure and management efficiency of the data lake.
[0040] The present invention uses knowledge graph technology to associate and integrate key information from different data sources to build a structured business knowledge network across source data. The knowledge graph can display the relationship and context between data, thereby helping users to deeply explore the potential value of cross-source data and realize automatic association and intelligent analysis in complex business scenarios.
[0041] By introducing AI technologies such as machine learning, deep learning and NLP, the present invention can realize the automated processing of tasks such as data annotation, classification, association and summary generation, greatly reducing the workload of manual participation. The automated data management process not only reduces labor costs, but also improves the efficiency and accuracy of data processing.
[0042] For the aforementioned embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the present application.
[0043] The above embodiments describe the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the changes and modifications made by those skilled in the art shall be within the scope of protection of the appended claims of the present invention without departing from the spirit and scope of the present invention.
Claims
1. An AI-driven intelligent data lake management and optimization system, characterized in that: include: A data ingestion module, used to automatically collect data from multiple data sources, including one or more of a database, a file system, an IOT, and an API interface; Data preprocessing module, used to preprocess the raw data, including cleaning, deduplication, and standardization operations; Natural language processing module, which is used to conduct in-depth analysis of unstructured text data and extract key business information and entities; The knowledge graph construction module is used to structure entities and relationships in various types of data, helping users quickly establish associations in large-scale data and gain business insights; Data classification and annotation module, which is used to automatically classify and annotate the data entering the data lake to ensure the organization and searchability of the data; The cross-source data association module is used to realize the intelligent association of cross-source data and ensure that data from different sources can be automatically connected together.
2. The AI-driven intelligent data lake management and optimization system according to claim 1, characterized in that: The data ingestion module includes a structured data ingestion submodule and an unstructured data ingestion submodule, wherein: The structured data ingestion submodule is used to automatically ingest structured data by connecting different data sources, and regularly update the content in the data lake using scheduled tasks or streaming processing; The unstructured data ingestion submodule is used to extract unstructured data from the Internet or corporate intranet through web crawler technology, and use content parsing tools to perform basic classification of unstructured data, identify different types of files and import them into the data lake.
3. The AI-driven intelligent data lake management and optimization system according to claim 2, characterized in that: The data preprocessing module includes a structured data preprocessing submodule and an unstructured data preprocessing submodule, wherein: The structured data preprocessing submodule is used to perform data cleaning operations, remove duplicate or redundant data entries, and repair or annotate missing data items. Through data normalization, heterogeneous data is unified into the same format; The unstructured data preprocessing submodule is used to perform word segmentation, stop word filtering, and extract key information through natural language processing technology. Image and audio data are preliminarily classified through deep learning models and multimodal large models to extract basic visual features.
4. The AI-driven intelligent data lake management and optimization system according to claim 3, characterized in that: The natural language processing module includes an entity recognition submodule, a speech detection submodule and a sentiment classification submodule, wherein: The entity recognition submodule is used to perform entity recognition on text data, extract information about names of people, places, and organizations, and determine important relationships in the text through syntactic analysis; The speech detection submodule is used to perform language detection on data from multiple languages and unify them into the target language through a translation model to ensure unified analysis of cross-language data; The sentiment classification submodule is used to classify the sentiment of user comments and feedback text data through the sentiment analysis model, and provide trend analysis and sentiment prediction for the business department.
5. The AI-driven intelligent data lake management and optimization system according to claim 4, characterized in that: The knowledge graph construction module includes a graph construction submodule, a mapping submodule and an interface submodule, wherein: A graph construction submodule is used to construct a knowledge graph based on the extracted entities and relationships after processing the text. The knowledge graph is stored in RDF format and can be queried by SPARQL. The mapping submodule is used to map entities from different data sources to the same vector space using graph embedding technology to ensure accurate association of cross-source data. The interface submodule is used to provide an API interface to support users to query and expand the knowledge graph, quickly find business-related entities and their relationships, and support intelligent search and reasoning functions.
6. The AI-driven intelligent data lake management and optimization system according to claim 5, characterized in that: The data classification and labeling module includes an automatic classification submodule, a labeling submodule and an information generation submodule, wherein: Automatic classification submodule, which uses supervised learning algorithms to classify different types of text, images, and videos based on pre-trained classification models; The annotation submodule is used to identify objects in images and videos through deep learning models and multimodal large models, and to label them specifically. For dynamic video content, the inter-frame difference method is combined with target detection technology to achieve automatic annotation of video content. The information generation submodule is used to generate rich label information for structured data through metadata analysis to ensure the searchability and operability of data in the data lake.
7. The AI-driven intelligent data lake management and optimization system according to claim 6, characterized in that: The cross-source data association module includes a fast query submodule, a data link submodule and an optimization submodule, wherein: The fast query submodule is used to automatically identify the same entity from different sources through the entity matching algorithm in the knowledge graph to ensure the relevance between data; The data linking submodule uses natural language processing technology to analyze entity relationships in text data, find potential connection points in data sources, and link isolated data from different sources together to provide users with more business insights; The optimization submodule is used to support users to define custom rules to associate data from different data sources, or to automatically learn potential association rules between data sources through machine learning models to continuously optimize the association effect.
8. An AI-driven intelligent data lake management and optimization method, implemented based on an AI-driven intelligent data lake management and optimization system according to any one of claims 1 to 7, characterized in that: The steps include: S1: Intelligent analysis and processing of unstructured data, specifically: for image data, intelligent processing and classification are performed through multimodal large models and deep learning models; for audio data, text conversion, sentiment analysis and semantic understanding are performed through high-precision multilingual speech recognition models; and natural speech models are used to perform preliminary cleaning and preprocessing of unstructured data to form structured data; S2: Intelligent labeling and classification of data lakes; S3: Automatic association of cross-source data.
9. The AI-driven intelligent data lake management and optimization method according to claim 8, characterized in that: Step S2 includes the following sub-steps: S21: Data feature extraction, using deep learning models to extract features of unstructured data and form a structured representation of the data; S22: Classification model training: training the classification model through supervised learning methods, automatically classifying data according to business needs, and labeling based on specific business rules; S23: Labeling and classification index generation: Generate classification index to facilitate quick retrieval and query.
10. The AI-driven intelligent data lake management and optimization method according to claim 9, characterized in that: Step S3 includes the following sub-steps: S31: Entity and relationship extraction, using natural language processing technology to identify entities and their relationships from unstructured data; S32: Knowledge graph construction: construct a knowledge graph based on entities and relationships, map cross-source data into a unified knowledge network, and break the phenomenon of data isolation; S33: Data query and semantic enhancement: Data query is performed through knowledge graphs to support data association and intelligent retrieval in complex business scenarios.