The invention provides a data lake construction method and
system based on multi-source distributed data, and relates to the field of data lake construction. The method comprises the following steps: firstly, deploying a
data acquisition agent at each node of a distributed
data source and establishing a distributed collaborative mechanism, and performing
structured analysis on data through the
data acquisition agent to obtain data characteristics and establish a relationship between fields; and then semantic description information is constructed, and an
RDF triple is generated based on the information and stored in a source end node. And then inputting the
RDF triple into an ontology registration service for
processing to generate a distributed ontology, and analyzing a corresponding relationship between the
RDF triple and concepts in the distributed ontology to obtain a
semantic equivalence rule and a mapping rule. And finally, organizing the distributed data sources into a unified
data view according to the rules, establishing a
distributed index, configuring an access permission, and finally constructing a data lake containing the unified
data view, the
distributed index and the access permission. According to the method, the performance
bottleneck problem caused by centralized
processing is avoided.