A library data management system based on data middle platform architecture

Abstract

The present invention discloses a library data management system based on a data middle platform architecture, wherein the data foundation layer is used to store library data and provide data sources for the data specification layer, the data management layer, the data service layer and the data display layer; the data specification layer is used to formulate and maintain the standards and specifications of the library data; the data management layer is used to manage the library data stored in the data foundation layer, including a data set management module, a data modeling management module, a data asset management module and a data quality management module; the data service layer is used to manage the API generated by the data foundation layer, the data specification layer and the data management layer in operation; the data display layer is used to provide visual data management of the library data. The present invention can break the information island, improve the administrative management efficiency and service quality, and enhance the sustainable development capacity of the library.

Description

Technical Field

[0001] The present invention relates to the technical field of library data management, and in particular to a library data management system based on a data middle platform architecture. Background Art

[0002] The informatization of libraries is in a stage of continuous development and change, and many libraries have made significant progress in the application of information technology. Automated management systems are widely used to realize the digital management of book collection, collection, circulation and other links. Readers can easily query book information, understand borrowing status, and make reservations and renewals through the online library catalog. The construction of electronic resources is also becoming increasingly rich. Libraries have purchased a large number of electronic books, academic journals, databases, etc., providing readers with convenient online reading and retrieval services. At the same time, some libraries are also actively building special digital resource libraries, such as local document databases, ancient book digitization projects, etc., to better preserve and disseminate precious cultural heritage.

[0003] However, the current situation of library informatization construction is that due to the lack of unified planning and unified data standards, the realization of various service functions requires the decentralized development or introduction of various application systems, and each department acts independently, duplicates construction, and collects and uses data for its own purposes. The resulting "information islands" and "information chimneys" are everywhere, making it impossible to achieve information resource sharing and business collaboration. These "information islands" have seriously hindered the flow of information within the library, making it difficult for different departments to achieve effective information sharing and business collaboration. At the same time, in terms of library administration, business collaboration between departments has become extremely difficult, with low work efficiency, and cannot meet the needs of modern libraries for efficient services. Summary of the invention

[0004] The present invention provides a library data management system based on a data middle platform architecture, which can break information silos, improve administrative management efficiency and service quality, and enhance the sustainable development capabilities of libraries. The specific technical solutions are as follows:

[0005] A library data management system based on a data middle platform architecture, including a data foundation layer, a data specification layer, a data management layer, a data service layer and a data display layer;

[0006] It includes a data foundation layer, which is used to store library data and provide data sources for the data specification layer, data management layer, data service layer and data display layer;

[0007] Data specification layer, the data specification layer is used to formulate and maintain the standards and specifications of library data, including data standard construction management module, data element management module and data dictionary management module; the data standard construction management module is used to set the standards of library data and build a data standard dictionary; the data element management module is used for metadata of library data and builds metadata standards; the data dictionary management module is used to build mapping rules based on the data standard dictionary and metadata standards obtained by the data standard construction management module and the data element management module;

[0008] The data management layer is used to manage the library data stored in the data foundation layer, including a data set management module, a data modeling management module, a data asset management module and a data quality management module; the data set management module is used to provide a unified data access service for the library data; the data modeling management module is used to build a data model for the management data specification layer; the data asset management module is used to convert the library data into data resources and data assets, and manage the permissions of the library data; the data quality management module is used to configure the library data management rules and perform quality management on the library data;

[0009] The data service layer is used to manage the APIs generated by the data foundation layer, the data specification layer, and the data management layer during operation;

[0010] The data display layer is used to provide visual data management of library data.

[0011] Preferably, the data foundation layer is used to store library data, including unstructured library data, semi-structured library data and structured library data; the data foundation layer stores unstructured library data through HDFS, stores semi-structured library data through HBase, and stores structured library data through Hive; the data foundation layer provides data sources for the data specification layer, data management layer, data service layer and data presentation layer through Flink stream computing.

[0012] Preferably, it also includes a data map; the data dictionary management module draws the data map according to the mapping rules constructed by the data standard dictionary and the metadata standard.

[0013] Preferably, when the data set management module is used to provide a unified data access service for library data, a text reading tool is set, and the text reading tool provides a data text input function for the library data; the internal association relationship of the library data is set, including setting the internal association relationship of the metadata and the internal association relationship of the big data cluster warehouse.

[0014] Preferably, the data modeling management module is used to construct a data model for managing a data specification layer, including establishing a data warehouse management model.

[0015] Preferably, the data quality management module is used to configure library data management rules, including pre-control prevention data management, in-process monitoring data management and post-supervision data management.

[0016] Preferably, the data service layer is used to manage the APIs generated during operation by the data base layer, data specification layer and data management layer, including establishing an API management module and an API market module; the API management module is used to provide API process management and security monitoring of the API; the API market module is used to provide users with API calling and query services.

[0017] Preferably, the data presentation layer is used to provide visual data management of library data, including visual data set management and chart management of library data.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The present invention provides a stable source for the system by setting up a data foundation layer to centrally store data. Each module of the data specification layer formulates standards and specifications to ensure that the data is accurate, consistent, and easy to understand and convert. The modules of the data management layer improve the data management level from the aspects of access service, modeling, assetization, and quality management. The data service layer management API promotes system interaction, and the data display layer realizes visual management. The system breaks the information island, improves administrative management efficiency and service quality, and enhances the sustainable development capacity of the library. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the prior art description. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0021] Figure 1 It is a system principle diagram of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0024] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0025] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] Example 1

[0027] As shown in the figure, a library data management system based on the data middle platform architecture includes a data foundation layer, a data specification layer, a data management layer, a data service layer and a data display layer;

[0028] It includes a data foundation layer, which is used to store library data and provide data sources for the data specification layer, data management layer, data service layer and data display layer;

[0029] Data specification layer, the data specification layer is used to formulate and maintain the standards and specifications of library data, including data standard construction management module, data element management module and data dictionary management module; the data standard construction management module is used to set the standards of library data and build a data standard dictionary; the data element management module is used for metadata of library data and builds metadata standards; the data dictionary management module is used to build mapping rules based on the data standard dictionary and metadata standards obtained by the data standard construction management module and the data element management module;

[0030] The data management layer is used to manage the library data stored in the data foundation layer, including a data set management module, a data modeling management module, a data asset management module and a data quality management module; the data set management module is used to provide a unified data access service for the library data; the data modeling management module is used to build a data model for the management data specification layer; the data asset management module is used to convert the library data into data resources and data assets, and manage the permissions of the library data; the data quality management module is used to configure the library data management rules and perform quality management on the library data;

[0031] The data service layer is used to manage the APIs generated by the data foundation layer, the data specification layer, and the data management layer during operation;

[0032] The data display layer is used to provide visual data management of library data.

[0033] The data base layer is the foundation of the entire library data management system. It is responsible for storing various types of library data and providing data sources for other layers, including but not limited to book borrowing records, reader information, collection resource data, library facility usage, etc.

[0034] The data specification layer is responsible for formulating and maintaining the standards and specifications of library data to ensure the consistency, accuracy and availability of data.

[0035] Among them, the data standard construction management module is used to set the standards of library data and construct a data standard dictionary. Through in-depth analysis of various types of library data, unified data formats, encoding rules, data types, and value ranges are determined. For example, for reader information, a unified format and value range for fields such as name, gender, and age are specified; for book borrowing records, unified standards for fields such as borrowing time, return time, and borrowing status are determined. The constructed data standard dictionary serves as a data standard reference for the entire system and provides a data specification basis for other modules.

[0036] The data element management module is used to manage the metadata of library data and build metadata standards. Metadata is data that describes data and is essential for understanding and using data. This module classifies, defines, and manages metadata for library data and determines the attributes, relationships, and semantics of metadata. For example, the metadata for a book may include information such as the title, author, publisher, publication date, and ISBN number. By building metadata standards, the consistency and accuracy of metadata is ensured, providing support for data integration, sharing, and analysis.

[0037] The data dictionary management module is used to build mapping rules based on the data standard dictionary and metadata standards obtained by the data standard construction management module and the data element management module. The data dictionary is a detailed description and explanation of data elements, which can help users better understand the meaning and purpose of the data. This module establishes mapping relationships between data elements based on the data standard dictionary and metadata standards, and clarifies the meaning and value range of each data element in different data sets. For example, the "gender" field in the reader information may have different values ​​in different data sets. Through the data dictionary management module, a unified mapping rule can be established to ensure the consistency of the "gender" field in the entire system.

[0038] The data management layer is responsible for the overall management of the library data stored in the data foundation layer to ensure the quality, availability and security of the data.

[0039] The data set management module provides a unified data access service for library data. This module is responsible for receiving data from different data sources, including library management systems, book lending systems, reader information systems, etc., and integrating data of different formats and sources into the data base layer through a unified data access interface. At the same time, this module is also responsible for cleaning, converting and verifying data to ensure data quality and consistency. For example, for reader information imported from different systems, there may be problems such as inconsistent formats and duplicate data. The data set management module can remove duplicate data and unify data formats through data cleaning and verification to ensure the accuracy and completeness of reader information.

[0040] The data modeling management module is used to build data models for managing the data specification layer. The data model is an abstraction and representation of data in the real world, which can help users better understand the structure and relationship of data. This module builds appropriate data models according to the characteristics and needs of library data, including entity relationship models, data warehouse models, etc.

[0041] The data asset management module is used to convert library data into data resources and data assets, and to manage the permissions of library data. Data assets are important resources of the library. This module is responsible for registering, evaluating and monitoring data assets to ensure the security and effective use of data assets. At the same time, this module is also responsible for managing the permissions of library data, and assigning different data access permissions according to the roles and needs of different users. For example, library administrators can access all library data, while ordinary readers can only access their own borrowing records and book query information. Through the data asset management module, the value of data assets can be maximized while ensuring the security and privacy of data.

[0042] The data quality management module is used to configure library data management rules and perform quality management on library data. Data quality is the core issue of library data management. This module is responsible for formulating and implementing data quality management rules to monitor, evaluate and improve data. For example, by setting data quality indicators such as data accuracy, completeness, consistency, etc., data can be regularly checked and evaluated. For data that does not meet quality standards, rectification and repair should be carried out in a timely manner. At the same time, this module can also improve the quality and availability of data through data cleaning, verification and error correction.

[0043] The data service layer is responsible for managing the APIs generated by the data foundation layer, data specification layer, and data management layer during operation. API (Application Programming Interface) is the abbreviation of application program interface, which is a bridge for data exchange and function calls between different software systems. This layer realizes the openness and scalability of the system through the management of API. On the one hand, the data service layer can provide library data to the outside world in the form of API, which is convenient for other systems and applications to call library data and realize data sharing and integration. For example, the library can provide book query API to the school's teaching management system to facilitate teachers and students to query the library's collection resources. On the other hand, the data service layer can also receive API requests from external systems to realize interaction and collaboration with external systems. For example, the library can receive API requests from book suppliers to obtain the latest book information and inventory status, and update the library's collection resources in a timely manner.

[0044] The data display layer is used to provide visual data management of library data. Library data is displayed to users in the form of intuitive and easy-to-understand charts, reports and maps, helping users to better understand and analyze data.

[0045] Example 2

[0046] This embodiment differs from Implementation 1 in that the data foundation layer is used to store library data, including unstructured library data, semi-structured library data and structured library data; the data foundation layer stores unstructured library data through HDFS, stores semi-structured library data through HBase, and stores structured library data through Hive; the data foundation layer provides data sources for the data specification layer, data management layer, data service layer and data presentation layer through Flink stream computing.

[0047] The data infrastructure layer stores library data in categories. HDFS is used to store unstructured library data, such as book cover images and lecture videos. HDFS's high fault tolerance and scalability ensure the safe storage of large-scale unstructured data. HBase is used to store semi-structured library data, such as book catalog data in XML format and reader comments in JSON format. HBase can flexibly adapt to the characteristics of semi-structured data and facilitate rapid query and update of data. As for structured library data, such as reader information tables and book information tables, Hive is used to store them. Hive provides a query language similar to SQL to facilitate data analysis.

[0048] At the same time, the data foundation layer provides data sources for other layers of the system through Flink stream computing. Flink can process data inflow and outflow in real time. For example, the records generated by readers' borrowing operations can be captured and processed in a timely manner. It can also integrate data from different data sources and unify format standards. In addition, Flink can perform real-time data analysis and mining to provide support for library decision-making.

[0049] The working principle of this embodiment is the same as that of Embodiment 1.

[0050] Example 3

[0051] This embodiment differs from Implementation 2 in that it further includes a data map; the data dictionary management module draws the data map according to the mapping rules constructed by the data standard dictionary and the metadata standard.

[0052] The data dictionary management module draws data maps based on the mapping rules constructed by the data standard dictionary and metadata standards. First, the data standard dictionary defines unified specifications and standards for various data types of the library, including data formats, naming rules, value ranges, etc. Metadata standards specify standards for describing data, such as the source of data, creation time, update frequency, etc. Through these standards and rules, the data dictionary management module can accurately identify different types of data and visualize them on the data map. The data map can present the overall structure and distribution of library data, allowing users to understand the relationship and hierarchy between data at a glance.

[0053] The existence of data maps not only facilitates the management and maintenance of data by library managers, but also provides better navigation and understanding for data users. When conducting data analysis and decision-making, users can use data maps to quickly find the required data and understand its background and related information, thereby improving the accuracy and efficiency of decision-making.

[0054] The working principle of this embodiment is the same as that of Embodiment 1.

[0055] Example 4

[0056] This embodiment differs from implementation 3 in that, when the data set management module is used to provide a unified data access service for library data, a text reading tool is set, and the text reading tool provides a data text input function for the library data; and internal associations of the library data are set, including setting internal associations of metadata and internal associations of big data cluster warehouses.

[0057] By setting up a text input tool, data text input function is provided for library data. This text input tool can easily receive text data in various formats, whether it is a text file imported from an external data source or text content entered manually by the user.

[0058] Secondly, in terms of setting the internal associations of library data, it includes setting the internal associations of metadata and the internal associations of big data cluster warehouses. For the internal associations of metadata, it ensures the logical connection between the data describing the library data. For example, the metadata of a book may include information such as the title, author, and publication date. By setting the internal associations of metadata, the relationship between these information can be clearly displayed, making it easier for users to query and analyze. For example, when a user queries the works of a certain author, the system can quickly locate all the books of the author through the internal associations of metadata.

[0059] The internal associations of the big data cluster warehouse strengthen the integration and coordination of library data in the big data environment. The big data cluster warehouse usually stores a large amount of library data. By setting internal associations, efficient connection and query between different data tables can be achieved. For example, by associating reader borrowing record data with book information data, it is possible to analyze which books are the most popular and which readers have more active borrowing behaviors.

[0060] The working principle of this embodiment is the same as that of Embodiment 1.

[0061] Example 5

[0062] This embodiment differs from Implementation 4 in that the data modeling management module is used to construct a data model for managing the data specification layer, including establishing a data warehouse management model.

[0063] The data modeling management module determines the subject area of ​​the data warehouse by analyzing the library's business needs and data characteristics. Establishing a data warehouse management model also involves the storage structure and index design of the data. Select appropriate database technology and storage methods to ensure efficient storage and fast retrieval of data. For example, the use of a distributed database system can improve the storage capacity and processing power of data, while a reasonable index design can speed up the query speed. In addition, the data modeling management module will also consider the data update and maintenance mechanism. With the continuous development of the library's business, new data will continue to be generated, and old data may need to be updated or cleaned up. Establish data update processes and data cleaning rules to ensure that the data in the data warehouse is always accurate and complete. Through data modeling, the data standards and specifications formulated by the data specification layer are applied to the actual data to ensure the consistency and availability of the data. For example, for the library's collection resource data, a data warehouse model can be constructed to integrate data such as book information, borrowing records, and reader evaluations into a data warehouse to facilitate data analysis and decision support.

[0064] The working principle of this embodiment is the same as that of Embodiment 1.

[0065] Example 6

[0066] This embodiment differs from Implementation 5 in that the data quality management module is used to configure library data management rules, including pre-control prevention data management, in-process monitoring data management, and post-supervision data management.

[0067] The aforementioned pre-control and preventive management is to establish data standards covering various business topics within the enterprise, unify indicator definitions, indicator calibers, and entry specifications covering each business field. For manually entered data, use non-open input methods as much as possible, such as drop-down menus, single check boxes, time controls, labels (supporting custom learning), etc., and perform necessary and timely verification on the input parts that must be open. In addition, for data quality problems caused by system reasons, we need to establish a data standard system. For production systems that can be transformed, they should be transformed under the guidance of data standards. For systems that cannot be transformed, they should be cleaned and converted through some technical means, and data quality should be controlled at the stage where data is generated. This will inevitably be the most efficient. Establishing a data responsibility system and a data quality management department within the enterprise, and formulating data quality monitoring processes and assessment methods will also help improve the pre-control and preventive mechanism for data quality.

[0068] The in-process monitoring and management mentioned above is to control the data quality in the process, that is, to monitor and process the data quality during the maintenance and use of the data. By establishing a process-based control system for data quality, process-based control is performed on each link such as data creation, change, collection, processing, loading, and application. In this process, the data quality of each node of the data flow can be monitored with the help of relevant modules in the data quality management tool, and real-time data quality warning can be provided. Data quality can be controlled from the source of the data, and management can be supported by combining system automated verification and manual review. In this process, the relevant processes and approval flows of the enterprise's data quality problem handling mechanism can also be embedded in the data quality management tool to effectively assist and monitor data quality.

[0069] For the post-event supervision and management, if quality problems are found for the data already stored in the data warehouse, data quality control tools must be used. When the data warehouse or data center is established, the key fields are uniformly named, formatted, and accurate according to the data standards to eliminate data ambiguity. According to the data standards, the corresponding rule model is established in the data quality management tool. For the imported historical data, data quality problems can be discovered by running the rule model, and the data quality problems can be tracked throughout the entire process in the platform.

[0070] The working principle of this embodiment is the same as that of Embodiment 1.

[0071] Example 7

[0072] This embodiment differs from implementation 6 in that the data service layer is used to manage the APIs generated by the data base layer, data specification layer and data management layer during operation, including establishing an API management module and an API market module; the API management module is used to provide API process management and security monitoring of the API; the API market module is used to provide users with API call and query services.

[0073] By establishing an API management module and providing API process management, starting from the API design stage, strictly plan its functions and interface specifications to ensure that the API has good scalability and compatibility. During the development of the API, strict code review and testing are carried out to ensure the stability and reliability of the API. When the API is launched, its performance indicators such as response time, throughput, etc. are continuously monitored to promptly discover and resolve potential performance issues. Secondly, the API market module provides users with API call and query services. Users can browse various available APIs in the API market to understand their functions and usage methods. Through a simple registration and authorization process, users can call the required APIs and integrate library data into their own applications or systems.

[0074] The working principle of this embodiment is the same as that of Embodiment 1.

[0075] Example 8

[0076] This embodiment differs from Implementation 7 in that the data presentation layer is used to provide visual data management of library data, including visual data set management and chart management of library data.

[0077] The visual data set management provides a data set details browsing page, which can clearly view the core information such as the data set sort, data set name, enabled status, creation time, etc., and can also perform corresponding modification operations on the corresponding data set.

[0078] The chart management includes providing various conventional and commonly used chart components such as bar charts, line charts, pie charts, scatter charts, funnel charts, radar charts, Sankey charts, and indicator charts. First, visual data set management allows library data to be presented in a more intuitive and easy-to-understand way. Through visualization, the overall structure, distribution, and relationship between library data can be clearly displayed. For example, the quantity distribution of different types of books, the frequency of borrowing, etc. can be displayed in a graphical way. In this way, library managers can quickly grasp the overall situation of the data and make more informed decisions.

[0079] The working principle of this embodiment is the same as that of Embodiment 1.

[0080] In summary, the present invention has the following beneficial effects:

[0081] The data foundation layer provides a stable data source for the entire library data management system, ensuring the normal operation of subsequent levels. By centrally storing library data, it avoids management confusion and resource waste caused by data dispersion, laying a solid foundation for information resource sharing and business collaboration.

[0082] Data standard construction management module: sets the standards for library data and builds a data standard dictionary, so that all kinds of data in the library have unified specifications. This not only facilitates data exchange and sharing between different departments, but also improves the accuracy and consistency of data.

[0083] Data Metadata Management Module: Manage the metadata of library data and build metadata standards. This helps to better understand and interpret the meaning of data and provides a guarantee for the effective use of data.

[0084] Data dictionary management module: Based on the data standard dictionary and metadata standards obtained by the data standard construction management module and the data element management module, the mapping rules are constructed to further ensure the accurate conversion and sharing of data between different systems and modules.

[0085] Dataset management module: provides unified data access services for library data, facilitating the acquisition and use of data by various departments. It is no longer necessary to develop independent data access channels, which reduces costs and improves efficiency. Data modeling management module: builds a data model for the management data specification layer to make data storage and management more scientific and reasonable. It can better adapt to the development and changes of library business and improve the scalability and flexibility of data. Data asset management module: transforms library data into data resources and assets, and manages data permissions. It clarifies the ownership and use rights of data, prevents data abuse and leakage, and also improves the value and utilization efficiency of data. Data quality management module: configures library data management rules and performs quality management on library data. It ensures the accuracy, completeness and timeliness of data, and provides reliable data support for various decisions and services of the library.

[0086] The data service layer manages the APIs generated by the data foundation layer, data specification layer and data management layer during operation, making the interaction between systems more convenient and efficient, facilitating the expansion and integration of the system and providing more service possibilities for the library.

[0087] The data display layer provides visual data management for library data, allowing managers and staff to intuitively understand the library's operating conditions and data status, which helps to identify problems and make decisions in a timely manner, and improves the efficiency and scientificity of management.

[0088] Those of ordinary skill in the art will appreciate that the units of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0089] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0090] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0091] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nlyMemory), random access memory (RAM, RandomAccessMemory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A library data management system based on a data middle platform architecture, characterized in that: It includes data foundation layer, data specification layer, data management layer, data service layer and data display layer; It includes a data foundation layer, which is used to store library data and provide data sources for the data specification layer, data management layer, data service layer and data display layer; Data specification layer, which is used to formulate and maintain the standards and specifications of library data, including data standard construction management module, data element management module and data dictionary management module; the data standard construction management module is used to set the standards of library data and build a data standard dictionary; the data element management module is used for the metadata of library data and builds metadata standards; The data dictionary management module is used to construct mapping rules based on the data standard dictionary and metadata standards obtained by the data standard construction management module and the data metadata management module; The data management layer is used to manage the library data stored in the data foundation layer, including a data set management module, a data modeling management module, a data asset management module and a data quality management module; the data set management module is used to provide a unified data access service for the library data; the data modeling management module is used to build a data model for the management data specification layer; the data asset management module is used to convert the library data into data resources and data assets, and manage the permissions of the library data; the data quality management module is used to configure the library data management rules and perform quality management on the library data; The data service layer is used to manage the APIs generated by the data foundation layer, the data specification layer, and the data management layer during operation; The data display layer is used to provide visual data management of library data.

2. According to claim 1, a library data management system based on a data middle platform architecture is characterized in that: The data foundation layer is used to store library data, including unstructured library data, semi-structured library data and structured library data; the data foundation layer stores unstructured library data through HDFS, stores semi-structured library data through HBase, and stores structured library data through Hive; the data foundation layer provides data sources for the data specification layer, data management layer, data service layer and data display layer through Flink stream computing.

3. According to claim 1, a library data management system based on a data middle platform architecture is characterized in that: It also includes a data map; the data dictionary management module draws the data map according to the mapping rules constructed by the data standard dictionary and metadata standards.

4. According to claim 1, a library data management system based on a data middle platform architecture is characterized in that: When the data set management module is used to provide unified data access services for library data, a text reading tool is set, and the text reading tool provides data text input function for library data; the internal association relationship of library data is set, including setting the internal association relationship of metadata and the internal association relationship of big data cluster warehouse.

5. According to claim 1, a library data management system based on a data middle platform architecture is characterized in that: The data modeling management module is used to construct a data model for managing a data specification layer, including establishing a data warehouse management model.

6. According to claim 1, a library data management system based on a data middle platform architecture is characterized in that: The data quality management module is used to configure library data management rules, including pre-control and preventive data management, in-process monitoring data management and post-supervision data management.

7. According to claim 1, a library data management system based on a data middle platform architecture is characterized in that: The data service layer is used to manage the APIs generated by the data basic layer, data specification layer and data management layer during operation, including establishing an API management module and an API market module; the API management module is used to provide API process management and security monitoring of the API; the API market module is used to provide users with API calling and query services.

8. According to claim 1, a library data management system based on a data middle platform architecture is characterized in that: The data presentation layer is used to provide visual data management of library data, including visual data set management and chart management of library data.

Images