Data access service construction method and device, equipment and storage medium

Abstract

The invention discloses a data access service construction method and device, equipment and a storage medium, and relates to the field of data structure.The data access service construction method comprises the steps that the attribute type, the attribute size and the attribute constraint of a target entity are determined, and an attribute definition is determined according to the attribute type, the attribute size and the attribute constraint; determining preset database connection information according to the database drive, the database access path, the user side name and the user side password, and constructing a target entity physical table based on a preset data structure, the preset database connection information and the attribute definition of the target entity; utilizing a preset data aggregation tool to map the fields of the to-be-governed data to the fields in the target entity physical table based on the field mapping relation to obtain a mapping result; loading the to-be-governed data to the target entity physical table based on a preset data processing rule and the mapping result to obtain governed data; and constructing a data access service based on the governed data so as to initiate access to the governed data based on the data access service. And the data service construction efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of data structures, and particularly to a method, apparatus, device, and storage medium for constructing a data access service. Background Art

[0002] In the process of constructing existing data services, there are many problems. First, there is a lack of a unified data modeling standard. Different departments or projects may adopt different ways to define data structures and attributes, resulting in poor compatibility between data and making it difficult to effectively integrate and share data. For example, in a large enterprise, the sales department and the finance department may respectively maintain their own customer data. Due to inconsistent data definitions, when cross-departmental data analysis is required, a large amount of time and effort is needed for data conversion and matching.

[0003] Secondly, the data governance link is weak. During the process of data collection, transmission, and storage, data quality problems are likely to occur, such as data missing, data errors, data duplication, etc. Traditional data governance methods often lack automated and intelligent means and are difficult to govern large-scale data in real time and effectively. This affects the accuracy and reliability of data services and cannot meet the needs of users for high-quality data.

[0004] In addition, the efficiency of data service construction is low. Traditional methods usually require developers to manually write a large amount of code to implement the data ETL (Extract, transform, load) process and the development of service interfaces. This method is not only time-consuming and laborious but also error-prone and difficult to quickly respond to changes in business requirements.

[0005] In summary, how to improve the efficiency of data service construction and quickly respond to changes in business requirements is an urgent problem to be solved at present. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a method, apparatus, device, and storage medium for constructing a data access service, which can improve the efficiency of data service construction and quickly respond to changes in business requirements. The specific solutions are as follows:

[0007] In the first aspect, the present application provides a method for constructing a data access service, including:

[0008] Obtain a target entity, determine the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in a preset data structure, and determine the attribute definition of the target entity according to the attribute type, the attribute size, and the attribute constraints;

[0009] Determine the preset database connection information based on the database driver, the database access path, the obtained client name, and the client password, and construct the corresponding target entity physical table in the preset database based on the preset data structure, the preset database connection information, and the attributes of the target entity;

[0010] Use the preset data aggregation tool to map the fields in the obtained data to be governed to the corresponding fields in the target entity physical table based on the field mapping relationship in the preset aggregation rule to obtain the corresponding mapping result;

[0011] Load the data to be governed into the target entity physical table in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result to obtain the governed data with unified data attributes;

[0012] Construct the corresponding data access service based on the governed data so as to initiate access to the governed data based on the data access service.

[0013] Optionally, the determining the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in the preset data structure includes:

[0014] Obtain the preset data structure based on the preset business scenario and / or preset data range;

[0015] Determine the corresponding data logic model according to the preset data structure;

[0016] Use the data logic model of the preset data structure to determine the attribute type, attribute size, and attribute constraints of the target entity;

[0017] Wherein, the attribute type includes any one or more of date attribute, text attribute, integer attribute, fixed-point number attribute, floating-point number attribute, and binary data attribute, the attribute size is the range of data that can be stored by the target entity, and the attribute constraints include any one or more of primary key, foreign key, and default value.

[0018] Optionally, the constructing the corresponding target entity physical table in the preset database based on the preset data structure, the preset database connection information, and the attributes of the target entity includes:

[0019] Convert the data logic model of the preset data structure into the preset physical table structure in the preset database based on the preset data structure and the preset database connection information;

[0020] Construct the corresponding initial entity physical table for the target entity according to the preset physical table structure;

[0021] Perform a preset parameter setting operation on the initial entity physical table using the attribute definition of the target entity to obtain the target entity physical table.

[0022] Optionally, the performing a preset parameter setting operation on the initial entity physical table using the attribute definition of the target entity to obtain the target entity physical table includes:

[0023] Select a corresponding data type in the preset database based on the attribute type in the attribute definition to obtain a first parameter setting result;

[0024] Set a corresponding field length for the attribute size in the attribute definition to obtain a second parameter setting result;

[0025] Determine corresponding constraint conditions based on the attribute constraints in the attribute definition using a preset SQL statement to obtain a third parameter setting result;

[0026] Determine the target entity physical table according to the first parameter setting result, the second parameter setting result, and the third parameter setting result.

[0027] Optionally, the loading the data to be governed into the target entity physical table in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result includes:

[0028] Perform a preset data conversion operation on the data to be governed based on the preset data processing rule in the preset aggregation rule and the mapping result to obtain data after attribute conversion;

[0029] Use a preset data storage processor to load the data after attribute conversion into the target entity physical table in the preset database based on the preset scheduling strategy in the preset aggregation rule;

[0030] Wherein, the preset scheduling strategy includes a preset timing execution strategy or a preset manual execution strategy.

[0031] Optionally, after the loading the data to be governed into the target entity physical table in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result, it further includes:

[0032] Use a preset format verification rule to correct the data after attribute conversion in the target entity physical table that does not meet the preset format requirements to obtain an updated target entity physical table;

[0033] Convert the data after attribute conversion in the updated target entity physical table into a unified encoding format to obtain data after attribute conversion with a unified format;

[0034] Verify whether the data after the attribute conversion with unified format meets the preset business rules;

[0035] If it does not meet the preset business rules, perform a preset data processing operation on the data after the attribute conversion with unified format based on the preset business requirements.

[0036] Optionally, constructing a corresponding data access service based on the governed data, so as to initiate an access to the governed data based on the data access service, includes:

[0037] Allocate the corresponding preset database access permissions to each client based on the permission control requirements;

[0038] Determine the API interface of the target entity physical table of the preset database, so that the client can query the corresponding governed data in the preset database through the API interface based on the preset database access permissions.

[0039] In a second aspect, the present application provides a data access service construction device, including:

[0040] An attribute definition determination module, configured to obtain a target entity, determine the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in the preset data structure, and determine the attribute definition of the target entity according to the attribute type, the attribute size, and the attribute constraints;

[0041] A physical table construction module, configured to determine the preset database connection information according to the database driver, the database access path, the obtained client name, and the client password, and construct a corresponding target entity physical table in the preset database based on the preset data structure, the preset database connection information, and the attribute definition of the target entity;

[0042] A result acquisition module, configured to use a preset data aggregation tool to map the fields in the obtained data to be governed to the corresponding fields in the target entity physical table based on the field mapping relationship in the preset aggregation rules to obtain a corresponding mapping result;

[0043] A definition unification module, configured to load the data to be governed into the target entity physical table in the preset database based on the preset data processing rules in the preset aggregation rules and the mapping result to obtain governed data with unified data attributes;

[0044] A service construction module, configured to construct a corresponding data access service based on the governed data, so as to initiate an access to the governed data based on the data access service.

[0045] In a third aspect, the present application provides an electronic device, including:

[0046] A memory for storing a computer program;

[0047] A processor for executing the computer program to implement the data access service construction method as described above.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the data access service construction method as described above is implemented.

[0049] In summary, the present application first obtains a target entity, determines the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in the preset data structure, and determines the attribute definition of the target entity according to the attribute type, the attribute size, and the attribute constraints; determines the preset database connection information according to the database driver, the database access path, the obtained client name, and the client password, and constructs a corresponding physical table of the target entity in the preset database based on the preset data structure, the preset database connection information, and the attribute definition of the target entity; uses a preset data aggregation tool to map the fields in the to-be-governed data obtained to the corresponding fields in the physical table of the target entity based on the field mapping relationship in the preset aggregation rule to obtain a corresponding mapping result; loads the to-be-governed data into the physical table of the target entity in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result to obtain the governed data with unified data attributes; constructs a corresponding data access service based on the governed data so as to initiate an access to the governed data based on the data access service. As can be seen from the above, the present application first obtains a target entity, determines its attribute type, attribute size, and attribute constraints according to the data logic model of the preset data structure, and then determines the attribute definition; determines the preset database connection information through the database driver, the database access path, the client name, and the password, and then constructs the physical table of the target entity in the preset database based on the preset data structure, the connection information, and the attribute definition; uses the preset data aggregation tool to map the fields of the to-be-governed data to the corresponding fields of the physical table of the target entity according to the field mapping relationship of the preset aggregation rule to obtain the mapping result; loads the to-be-governed data into the physical table of the target entity according to the preset data processing rule and the mapping result to obtain the governed data with unified data attributes; finally, constructs a data access service based on the governed data to realize the access to the governed data. In this way, the present application establishes a unified data standard through a standardized data modeling process, improves the compatibility and shareability of data; uses automated data aggregation and governance tools to improve data quality and ensure the accuracy and reliability of data; adopts a modular design concept to improve the efficiency of data service construction and can quickly respond to changes in business requirements. By achieving these goals, more efficient, accurate, and reliable data services are provided for users, and the competitiveness of enterprises and institutions is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0051] Figure 1 Flowchart of a method for constructing a data access service disclosed in this application;

[0052] Figure 2 Flowchart of a specific method for constructing a data access service disclosed in this application;

[0053] Figure 3 Schematic structural diagram of a device for constructing a data access service disclosed in this application;

[0054] Figure 4 Structural diagram of an electronic device disclosed in this application. Detailed implementation manners

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0056] Currently, in the existing data service construction process, there is a lack of a unified data modeling standard. Different departments or projects may adopt different ways to define data structures and attributes, resulting in poor compatibility between data and making it difficult to effectively integrate and share data. Secondly, the data governance link is weak. During the data collection, transmission, and storage processes, data quality problems are likely to occur, such as data missing, data errors, data duplication, etc. Traditional data governance methods often lack automated and intelligent means and are difficult to govern large-scale data in real time and effectively. This affects the accuracy and reliability of data services and cannot meet the needs of users for high-quality data. In addition, the efficiency of data service construction is low. Traditional methods usually require developers to manually write a large amount of code to implement the data ETL process and the development of service interfaces. This method not only takes time and effort but is also error-prone and difficult to quickly respond to changes in business requirements. To solve the above technical problems, this application discloses a method, device, equipment, and storage medium for constructing a data access service, which can improve the efficiency of data service construction and quickly respond to changes in business requirements.

[0057] See Figure 1 As shown, an embodiment of the present invention discloses a method for constructing a data access service, including:

[0058] Step S11, obtain a target entity, determine the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in the preset data structure, and determine the attribute definition of the target entity according to the attribute type, the attribute size, and the attribute constraints.

[0059] In this embodiment, first, it is necessary to declare the data model D, clarify the applicable business scenarios and data scope, and determine the core component logical model L of the data model D, which is used to describe entities and their attributes. Then, obtain the entity set as , where n is the number of entities. Each entity has a unique name to identify the entity, and the target entity is determined from the entity set. After obtaining the target entity, obtain the preset data structure based on the preset business scenario and / or preset data scope; determine the corresponding data logical model according to the preset data structure; use the data logical model of the preset data structure to determine the attribute type, attribute size, and attribute constraints of the target entity; where the attribute type includes any one or more of date attribute, text attribute, integer attribute, fixed-point number attribute, floating-point number attribute, and binary data attribute, the attribute size is the range of data that the target entity can store, and the attribute constraints include any one or more of primary key, foreign key, and default value. Specifically, clarify the corresponding preset data model D according to the preset business scenario and / or preset data scope, then determine the data logical model L of the core component in the preset data model D, and use the data logical model L to parse the attribute type , attribute size and attribute constraints of the target entity. Among them, the attribute type can be divided into multiple types, such as date attribute (representing date and time information), text attribute (used to store character data), integer attribute (storing integer values), fixed-point number attribute (accurately representing decimals), floating-point number attribute (approximately representing decimals), binary data attribute (storing binary files, etc.); the attribute size is used to specify the maximum length or range of data that the attribute can store. For text attributes, the attribute size represents the maximum number of characters; for integer attributes and fixed-point number attributes, the attribute size represents the number of digits of the value, etc.; the attribute constraints are used to ensure the integrity and consistency of the data. Common attribute constraints include primary key (the attribute that uniquely identifies an entity), unique (the attribute value must be unique in the entity set), foreign key (establishing the association relationship between entities), default value (when the attribute value is not specified, the default value is used), etc.

[0060] After determining the attribute type , attribute size and attribute constraints of the target entity, determine the attribute definition of the target entity according to the attribute type , attribute size and attribute constraints :

[0061] ;

[0062] wherein, is the attribute type; is the attribute size; is the attribute constraint; is the attribute definition.

[0063] Step S12: Determine the preset database connection information according to the database driver, the database access path, the obtained client name, and the client password, and construct a corresponding physical table of the target entity in the preset database based on the preset data structure, the preset database connection information, and the attribute definition of the target entity.

[0064] In this embodiment, to determine the database connection information , is the database driver, which is used to communicate with a specific database system; U is the URL of the database, specifying the location and access path of the database; N is the user name, which is used for authentication; P is the password, which is used to protect the access rights of the database. Subsequently, based on the preset data structure and the preset database connection information, convert the data logical model of the preset data structure into a preset physical table structure in the preset database; construct a corresponding initial physical table of the target entity according to the preset physical table structure; perform a preset parameter setting operation on the initial physical table of the target entity to obtain the physical table of the target entity. Specifically, according to the data model D and the database connection information C, perform the materialization operation of the data model in the corresponding preset database, that is, convert the data logical model L into a physical table structure in the database, that is, the initial physical table of the entity, and then construct the initial physical table of the entity by using the attribute definition of the target entity. For example, through the SQL (Structured Query Language) statements or API (Application Programming Interface) interfaces provided by the DBMS (Database Management System), create the corresponding initial physical table of the entity according to the entity and attribute information in the data logical model L. For each entity , create a corresponding physical table , and each column in the table corresponds to an attribute definition of the entity . .

[0065] Further, after determining the initial entity physical table, select the corresponding data type in the preset database based on the attribute type in the attribute definition to obtain the first parameter setting result; set the corresponding field length for the attribute size in the attribute definition to obtain the second parameter setting result; determine the corresponding constraint conditions based on the attribute constraints in the attribute definition using a preset SQL statement to obtain the third parameter setting result; determine the target entity physical table according to the first parameter setting result, the second parameter setting result, and the third parameter setting result. Specifically, perform corresponding settings on the initial entity physical table according to the definition parameters of the attributes. For the attribute type , select a suitable data type in the preset database to store the attribute; for the attribute size , set the corresponding field length; for the attribute constraint , add the corresponding constraint conditions through SQL statements. After the settings are completed, the target entity physical table is obtained.

[0066] Step S13: Use a preset data aggregation tool to map the fields in the to-be-governed data obtained based on the field mapping relationship in the preset aggregation rule to the corresponding fields in the target entity physical table to obtain the corresponding mapping result.

[0067] In this embodiment, data aggregation operations are performed based on a preset aggregation rule through a preset data aggregation tool. Assume the data aggregation rule is , including a field mapping relationship and a data processing rule. The field mapping relationship is used to map the fields in the existing data resources to the corresponding fields in the target entity physical table. In addition, the data aggregation operation can be set to execute periodically or manually. Periodic execution can be achieved through the scheduling function of NiFi, for example, performing a data aggregation task once at 2 am every day. Manual execution allows users to trigger the data aggregation operation at any time when needed.

[0068] Step S14: Load the to-be-governed data into the target entity physical table in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result to obtain the governed data with unified data attributes.

[0069] In this embodiment, after mapping the fields in the data to be governed to the corresponding fields in the target entity physical table, a preset data conversion operation is performed on the data to be governed based on the preset data processing rules in the preset aggregation rule and the mapping result to obtain the data with attributes converted; a preset data storage processor is used to load the data with attributes converted into the target entity physical table in the preset database based on the preset scheduling strategy in the preset aggregation rule; wherein, the preset scheduling strategy includes a preset timing execution strategy or a preset manual execution strategy. Specifically, operations such as cleaning, converting, and filtering the data in the target entity are performed according to the preset data processing rules in the preset aggregation rule and the obtained mapping result. For example, for date fields, different date formats need to be uniformly converted into the format supported by the target database; for numerical fields, data type conversion or data range checking is required.

[0070] Furthermore, the data with attributes converted in the target entity physical table that does not meet the preset format requirements is corrected using the preset format verification rule to obtain the updated target entity physical table; the data with attributes converted in the updated target entity physical table is converted into a unified coding format to obtain the data with attributes converted with a unified format; it is verified whether the data with attributes converted with a unified format meets the preset business rules; if it does not meet the preset business rules, a preset data processing operation is performed on the data with attributes converted with a unified format based on the preset business requirements. Specifically, noise, duplicate data, and error data in the data are removed. For example, regular expressions or rule engines are used to check whether the format of the data meets the requirements, and the data that does not meet the requirements is corrected or deleted. Then, the data is converted into a unified format and standard. For example, text data with different encodings is converted into a unified coding format, and numerical data with different units is subjected to unit conversion. Then, the integrity and consistency of the data are verified. For example, it is checked whether the primary key is unique, whether the foreign key references valid records, and whether the value range of the data meets the business rules, etc. Performing a data governance task on the target entity to obtain the governed data with unified data attributes can be executed regularly or manually. Regular execution is set according to the update frequency of the data and business requirements. For example, the data governance task is executed once a week; manual execution allows users to process it in a timely manner when data quality problems are found.

[0071] Step S15: Construct a corresponding data access service based on the governed data, so as to initiate an access to the governed data based on the data access service.

[0072] In this embodiment, corresponding data access services are constructed based on the governed data with unified data attributes, and library tables or interface services are provided externally. Corresponding preset database access permissions can be assigned to each client based on permission control requirements; the API interface of the target entity physical table of the preset database is determined so that the client can query the corresponding governed data in the preset database through the API interface based on the preset database access permissions. Specifically, users can directly access the governed database tables and perform data query and analysis through SQL statements. To improve the security and efficiency of data access, corresponding permission control and index optimization can be set. For example, different database access permissions are assigned to different user roles; indexes are created for frequently queried fields to speed up querying. At the same time, by developing corresponding API interfaces, users can initiate access to the governed data by calling the interfaces.

[0073] As can be seen from the above, in the embodiment of the present application, first, the target entity is obtained, and its attribute type, attribute size, and attribute constraints are determined according to the data logic model of the preset data structure, and then the attribute definition is determined; the preset database connection information is determined through the database driver, database access path, client name, and password, and then the target entity physical table is constructed in the preset database based on the preset data structure, connection information, and attribute definition; using the preset data aggregation tool, according to the field mapping relationship of the preset aggregation rules, the fields of the data to be governed are mapped to the corresponding fields of the target entity physical table to obtain the mapping result; according to the preset data processing rules and the mapping result, the data to be governed is loaded into the target entity physical table to obtain the governed data with unified data attributes; finally, a data access service is constructed based on the governed data to realize the access to the governed data. In this way, through the standardized data modeling process, the embodiment of the present application establishes a unified data standard, improves data compatibility and shareability; uses automated data aggregation and governance tools to improve data quality and ensure data accuracy and reliability; adopts a modular design concept to improve the efficiency of data service construction and can quickly respond to changes in business requirements. By achieving these goals, more efficient, accurate, and reliable data services are provided for users, enhancing the competitiveness of enterprises and institutions.

[0074] Based on the previous embodiment, it can be known that the present application discloses a method for constructing a data access service, which can improve the efficiency of data service construction and quickly respond to changes in business requirements. Next, the Figure 2 data access service construction method shown will be described in detail.

[0075] First, this application declares a data model D, clarifies the applicable business scenarios and data scope, and determines the core component logical model L of the data model D. Obtain the entity set and determine the target entity in the entity set. According to the preset business scenario and / or preset data scope, clarify the corresponding preset data model D, and then determine the data logical model L of the core components in the preset data model D. Use the data logical model L to analyze the attribute type, attribute size, and attribute constraints of the target entity, and determine the attribute definition of the target entity according to the attribute type, attribute size, and attribute constraints. The structure and attributes of the data are clarified, providing clear guidance for subsequent data processing and service construction.

[0076] Secondly, determine the database connection information. According to the data model and the database connection information, perform the materialization operation of the data model in the corresponding preset database, that is, convert the data logical model into the physical table structure in the database, namely the initial entity physical table, and then use the attribute definition of the target entity to construct the initial entity physical table. After determining the initial entity physical table, perform corresponding settings on the initial entity physical table according to the definition parameters of the attributes to obtain the target entity physical table.

[0077] Then, perform a data aggregation operation through a preset data aggregation tool based on preset aggregation rules, where the data aggregation rules include field mapping relationships and data processing rules. Use the field mapping relationship to map the fields in the existing data resources to the corresponding fields in the target entity physical table to obtain the corresponding mapping results. Through field mapping and data conversion, convert data in different formats and standards into a unified format, facilitating the circulation and use of data.

[0078] Next, after mapping the fields in the data to be governed to the corresponding fields in the target entity physical table, perform operations such as cleaning, conversion, and filtering on the data in the target entity according to the preset data processing rules in the preset aggregation rules and the obtained mapping results to obtain the governed data with unified data attributes. In this way, the accuracy, integrity, and consistency of the data are ensured.

[0079] Finally, construct the corresponding data access service according to the governed data, so as to provide library tables or interface services outward using the data access service. Users can select appropriate service methods according to their technical capabilities and business needs, improving the convenience and flexibility of data acquisition.

[0080] See Figure 3 As shown in

[0081] An attribute definition determination module 11, configured to obtain a target entity, determine the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in a preset data structure, and determine the attribute definition of the target entity according to the attribute type, the attribute size, and the attribute constraints;

[0082] A physical table construction module 12, configured to determine preset database connection information according to a database driver, a database access path, the obtained client name, and the client password, and construct a corresponding physical table of the target entity in a preset database based on the preset data structure, the preset database connection information, and the attribute definition of the target entity;

[0083] A result acquisition module 13, configured to use a preset data aggregation tool to map the fields in the to-be-governed data obtained to the corresponding fields in the physical table of the target entity based on the field mapping relationship in a preset aggregation rule to obtain a corresponding mapping result;

[0084] A definition unification module 14, configured to load the to-be-governed data into the physical table of the target entity in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result to obtain governed data with unified data attributes;

[0085] A service construction module 15, configured to construct a corresponding data access service based on the governed data, so as to initiate an access to the governed data based on the data access service.

[0086] As can be seen from the above, in this application, first, the target entity is obtained, and its attribute type, attribute size, and attribute constraints are determined based on the data logic model of the preset data structure, and then the attribute definition is determined; the preset database connection information is determined through the database driver, the database access path, the client name, and the password, and then the physical table of the target entity is constructed in the preset database based on the preset data structure, the connection information, and the attribute definition; the preset data aggregation tool is used to map the fields of the to-be-governed data to the corresponding fields of the physical table of the target entity according to the field mapping relationship of the preset aggregation rule to obtain a mapping result; the to-be-governed data is loaded into the physical table of the target entity according to the preset data processing rule and the mapping result to obtain governed data with unified data attributes; finally, a data access service is constructed based on the governed data to realize the access to the governed data. In this way, this application establishes a unified data standard through a standardized data modeling process, improves the compatibility and shareability of data; uses automated data aggregation and governance tools to improve data quality and ensure the accuracy and reliability of data; adopts a modular design concept to improve the efficiency of data service construction and can quickly respond to changes in business requirements. By achieving these goals, more efficient, accurate, and reliable data services are provided for users, and the competitiveness of enterprises and institutions is enhanced.

[0087] In some specific embodiments, the attribute definition determination module 11 may specifically include:

[0088] A preset data structure acquisition unit, configured to acquire a preset data structure based on a preset business scenario and / or a preset data range;

[0089] A data logic model determination unit, configured to determine a corresponding data logic model according to the preset data structure;

[0090] An entity attribute determination unit, configured to determine the attribute type, attribute size, and attribute constraints of the target entity by using the data logic model of the preset data structure; wherein, the attribute type includes any one or more of a date attribute, a text attribute, an integer attribute, a fixed-point number attribute, a floating-point number attribute, and a binary data attribute, the attribute size is the range in which the target entity can store data, and the attribute constraints include any one or more of a primary key, a foreign key, and a default value.

[0091] In some specific embodiments, the physical table construction module 12 may specifically include:

[0092] A data logic model conversion unit, configured to convert the data logic model of the preset data structure into a preset physical table structure in a preset database based on the preset data structure and the preset database connection information;

[0093] An initial entity physical table construction unit, configured to construct a corresponding initial entity physical table for the target entity according to the preset physical table structure;

[0094] A target entity physical table acquisition unit, configured to perform a preset parameter setting operation on the initial entity physical table by using the attribute definition of the target entity to obtain a target entity physical table.

[0095] In some specific embodiments, the target entity physical table acquisition unit may specifically include:

[0096] A first parameter setting result acquisition subunit, configured to select a corresponding data type in the preset database based on the attribute type in the attribute definition to obtain a first parameter setting result;

[0097] A second parameter setting result acquisition subunit, configured to set a corresponding field length for the attribute size in the attribute definition to obtain a second parameter setting result;

[0098] A third parameter setting result acquisition subunit, configured to determine corresponding constraint conditions by using a preset SQL statement based on the attribute constraints in the attribute definition to obtain a third parameter setting result;

[0099] A target entity physical table determination subunit, configured to determine the target entity physical table according to the setting results of the first parameter, the second parameter, and the third parameter.

[0100] In some specific embodiments, the definition unified module 14 may specifically include:

[0101] An attribute-converted data acquisition unit, configured to perform a preset data conversion operation on the data to be governed based on the preset data processing rules in the preset aggregation rule and the mapping result, so as to obtain attribute-converted data;

[0102] An attribute-converted data loading unit, configured to use a preset data storage processor to load the attribute-converted data into the target entity physical table in the preset database based on the preset scheduling policy in the preset aggregation rule; wherein, the preset scheduling policy includes a preset timing execution policy or a preset manual execution policy.

[0103] In some specific embodiments, the data access service construction device may further include:

[0104] An updated target entity physical table acquisition module, configured to correct the attribute-converted data in the target entity physical table that does not meet the preset format requirements by using a preset format verification rule, so as to obtain an updated target entity physical table;

[0105] A uniformly formatted attribute-converted data acquisition module, configured to convert the attribute-converted data in the updated target entity physical table into a unified coding format, so as to obtain uniformly formatted attribute-converted data;

[0106] A data verification module, configured to verify whether the uniformly formatted attribute-converted data meets the preset business rules;

[0107] A preset data processing operation execution module, configured to, if the preset business rules are not met, perform a preset data processing operation on the uniformly formatted attribute-converted data based on the preset business requirements.

[0108] In some specific embodiments, the service construction module 15 may specifically include:

[0109] A preset database access permission allocation unit, configured to allocate corresponding preset database access permissions to each client based on the permission control requirements;

[0110] The post-governance data query unit is used to determine the API interface of the target entity physical table in the preset database, so that the client can query the corresponding post-governance data in the preset database through the API interface based on the preset database access permission.

[0111] Furthermore, an embodiment of the present application also discloses an electronic device. Figure 4 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment. The content in the figure should not be considered as any limitation on the scope of use of the present application.

[0112] Figure 4 It is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the data access service construction method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0113] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and specific limitations are not imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0114] In addition, as a carrier for resource storage, the memory 22 may be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0115] Among them, the operating system 221 is used to manage and control each hardware device and the computer program 222 on the electronic device 20, and it may be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the data access service construction method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs that can be used to complete other specific tasks.

[0116] Further, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the data access service construction method disclosed above. For the specific steps of this method, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated herein.

[0117] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and reference may be made to the description in the method part for relevant parts.

[0118] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0119] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0120] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0121] The above has introduced the technical solution provided by the present application in detail. Specific examples are used in this text to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for constructing a data access service, characterized in that, Including: Obtain a target entity, determine the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in the preset data structure, and determine the attribute definition of the target entity according to the attribute type, the attribute size, and the attribute constraints; Determine the preset database connection information according to the database driver, the database access path, the obtained client name, and the client password, and construct a corresponding physical table of the target entity in the preset database based on the preset data structure, the preset database connection information, and the attribute definition of the target entity; Use a preset data aggregation tool to map the fields in the obtained data to be governed to the corresponding fields in the physical table of the target entity based on the field mapping relationship in the preset aggregation rules to obtain a corresponding mapping result; Load the data to be governed into the physical table of the target entity in the preset database based on the preset data processing rules in the preset aggregation rules and the mapping result to obtain the governed data with unified data attributes; Construct a corresponding data access service based on the governed data, so as to initiate an access to the governed data based on the data access service.

2. The data access service construction method according to claim 1, wherein The determining the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in the preset data structure includes: Obtain a preset data structure based on a preset business scenario and / or a preset data range; Determine the corresponding data logic model according to the preset data structure; Use the data logic model of the preset data structure to determine the attribute type, attribute size, and attribute constraints of the target entity; Wherein, the attribute type includes any one or more of a date attribute, a text attribute, an integer attribute, a fixed-point number attribute, a floating-point number attribute, and a binary data attribute, the attribute size is the range of data that can be stored by the target entity, and the attribute constraints include any one or more of a primary key, a foreign key, and a default value.

3. The method for constructing a data access service according to claim 1, wherein The constructing a corresponding physical table of the target entity in the preset database based on the preset data structure, the preset database connection information, and the attribute definition of the target entity includes: Convert the data logic model of the preset data structure into a preset physical table structure in the preset database based on the preset data structure and the preset database connection information; Construct an initial physical table of the target entity according to the preset physical table structure; Perform a preset parameter setting operation on the initial physical table of the target entity using the attribute definition of the target entity to obtain a physical table of the target entity.

4. The method for constructing a data access service according to claim 3, wherein The performing a preset parameter setting operation on the initial physical table of the target entity using the attribute definition of the target entity to obtain a physical table of the target entity includes: Select a corresponding data type in the preset database based on the attribute type in the attribute definition to obtain a first parameter setting result; Set the corresponding field length for the attribute size in the attribute definition to obtain a second parameter setting result; Determine the corresponding constraint conditions using a preset SQL statement based on the attribute constraints in the attribute definition to obtain a third parameter setting result; Determine the target entity physical table according to the setting results of the first parameter, the second parameter, and the third parameter.

5. The method for constructing a data access service according to claim 1, wherein Loading the data to be governed into the target entity physical table in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result includes: Perform a preset data conversion operation on the data to be governed based on the preset data processing rule in the preset aggregation rule and the mapping result to obtain data with attributes converted; Use a preset data storage processor to load the data with attributes converted into the target entity physical table in the preset database based on the preset scheduling policy in the preset aggregation rule; Wherein, the preset scheduling policy includes a preset timing execution policy or a preset manual execution policy.

6. The method for constructing a data access service according to claim 5, wherein After loading the data to be governed into the target entity physical table in the preset database based on the preset data processing rule in the preset aggregation rule and the mapping result, it further includes: Use a preset format verification rule to correct the data with attributes converted in the target entity physical table that does not meet the preset format requirements to obtain an updated target entity physical table; Convert the data with attributes converted in the updated target entity physical table into a unified encoding format to obtain data with attributes converted in a unified format; Verify whether the data with attributes converted in a unified format meets the preset business rules; If it does not meet the preset business rules, perform a preset data processing operation on the data with attributes converted in a unified format based on the preset business requirements.

7. The method for constructing a data access service according to any one of claims 1 to 6, characterized in that, Constructing a corresponding data access service based on the governed data so as to initiate access to the governed data based on the data access service includes: Allocate corresponding access permissions to the preset database for each client based on the permission control requirements; Determine the API interface of the target entity physical table of the preset database so that the client can query the corresponding governed data in the preset database through the API interface based on the access permissions to the preset database.

8. A data access service construction device, characterized in that Include: An attribute definition determination module, configured to obtain a target entity, determine the attribute type, attribute size, and attribute constraints of the target entity based on the data logic model in the preset data structure, and determine the attribute definition of the target entity according to the attribute type, the attribute size, and the attribute constraints; A physical table construction module, configured to determine preset database connection information according to the database driver, the database access path, the obtained client name, and the client password, and construct a corresponding target entity physical table in the preset database based on the preset data structure, the preset database connection information, and the attribute definition of the target entity; A result acquisition module, configured to use a preset data aggregation tool to map the fields in the obtained data to be governed to the corresponding fields in the target entity physical table based on the field mapping relationship in the preset aggregation rule to obtain a corresponding mapping result; Define a unified module for loading the data to be governed into the target entity physical table in the preset database based on the preset data processing rules in the preset aggregation rules and the mapping result, so as to obtain the governed data with unified data attributes; A service construction module for constructing a corresponding data access service based on the governed data, so as to initiate access to the governed data based on the data access service.

9. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor for executing the computer program to implement the data access service construction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by the processor, the data access service construction method according to any one of claims 1 to 7 is implemented.

Images