Data modeling method and apparatus, and computer device and storage medium

Abstract

Provided in the present disclosure are a data modeling method and apparatus, and a computer device and a storage medium. The method comprises: displaying a modeling interface, wherein the modeling interface is used for displaying a plurality of initial data sources; in response to a target data source being selected, displaying a plurality of pieces of source data under the target data source, and in response to a virtual table creation operation for the source data, displaying first identifiers of first virtual tables corresponding to the source data, wherein the first virtual tables are generated on the basis of data attribute information corresponding to the source data; in response to a selection operation for the first identifiers, displaying the first virtual tables respectively corresponding to the selected first identifiers, and in response to a data processing logic creation operation for the selected first virtual tables, displaying second identifiers of second virtual tables; and in response to a selection operation for third identifiers, displaying third virtual tables respectively corresponding to the selected third identifiers, and in response to a modeling relationship creation operation for the third virtual tables, displaying a target virtual table.

Description

Data modeling method and device, computer device, and storage medium

[0001] The present disclosure claims priority to a Chinese patent application No. 202410738186.4, filed on June 7, 2024, entitled "A data modeling method, device, computer device, and storage medium", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of database, in particular, to a data modeling method, device, computer device, and storage medium. BACKGROUND

[0003] Data from different data sources is often stored in different databases. In order to better manage data in external data sources, for example, data in data systems corresponding to different enterprises, a data modeling method can often be used. The purpose of data modeling is to create a data table with a new database structure or a backup database structure for the data in the database corresponding to the external data source, and then use the data table to implement query and retrieval of the data. A comprehensive optimized data model helps to create a simplified logical database, eliminates redundant data, reduces storage requirements, and enables efficient retrieval.

[0004] In a traditional data modeling process, data is loaded from the source end to the target end using an extract-transform-load (ELT) process before modeling. ETL is mainly responsible for extracting data (such as relational data, data files, etc.) in various data sources (such as various databases) to a temporary intermediate layer for transformation processing, and finally loading the transformed data to the target end. After loading the data to the target end, data modeling is performed on the data to obtain a modeling result. However, the traditional modeling method not only needs to process data under different data sources, but also has the problem of resource waste and security risk caused by the need to load data to the target end, which affects the efficiency of data modeling. SUMMARY

[0005] The present disclosure provides at least a data modeling method, device, computer device, and storage medium.

[0006] In a first aspect, the present disclosure provides a computer device, comprising a processor, a memory, and a bus, the memory storing machine-readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, the machine-readable instructions are executed by the processor to perform a data modeling process, the data modeling process comprises:

[0007] displaying a modeling interface, the modeling interface being used to display a plurality of initial data sources;

[0008] in response to a target data source in the plurality of initial data sources being selected, displaying a plurality of source data under the target data source, and in response to a virtual table creation operation for at least one of the source data, displaying a first identifier of a first virtual table corresponding to each of the source data; the first virtual table being generated according to data attribute information corresponding to the source data;

[0009] in response to a selection operation for the first identifier, displaying first virtual tables corresponding to each of the selected first identifiers respectively, and in response to a data processing logic creation operation for the selected first virtual table, displaying a second identifier of a second virtual table corresponding to the selected first virtual table; the second virtual table being generated according to a data processing logic configured for the selected first virtual table, the data processing logic being used to perform logical processing on data attribute information in the first virtual table;

[0010] in response to a selection operation for the third identifier, displaying third virtual tables corresponding to each of the selected third identifiers respectively, and in response to a modeling relationship creation operation for the third virtual table, displaying a target virtual table; the third identifier including the first identifier and / or the second identifier; the target virtual table being generated according to a modeling relationship configured by a user for the third virtual table and a hierarchical relationship of the third virtual table, the modeling relationship being used to indicate a connection relationship between the third virtual tables and / or associated data attribute information.

[0011] In a possible implementation, in a data modeling process performed by the processor, the target virtual table is generated according to the following steps:

[0012] in a case where the third virtual table includes a second virtual table, according to a hierarchical relationship of the third virtual table, determining a data reference relationship corresponding to the target virtual table and a target data processing logic, the data reference relationship being used to locate source data corresponding to the target virtual table;

[0013] generating the target virtual table according to data attribute information in the third virtual table, a modeling relationship corresponding to the third virtual table, a data processing logic, and the data reference relationship.

[0014] In a possible implementation, in a case where the target data source is an unstructured data source, the source data includes a plurality of data topics, and each data topic includes at least one string data; the processor is configured to determine data attribute information of the source data according to the following steps:

[0015] For any data subject, any string data generated under the data subject is structured to obtain target structured data corresponding to the string data;

[0016] According to the target structured data, determine the data attribute information corresponding to the data subject.

[0017] In a possible implementation, in the case where the target data source is a structured data source, the source data includes various data tables, and each data table includes at least one structured data;

[0018] The processor is configured to determine the data attribute information of the source data according to the following steps:

[0019] Attribute information extraction is performed on any data table to obtain the data attribute information corresponding to each data table.

[0020] In a possible implementation, in the data modeling process performed by the processor, after the target virtual table is displayed, the following steps are further included:

[0021] In response to a query request for the target virtual table, data results queried from the target data source are displayed; the data results are obtained by using a source data engine of the target data source to perform data query on source data positioned by a data reference relationship corresponding to the target virtual table according to the modeling relationship and the data processing logic corresponding to the target virtual table.

[0022] In a possible implementation, in the data modeling process performed by the processor, in response to the data processing logic creation operation for the selected first virtual table, a second identifier of a second virtual table corresponding to the selected first virtual table is displayed, including:

[0023] In response to the data processing logic creation operation for the selected first virtual table, a logic setting page is displayed; the logic setting page is used to display a plurality of preset processing logics.

[0024] Each first preset processing logic selected by a user and a first target logic parameter set for each first preset processing logic are obtained.

[0025] In response to a logic determination operation, a second identifier of a second virtual table of a next level corresponding to the selected first virtual table is displayed; the second virtual table is obtained by performing logic processing on data attribute information in the selected first virtual table by using a data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter.

[0026] In a possible implementation, the data modeling process performed by the processor further comprises, before the operation of selecting the third identifier, the following operation:

[0027] In response to the operation of selecting any second identifier, the second virtual table corresponding to the selected second identifier is displayed.

[0028] In response to the operation of creating a logic for the selected second virtual table, a logic setting page is displayed, and the logic setting page is used to display a plurality of preset processing logics.

[0029] The second preset processing logic selected by the user and the second target logic parameter set for each second preset processing logic are obtained.

[0030] In response to the operation of determining a logic, the second identifier of the newly generated second virtual table is displayed, the newly generated second virtual table is a virtual table at a next level of the selected second virtual table, and the newly generated second virtual table is obtained by performing logic processing on the data attribute information in the selected second virtual table by using the data processing logic generated according to each second preset processing logic and the corresponding second target logic parameter.

[0031] In a possible implementation, the operation of selecting includes an identifier dragging operation.

[0032] The operation of displaying the third virtual table corresponding to each selected third identifier includes the following operation:

[0033] In response to the operation of dragging the third identifier to the table display area, the third virtual table corresponding to each dragged third identifier is displayed.

[0034] The operation of displaying the target virtual table in response to the operation of creating a modeling relationship for the third virtual table includes the following operation:

[0035] In response to the operation of connecting the target data attribute information in different third virtual tables, an association relationship configuration page is displayed, and the association relationship configuration page includes a connection relationship selection control.

[0036] In response to the operation of triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the preset connection relationship selected by the user is used as the first target connection relationship between different third virtual tables.

[0037] In response to the operation of first confirming the creation, the target virtual table is displayed, the association data attribute information in the modeling relationship of the target virtual table is the two target data attribute information connected by the operation of connecting, and the connection relationship in the modeling relationship of the target virtual table includes the first target connection relationship.

[0038] In a possible implementation, in the data modeling process performed by the processor, in response to the modeling relationship creation operation for the third virtual table, the target virtual table is displayed, including:

[0039] In response to the trigger operation for the displayed third virtual table, a modeling relationship setting page is displayed; the modeling relationship setting page displays an associated node setting control and a connection relationship selection control;

[0040] In response to the trigger of the associated node setting control, fourth identifiers of respective fourth virtual tables are displayed, and the fourth identifier selected by the user is determined; the fourth virtual table at least includes the third virtual table;

[0041] In response to the trigger of the connection relationship selection control, a plurality of preset connection relationships are displayed, and the preset connection relationship selected by the user is set as a second target connection relationship between the triggered third virtual table and the fourth virtual table corresponding to the selected fourth identifier;

[0042] In response to the second determination creation operation, a target virtual table is displayed; the connection relationship in the modeling relationship of the target virtual table includes the second target connection relationship.

[0043] In a possible implementation, in the data modeling process performed by the processor, after obtaining the respective first preset processing logic selected by the user and the first target logic parameter set for each first preset processing logic, the data modeling process further includes:

[0044] In response to the logic test operation, a logic test result is displayed; the logic test result is obtained by performing logic processing on source data located by the selected first virtual table corresponding data reference relationship by using the data processing logic generated according to the respective first preset processing logic and the corresponding first target logic parameter.

[0045] In a possible implementation, in the data modeling process performed by the processor, in response to the target data source in the plurality of initial data sources being selected, a plurality of pieces of source data under the target data source are displayed, including:

[0046] In response to the target data source in the plurality of initial data sources being selected, registration information input by the user is obtained;

[0047] In a case where the registration information indicates that the user has access permission for the target data source, a plurality of pieces of source data corresponding to the target data source are displayed.

[0048] In a second aspect, the embodiments of the present disclosure provide a data modeling method, including:

[0049] A modeling interface is displayed, which is used to display a plurality of initial data sources;

[0050] In response to a target data source in the plurality of initial data sources being selected, a plurality of source data under the target data source is displayed, and in response to a virtual table creation operation for at least one source data, a first identifier of a first virtual table corresponding to each source data is displayed; the first virtual table is generated according to data attribute information corresponding to the source data;

[0051] In response to a selection operation for the first identifier, first virtual tables corresponding to each selected first identifier are displayed, and in response to a data processing logic creation operation for the selected first virtual table, a second identifier of a second virtual table corresponding to the selected first virtual table is displayed; the second virtual table is generated according to a data processing logic configured for the selected first virtual table, and the data processing logic is used for logical processing on data attribute information in the first virtual table;

[0052] In response to a selection operation for the third identifier, third virtual tables corresponding to each selected third identifier are displayed, and in response to a modeling relationship creation operation for the third virtual table, a target virtual table is displayed; the third identifier includes the first identifier and / or the second identifier; the target virtual table is generated according to a modeling relationship configured by a user for the third virtual table and a hierarchical relationship of the third virtual table, and the modeling relationship is used to indicate a connection relationship between the third virtual tables and / or associated data attribute information.

[0053] In a possible implementation, the target virtual table is generated according to the following steps:

[0054] In a case where the third virtual table includes a second virtual table, according to a hierarchical relationship of the third virtual table, a data reference relationship corresponding to the target virtual table and a target data processing logic are determined, and the data reference relationship is used to locate source data corresponding to the target virtual table;

[0055] According to data attribute information in the third virtual table, a modeling relationship corresponding to the third virtual table, a data processing logic, and the data reference relationship, the target virtual table is generated.

[0056] In a possible implementation, in a case where the target data source is an unstructured data source, the source data includes a plurality of data topics, and each data topic includes at least one string data;

[0057] The data attribute information of the source data is determined according to the following steps:

[0058] For any of the data topics, any of the string data generated under the data topic is structured to obtain target structured data corresponding to the string data;

[0059] According to the target structured data, the data attribute information corresponding to the data topic is determined.

[0060] In a possible implementation, in the case that the target data source is a structured data source, the source data includes respective data tables, and each data table includes at least one piece of structured data;

[0061] The data attribute information of the source data is determined according to the following steps:

[0062] Attribute information extraction is performed on any of the data tables to obtain the data attribute information corresponding to each data table.

[0063] In a possible implementation, after the target virtual table is displayed, the following further includes:

[0064] In response to a query request for the target virtual table, data results queried from the target data source are displayed; the data results are obtained by using a source data engine of the target data source to perform data query on source data positioned by a data reference relationship corresponding to the target virtual table according to the modeling relationship and the data processing logic corresponding to the target virtual table.

[0065] In a possible implementation, in response to the data processing logic creation operation for the selected first virtual table, a second identifier of a second virtual table corresponding to the selected first virtual table is displayed, including:

[0066] In response to the data processing logic creation operation for the selected first virtual table, a logic setting page is displayed; the logic setting page is used to display a plurality of preset processing logics.

[0067] Each first preset processing logic selected by a user and a first target logic parameter set for each first preset processing logic are obtained.

[0068] In response to a logic determination operation, a second identifier of a second virtual table of a next level corresponding to the selected first virtual table is displayed; the second virtual table is obtained by performing logic processing on data attribute information in the selected first virtual table by using a data processing logic generated according to each first preset processing logic and a corresponding first target logic parameter.

[0069] In a possible implementation, before the third identifier is selected, the following further includes:

[0070] In response to selection of any second identifier, a second virtual table corresponding to the selected second identifier is displayed;

[0071] In response to a logical creation operation on the selected second virtual table, a logical setting page is displayed; the logical setting page is used to display a plurality of preset processing logics;

[0072] Each second preset processing logic selected by the user and a second target logic parameter set for each second preset processing logic are acquired;

[0073] In response to a logical determination operation, a second identifier of a newly generated second virtual table is displayed; the newly generated second virtual table is a virtual table at a next level of the selected second virtual table, and the newly generated second virtual table is obtained by logically processing data attribute information in the selected second virtual table by using a data processing logic generated according to each second preset processing logic and a corresponding second target logic parameter.

[0074] In a possible implementation, the selection operation includes an identifier dragging operation;

[0075] In response to the selection operation on the third identifier, a third virtual table corresponding to each selected third identifier is displayed, including:

[0076] In response to a dragging operation of dragging the third identifier to a table display area, a third virtual table corresponding to each dragged third identifier is displayed;

[0077] In response to the modeling relationship creation operation on the third virtual table, a target virtual table is displayed, including:

[0078] In response to a connection operation on target data attribute information in different third virtual tables, an association relationship configuration page is displayed; the association relationship configuration page displays a connection relationship selection control;

[0079] In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the preset connection relationship selected by the user is used as a first target connection relationship between different third virtual tables;

[0080] In response to a first confirmation creation operation, a target virtual table is displayed; association data attribute information in a modeling relationship of the target virtual table is two target data attribute information connected by the connection operation, and a connection relationship in the modeling relationship of the target virtual table includes the first target connection relationship.

[0081] In a possible implementation, in response to the modeling relationship creation operation on the third virtual table, a target virtual table is displayed, including:

[0082] In response to a triggering operation on the third virtual table, a modeling relationship setting page is displayed; the modeling relationship setting page displays an associated node setting control and a connection relationship selection control;

[0083] In response to triggering the associated node setting control, a fourth identifier of a fourth virtual table is displayed, and the fourth identifier selected by the user is determined; the fourth virtual table at least includes the third virtual table;

[0084] In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and a preset connection relationship selected by the user is set as a second target connection relationship between the triggered third virtual table and the fourth virtual table corresponding to the selected fourth identifier;

[0085] In response to a second determination creation operation, a target virtual table is displayed; the connection relationship in the modeling relationship of the target virtual table includes the second target connection relationship.

[0086] In a possible implementation, after obtaining the first preset processing logic selected by the user and the first target logic parameter set for each first preset processing logic, the method further includes:

[0087] In response to a logic test operation, a logic test result is displayed; the logic test result is obtained by performing logic processing on source data located by the data reference relationship corresponding to the selected first virtual table by using the data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter.

[0088] In a possible implementation, in response to the target data source in the plurality of initial data sources being selected, the plurality of source data under the target data source is displayed, including:

[0089] In response to the target data source in the plurality of initial data sources being selected, registration information input by the user is obtained;

[0090] In a case where the registration information indicates that the user has access permission for the target data source, the plurality of source data corresponding to the target data source is displayed.

[0091] In a third aspect, the embodiments of the present disclosure further provide a data modeling device, including:

[0092] A first display module is configured to display a modeling interface, and the modeling interface is configured to display a plurality of initial data sources;

[0093] The second display module is configured to display a plurality of source data under a target data source in the plurality of initial data sources in response to the target data source being selected, and display a first identifier of a first virtual table corresponding to each of the source data in response to a virtual table creation operation for at least one of the source data; the first virtual table is generated according to data attribute information corresponding to the source data;

[0094] The third display module is configured to display the first virtual tables corresponding to the selected first identifiers respectively in response to a selection operation for the first identifiers, and display a second identifier of a second virtual table corresponding to the selected first virtual table in response to a data processing logic creation operation for the selected first virtual table; the second virtual table is generated according to a data processing logic configured for the selected first virtual table, and the data processing logic is used for performing logical processing on data attribute information in the first virtual table.

[0095] The fourth display module is configured to display the third virtual tables corresponding to the selected third identifiers respectively in response to a selection operation for the third identifiers, and display a target virtual table in response to a modeling relationship creation operation for the third virtual tables; the third identifiers include the first identifiers and / or the second identifiers; the target virtual table is generated according to a modeling relationship configured by a user for the third virtual tables and a hierarchical relationship of the third virtual tables, and the modeling relationship is used for indicating a connection relationship between the third virtual tables and / or associated data attribute information.

[0096] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, the steps of the above-mentioned second aspect or any possible implementation manner of the second aspect are executed.

[0097] For the effects of the above-mentioned data modeling device, computer equipment and computer readable storage medium, refer to the description of the above-mentioned data modeling method, which will not be repeated here.

[0098] The data modeling method, device, computer device and storage medium provided by the embodiments of the present disclosure correspond different initial data sources to different databases, and after any data source in the plurality of initial data sources is selected as a target data source, a first virtual table is generated according to data attribute information, and the first virtual table does not include specific source data, so that in the visual modeling interface, the conversion and modeling of the source data in the database can be realized without copying the source data in the target data source by using the first virtual table created for the source data in the target data source, and the problem of resource waste and security risk caused by repeated data copying is avoided, thereby improving the data modeling efficiency. Since the target data source can be any one of the plurality of initial data sources, and after the target data source is selected, the modeling of the target data source can be realized by logically configuring the first virtual table and configuring the modeling relationship in the visual modeling interface, so that the differences between different data sources can be shielded by creating a virtual table, and the data in different target data sources can be processed by using the visual modeling interface and the first virtual table, thereby realizing non-discriminatory data processing and modeling, and thus the data under different data sources does not need to be processed specifically. Moreover, by using the modeling method of creating a virtual table, the data conversion (i.e. logical processing) and data modeling processes can be combined together, thereby ensuring the consistency between data conversion and data modeling, and avoiding invalid data conversion work. After the modeling is completed, not only the data logic in the database can be simplified and the redundant data in the database can be eliminated, but also the data in the database can be queried and searched by using the modeling result, thereby improving the query and search efficiency of the data in the database.

[0099] In order to make the above objectives, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0100] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. The drawings are incorporated into the specification and form a part of the specification, which show the embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as a limitation on the scope, and other related drawings can also be obtained by those skilled in the art without creating any creative labor.

[0101] FIG. 1 shows a flowchart of a data modeling method according to an embodiment of the present disclosure;

[0102] FIG. 2 shows a schematic diagram of a modeling interface according to an embodiment of the present disclosure;

[0103] FIG. 3 shows a display diagram of source data according to an embodiment of the present disclosure;

[0104] FIG. 4 shows a detail page diagram of a data table according to an embodiment of the present disclosure;

[0105] FIG. 5 shows another modeling interface diagram according to an embodiment of the present disclosure;

[0106] FIG. 6 shows a logical setting page diagram according to an embodiment of the present disclosure;

[0107] FIG. 7 shows a setting page of multiple data processing logics and a logical test result display diagram according to an embodiment of the present disclosure;

[0108] FIG. 8 shows a modeling relationship setting page diagram according to an embodiment of the present disclosure;

[0109] FIG. 9 shows a query diagram according to an embodiment of the present disclosure;

[0110] FIG. 10 shows a modeling flow diagram according to an embodiment of the present disclosure;

[0111] FIG. 11 shows a data modeling device diagram according to an embodiment of the present disclosure;

[0112] FIG. 12 shows a computer device structure diagram according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0113] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The components of the embodiments of the present disclosure described and shown herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present disclosure.

[0114] It is found through research that in the traditional data modeling process, after the data from different databases is loaded into the system by the ETL process, the data modeler can associate the data tables loaded in the system in the modeling process, and through different modeling methods, the relationship between different data tables is determined. However, since the data tables in the system are tables loaded by the ETL process, the data modeler cannot perform secondary processing on the data tables during the modeling process. If the data modeler wants to perform secondary processing on part of the loaded data tables during the modeling process, the modeling process needs to be interrupted and the ETL process needs to be performed again, and then the data tables obtained after the ETL process are used to perform modeling again, which greatly affects the efficiency of data modeling. In order to avoid the problem of repeated ETL and improve the modeling efficiency, a tool, namely data build tool (dbt), is proposed. Dbt can realize data modeling by combining ETL and data modeling together, but when the data modeler uses dbt, there are many dbt concepts to learn, which increases the difficulty of use. Moreover, when using dbt modeling, not only the data under different data sources needs to be processed, but also the source data needs to be loaded into the system, which causes resource waste and security risk, and the effect of improving the efficiency of data modeling is limited.

[0115] Based on the above research, the present disclosure provides a data modeling method and device, computer equipment and storage medium. Different initial data sources correspond to different databases. After any data source in the plurality of initial data sources is selected as a target data source, a first virtual table is generated according to data attribute information, and the first virtual table does not include specific source data. In the visual modeling interface, the first virtual table created for the source data in the target data source can be used to realize the conversion and modeling of the source data in the database without copying the source data in the target data source, thereby avoiding the problem of resource waste and security risk caused by repeated data copying, and improving the data modeling efficiency. Since the target data source can be any one of the plurality of initial data sources, and after the target data source is selected, the first virtual table is logically configured and the modeling relationship is configured in the visual modeling interface, the modeling of the target data source can be realized. Therefore, by creating a virtual table, the differences between different data sources can be shielded, and the data in different target data sources can be processed using the visual modeling interface and the first virtual table, thereby realizing non-discriminatory data processing and modeling, and eliminating the need for targeted processing of data under different data sources. Moreover, using the modeling method of creating a virtual table can combine the data conversion (i.e. logical processing) and data modeling processes together, thereby ensuring the consistency between data conversion and data modeling, and avoiding invalid data conversion work. After the modeling is completed, not only can the data logic in the database be simplified and the redundant data in the database be eliminated, but also the modeling results can be used to query and search the data in the database, thereby improving the query and search efficiency of the data in the database.

[0116] It should be noted that similar reference numerals and letters refer to like items in the accompanying drawings, and thus, once an item is defined in one drawing, it is not necessary to further define and explain it in subsequent drawings.

[0117] The term "and / or" herein is merely descriptive and represents three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the term "at least one" herein represents any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0118] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the personal information (including but not limited to attribute information, face image, etc.) involved in the present disclosure is obtained with the authorization of the user. Specifically, the user can be prompted to authorize by a pop-up window in the page, information push, etc., and the above personal information can be obtained after the user agrees to authorize.

[0119] It should be noted that the specific terms mentioned in the embodiments of the present disclosure include:

[0120] Hive: a data warehouse tool used for data extraction, transformation, and loading, which is a mechanism for storing, querying, and analyzing large-scale data;

[0121] Oracle: a relational database management system, which is portable, easy to use, and powerful, and is suitable for various large, medium, and small computer environments;

[0122] OceanBase: a native distributed database;

[0123] Hologress: a relational database focusing on real-time query and analysis, which uses columnar storage and indexing optimization techniques to provide low query latency and high-performance query response;

[0124] MySQL: a relational database management system, which saves data in different tables instead of storing all data in a large warehouse;

[0125] StarRocks: an open-source new-generation ultra-fast all-scenario database that can efficiently support multi-dimensional analysis, real-time analysis, high-concurrency analysis, and other data analysis scenarios at large data levels;

[0126] Kafka: a high-throughput distributed publish-subscribe message system that can handle all action stream data of consumers in a website;

[0127] Message Queue: Message Queue, abbreviated as MQ, refers to a container that saves messages, which is essentially a queue that saves data;

[0128] Kerberos: a computer network authorization protocol used to authenticate the identity of individuals communicating in a non-secure network in a secure manner;

[0129] SQL: Structured Query Language, a database query and programming language used to access data and query, update, and manage relational database systems;

[0130] Case when: a SQL syntax used to indicate that once a certain WHEN is met, the data will exit the CASE WHEN and no longer consider other CASEs.

[0131] For the convenience of understanding the present embodiment, first, a data modeling method disclosed by the present embodiment is introduced in detail, and the execution subject of the data modeling method provided by the present embodiment is generally a terminal device or other processing device with certain computing power, wherein the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a personal digital processing device (Personal Digital Assistant, PDA), a handheld device, a computer device, etc.; in some possible implementation manners, the data modeling method can be realized by a processor calling computer readable instructions stored in a memory.

[0132] Next, the data modeling method provided by the present embodiment is described by taking a client as an example.

[0133] As shown in FIG. 1, a flowchart of the data modeling method provided by the present embodiment can include the following steps:

[0134] S101: display a modeling interface, the modeling interface is used to display a plurality of initial data sources.

[0135] Here, the modeling interface is an interface provided to a user (also referred to as a data modeler) for data modeling. In the modeling interface, a plurality of types of initial data sources can be displayed, and different types of initial data sources can have different data storage manners and be related to different databases or data queues. Exemplarily, the initial data sources can include but are not limited to Hive, Oracle, OcaeanBase, Hologress, Postgres, MySQL, StarRocks, Kafka and message queue.

[0136] As shown in FIG. 2, a schematic diagram of a modeling interface provided by the present embodiment is shown. Specifically, the data modeling method provided by the present embodiment can take a developed modeling tool as a carrier, and after a user opens the modeling tool, the modeling interface can be displayed. In FIG. 2, a data source type selection control is displayed in the modeling interface. When the user triggers the data source type selection control, a plurality of types of initial data sources can be displayed in response to the triggering operation. In FIG. 2, four types of data sources (i.e., Hive, Oracle, OcaeanBase and Hologress) are shown, but in actual application, various types of data sources supported by the present embodiment can be displayed.

[0137] S102: in response to a target data source in the plurality of initial data sources being selected, displaying a plurality of source data under the target data source, and in response to a virtual table creation operation for at least one source data, displaying a first identifier of a first virtual table corresponding to each source data; the first virtual table is generated according to data attribute information corresponding to the source data.

[0138] Here, the target data source is one of the plurality of initial data sources that is selected. There can be at least one source data under each initial data source, and different source data can come from different business scenarios, different business lines, etc. For example, one application scenario of the data modeling method provided by the embodiments of the present disclosure is to model data from any external database. Different types of data sources have different structural attribute information, and different source data corresponding to different structural attributes. In the case of unstructured data attributes of the structural attribute information of the data source, the source data can include various data topics, and each data topic includes at least one string data; in the case of structured data attributes of the structural attribute information of the data source, the source data can include various data tables, and each data table includes at least one structured data. For example, in the case of a kafka queue as a data source, it can be known that the data in the kafka queue is streaming string data (i.e. string data arriving one by one at an indefinite time), and different string data can correspond to different data topics; in the case of a Hive database as a data source, the Hive database can include a plurality of divided data tables, each data table includes a certain amount of structured data, and each data table has corresponding data attribute information. For example, in specific implementation, the data attribute information can be the schema information corresponding to the data table. One source data can correspond to one first virtual table, one virtual table corresponds to one first identifier, and different virtual tables correspond to different first identifiers.

[0139] In implementation, the selected initial data source can be taken as a target data source in response to a selection operation (e.g., clicking) of any one of a plurality of initial data sources displayed in the modeling interface. Then, a plurality of source data matching the structural attribute information of the target data source can be acquired and displayed. The source data matching the structural attribute information of the target data source can include data tables or data topics. For example, in the case that the target data source has unstructured data attributes, the source data is a data topic; in the case that the target data source has structured data attributes, the source data is a data table. For displaying the source data, specific information corresponding to each source data can be directly displayed, or identification information of each source data can be displayed. For example, in the case that the source data is a data table, a complete data table can be directly displayed, or identification information of each data table can be displayed. For example, in response to a selection operation on a Hive data source displayed in the modeling interface, identification information of a plurality of data tables under the Hive data source is acquired and displayed. Alternatively, in response to a selection operation on a kafka data source displayed in the modeling interface, identification information of a plurality of data topics under the kafka data source is acquired and displayed.

[0140] As shown in FIG. 3, a display diagram of source data provided by an embodiment of the present application is shown, wherein the target data source is a hive data source, the database corresponding to the data source is an abcde database, each source data shown in FIG. 3 is a data table name of each data table, in addition, the data source to which each data table belongs, the database to which each data table belongs, the button (i.e., the "details" mark in FIG. 3) for querying the details of each data table, and the button (i.e., the "create a virtual table attached to the source" mark in FIG. 3) for creating a first virtual table corresponding to each data table are also shown. If a user triggers the details button of a certain data table, one or more of the table data, data attribute information, and data partition information corresponding to the data table can be queried from the database to which the data table belongs and displayed to the user, wherein the data attribute information can be the schema information of the data table. If the user does not trigger the details button of the data table, the corresponding table data or data attribute information does not need to be queried from the database to which the data table belongs. As shown in FIG. 4, a details page diagram of a data table provided by an embodiment of the present application is shown, wherein the page shown in FIG. 4 can be a page displayed in response to the user triggering the details button corresponding to the table name 1 in FIG. 3, the page shows the schema information preview mark, the data preview mark, and the data partition preview mark of the data table corresponding to the table name 1, and the user can view the corresponding information by triggering the three marks. In FIG. 4, the default triggered mark is the schema information preview mark, and the field information of each field under the schema information is shown in the diagram, the field information can include the field name, the field type, whether it can be null, the primary key, the partition key, the partition format, the field description, and the like. In FIG. 4, the "create a virtual table" mark is also shown, and by triggering the mark, a first virtual table corresponding to the data table with the table name 1 can be created.

[0141] In addition, in FIG. 3, the mark (i.e., the "batch create a virtual table attached to the source" mark) for simultaneously creating a first virtual table for multiple data tables is also shown, and by triggering the mark, the user can select multiple data tables and synchronously create corresponding first virtual tables for the multiple data tables.

[0142] After the plurality of pieces of source data are displayed, in response to a virtual table creation operation for at least one piece of source data, a first virtual table corresponding to the source data can be created by using a virtualization modeling engine corresponding to the modeling tool according to data attribute information of the source data, and a first identifier of the first virtual table is determined. Then, the client can obtain the first identifier of the generated first virtual table and display it to the user. It should be noted that when the first virtual table corresponding to a certain data table is generated, only the data attribute information (specifically, the schema information) of the data table is used, and neither the specific data included in the data table nor the specific data included in the data table is loaded locally (i.e., the modeling tool end provided by the embodiment of the present disclosure). Therefore, the first virtual table generated does not include any specific data, but only includes the data attribute information corresponding to the data table. In this way, the data table in the external data source is mapped to a virtualization system by creating a first virtual table that is attached to the source, the data table in the external data source is referenced, and the conversion from a physical entity table to a virtualization is completed.

[0143] In addition, after the first virtual table corresponding to a data table is created, a data reference relationship between the data table and the first virtual table can be created. The data reference relationship can be used to locate the data table corresponding to the first virtual table and query specific data from the data table when the first virtual table is queried.

[0144] Taking FIG. 3 as an example, in response to a triggering operation of the user on the "create a virtual table attached to the source" identifier corresponding to the name 1 in FIG. 3, a first virtual table corresponding to the data table with the name 1 can be created by using the virtualization modeling engine according to the data attribute information of the data table, and a corresponding first identifier is determined. Then, the client can obtain the first identifier and display it. In implementation, the user can trigger the "create a virtual table attached to the source" identifier corresponding to different data table names according to specific data processing requirements, so as to realize the virtual table creation of the data tables corresponding to the data table names. Then, the client can display the first identifiers of the generated first virtual tables corresponding to the data table names to the user. Of course, the user can also use the "batch create a virtual table attached to the source" identifier in FIG. 3 to realize the batch creation of a plurality of first virtual tables corresponding to a plurality of data tables.

[0145] In an embodiment, the step of displaying a plurality of pieces of source data can also be implemented according to the following steps:

[0146] In response to the target data source in the plurality of initial data sources being selected, the registration information input by the user is obtained; in a case where the registration information indicates that the user has access authority for the target data source, a plurality of pieces of source data corresponding to the target data source are displayed.

[0147] Here, the registration information can be permission verification information input by the user, such as a username and password, a verification code, a verification password, Kerberos authentication information, and the like. In a case where the registration information indicates that the user has access permission, it can be indicated that the registration is successful, and the user has permission to reference the target data source.

[0148] For example, in FIG. 2, there can also be alternative authentication methods, i.e., username / password authentication and Kerberos authentication, and the user can select any authentication method to input registration information. In FIG. 2, taking the selected authentication method, i.e., username / password authentication, as an example, in a case where this authentication method is selected, there can also be a username input box, a password input box, a data source note information input box, a test data source identifier for testing the data source, and a connection data source identifier for connecting the data source. Among them, the test data source identifier, when triggered, can be used to test the security and stability of the target data source selected by the user; the connection data source identifier, when triggered, can be used to determine whether the registration information input by the user is correct, in a case where it is correct, it is determined that the user has access permission to the target data source, the target data source is connected to realize the reference of the target data source, and then a plurality of source data corresponding to the target data source can be obtained and displayed; in a case where the registration information is incorrect, the target data source is refused to be connected, and a registration failure prompt information is displayed.

[0149] For example, after the user selects the target data source, the username and password information input by the user can be obtained, and in response to the user triggering the operation of the connection data source identifier, the username and password information are used as registration information by the virtualization modeling engine to determine whether the registration information indicates that the user has access permission, if yes, the target data source is connected and a plurality of data sources as shown in FIG. 3 are displayed, if not, a registration failure prompt information is generated and displayed.

[0150] In this way, by inputting registration information and verifying the user using the registration information, it can be accurately determined whether the user has permission to use the external data source. In a case where the user has the use permission, by connecting the data source, the reference of the data source can be realized without copying the entity data in the data source to the local. Moreover, no matter which kind of external initial data source is selected as the target data source, the reference of the external data source can be completed by using the registration information, and there is no need to perform a complex EL process on the external data source, thereby reducing the reference complexity of different types of data sources and quickly supporting a large number of data sources.

[0151] S103: in response to the selection operation on the first identifier, display the first virtual table corresponding to each selected first identifier, respectively, and in response to the data processing logic creation operation on the selected first virtual table, display the second identifier of the second virtual table corresponding to the selected first virtual table; the second virtual table is generated according to the data processing logic configured for the selected first virtual table, and the data processing logic is used for logical processing of the data attribute information in the first virtual table.

[0152] Here, the selection operation on the first identifier can be a single-click operation, a double-click operation, a drag operation, etc. The specific selection operation can be set according to experience, which is not limited here. The specific first identifier to be selected can be determined according to the actual data processing business needs. Different needs require processing of different first virtual tables. The data modeling process can include two processes, namely the virtual table data processing process and the virtual table modeling process. The virtual table data processing process is to execute the data processing logic creation operation on the virtual table, that is, to process the data attribute information in the virtual table using the data processing logic. The virtual table modeling process is to execute the modeling relationship creation operation on the virtual table, that is, to build the modeling relationship between the virtual tables to obtain the target virtual table. The data modeling method provided by the embodiment of the application supports completing the virtual table data processing process and the virtual table modeling process in one modeling interface.

[0153] The data processing logic creation operation is used to create a conversion logic (i.e. T logic in ELT) for the data attribute information in the first virtual table. For example, the data processing logic can be a logic for filtering fields in the first virtual table, a logic for weighted summation of fields in the first virtual table, etc.

[0154] The second virtual table is a new virtual table obtained by processing the data attribute information in the first virtual table using the data processing logic created by the user. Since the data processing logic is created according to the business needs, each second virtual table obtained by creation can also be called a business virtual table. The second identifier is used to identify the second virtual table, and one second virtual table corresponds to one second identifier.

[0155] Taking the drag operation as an example, in response to the operation of the user dragging any first identifier to the modeling area in the modeling page, the first virtual table corresponding to the first identifier is displayed in the modeling area. Then, in response to the data processing logic creation operation of the user on the displayed first virtual table, the data processing logic created by the user is obtained. Using the virtualization modeling engine, the data attribute information in the first virtual table is logically processed according to the data processing logic, and the second virtual table and the corresponding second identifier are generated. Then, the client can display the second identifier in the table identifier display area corresponding to the business virtual table.

[0156] For ease of understanding, as shown in FIG. 5, another modeling interface provided by the embodiment of the present application is shown, wherein a virtual table identifier is displayed in the modeling interface, after triggering the virtual table identifier, a first virtual table identifier (i.e., a source virtual table identifier) and a second virtual table identifier (i.e., a business virtual table identifier) can be displayed. After triggering the first virtual table identifier, the first identifiers of each first virtual table can be displayed in the table identifier display area of the modeling interface; after triggering the second virtual table identifier, the first identifiers of each second virtual table can be displayed in the table identifier display area of the modeling interface. In FIG. 5, the second virtual table identifier is triggered, the second identifiers of a plurality of second virtual tables that have been created are displayed in the table identifier display area of the modeling interface, and three second virtual tables are displayed in the modeling area of the modeling interface.

[0157] S104: in response to a selection operation on the third identifiers, the third virtual tables corresponding to the selected third identifiers are displayed, and in response to a modeling relationship creation operation on the third virtual tables, a target virtual table is displayed; the third identifiers include the first identifiers and / or the second identifiers; the target virtual table is generated according to the modeling relationship configured by the user for the third virtual tables and the hierarchical relationship of the third virtual tables, and the modeling relationship is used to indicate the connection relationship between the third virtual tables and / or the associated data attribute information.

[0158] Here, the third identifiers can be the second identifiers and / or the first identifiers, and the target virtual table is the final modeling result of the data after the user performs modeling. The third virtual table is a virtual table with a third identifier. The modeling relationship is used to indicate the table connection relationship between the third virtual tables for creating the target virtual table and / or the associated data attribute information. The connection relationship can include left join, right join, inner join, full outer join, cross join, etc. The associated data attribute information is used to represent the fields associated between the third virtual tables, also known as the association key between the third virtual tables. For example, in the case of schema information as the data attribute information, the associated data attribute information can be the field information associated between different third virtual tables. The hierarchical relationship is a nested relationship, and the hierarchical relationship of the third virtual table is used to represent the virtual tables of each layer corresponding to the third virtual table. For example, in the case that the third virtual table is created by using the second virtual table 1, the second virtual table 1 is created by using the second virtual table 2, and the second virtual table 2 is created by using the first virtual table 1, the hierarchical relationship of the third virtual table can be that the first layer is the first virtual table 1, the second layer is the second virtual table 2, the third layer is the second virtual table 1, and the fourth layer is the third virtual table.

[0159] The modeling relationship creation operation is used to create an association between different third virtual tables. The specific modeling relationship and the third identifier to be used are determined according to specific business requirements. The target virtual table does not include any entity data in the target data source, but only includes data attribute information, data processing logic, and data reference relationships. Therefore, the target virtual table does not occupy entity data, and has very low storage costs. However, because the target virtual table has corresponding representation information, a user can still perform a series of operations such as querying and referencing the target virtual table through a common manner.

[0160] For example, when the third identifier includes only the second identifier, in implementation, in response to an operation of dragging a certain number of second identifiers to the modeling area of the modeling interface by a user, the second virtual table corresponding to each of the dragged second identifiers is displayed as a third virtual table in the modeling area. Alternatively, for example, when the third identifier includes the second identifier and the first identifier, in implementation, in response to an operation of dragging a certain number of first identifiers to the modeling area of the modeling interface by a user, and in response to an operation of dragging a certain number of second identifiers to the modeling area of the modeling interface by the user, the second virtual table corresponding to each of the dragged second identifiers and the first virtual table corresponding to each of the dragged first identifiers are all displayed as third virtual tables in the modeling area.

[0161] Then, after displaying the third virtual tables in the modeling area, in response to a modeling relationship creation operation of the user on each of the displayed third virtual tables, the connection relationship and / or the associated data attribute information between different third virtual tables indicated by the user are obtained. Using the virtualization modeling engine, the target virtual table and the corresponding target identifier are generated according to the obtained connection relationship and / or associated data attribute information, the data attribute information in the third virtual table, the data processing logic corresponding to the third virtual table, and the hierarchical relationship of the third virtual table. Then, the client can display the target identifier in the table identifier display area of the modeling interface, and display the target virtual table in the modeling area of the modeling interface.

[0162] For example, in FIG. 5, three second virtual tables are displayed in the modeling area of the modeling interface. The three second virtual tables include one main table and two sub-tables (sub-table 1 and sub-table 2). The associated data attribute information of the main table and the sub-table 1 is current_code and OPDATE, and the connection relationship is left join. The associated data attribute information of the main table and the sub-table 2 is current_name and OPMODE, and the connection relationship is left join.

[0163] In this way, the modeling relationship between multiple virtual tables can be quickly created by means of dragging and graphical modeling, and the graphical modeling can be performed by means of virtualization. In addition, the new virtual table can be quickly and conveniently constructed by selecting the virtual table by means of the graphical method, selecting the associated key between the virtual tables, and configuring the connection relationship, and the complex modeling can be completed based on the nesting relationship of the virtual tables.

[0164] Optionally, after the first virtual table corresponding to the selected first identifier is displayed, the virtualization modeling engine can be used to generate a second virtual table corresponding to the first virtual table and a second identifier corresponding to the second virtual table according to the modeling relationship configured by the user in response to the modeling relationship creation operation of the user for the displayed first virtual table. Then, the generated second identifier can be displayed to the user by the client. In this way, the data modeling method provided in the embodiments of the present application can use the created virtual table to combine the data modeling process and the data processing logic according to the business requirements in the modeling process by means of virtualization modeling, and the data modeling process is simplified.

[0165] In this way, different initial data sources correspond to different databases, and after any data source in the multiple initial data sources is selected as a target data source, the first virtual table is generated according to the data attribute information, and the first virtual table does not include specific source data. Therefore, in the visual modeling interface, the source data in the database can be converted and modeled without copying the source data in the target data source by using the first virtual table created for the source data in the target data source, thereby avoiding the problem of resource waste and security risk caused by repeated copying of data, and improving the data modeling efficiency. Since the target data source can be any one of the multiple initial data sources, and the modeling of the target data source can be implemented by logically configuring the first virtual table and configuring the modeling relationship in the visual modeling interface after the target data source is selected, the differences between different data sources can be shielded by creating the virtual table, and the data in different target data sources can be processed by using the visual modeling interface and the first virtual table, thereby realizing non-discriminatory data processing and modeling, and thus the data under different data sources does not need to be processed specifically. In addition, the data conversion (i.e., logical processing) and the data modeling can be combined by using the modeling method of creating the virtual table, thereby ensuring the consistency between the data conversion and the data modeling, and avoiding invalid data conversion work. In addition, after the modeling is completed, not only the data logic in the database can be simplified, and the redundant data in the database can be eliminated, but also the data in the database can be queried and searched by using the modeling result, thereby improving the query and search efficiency of the data in the database.

[0166] And, by means of virtualization modeling, for any external target data source, it is no longer necessary to copy data repeatedly to the local through data import, and the data application can be quickly performed on the existing data system of the customer, reducing the resource waste and security risks caused by repeated copying of data between different systems. In addition, the data system of the product can be combined with the existing data system of the customer, avoiding the complex docking process of the traditional data system, and through the virtualization method, the data logic processing and modeling process are greatly simplified, and the user can quickly output and deposit data assets.

[0167] In an embodiment, the target virtual table can be generated by the virtualization modeling engine according to the following steps:

[0168] In the case where the third virtual table includes the second virtual table, according to the hierarchical relationship of the third virtual table, the data reference relationship and the target data processing logic corresponding to the target virtual table are determined, and the data reference relationship is used to locate the source data corresponding to the target virtual table;

[0169] According to the data attribute information in the third virtual table, the modeling relationship corresponding to the third virtual table, the data processing logic, and the data reference relationship, the target virtual table is generated.

[0170] Here, since the second virtual table is often obtained by logically processing or modeling processing the first virtual table, the second virtual table often belongs to the virtual table under the first virtual table. At this time, in order to ensure that the source data corresponding to the second virtual table in the target data source can be accurately located during data query, the hierarchical relationship of the second virtual table is used to determine the data reference relationship corresponding to the second virtual table.

[0171] Each virtual table corresponds to a data reference relationship. For the first virtual table, the data reference relationship of the first virtual table can be created when the virtualization modeling engine creates the first virtual table according to the data attribute information of the source data, and the data reference relationship is used to indicate the mapping relationship between the first virtual table and the source data, so that when the first virtual table is queried, the corresponding source data in the target data source can be located and queried according to the data reference relationship. For the second virtual table, the data reference relationship corresponding to the second virtual table can be determined according to the data reference relationship of each first virtual table used when the second virtual table is created. For example, when the second virtual table is obtained by logically processing the first virtual table using user-configured data processing logic, the data reference relationship of the second virtual table can be determined according to the source data indicated by the data reference relationship corresponding to the first virtual table. For example, the second virtual table 1 is created according to the first virtual table 1, the data reference relationship of the first virtual table 1 indicates that the first virtual table 1 is related to the data table 1 in the target data source, and the data reference relationship of the second virtual table 1 is the mapping relationship between the second virtual table 1 and the data table 1.

[0172] When the second virtual table is obtained by modeling and processing a plurality of first virtual tables, the data reference relationship of the second virtual table can be determined according to the source data indicated by the data reference relationship corresponding to each first virtual table. For example, the second virtual table 1 is created according to the first virtual table 1, the first virtual table 2 and the first virtual table 3, the data reference relationship of the first virtual table 1 indicates that the first virtual table 1 is related to the data table 1 in the target data source, the data reference relationship of the first virtual table 2 indicates that the first virtual table 2 is related to the data table 2 in the target data source, and the data reference relationship of the first virtual table 3 indicates that the first virtual table 3 is related to the data table 3 in the target data source. The data reference relationship of the second virtual table 1 is the mapping relationship between the second virtual table 1 and the data table 1, the data table 2 and the data table 3.

[0173] In implementation, when the virtualization modeling engine creates the target virtual table, it can be determined whether the third virtual table includes the second virtual table. If not, it means that the modeling is directly performed on each first virtual table, and there is no setting of data processing logic. Then, the target virtual table can be generated according to the data attribute information in each third virtual table, the modeling relationship corresponding to the third virtual table, and the data reference relationship.

[0174] In the case of including the second virtual table in the third virtual table, the hierarchical relationship of the second virtual table needs to be determined first. According to the hierarchical relationship, the virtual tables under each upper hierarchical level of the second virtual table are determined, and the data reference relationship corresponding to the target virtual table and the target data processing logic are determined according to the data reference relationship and the data processing logic corresponding to the virtual tables respectively. Among them, the target data processing logic can be determined according to the data processing logic of each virtual table above the target virtual table in the hierarchy. Then, after determining the data reference relationship corresponding to the target virtual table and the target data processing logic, the target virtual table corresponding to the data reference relationship and the target data processing logic can be generated according to the data attribute information and the modeling relationship of the third virtual table.

[0175] In an embodiment, in the case that the structure attribute information of the target data source is different, the determination process of the data attribute information of the source data is also different. Specifically, the structure attribute information can be divided into two types, one is the unstructured data attribute indicating that the target data source is an unstructured data source, and the other is the structured data attribute indicating that the target data source is a structured data source. The method of determining data attribute information under different conditions will be introduced as follows:

[0176] Case 1, in the case that the target data source is an unstructured data source, the source data includes each data topic, and each data topic includes at least one string data. For example, in the case that the target data source is a kafka queue, it indicates that the target data source is an unstructured data source, and the structure attribute information of the target data source is an unstructured data attribute. As for the data in the kafka queue, the data in the queue arrives one by one at different times, and the data is all string data. Although the arrival time of different string data in the queue is different, the queue can be subdivided into different data topics (topics), and the string data under each data topic can be updated in real time. However, any string data under the data topic is split, and the result obtained by performing data attribute extraction based on the split result is the same. Based on this attribute of the unstructured data source, the data attribute information can be determined according to the following steps when creating a virtual table:

[0177] For any data topic, any string data generated under the data topic is structured processed to obtain target structured data corresponding to the string data; and the data attribute information corresponding to the data topic is determined according to the target structured data.

[0178] Here, the structured processing is used to parse and split the string data, wherein the structured processing mode corresponding to different types of unstructured data sources is different. The target structured data is each field obtained by structuring the string data.

[0179] Exemplarily, in a case where the target data source is an unstructured data source, each piece of source data displayed on the modeling interface can be a topic identifier of each data topic, and then in response to a virtual table creation operation of any data topic by a user, the virtual table modeling engine can obtain any string data under the data topic from the target data source, and then perform structured processing on the string data to obtain corresponding target structured data. Subsequently, data attribute information of the target structured data can be extracted to obtain data attribute information corresponding to the data topic. Then, the data attribute information can be used to construct a first virtual table corresponding to the data topic. Here, the constructed first virtual table can be a streaming table, which can indicate that the data in the target data source is streaming data, and the table only includes data attribute information.

[0180] In this way, for Kafka unstructured data, the structured management of data under Kafka can be implemented by using a streaming table as a kind of virtual table, the standardized use of non-standard data sources can be implemented, and the data modeling of real-time scenarios can be implemented.

[0181] Case 2, in a case where the target data source is a structured data source, the source data includes each data table, and each data table includes at least one structured data. Here, in a case where the target data source is a Hive data source, it is indicated that the target data source is a structured data source, and the structure attribute information of the target data source is structured data attribute. Since the data included in the structured data source is usually batch data, and each batch of data has been divided into different data tables, and since each data table has corresponding data attribute information, when creating a virtual table, the data attribute information is determined according to the following steps:

[0182] Attribute information extraction is performed on any data table to obtain data attribute information corresponding to the data table.

[0183] Exemplarily, in a case where the target data source is a structured data source, each piece of source data displayed on the modeling interface can be identifier information of each data table, and then in response to a virtual table creation operation of any data table corresponding to the identifier information by a user, the virtual table modeling engine can extract schema information of the data table from the data table provided by the target data source, use the schema information as data attribute information of the data table, and create a first virtual table corresponding to the data table according to the data attribute information.

[0184] In this way, in the case that the target data source respectively provides streaming string data and batch data table data, by using different data attribute information determination methods, virtualization modeling processing can be implemented on the streaming data and the batch data, thereby avoiding the influence of data sources with different structure attribute information on the modeling process, and improving the universality of data modeling.

[0185] In an embodiment, for the step of displaying the second identifier in S103, the following steps can be implemented:

[0186] S103-1: In response to a data processing logic creation operation on the selected first virtual table, a logic setting page is displayed; the logic setting page is used to display a plurality of preset processing logics.

[0187] Here, different preset processing logics have different logic processing functions, and for the same preset processing logic, the corresponding logic function also differs when the set parameters are different. Specifically, the plurality of preset processing logics can include expression processing logic, case when processing logic and data filtering processing logic; wherein the expression processing logic is used to provide a simple field-to-field, field-to-simple number operation rule, which can facilitate simple data processing and can set units to achieve simple data calculation; the case when processing logic is a common data processing method, which can classify data by bucketing to implement classification processing on the data; the data filtering processing logic is another common data processing method, which can set data cleaning logic by setting filtering conditions to clean data that does not meet the conditions and leave data that meets the conditions.

[0188] The data processing logic creation operation can be, for example, an operation of a "data processing" identifier in the modeling interface.

[0189] As shown in FIG. 6, a schematic diagram of a logic setting page provided by an example of the present application is shown. After the first virtual table is displayed, the logic setting page shown in FIG. 6 can be displayed in response to the user clicking the "data processing" icon in the modeling interface. In FIG. 6, a data processing logic setting area is shown, in which a preset processing logic selection control, a new column name setting box of the second virtual table obtained after processing according to the preset data processing logic, an alias setting box, a unit setting box, a field description setting box, and an add data processing logic control are shown. After the preset processing logic selection control is triggered, various preset processing logics can be displayed, and then a preset processing logic selected by the user can be set as the first preset processing logic. After the add data processing logic control is triggered, a new data processing logic setting area can be added below the existing data processing logic setting area in FIG. 6, and then a new data processing logic can be set in the data processing logic setting area. The data processing logics set in different data processing logic setting areas are parallel processing logics for the first virtual table. Meanwhile, in the case that the user needs to set multiple data processing logics, the logic relationship between the multiple data processing logics can be input through a logic relationship setting box displayed in the logic setting page. For example, the data filtering relationship setting box shown in FIG. 7 can be used to set the logic relationship between multiple data filtering logics.

[0190] In addition, a cancel control, a confirm control, and a test control are also displayed in the logic setting page. The cancel control is triggered to cancel the setting of the data processing logic and close the logic setting page. The confirm control is triggered to process the first virtual table according to the data processing logic configured by the user to obtain the second virtual table. The test control is triggered to test the data processing logic configured by the user.

[0191] S103-2: Obtain each first preset processing logic selected by the user, and a first target logic parameter set for each first preset processing logic.

[0192] Here, the first preset processing logic can be regarded as a preset processing logic used in the data processing logic configured for the first virtual table, and the first target logic parameter can be regarded as a logic parameter configured for the first preset processing logic.

[0193] In implementation, if after the logic setting page shown in FIG. 6 is displayed, the user only selects preset processing logic in the data processing logic setting area in FIG. 6 and sets specific logic parameters, the first preset processing logic and the first target logic parameter obtained at this time each include only one; if the user has triggered the add data processing logic control at least once, the preset processing logic selected by the user in the data processing logic setting area in FIG. 6 and the preset processing logic selected each time the add data processing logic control is triggered can each be taken as the first preset processing logic, and the logic parameter set by the user in the data processing logic setting area in FIG. 6 and the logic parameter set each time the add data processing logic control is triggered can each be taken as the first target logic parameter.

[0194] For example, the data processing logic set by the user in each data processing logic setting area can be determined according to the processing logic selection operation and the logic parameter setting operation performed by the user in each data processing logic setting area in the logic setting page.

[0195] S103-3: In response to the logic determination operation, a second identifier of a second virtual table corresponding to a next level of the selected first virtual table is displayed; the second virtual table is obtained by performing logic processing on the data attribute information in the selected first virtual table by using the data processing logic generated according to each first preset processing logic and corresponding first target logic parameter.

[0196] Here, the logic determination operation can be an operation of triggering a determination control in the logic setting page. Since the first virtual table is processed by using the data processing logic configured by the user, the second virtual table obtained can be regarded as a virtual table of a next level of the first virtual table.

[0197] For example, in response to triggering the determination control in the logic setting page, the data attribute information in the first virtual table can be processed by using the virtualization modeling engine according to the first preset processing logic and the first target logic parameter set by the user in each data processing logic setting area, to obtain the second virtual table and the corresponding second identifier, and the data reference relationship of the second virtual table can be determined according to the data reference relationship of the first virtual table.

[0198] In this way, the user can quickly configure data processing logic in a graphical manner, and the difference in SQL statements between different data systems is shielded, and the user can process data in different data sources by using one graphical configuration, and the data processing is realized without difference. In addition, the user can process data by defining data processing logic, which avoids a large amount of data generated by each ETL in the prior art, and instead uses the engine to meet the user's query data requirements, thereby reducing the resources consumed by invalid data calculation.

[0199] In an embodiment, after S103-2 is performed, that is, after the user configures at least one data processing logic for the first virtual table, each configured data processing logic can be tested, and specifically, the logic test result can be displayed in response to a logic test operation. The logic test result is obtained by using the data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter to perform logic processing on the source data positioned by the selected data reference relationship corresponding to the first virtual table.

[0200] Here, the logic test operation can specifically be an operation of triggering a test control in the logic setting page. The logic test result is data filtered from the source data after the source data positioned by the data reference relationship corresponding to the first virtual table is processed according to each data processing logic configured by the user.

[0201] As shown in FIG. 7, it is a setting page of multiple data processing logics and a display of a logic test result provided by an embodiment of the present application. In FIG. 7, three data processing logic setting areas are displayed, and in each data processing logic setting area, the first preset processing logic selected by the user is a data filtering logic. After the user triggers the test control in FIG. 7, the logic test result can be displayed in the right area in FIG. 7. In FIG. 7, the logic test result includes 6 columns, and each column includes 5 value data. The logic test result can be obtained according to the following process: the virtualization modeling engine generates each data processing logic configured by the user according to the first preset processing logic and the corresponding first target logic parameter configured in each data processing logic setting area, and then calls a source processing engine of the target data source to perform a data filtering operation on the source data corresponding to the first virtual table according to the data processing logic, to obtain each filtered data. Then, the virtualization modeling engine can obtain the filtered data and feed back the filtered data to the client as the logic test result. In this way, the client can display the logic test result to the user.

[0202] In this way, by displaying the logic test result, the user can test the configured data processing logic in real time, and determine the rationality of the data processing logic according to the logic test result, and then determine whether the data processing logic needs to be modified, thereby effectively improving the rationality of the configured data processing logic.

[0203] In an embodiment, the modeling method provided by the embodiments of the present disclosure is also applicable to multi-layer data processing logic of a virtual table, so as to implement complex data processing or nested processing. Specifically, before responding to the selection operation for the third identifier, the multi-layer data processing logic can also be set according to the following steps:

[0204] Step one, in response to selecting any second identifier, display the second virtual table corresponding to the selected second identifier.

[0205] For example, after the user triggers the second virtual table identifier in FIG. 5, the second identifiers of the plurality of second virtual tables that have been created can be displayed in the table identifier display area. Then, in response to the operation of the user dragging any second identifier to the modeling area of the modeling interface, the second virtual table corresponding to the second identifier can be displayed in the modeling area.

[0206] Step two, in response to a logic creation operation for the selected second virtual table, display a logic setting page; the logic setting page is used to display a plurality of preset processing logics.

[0207] For example, after the selected second virtual table is displayed, in response to the operation of the user clicking the “data processing” identifier in the modeling interface, the logic setting page shown in FIG. 6 can be displayed.

[0208] Step three, obtain each second preset processing logic selected by the user, and a second target logic parameter respectively set for each second preset processing logic.

[0209] Here, the second preset processing logic can be regarded as a preset processing logic used in the data processing logic configured for the second virtual table; and the second target logic parameter can be regarded as a logic parameter configured for the second preset processing logic.

[0210] In a specific implementation, for the step of obtaining the second preset processing logic and the second target logic parameter, reference can be made to the step of obtaining the first preset processing logic and the first target logic parameter in S103-2 described above, which will not be described herein again.

[0211] Step four, in response to the logical determination operation, display the second identification of the newly generated second virtual table; the newly generated second virtual table is a virtual table of the next level of the selected second virtual table, and the newly generated second virtual table is generated by using the data processing logic according to each second preset processing logic and the corresponding second target logic parameter, and the data attribute information in the selected second virtual table is logically processed.

[0212] Here, the logical determination operation can be an operation of triggering the determination control in the logic setting page. Since the second virtual table is logically processed by using the data processing logic configured by the user, the new second virtual table obtained can be regarded as a virtual table of the next level of the second virtual table.

[0213] Exemplarily, in response to the determination control in the logic setting page, the data attribute information of the second virtual table can be logically processed by using the virtualization modeling engine according to the second preset processing logic and the second target logic parameter respectively set by the user in each data processing logic setting area, to obtain a new second virtual table and the corresponding second identification, and the data reference relationship of the new second virtual table can be determined according to the data reference relationship of the selected second virtual table. Meanwhile, the new second virtual table can be regarded as a virtual table of the next level of the selected second virtual table, and the second identification of the new second virtual table can be displayed in the table identification display area as an identification of the next level of the second identification of the selected second virtual table.

[0214] Optionally, for a certain second virtual table, when the second virtual table is obtained after logically processing the first virtual table, the second identification of the second virtual table can also be displayed in the table identification display area as an identification of the next level of the first identification of the first virtual table.

[0215] It should be noted that after the new second virtual table is generated, if further data processing is needed, the second virtual table corresponding to the selected second identifier can be displayed in response to selection of any new second identifier, and then the next level of second virtual table of the new second virtual table is obtained by returning to step two. In this way, through repeated execution of steps one to four, multiple second virtual tables with multi-level nested relationship can be obtained, and multi-level data processing logic can be implemented. For example, according to data processing logic 1, the second virtual table 1 is processed, and the virtual table at the next level of the second virtual table 1 is the second virtual table 11; according to data processing logic 2, the second virtual table 11 is processed, and the virtual table at the next level of the second virtual table 11 is the second virtual table 111; according to data processing logic 3, the second virtual table 111 is processed, and the virtual table at the next level of the second virtual table 111 is the second virtual table 1111, wherein the nested relationship of the multi-level data processing logic is that the next level of the data processing logic 1 is the data processing logic 2, and the next level of the data processing logic 2 is the data processing logic 3.

[0216] In an embodiment, the selection operation for the first identifier, the second identifier or the third identifier can include an identifier dragging operation. Specifically, the identifier dragging operation can be an operation of dragging the identifier from the table identifier display area to the modeling area. In this case, the operation of displaying the virtual table corresponding to the third identifier in S104 can be implemented according to the following steps:

[0217] In response to the dragging operation of dragging the third identifier to the table display area, the third virtual table corresponding to each third identifier dragged is displayed.

[0218] For example, in response to the identifier dragging operation for any identifier displayed in the table identifier display area in FIG. 5, the identifier is taken as the third identifier, and the third virtual table corresponding to the identifier is displayed in the modeling area.

[0219] Further, the step of displaying the target virtual table in S104 can also be implemented according to the following steps:

[0220] S104-1: In response to the connection operation for the target data attribute information in the different third virtual tables, a correlation relationship configuration page is displayed; the correlation relationship configuration page displays a connection relationship selection control.

[0221] Here, the modeling relationship refers to the association relationship and the associated data attribute information between at least two virtual tables. In modeling, the selected multiple third virtual tables can include one master table and multiple slave tables.

[0222] The target data attribute information is data attribute information selected by the user and required to establish a connection relationship. Specifically, the target data attribute information can be two data attribute information required to establish a connection relationship between any two third virtual tables. For example, attribute 1 of the third virtual table 1 and attribute 2 of the third virtual table 2 are required to establish a connection relationship, and the attribute 1 and the attribute 2 can be a group of target data attribute information. The attribute 1 of the third virtual table 1 and the attribute 2 of the third virtual table 2 are required to establish a connection relationship, and the attribute 1 and the attribute 2 can be a group of target data attribute information. The attribute 2 of the third virtual table 1 and the attribute 3 of the third virtual table 3 are required to establish a connection relationship, and the attribute 1 and the attribute 3 can be a group of target data attribute information.

[0223] For any virtual table displayed in the modeling area, each data attribute information of the virtual table can be displayed to the user. For example, in FIG. 5, the 10 data attribute information of the main table is displayed to the user in the form of a column. The connection operation can be an operation of establishing a connection between data attribute information in different third virtual tables. The association relationship configuration page is used to configure the connection relationship between any group of target data attribute information. The connection relationship selection control can display various preset connection relationships for selection after being triggered, wherein the preset connection relationship can include left join, right join, inner join, full join, cross join, etc.

[0224] For example, after displaying each third virtual table selected by the user in the modeling area, the connection between the target data attribute information in any two third virtual tables can be displayed in response to a connection operation of the user establishing a connection between the target data attribute information, and the association relationship configuration page can be displayed.

[0225] S104-2: In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the preset connection relationship selected by the user is used as the first target connection relationship between different third virtual tables.

[0226] For example, in response to the operation of the user clicking the connection relationship selection control, a plurality of selectable preset connection relationships can be displayed in the association relationship configuration page. Then, in response to the selection operation (such as a click operation) of the user for any preset connection relationship, the selected preset connection relationship is used as the first target connection relationship between the target data attribute information, and the first target connection relationship is also used as the connection relationship between the two third virtual tables corresponding to the target data attribute information.

[0227] Taking the three second virtual tables shown in FIG. 5 as an example of third virtual tables, in FIG. 5, there is a connection between the current_code attribute of the main table and the OPDATE attribute of the secondary table 1, and thus the current_code attribute and the OPDATE attribute are a set of target data attribute information, and the first target connection relationship of the set of target data attribute information is a left join. There is a connection between the current_name attribute of the main table and the OPMODE attribute of the secondary table 2, and thus the current_name attribute and the OPMODE attribute are a set of target data attribute information, and the first target connection relationship of the set of target data attribute information is also a left join.

[0228] In this way, based on the initiated connection operation between the data attribute information of the user in different third virtual tables, and the first target connection relationship selected for different connection operations, different modeling relationships can be established between the displayed third virtual tables.

[0229] Optionally, the modeling method provided by the embodiments of the present disclosure also aims at multi-layer modeling relationships of virtual tables to implement complex data modeling or nested modeling. Still taking the virtual tables shown in FIG. 5 as an example, after the modeling relationship between the main table 1 and the secondary table 1 or the secondary table 2, if multi-layer modeling needs to be performed, a new third virtual table can be displayed in the modeling area, for example, a secondary table 3 is displayed in FIG. 5. Then, in response to the connection operation on the target data attribute information in the secondary table 1 (or the secondary table 2) and the secondary table 3, and according to the preset connection relationship selected by the user, the connection relationship between the target data attribute information in the secondary table 1 (or the secondary table 2) and the secondary table 3 is determined, and the connection relationship is simultaneously used as the connection relationship between the secondary table 1 (or the secondary table 2) and the secondary table 3. In this way, one layer of modeling relationship between the main table and the secondary table 1 (or the secondary table 2) can be formed, and the secondary table 1 (or the secondary table 2) and the secondary table 3 have the next layer of modeling relationship.

[0230] It can be understood that the number of layers of modeling relationships that need to be established can be determined according to specific business needs, and the embodiments of the present disclosure do not make specific limitations.

[0231] S104-3: In response to the first confirmation creation operation, display the target virtual table; the associated data attribute information in the modeling relationship of the target virtual table is the two target data attribute information connected by the connection operation, and the connection relationship in the modeling relationship of the target virtual table includes the first target connection relationship.

[0232] Here, the association relationship configuration page can display a creation identifier, and the first confirmation creation operation can be an operation of the user clicking the creation identifier.

[0233] For example, in response to a user clicking the operation of creating the identifier in the association relationship configuration page, the virtualization modeling engine is utilized to determine the group target data attribute information determined in S104-1 as the association data attribute information, and to determine the first target connection relationship determined in S104-2 as the connection relationship, and to generate the target virtual table and the corresponding target identifier according to the data processing logic and the data attribute information corresponding to the third virtual table. Then, the client can display the target identifier in the table identifier display area, and display the target virtual table in the modeling area.

[0234] In this way, the virtual table provides the modeling capability for virtualization modeling, and the user can model between the offline table and the real-time table by selecting the third virtual table. The modeling through the interface is very simple, and the user only needs to select the third virtual table, select the corresponding join relationship and the association key, and can quickly complete the modeling between the virtual tables. Moreover, through the virtualization and graphical modeling, the virtual table data processing process and the virtual table modeling process can be integrated together, ensuring the consistency of the virtual table data processing and the data modeling, and avoiding invalid virtual table data processing work.

[0235] In an embodiment, the step of displaying the target virtual table in S104 can also be implemented according to the following steps:

[0236] A1: In response to a triggering operation for the displayed third virtual table, a modeling relationship setting page is displayed; the modeling relationship setting page displays an association node setting control and a connection relationship selection control.

[0237] Here, the triggering operation can include, for example, an operation of clicking a preset control on the third virtual table, an operation of double-clicking the third virtual table, etc. The modeling relationship setting page is used to set the modeling relationship of the third virtual table, specifically to set the association node (i.e. the associated other third virtual table) of the third virtual table, the connection relationship between the association node, and the association data attribute information between the association node, etc. The modeling relationship setting page at least displays the association node setting control and the connection relationship selection control, the association node setting control is used to display the selectable nodes after being triggered, and the connection relationship selection control is used to display the selectable preset connection relationship after being triggered.

[0238] Still taking the main table, the secondary table 1 and the secondary table 2 as an example shown in FIG. 5, a modeling relationship setting page shown in FIG. 8 can be displayed in response to a trigger operation of the user for the secondary table 1. The association node setting control in FIG. 8 is the control corresponding to "please select the parent node of the current secondary table", and the connection relationship selection control in FIG. 8 is the control corresponding to "the join relationship with the parent node". In addition, the time column setting control (i.e., the control corresponding to "please select the time column of the secondary table"), the confirm button and the cancel button are also displayed in FIG. 8. The time column setting control is used to set the time column of the virtual table after being triggered, the confirm button is used to determine to create the modeling relationship after being triggered, and the cancel button is used to cancel to create the modeling relationship after being triggered.

[0239] A2: in response to triggering the association node setting control, displaying fourth identifiers of each fourth virtual table, and determining the fourth identifier selected by the user; the fourth virtual table at least includes the third virtual table.

[0240] Here, the fourth virtual table can at least include each of the third virtual tables displayed. Optionally, the fourth virtual table can include the third virtual table, the second virtual table and the first virtual table. The fourth identifier is the identifier of the fourth virtual table.

[0241] For example, in response to the user triggering the association node setting control in FIG. 8, the fourth identifiers of each fourth virtual table that can be associated with the secondary table 1 and can be the main table of the secondary table 1 are displayed. Then, in response to the selection operation of the user for any fourth identifier displayed, the fourth identifier is taken as the fourth identifier selected by the user, and the fourth virtual table corresponding to the fourth identifier selected by the user is taken as the associated virtual table of the secondary table 1.

[0242] A3: in response to triggering the connection relationship selection control, displaying a plurality of preset connection relationships, and taking the preset connection relationship selected by the user as the second target connection relationship between the third virtual table triggered and the fourth virtual table corresponding to the fourth identifier selected.

[0243] For example, in response to the user triggering the connection relationship selection control in FIG. 8, a plurality of preset connection relationships are displayed, and then the preset connection relationship selected by the user is taken as the second target connection relationship between the secondary table 1 and the fourth virtual table corresponding to the fourth identifier selected by the user.

[0244] A4: in response to a second determination creation operation, displaying a target virtual table; the connection relationship in the modeling relationship of the target virtual table includes the second target connection relationship.

[0245] Here, the second determination creation operation can be the operation of clicking the confirm button in the modeling relationship setting page.

[0246] Exemplarily, in response to an operation of the user clicking the determination button, the virtualization modeling engine can be used to generate the target virtual table and the target identifier corresponding to the target virtual table according to the second target connection relationship, the data attribute information and the data processing logic in the secondary table 1, and the data attribute information and the data processing logic in the fourth virtual table corresponding to the fourth identifier selected by the user. Then, the client can display the target identifier in the table identifier display area and display the target virtual table in the modeling area.

[0247] Optionally, before performing A4, in response to the user triggering the time column setting control in FIG. 8, the modeling engine can be used to determine the columns in the secondary table 1 that can be used as time columns, and then the column selected by the user can be used as the time column of the secondary table 1. Then, in response to an operation of the user clicking the determination button, the virtualization modeling engine can be used to generate the target virtual table and the target identifier corresponding to the target virtual table according to the second target connection relationship, the time column selected by the user, the data attribute information and the data processing logic corresponding to the secondary table 1, and the data attribute information and the data processing logic in the fourth virtual table corresponding to the fourth identifier selected by the user.

[0248] In this way, after the virtual table is displayed by being dragged, the connection relationship and the associated node between the virtual tables can be set by using the modeling relationship setting page, and the snowflake or star modeling can be easily implemented.

[0249] In an embodiment, after the target virtual table is displayed, the data modeling method provided by the embodiment of the present disclosure further supports querying the modeling result. Specifically, in response to a query request for the target virtual table, the data result queried from the target data source can be displayed. The data result is obtained by using a source data engine of the target data source to perform data query on source data positioned by the data reference relationship corresponding to the target virtual table according to the modeling relationship and the data processing logic corresponding to the target virtual table.

[0250] Here, the query request can be a SQL query initiated for the target virtual table. The data result is entity data queried from the source data positioned by the data reference relationship corresponding to the target data source.

[0251] As shown in FIG. 9, a query schematic diagram provided by an embodiment of the present application is shown, wherein a SQL query identifier is displayed in a modeling page, and after the identifier is triggered, a SQL statement writing area and target identifiers of each target virtual table created are displayed in a table identifier display area in the modeling interface. Then, the SQL statement input by a user in the SQL statement writing area for the target virtual table can be obtained, and the SQL statement input by the user is sent to the virtualization modeling engine as a query request in response to a query operation. Then, the virtualization modeling engine invokes a source data engine of the target data source, and performs data query on source data positioned by a data reference relationship corresponding to the target virtual table according to a modeling relationship and data processing logic corresponding to the target virtual table, to obtain data results. Then, the virtualization modeling engine can obtain the data results queried by the source data engine and feed back to the client, so that the client displays the data results in the modeling page.

[0252] In this way, through the SQL query, the data of the target data source can be queried, and the target virtual table constructed can also be queried. When the target virtual table is queried, the source data engine of the target data source can be used to query the source data associated with the target virtual table according to the modeling relationship and data processing logic corresponding to the target virtual table, to obtain data results. The entire query process has a lower requirement on the computing resources of the virtualization modeling engine, and mainly relies on the source data engine. Since the source data engine has reliable computing resources, the data query efficiency can be effectively improved.

[0253] In addition, when the source data engine is used to query data, all entity data in the source data associated with the target virtual table will not be loaded to the local, but only part of the data results queried by the source data engine will be loaded to the local for display, effectively avoiding the problem of resource waste and security risk caused by repeated copying of data. In addition, the data results obtained by the query are not stored persistently in the local, effectively reducing the storage pressure of the local.

[0254] To facilitate understanding of the data modeling method provided by the embodiments of the present application, the disclosure also provides a modeling process schematic diagram. As shown in FIG. 10, a modeling process schematic diagram provided by an embodiment of the present application can include the following steps:

[0255] S1001: Register an external data source.

[0256] Specifically, this step is to use the registration information input by the user to determine whether the user has access rights to the target data source. In the case where it is determined that the user has access rights, the virtualization modeling system can be connected with the external data source system.

[0257] S1002: Create a data table in the target data source as a source-attached virtual table.

[0258] Specifically, the step is to create a corresponding first virtual table for the data table selected by the user.

[0259] In this way, the virtualization process of the data table under various data sources is realized through the pasting source virtual table, so that it is included in the virtualization system.

[0260] S1003: Create data processing logic based on the virtual table to realize data processing for the virtual table.

[0261] Here, the data attribute information in the virtual table can be logically processed according to the data processing logic configured by the user in the modeling interface, to obtain a second virtual table.

[0262] S1004: Determine whether multi-layer data processing logic is needed.

[0263] If yes, return to execute S1003, that is, create multi-layer data processing logic according to the steps of S1003, and if no, execute the following S1005.

[0264] In this way, the graphical data processing logic configuration is performed through the virtualization manner, the virtual table after executing the data processing logic is defined through the logical definition manner, the convenience and efficiency of data logic processing are improved. And based on the nesting relationship of the virtual table, complex data processing can be realized.

[0265] S1005: Modeling according to the virtual table.

[0266] This step is to create a target virtual table according to the modeling relationship configured by the user in the modeling interface.

[0267] S1006: Determine whether multi-layer modeling is needed.

[0268] If yes, return to execute S1005, that is, perform multi-layer modeling according to the steps of S1005, and if no, execute the following S1007.

[0269] In this way, the graphical modeling is performed through the virtualization manner, the virtual table is selected through the graphical manner, the connection relationship is configured by selecting the association key between the virtual tables, so as to build a new virtual table, the convenience and efficiency of data modeling are improved. And complex modeling can be completed based on the nesting relationship of the virtual table, so as to realize complex data processing.

[0270] S1007: Modeling result query.

[0271] This step can respond to a query request for the target virtual table, and display the data result queried from the target data source.

[0272] In this way, the virtual table model that has been constructed can be queried through the modeling result query, and the virtual table can find the corresponding data processing result according to the data table in the associated target data source and the constructed data processing logic, and the data processing result is not persisted, thereby providing a new logical data application method.

[0273] Those skilled in the art can understand that, in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined according to its function and possible internal logic.

[0274] Based on the same inventive concept, the data modeling device corresponding to the data modeling method is also provided in the embodiments of the disclosure. Since the principle of solving problems of the device in the embodiments of the disclosure is similar to the above-mentioned data modeling method of the embodiments of the disclosure, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0275] As shown in FIG. 11, a schematic diagram of a data modeling device provided in the embodiments of the disclosure includes:

[0276] The first display module 1101 is configured to display a modeling interface, and the modeling interface is configured to display a plurality of initial data sources;

[0277] The second display module 1102 is configured to display a plurality of source data under a target data source in the plurality of initial data sources in response to the target data source being selected, and display a first identifier of a first virtual table corresponding to each source data in response to a virtual table creation operation for at least one source data; the first virtual table is generated according to data attribute information corresponding to the source data;

[0278] The third display module 1103 is configured to display the first virtual table corresponding to each selected first identifier in response to a selection operation for the first identifier, and display a second identifier of a second virtual table corresponding to the selected first virtual table in response to a data processing logic creation operation for the selected first virtual table; the second virtual table is generated according to the data processing logic configured for the selected first virtual table, and the data processing logic is configured to perform logical processing on the data attribute information in the first virtual table;

[0279] The fourth display module 1104 is configured to display third virtual tables corresponding to the selected third identifiers respectively in response to a selection operation on the third identifiers, and display a target virtual table in response to a modeling relationship creation operation on the third virtual tables; the third identifiers include the first identifiers and / or the second identifiers; the target virtual table is generated according to modeling relationships configured by a user for the third virtual tables and hierarchical relationships of the third virtual tables, and the modeling relationships are used to indicate connection relationships between the third virtual tables and / or associated data attribute information.

[0280] In a possible implementation, the apparatus further includes a generation module 1105 configured to generate the target virtual table according to the following steps:

[0281] In a case where the third virtual tables include second virtual tables, the data reference relationships and target data processing logic corresponding to the target virtual table are determined according to hierarchical relationships of the third virtual tables, and the data reference relationships are used to locate source data corresponding to the target virtual table.

[0282] The target virtual table is generated according to data attribute information in the third virtual tables, modeling relationships corresponding to the third virtual tables, data processing logic, and the data reference relationships.

[0283] In a possible implementation, in a case where the target data source is an unstructured data source, the source data includes each data subject, and each data subject includes at least one string data.

[0284] The apparatus further includes a first determination module 1106 configured to determine data attribute information of the source data according to the following steps:

[0285] For any data subject, any string data generated under the data subject is subjected to structured processing to obtain target structured data corresponding to the string data.

[0286] The data attribute information corresponding to the data subject is determined according to the target structured data.

[0287] In a possible implementation, in a case where the target data source is a structured data source, the source data includes each data table, and each data table includes at least one structured data.

[0288] The apparatus further includes a second determination module 1107 configured to determine data attribute information of the source data according to the following steps:

[0289] For any data table, attribute information extraction is performed to obtain the data attribute information corresponding to the data table.

[0290] In a possible implementation, the apparatus further includes a query module 1108, configured to:

[0291] In response to a query request for the target virtual table, display data results queried from the target data source; the data results are obtained by using a source data engine of the target data source to query source data positioned by the data reference relationship corresponding to the target virtual table according to the modeling relationship and the data processing logic corresponding to the target virtual table.

[0292] In a possible implementation, the third display module 1103, in response to the data processing logic creation operation for the selected first virtual table, is configured to:

[0293] In response to the data processing logic creation operation for the selected first virtual table, display a logic setting page; the logic setting page is configured to display a plurality of preset processing logics.

[0294] Obtain each first preset processing logic selected by a user and a first target logic parameter set for each first preset processing logic;

[0295] In response to the logic determination operation, display a second identifier of a second virtual table of a next level corresponding to the selected first virtual table; the second virtual table is obtained by logically processing data attribute information in the selected first virtual table by using a data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter.

[0296] In a possible implementation, the third display module 1103, before the response to the selection operation for the third identifier, is further configured to:

[0297] In response to selection of any second identifier, display a second virtual table corresponding to the selected second identifier;

[0298] In response to the logic creation operation for the selected second virtual table, display a logic setting page; the logic setting page is configured to display a plurality of preset processing logics.

[0299] Obtain each second preset processing logic selected by a user and a second target logic parameter set for each second preset processing logic;

[0300] in response to the logic determining operation, display a second identifier of a newly generated second virtual table; the newly generated second virtual table is a virtual table of a next level of the selected second virtual table, and the newly generated second virtual table is generated by performing logic processing on data attribute information in the selected second virtual table by using data processing logic generated according to each second preset processing logic and corresponding second target logic parameter.

[0301] In a possible implementation, the selecting operation includes an identifying drag operation.

[0302] The fourth display module 1104, in response to the selecting operation on the third identifier, displays a third virtual table corresponding to each selected third identifier, is used for:

[0303] In response to a drag operation of dragging the third identifier to the table display area, display the third virtual table corresponding to each dragged third identifier;

[0304] The fourth display module 1104, in response to the modeling relationship creation operation on the third virtual table, displays a target virtual table, is used for:

[0305] In response to a connection operation on target data attribute information in different third virtual tables, display an association relationship configuration page; the association relationship configuration page displays a connection relationship selection control;

[0306] In response to triggering the connection relationship selection control, display a plurality of preset connection relationships, and select the preset connection relationship selected by the user as a first target connection relationship between different third virtual tables;

[0307] In response to a first confirmation creation operation, display a target virtual table; the association data attribute information in the modeling relationship of the target virtual table is the two target data attribute information connected by the connection operation, and the connection relationship in the modeling relationship of the target virtual table includes the first target connection relationship.

[0308] In a possible implementation, the fourth display module 1104, in response to the modeling relationship creation operation on the third virtual table, displays a target virtual table, is used for:

[0309] In response to a trigger operation on the displayed third virtual table, display a modeling relationship setting page; the modeling relationship setting page displays an association node setting control and a connection relationship selection control;

[0310] In response to triggering the association node setting control, display a fourth identifier of each fourth virtual table, and determine the fourth identifier selected by the user; the fourth virtual table at least includes the third virtual table;

[0311] In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the user-selected preset connection relationship is taken as a second target connection relationship between a third virtual table triggered and a fourth virtual table selected according to the fourth identifier.

[0312] In response to the second determination creation operation, a target virtual table is displayed; and a connection relationship in a modeling relationship of the target virtual table includes the second target connection relationship.

[0313] In a possible implementation, the apparatus further includes a test module 1109, after obtaining each first preset processing logic selected by the user and the first target logic parameter set for each first preset processing logic, the test module 1109 is configured to:

[0314] In response to the logic test operation, a logic test result is displayed; and the logic test result is obtained by performing logic processing on source data positioned by the selected first virtual table corresponding data reference relationship by using the data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter.

[0315] In a possible implementation, the second display module 1102, in response to the target data source in the plurality of initial data sources being selected, displays a plurality of pieces of source data under the target data source, is configured to:

[0316] In response to the target data source in the plurality of initial data sources being selected, the registration information input by the user is obtained.

[0317] In a case where the registration information indicates that the user has access authority for the target data source, a plurality of pieces of source data corresponding to the target data source are displayed.

[0318] The description of the processing flow of each module in the apparatus and the interaction flow between the modules can refer to the related description in the above method embodiments, and will not be described in detail here.

[0319] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Referring to FIG. 12, a structural schematic diagram of a computer device provided by the embodiment of the present disclosure is shown, which includes a processor 1201, a memory 1202, and a bus 1203. The memory 1202 is used to store execution instructions, including an internal memory 1221 and an external memory 1222. The internal memory 1221 is also referred to as an internal storage, which is used to temporarily store operation data in the processor 1201 and exchange data with the external memory 1222 such as a hard disk. The processor 1201 exchanges data with the external memory 1222 through the internal memory 1221. When the computer device is running, the processor 1201 and the memory 1202 communicate through the bus 1203, so that the processor 1201 performs a data modeling process, which includes:

[0320] displaying a modeling interface, the modeling interface being used to display a plurality of initial data sources;

[0321] in response to a target data source in the plurality of initial data sources being selected, displaying a plurality of source data under the target data source, and in response to a virtual table creation operation for at least one source data, displaying a first identifier of a first virtual table corresponding to each source data; the first virtual table being generated according to data attribute information corresponding to the source data;

[0322] in response to a selection operation for the first identifier, displaying first virtual tables corresponding to each selected first identifier respectively, and in response to a data processing logic creation operation for the selected first virtual table, displaying a second identifier of a second virtual table corresponding to the selected first virtual table; the second virtual table being generated according to a data processing logic configured for the selected first virtual table, the data processing logic being used to perform logical processing on data attribute information in the first virtual table;

[0323] in response to a selection operation for the third identifier, displaying third virtual tables corresponding to each selected third identifier respectively, and in response to a modeling relationship creation operation for the third virtual table, displaying a target virtual table; the third identifier including the first identifier and / or the second identifier; the target virtual table being generated according to a modeling relationship configured by a user for the third virtual table and a hierarchical relationship of the third virtual table, the modeling relationship being used to indicate a connection relationship between the third virtual tables and / or associated data attribute information.

[0324] In a possible implementation, in the data modeling process performed by the processor 1201, the target virtual table is generated according to the following steps:

[0325] In a case that the third virtual table comprises a second virtual table, according to a hierarchical relationship of the third virtual table, a data reference relationship corresponding to the target virtual table and target data processing logic are determined, the data reference relationship is used for locating source data corresponding to the target virtual table;

[0326] According to data attribute information in the third virtual table, modeling relationship corresponding to the third virtual table, data processing logic, and the data reference relationship, the target virtual table is generated.

[0327] In a possible implementation, in a case that the target data source is an unstructured data source, the source data comprises each data subject, and each data subject comprises at least one string data; the processor 1201 is further configured to determine data attribute information of the source data according to the following steps:

[0328] For any data subject, any string data generated under the data subject is subjected to structured processing, to obtain target structured data corresponding to the string data;

[0329] According to the target structured data, data attribute information corresponding to the data subject is determined.

[0330] In a possible implementation, in a case that the target data source is a structured data source, the source data comprises each data table, and each data table comprises at least one structured data;

[0331] The processor 1201 is further configured to determine data attribute information of the source data according to the following steps:

[0332] Attribute information extraction is performed on any data table, to obtain the data attribute information corresponding to each data table.

[0333] In a possible implementation, in the data modeling process performed by the processor 1201, after the target virtual table is displayed, the following steps are further included:

[0334] In response to a query request for the target virtual table, data results queried from the target data source are displayed; the data results are obtained by using a source data engine of the target data source to perform data query on source data located by the data reference relationship corresponding to the target virtual table, according to the modeling relationship and the data processing logic corresponding to the target virtual table.

[0335] In a possible implementation, in the data modeling process performed by the processor 1201, in response to the data processing logic creation operation on the selected first virtual table, a second identifier of a second virtual table corresponding to the selected first virtual table is displayed, including:

[0336] in response to a data processing logic creation operation for the selected first virtual table, display a logic setting page; the logic setting page is used to display a plurality of preset processing logics;

[0337] obtain each first preset processing logic selected by the user, and a first target logic parameter set for each of the first preset processing logics;

[0338] in response to a logic determination operation, display a second identifier of a second virtual table corresponding to a next level of the selected first virtual table; the second virtual table is obtained by performing logic processing on data attribute information in the selected first virtual table by using data processing logics generated according to each of the first preset processing logics and the corresponding first target logic parameter;

[0339] In a possible implementation, before the operation of selecting the third identifier, the data modeling process performed by the processor 1201 further includes:

[0340] in response to selection of any second identifier, display a second virtual table corresponding to the selected second identifier;

[0341] in response to a logic creation operation for the selected second virtual table, display a logic setting page; the logic setting page is used to display a plurality of preset processing logics;

[0342] obtain each second preset processing logic selected by the user, and a second target logic parameter set for each of the second preset processing logics;

[0343] in response to a logic determination operation, display a second identifier of a newly generated second virtual table; the newly generated second virtual table is a virtual table of a next level of the selected second virtual table, and the newly generated second virtual table is obtained by performing logic processing on data attribute information in the selected second virtual table by using data processing logics generated according to each of the second preset processing logics and the corresponding second target logic parameter;

[0344] In a possible implementation, the operation of selecting the third identifier includes an identifier dragging operation.

[0345] in response to the operation of selecting the third identifier, display a third virtual table corresponding to each of the selected third identifiers, including:

[0346] in response to a dragging operation of dragging the third identifier to a table display area, display a third virtual table corresponding to each of the dragged third identifiers;

[0347] The response to the modeling relationship creation operation for the third virtual table displays a target virtual table, including:

[0348] In response to a connection operation for target data attribute information in different third virtual tables, an association relationship configuration page is displayed; the association relationship configuration page displays a connection relationship selection control;

[0349] In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the user-selected preset connection relationship is used as a first target connection relationship between different third virtual tables;

[0350] In response to a first confirmation creation operation, a target virtual table is displayed; the association data attribute information in the modeling relationship of the target virtual table is the two target data attribute information connected by the connection operation, and the connection relationship in the modeling relationship of the target virtual table includes the first target connection relationship.

[0351] In a possible implementation, in the data modeling process performed by the processor 1201, the response to the modeling relationship creation operation for the third virtual table displays a target virtual table, including:

[0352] In response to a trigger operation for the displayed third virtual table, a modeling relationship setting page is displayed; the modeling relationship setting page displays an association node setting control and a connection relationship selection control;

[0353] In response to triggering the association node setting control, a fourth identifier of each fourth virtual table is displayed, and the user-selected fourth identifier is determined; the fourth virtual table at least includes the third virtual table;

[0354] In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the user-selected preset connection relationship is used as a second target connection relationship between the triggered third virtual table and the fourth virtual table corresponding to the selected fourth identifier;

[0355] In response to a second determination creation operation, a target virtual table is displayed; the connection relationship in the modeling relationship of the target virtual table includes the second target connection relationship.

[0356] In a possible implementation, in the data modeling process performed by the processor 1201, after obtaining the user-selected each first preset processing logic and the first target logic parameter set for each first preset processing logic, the process further includes:

[0357] In response to the logic test operation, a logic test result is displayed; the logic test result is obtained by using the data processing logic generated according to the first preset processing logic and the corresponding first target logic parameter to perform logic processing on the source data positioned by the data reference relationship corresponding to the selected first virtual table.

[0358] In a possible implementation, in the data modeling process performed by the processor 1201, in response to a target data source in the plurality of initial data sources being selected, a plurality of pieces of source data under the target data source are displayed, including:

[0359] In response to the target data source in the plurality of initial data sources being selected, registered information input by a user is obtained.

[0360] In a case where the registered information indicates that the user has access authority for the target data source, a plurality of pieces of source data corresponding to the target data source are displayed.

[0361] The disclosure also provides a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the data modeling method described in the above method embodiments are performed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0362] The computer program product of the data modeling method provided by the disclosure includes a computer-readable storage medium storing program codes. The instructions included in the program codes can be used to perform the steps of the data modeling method described in the above method embodiments. For details, refer to the above method embodiments, which will not be described here.

[0363] The computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0364] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here. In several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and another division can be made in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0365] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0366] In addition, the functional units in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0367] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part of the prior art or the part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present disclosure. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program codes that can be stored in the medium.

[0368] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit the present disclosure. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easy changes to the technical solutions described in the foregoing embodiments, or easily think of changes or equivalent replacements for some of the technical features; and these modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A computer device, characterized by, The application relates to a computer device, comprising: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating through the bus when the computer device is running, the machine readable instructions being executed by the processor to perform a data modeling process, the data modeling process comprising: displaying a modeling interface, the modeling interface being used to display a plurality of initial data sources; in response to a target data source in the plurality of initial data sources being selected, displaying a plurality of source data under the target data source, and in response to a virtual table creation operation for at least one piece of the source data, displaying a first identifier of a first virtual table corresponding to each piece of the source data; the first virtual table being generated according to data attribute information corresponding to the source data; in response to a selection operation for the first identifier, displaying first virtual tables corresponding to each selected first identifier respectively, and in response to a data processing logic creation operation for the selected first virtual table, displaying a second identifier of a second virtual table corresponding to the selected first virtual table; the second virtual table being generated according to a data processing logic configured for the selected first virtual table, the data processing logic being used to perform logical processing on data attribute information in the first virtual table; in response to a selection operation for the third identifier, displaying third virtual tables corresponding to each selected third identifier respectively, and in response to a modeling relationship creation operation for the third virtual table, displaying a target virtual table; the third identifier comprising the first identifier and / or the second identifier; the target virtual table being generated according to a modeling relationship configured by a user for the third virtual table and a hierarchical relationship of the third virtual table, the modeling relationship being used to indicate a connection relationship between the third virtual tables and / or associated data attribute information.

2. The computer device of claim 1, wherein, In the data modeling process executed by the processor, the target virtual table is generated according to the following steps: in the case that the third virtual table comprises a second virtual table, according to a hierarchical relationship of the third virtual table, determining a data reference relationship and a target data processing logic corresponding to the target virtual table, the data reference relationship being used to locate source data corresponding to the target virtual table; generating the target virtual table according to data attribute information in the third virtual table, a modeling relationship corresponding to the third virtual table, data processing logic and the data reference relationship.

3. The computer device of claim 1, wherein, in the case that the target data source is an unstructured data source, the source data comprising a plurality of data subjects, each data subject comprising at least one piece of string data; the processor being used to determine data attribute information of the source data according to the following steps: for any data subject, performing structural processing on any piece of string data generated under the data subject to obtain target structured data corresponding to the string data; determining data attribute information corresponding to the data subject according to the target structured data.

4. The computer device of claim 1, wherein, in the case that the target data source is a structured data source, the source data comprising a plurality of data tables, each data table comprising at least one piece of structured data; The processor is configured to determine data attribute information of the source data according to the following steps: Attribute information extraction is performed on any of the data tables to obtain the data attribute information corresponding to each of the data tables.

5. The computer device of claim 2, wherein, In the data modeling process performed by the processor, after the target virtual table is displayed, the following further includes: In response to a query request for the target virtual table, a data result queried from the target data source is displayed; the data result is obtained by using a source data engine of the target data source to perform data query on source data positioned by a data reference relationship corresponding to the target virtual table according to the modeling relationship and the data processing logic corresponding to the target virtual table.

6. The computer device of claim 1, wherein, In the data modeling process performed by the processor, in response to the data processing logic creation operation for the selected first virtual table, a second identifier of a second virtual table corresponding to the selected first virtual table is displayed, including: In response to the data processing logic creation operation for the selected first virtual table, a logic setting page is displayed; the logic setting page is configured to display a plurality of preset processing logics. Each first preset processing logic selected by a user and a first target logic parameter set for each of the first preset processing logics are obtained. In response to a logic determination operation, a second identifier of a second virtual table of a next level corresponding to the selected first virtual table is displayed; the second virtual table is obtained by performing logic processing on data attribute information in the selected first virtual table by using a data processing logic generated according to each of the first preset processing logics and the corresponding first target logic parameter.

7. The computer device of claim 1, wherein, In the data modeling process performed by the processor, before the selection operation for the third identifier is responded to, the following further includes: In response to selection of any of the second identifiers, a second virtual table corresponding to the selected second identifier is displayed. In response to the logic creation operation for the selected second virtual table, a logic setting page is displayed; the logic setting page is configured to display a plurality of preset processing logics. Each second preset processing logic selected by a user and a second target logic parameter set for each of the second preset processing logics are obtained. In response to a logic determination operation, a second identifier of a newly generated second virtual table is displayed; the newly generated second virtual table is a virtual table of a next level of the selected second virtual table, and the newly generated second virtual table is obtained by performing logic processing on data attribute information in the selected second virtual table by using a data processing logic generated according to each of the second preset processing logics and the corresponding second target logic parameter.

8. The computer device of claim 1, wherein, In the data modeling process performed by the processor, the selection operation includes an identifier dragging operation. In response to the selection operation for the third identifier, third virtual tables corresponding to the selected third identifiers are displayed, including: In response to a dragging operation of dragging the third identifier to a table display area, third virtual tables corresponding to the dragged third identifiers are displayed. In response to the modeling relationship creation operation for the third virtual table, a target virtual table is displayed, including: In response to a connection operation for target data attribute information in different third virtual tables, a connection relationship configuration page is displayed; the connection relationship configuration page displays a connection relationship selection control; In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the user-selected preset connection relationship is used as a first target connection relationship between different third virtual tables; In response to a first confirmation creation operation, a target virtual table is displayed; the associated data attribute information in the modeling relationship of the target virtual table is the two target data attribute information connected by the connection operation, and the connection relationship in the modeling relationship of the target virtual table includes the first target connection relationship.

9. The computer device of claim 1, wherein, In the data modeling process performed by the processor, in response to a modeling relationship creation operation for the third virtual table, a target virtual table is displayed, including: In response to a trigger operation for the displayed third virtual table, a modeling relationship setting page is displayed; the modeling relationship setting page displays an association node setting control and a connection relationship selection control; In response to triggering the association node setting control, fourth identifiers of each fourth virtual table are displayed, and the user-selected fourth identifier is determined; the fourth virtual table includes at least the third virtual table; In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the user-selected preset connection relationship is used as a second target connection relationship between the triggered third virtual table and the fourth virtual table corresponding to the selected fourth identifier; In response to a second determination creation operation, a target virtual table is displayed; the connection relationship in the modeling relationship of the target virtual table includes the second target connection relationship.

10. The computer device of claim 6, wherein, In the data modeling process performed by the processor, after obtaining the user-selected each first preset processing logic and the first target logic parameter set for each first preset processing logic, the process further includes: In response to a logic test operation, a logic test result is displayed; the logic test result is obtained by using the data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter to perform logic processing on the source data positioned by the selected data reference relationship of the first virtual table.

11. The computer device of claim 1, wherein, In the data modeling process performed by the processor, in response to the target data source in the plurality of initial data sources being selected, a plurality of source data under the target data source are displayed, including: In response to the target data source in the plurality of initial data sources being selected, the registration information input by the user is obtained; In the case where the registration information indicates that the user has access authority for the target data source, a plurality of source data corresponding to the target data source are displayed.

12. A data modeling method, characterized by, It includes: Display a modeling interface, the modeling interface is used to display a plurality of initial data sources; In response to the target data source in the plurality of initial data sources being selected, a plurality of source data under the target data source are displayed, and in response to a virtual table creation operation for at least one source data, a first identifier of a first virtual table corresponding to each source data is displayed; The first virtual table is generated according to the data attribute information corresponding to the source data; In response to a selection operation on the first identifiers, first virtual tables corresponding to the selected first identifiers are displayed respectively, and in response to a data processing logic creation operation on the selected first virtual tables, second identifiers of second virtual tables corresponding to the selected first virtual tables are displayed; The second virtual tables are generated according to data processing logics configured for the selected first virtual tables, and the data processing logics are used for logically processing data attribute information in the first virtual tables; In response to a selection operation on the third identifiers, third virtual tables corresponding to the selected third identifiers are displayed respectively, and in response to a modeling relationship creation operation on the third virtual tables, a target virtual table is displayed; the third identifiers include the first identifiers and / or the second identifiers; The target virtual table is generated according to a modeling relationship configured by a user for the third virtual tables and a hierarchical relationship of the third virtual tables, and the modeling relationship is used for indicating a connection relationship between the third virtual tables and / or associated data attribute information.

13. The method of claim 12, wherein, The target virtual table is generated according to the following steps: In a case where the third virtual tables include the second virtual tables, a data reference relationship corresponding to the target virtual table and a target data processing logic are determined according to a hierarchical relationship of the third virtual tables, and the data reference relationship is used for locating source data corresponding to the target virtual table; The target virtual table is generated according to data attribute information in the third virtual tables, a modeling relationship corresponding to the third virtual tables, a data processing logic, and the data reference relationship.

14. The method of claim 12, wherein, In a case where the target data source is an unstructured data source, the source data includes each data subject, and each data subject includes at least one string data; The data attribute information of the source data is determined according to the following steps: For any data subject, any string data generated under the data subject is structured to obtain target structured data corresponding to the string data; According to the target structured data, data attribute information corresponding to the data subject is determined.

15. The method of claim 12, wherein, In a case where the target data source is a structured data source, the source data includes each data table, and each data table includes at least one structured data; The data attribute information of the source data is determined according to the following steps: For any data table, attribute information extraction is performed to obtain the data attribute information corresponding to the data table.

16. The method of claim 13, wherein, After the target virtual table is displayed, the following steps are further included: In response to a query request for the target virtual table, a data result queried from the target data source is displayed; the data result is obtained by using a source data engine of the target data source to perform data query on source data located by the data reference relationship corresponding to the target virtual table according to the modeling relationship and the data processing logic corresponding to the target virtual table.

17. The method of claim 12, wherein, The second identifiers of the second virtual tables corresponding to the selected first virtual tables are displayed in response to the data processing logic creation operation on the selected first virtual tables, and the second identifiers of the second virtual tables corresponding to the selected first virtual tables are displayed in response to the data processing logic creation operation on the selected first virtual tables. In response to the data processing logic creation operation for the selected first virtual table, a logic setting page is displayed; the logic setting page is used to display a plurality of preset processing logics; Each first preset processing logic selected by the user and a first target logic parameter set for each first preset processing logic are obtained; In response to the logic determination operation, a second identifier of a second virtual table corresponding to the next level of the selected first virtual table is displayed; the second virtual table is obtained by performing logic processing on the data attribute information in the selected first virtual table by using the data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter.

18. The method of claim 12, wherein, Before the selection operation for the third identifier is responded to, the method further comprises: In response to the selection of any second identifier, a second virtual table corresponding to the selected second identifier is displayed; In response to the logic creation operation for the selected second virtual table, a logic setting page is displayed; the logic setting page is used to display a plurality of preset processing logics; Each second preset processing logic selected by the user and a second target logic parameter set for each second preset processing logic are obtained; In response to the logic determination operation, a second identifier of a newly generated second virtual table is displayed; the newly generated second virtual table is a virtual table of the next level of the selected second virtual table, and the newly generated second virtual table is obtained by performing logic processing on the data attribute information in the selected second virtual table by using the data processing logic generated according to each second preset processing logic and the corresponding second target logic parameter.

19. The method of claim 12, wherein, The selection operation comprises an identifier dragging operation; The response to the selection operation for the third identifier to display the third virtual table corresponding to each selected third identifier comprises: In response to the dragging operation of dragging the third identifier to the table display area, the third virtual table corresponding to each dragged third identifier is displayed; The response to the modeling relationship creation operation for the third virtual table to display a target virtual table comprises: In response to the connection operation of different target data attribute information in the third virtual table, an association relationship configuration page is displayed; the association relationship configuration page displays a connection relationship selection control; In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and the preset connection relationship selected by the user is used as a first target connection relationship between different third virtual tables; In response to the first confirmation creation operation, a target virtual table is displayed; the associated data attribute information in the modeling relationship of the target virtual table is the two target data attribute information connected by the connection operation, and the connection relationship in the modeling relationship of the target virtual table comprises the first target connection relationship.

20. The method of claim 12, wherein, The response to the modeling relationship creation operation for the third virtual table to display a target virtual table comprises: In response to the triggering operation for the displayed third virtual table, a modeling relationship setting page is displayed; the modeling relationship setting page displays an association node setting control and a connection relationship selection control; In response to triggering the association node setting control, fourth identifiers of respective fourth virtual tables are displayed, and a fourth identifier selected by the user is determined; the fourth virtual tables at least include the third virtual tables; In response to triggering the connection relationship selection control, a plurality of preset connection relationships are displayed, and a preset connection relationship selected by the user is taken as a second target connection relationship between the triggered third virtual table and the fourth virtual table corresponding to the selected fourth identifier; In response to the second determination creation operation, a target virtual table is displayed; a connection relationship in a modeling relationship of the target virtual table includes the second target connection relationship.

21. The method of claim 17, wherein, After obtaining each first preset processing logic selected by the user and a first target logic parameter set for each first preset processing logic, the method further includes: In response to a logic test operation, a logic test result is displayed; the logic test result is obtained by performing logic processing on source data located by a data reference relationship corresponding to the selected first virtual table, by using a data processing logic generated according to each first preset processing logic and the corresponding first target logic parameter.

22. The method of claim 12, wherein, The displaying, in response to the target data source in the plurality of initial data sources being selected, of the plurality of pieces of source data under the target data source includes: In response to the target data source in the plurality of initial data sources being selected, obtaining registration information input by the user; In a case where the registration information indicates that the user has access authority for the target data source, displaying the plurality of pieces of source data corresponding to the target data source.

23. A data modeling apparatus, comprising: The method includes: A first display module is configured to display a modeling interface, the modeling interface being configured to display a plurality of initial data sources; A second display module is configured to, in response to a target data source in the plurality of initial data sources being selected, display a plurality of pieces of source data under the target data source, and in response to a virtual table creation operation for at least one piece of source data, display a first identifier of a first virtual table corresponding to each piece of source data; The first virtual table is generated according to data attribute information corresponding to the source data; A third display module is configured to, in response to a selection operation for the first identifier, display respective first virtual tables corresponding to each selected first identifier, and in response to a data processing logic creation operation for the selected first virtual table, display a second identifier of a second virtual table corresponding to the selected first virtual table; The second virtual table is generated according to data processing logic configured for the selected first virtual table, the data processing logic being configured to perform logic processing on data attribute information in the first virtual table; A fourth display module is configured to, in response to a selection operation for a third identifier, display respective third virtual tables corresponding to each selected third identifier, and in response to a modeling relationship creation operation for the third virtual table, display a target virtual table; the third identifier includes the first identifier and / or the second identifier; The target virtual table is generated according to a modeling relationship configured by the user for the third virtual table and a hierarchical relationship of the third virtual table, the modeling relationship being configured to indicate a connection relationship between the third virtual tables and / or associated data attribute information.

24. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which, when executed by the processor, performs the steps of the data modeling method according to any one of claims 12 to 22.

Images