A method and device for creating a data table

Abstract

The present disclosure relates to a method and device for creating a data table. The method comprises: receiving a data table creation instruction input by a user, wherein the data table creation instruction includes a data source identifier; determining data structure information in a target data source indicated by the data source identifier; and generating a table creation statement for the target data source according to the data structure information in the target data source, wherein the table creation statement is used to create a data table corresponding to the target data source. The present disclosure can avoid users from manually inputting fields, thereby improving table creation efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for creating a data table. Background Art

[0002] At present, most databases provide table creation statements, such as create table A like B, which can be used to copy a new data table B with the same structure but different table name from a data table A already in the database. However, when creating a new data table directly based on the original data stored in the database, the user needs to manually enter each field in the new data table, resulting in low table creation efficiency. Summary of the invention

[0003] In view of this, the present disclosure proposes a method and device for creating a data table, so as to avoid users from manually inputting fields, thereby improving table building efficiency.

[0004] According to a first aspect of the present disclosure, a method for creating a data table is provided, comprising: receiving a data table creation instruction input by a user, the data table creation instruction including a data source identifier; determining data structure information in a target data source indicated by the data source identifier; and generating a table creation statement for the target data source based on the data structure information in the target data source, the table creation statement being used to create a data table corresponding to the target data source.

[0005] In a possible implementation, determining the data structure information in the target data source indicated by the data source identifier includes: extracting at least one data record stored in the target data source; and determining the data structure information in the target data source based on the at least one data record.

[0006] In a possible implementation, determining the data structure information in the target data source indicated by the data source identifier includes: determining each data item in the target data source and a first data type identifier corresponding to any data item as the data structure information.

[0007] In a possible implementation, a table creation statement for the target data source is generated based on the data structure information in the target data source, including: determining at least one field in the table creation statement based on each data item in the target data source; and determining a second data type identifier for the field corresponding to the data item based on a first data type identifier corresponding to any data item, wherein the data type indicated by the first data type identifier is the same as the data type indicated by the second data type identifier.

[0008] In a possible implementation, the second data type identifiers determined according to the first data type identifiers indicating the same data type are the same.

[0009] In a possible implementation, the target data source includes at least one of the following: a database, and a storage service with a data storage function.

[0010] In a possible implementation, the target data source includes at least one of the following: object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis.

[0011] According to a second aspect of the present disclosure, a device for creating a data table is provided, comprising: a receiving module for receiving a data table creation instruction input by a user, wherein the data table creation instruction includes a data source identifier; a determining module for determining data structure information in a target data source indicated by the data source identifier; and a creating module for generating a table creation statement for the target data source based on the data structure information in the target data source, wherein the table creation statement is used to create a data table corresponding to the target data source.

[0012] In a possible implementation, the determination module includes: an extraction submodule, used to extract at least one data record stored in the target data source; and a first determination submodule, used to determine data structure information in the target data source based on the at least one data record.

[0013] In a possible implementation, the determination module is specifically configured to: determine each data item in the target data source, and a first data type identifier corresponding to any data item, as the data structure information.

[0014] In one possible implementation, the creation module includes: a second determination submodule, used to determine at least one field in the table creation statement based on each data item in the target data source; and a third determination submodule, used to determine a second data type identifier of a field corresponding to the data item based on a first data type identifier corresponding to any data item, wherein the data type indicated by the first data type identifier is the same as the data type indicated by the second data type identifier.

[0015] In a possible implementation, the second data type identifiers determined according to the first data type identifiers indicating the same data type are the same.

[0016] In a possible implementation, the target data source includes at least one of the following: a database, and a storage service with a data storage function.

[0017] In a possible implementation, the target data source includes at least one of the following: object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis

[0018] According to a third aspect of the present disclosure, a device for creating a data table is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the method for creating a data table described in the first aspect above.

[0019] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the method for creating a data table described in the first aspect above.

[0020] By receiving a data table creation instruction including a data source identifier input by a user, determining the data structure information in the target data source indicated by the data source identifier, and then automatically generating a table creation statement for creating a data table corresponding to the target data source based on the data structure information in the target data source, it is possible to avoid manual input of fields by the user, thereby improving table creation efficiency.

[0021] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0023] Figure 1 A schematic diagram showing a flow chart of a method for creating a data table according to an embodiment of the present disclosure;

[0024] Figure 2 A schematic diagram showing a method for creating a data table according to an embodiment of the present disclosure;

[0025] Figure 3 A schematic diagram showing the structure of a device for creating a data table according to an embodiment of the present disclosure;

[0026] Figure 4 A schematic structural diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0027] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0028] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0029] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0030] Data lake is a new type of data warehouse solution. Data lake itself does not store data, but reads data from the underlying data storage when users need it. For example, data lake obtains raw data from multiple data sources of an enterprise, and for different purposes, the same raw data may have multiple copies of data that meet specific internal model formats. Therefore, the data processed in the data lake may be structured data or unstructured data.

[0031] Currently, in the field of data lakes, as stated in the background technology section, creating a data table for a data source requires users to manually enter each field in the data table. However, since the data source (for example, an enterprise's business database) includes many fields, manually entering fields results in low table creation efficiency.

[0032] The data table creation method provided by the present disclosure can be applied to the data table creation scenario in the data lake field to realize the automatic generation of table creation statements, avoid users manually entering fields, and improve table creation efficiency. The following is a detailed introduction to the data table creation method provided by the present disclosure using the data lake field as an example. Those skilled in the art should understand that the data lake field is only an example of an application scenario of the present disclosure and does not constitute a limitation of the present disclosure. The data table creation method provided by the present disclosure can also be applied to other application scenarios.

[0033] Figure 1 A schematic diagram showing a flow chart of a method for creating a data table according to an embodiment of the present disclosure is shown. Figure 1 As shown, the method may include:

[0034] Step S11, receiving a data table creation instruction input by a user, wherein the data table creation instruction includes a data source identifier.

[0035] Step S12: determine the data structure information in the target data source indicated by the data source identifier.

[0036] Step S13: Generate a table creation statement for the target data source according to the data structure information in the target data source, where the table creation statement is used to create a data table corresponding to the target data source.

[0037] When a user wants to create a data table for a target data source, he or she inputs a data table creation instruction including a data source identifier into the data lake, and then the data lake determines the data structure in the target data source indicated by the data source identifier. Based on the data structure information, a table creation statement for creating a data table corresponding to the target data source can be automatically generated in the data lake, thereby avoiding users from manually entering fields in the data table and improving table creation efficiency.

[0038] In a possible implementation, the target data source includes at least one of the following: a database, and a storage service with a data storage function.

[0039] The target data source may be a database for storing structured data, a storage service with data storage function for storing unstructured data, or other data storage, which is not specifically limited in the present disclosure.

[0040] In a possible implementation, the target data source includes at least one of the following: object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis.

[0041] Figure 2 A schematic diagram showing a flow chart of a method for creating a data table according to an embodiment of the present disclosure is shown. Figure 2 As shown in the figure, the underlying data sources include object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis. The data lake can create data tables for the underlying data sources, so that the data in the underlying data sources can be queried and read based on the data tables.

[0042] In one example, the data table creation instruction is Structured Query Language (SQL).

[0043] In one example, if Figure 2As shown in the figure, when a user wants to create a data table for the distributed document storage database MangoDB (target data source), the user enters the data table creation instruction into the data lake:

[0044] create external table data_lake_table like mapping('mongodb_collection').

[0045] Among them, 'mongodb_collection' is the data source identifier, which is used to indicate the target data source MangoDB.

[0046] In one example, when a user wants to create a data table for the object storage service OSS (target data source), the user enters a data table creation instruction into the data lake:

[0047] create external table data_lake_table like mapping('oss: / / test-bucket / my-file').

[0048] Among them, 'oss: / / test-bucket / my-file' is the data source identifier, which is used to indicate the target data source OSS.

[0049] In one example, when a user wants to create a data table for the MySQL database (target data source), the user enters a data table creation instruction into the data lake:

[0050] create external table data_lake_table like mapping('mysql_table').

[0051] Among them, 'mysql_table' is the data source identifier, which is used to indicate the target data source MySQL.

[0052] The syntax format of the data table creation instruction can be changed according to actual conditions, and this disclosure does not make any specific limitation on this.

[0053] For example, when a user wants to create a data table for the distributed document storage database MangoDB, the data table creation instruction can be, in addition to the above records, also: create external table data_lake_table likemapping mongodb_collection; or create external table data_lake_table likeunderlying mongodb_collection.

[0054] In a possible implementation, determining data structure information in a target data source indicated by a data source identifier includes: determining each data item in the target data source and a first data type identifier corresponding to any data item as data structure information.

[0055] After the data lake receives the data table creation instruction input by the user including the data source identifier, the data lake engine can automatically analyze and deduce the data structure in the target data source indicated by the data source identifier, and determine the data structure information in the target data source: each data item, and the first data type identifier corresponding to any data item.

[0056] In a possible implementation, determining data structure information in a target data source indicated by a data source identifier includes: extracting at least one data record stored in the target data source; and determining data structure information in the target data source based on the at least one data record.

[0057] Still with the above Figure 2 For example, after the data lake receives the data table creation instruction input by the user: create externaltable data_lake_table like mapping('mongodb_collection'), the data lake extracts a data record from the distributed document storage database MangoDB for analysis and deduction, and determines the data structure information in the distributed document storage database MangoDB.

[0058] For example, {"_id":"ObjectId("5c134c3f36d9cf6ad7077043")""id":1"name":"james""age":10"create_time":"ISODate("2018-12-14T06:22:54.369Z")"} is the extracted data record. According to the data record, the data lake analysis deduces that the distributed document storage database MangoDB includes four data items: age, creation time creat_time, identifier id and name name; among which, the first data type corresponding to age age is identified as double, and the data type used to indicate is a double-precision floating point type; the first data type corresponding to the creation time creat_time is identified as timestamp, and the data type used to indicate is a timestamp character sequence type; the first data type corresponding to the identifier id is identified as double, and the data type used to indicate is a double-precision floating point type; the first data type corresponding to the name name is identified as varchar, and the data type used to indicate is a string type.

[0059] In one possible implementation, a table creation statement for a data source is generated based on data structure information in a target data source, including: determining at least one field in the table creation statement based on each data item in the target data source; and determining a second data type identifier for a field corresponding to any data item based on a first data type identifier corresponding to the data item, wherein a data type indicated by the first data type identifier is the same as a data type indicated by the second data type identifier.

[0060] In a possible implementation, the second data type identifiers determined according to the first data type identifiers indicating the same data type are the same.

[0061] In different data sources, the first data type identifier indicating the same data type may be different. For example, in the distributed document storage database MangoDB, the first data type identifier indicating the string type is string, while in the database MySQL, the first data type identifier indicating the string type is varchar. In the data lake, different first data type identifiers indicating the same data type are uniformly mapped, that is, when determining the second data type identifier of the field corresponding to any data item based on the first data type identifier corresponding to the data item, the second data type identifier determined based on the first data type identifier indicating the same data type is the same.

[0062] For example, in the data lake, the second data type indicating the string type is identified as varchar. When creating a data table corresponding to the distributed document storage database MangoDB, for a data item whose first data type is identified as string (used to indicate the string type) in the distributed document storage database MangoDB, the second data type of the field corresponding to the data item is determined to be varchar; when creating a data table corresponding to the database MySQL, for a data item whose first data type is identified as varchar (used to indicate the string type) in the database MySQL, the second data type of the field corresponding to the data item is determined to still be varchar.

[0063] Still with the above Figure 2 For example, after the data lake determines the data structure information in the distributed document storage database MangoDB, it automatically generates a table creation statement for creating a data table corresponding to the distributed document storage database MangoDB based on the data structure information.

[0064] For example, the table creation statement for creating a data table corresponding to the distributed document storage database MangoDB is:

[0065] CREATE EXTERNAL TABLE 'data_lake_table', / / Create a data table in the data lake;

[0066] 'age' double NULL COMMENT, / / Create a field age, the data type is double precision floating point type;

[0067] 'create_time' timestamp NULL COMMENT, / / Create a field create_time, the data type is timestamp character sequence type;

[0068] 'id' double NULL COMMENT, / / Create a field id, the data type is double precision floating point type;

[0069] 'name'varchar NULL COMMENT, / / Create a field name, the data type is string type;

[0070] TBLPROPERTIES(TABLE_MAPPING='mongo_test.mongo_collection').

[0071] The data lake creates a data table corresponding to the target data source based on the table creation statement, and then users can query the data in the target data source based on the data table in the data lake.

[0072] Still with the above Figure 2 For example, after the data lake automatically generates a table creation statement for creating a data table corresponding to the distributed document storage database MangoDB, it creates a data table corresponding to the distributed document storage database MangoDB based on the table creation statement. When the data lake receives a data query instruction for the distributed document storage database MangoDB input by the user, the data lake can query the data in the distributed document storage database MangoDB based on the data table.

[0073] By receiving a data table creation instruction including a data source identifier input by a user, determining the data structure information in the target data source indicated by the data source identifier, and then automatically generating a table creation statement for creating a data table corresponding to the target data source based on the data structure information in the target data source, it is possible to avoid manual input of fields by the user, thereby improving table creation efficiency.

[0074] Figure 3 A schematic diagram showing the structure of a device for creating a data table according to an embodiment of the present disclosure. Figure 3 The device 30 shown can be used to perform Figure 1 In the steps of the method embodiment shown, the device 30 includes:

[0075] A receiving module 31 is used to receive a data table creation instruction input by a user, wherein the data table creation instruction includes a data source identifier;

[0076] A determination module 32, used to determine data structure information in a target data source indicated by a data source identifier;

[0077] The creation module 33 is used to generate a table creation statement of the target data source according to the data structure information in the target data source, and the table creation statement is used to create a data table corresponding to the target data source.

[0078] In a possible implementation, the determination module 32 includes:

[0079] An extraction submodule, used to extract at least one data record stored in a target data source;

[0080] The first determination submodule is used to determine data structure information in the target data source according to the at least one data record.

[0081] In a possible implementation, the determination module 32 is specifically configured to:

[0082] Each data item in the target data source and a first data type identifier corresponding to any data item are determined as data structure information.

[0083] In a possible implementation, the creation module 33 includes:

[0084] A second determination submodule is used to determine at least one field in the table creation statement according to each data item in the target data source;

[0085] The third determination submodule is used to determine the second data type identifier of the field corresponding to any data item based on the first data type identifier corresponding to the data item, wherein the data type indicated by the first data type identifier is the same as the data type indicated by the second data type identifier.

[0086] In a possible implementation, the second data type identifiers determined according to the first data type identifiers indicating the same data type are the same.

[0087] In a possible implementation, the target data source includes at least one of the following:

[0088] Databases and storage services with data storage capabilities.

[0089] In a possible implementation, the target data source includes at least one of the following:

[0090] Object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis.

[0091] The device 30 provided by the present disclosure can achieve Figure 1 The various steps in the method embodiment shown achieve the same technical effect and will not be described again here to avoid repetition.

[0092] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Figure 4 As shown, at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services.

[0093] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0094] The memory stores the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0095] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a data table creation device at the logical level. The processor executes the program stored in the memory and specifically performs: receiving a data table creation instruction input by a user, the data table creation instruction including a data source identifier; determining the data structure information in the target data source indicated by the data source identifier; generating a table creation statement of the target data source according to the data structure information in the target data source, the table creation statement is used to create a data table corresponding to the target data source.

[0096] In a possible implementation manner, the processor is specifically configured to execute: extracting at least one data record stored in the target data source; and determining data structure information in the target data source according to the at least one data record.

[0097] In a possible implementation, the processor is specifically configured to execute: determining each data item in the target data source, and a first data type identifier corresponding to any data item as data structure information.

[0098] In one possible implementation, the processor is specifically configured to execute: determining at least one field in a table creation statement based on each data item in a target data source; determining a second data type identifier of a field corresponding to any data item based on a first data type identifier corresponding to the data item, wherein the data type indicated by the first data type identifier is the same as the data type indicated by the second data type identifier.

[0099] In a possible implementation, the second data type identifiers determined according to the first data type identifiers indicating the same data type are the same.

[0100] In a possible implementation, the target data source includes at least one of the following: a database, and a storage service with a data storage function.

[0101] In a possible implementation, the target data source includes at least one of the following: object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis.

[0102] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of this specification may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of this specification may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0103] The electronic device may perform Figure 1 The method shown in the method embodiment is executed to achieve the above Figure 1 The functions of the method embodiment shown are not described in detail in this specification.

[0104] The embodiment of the present specification also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method for creating a data table in the embodiment shown in the figure is specifically implemented Figure 1 Steps of the method embodiment are shown.

[0105] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0106] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0107] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0108] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0109] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0110] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0111] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0112] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0113] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for creating a data table, characterized in that: Applied to a data lake, the data lake itself does not store data, but reads data from the underlying data storage when the user needs it. The data lake corresponds to multiple data sources, and the data sources are used to store data. The multiple data sources include a target data source, including: Receive a first SQL table creation statement input by a user, wherein the first SQL table creation statement includes a data source identifier and a table name but does not include fields, and the first SQL table creation statement is used to indicate that a data table with the table name is created for a target data source indicated by the data source identifier; Extracting at least one data record stored in the target data source from the target data source; Determine, according to the at least one data record, each data item of the data record and its corresponding first data type identifier; Performing unified data type mapping on the first data type identifier corresponding to the data item to obtain a second data type identifier of the field corresponding to the data item, so as to generate a second SQL table creation statement, wherein the second SQL table creation statement includes the table name, the field and its second data type identifier; Based on the second SQL table creation statement, a data table corresponding to the target data source is created so that the user can query the data in the target data source based on the data table in the data lake.

2. The method according to claim 1, characterized in that The second data type identifiers determined based on the first data type identifiers indicating the same data type are the same.

3. The method according to claim 1, characterized in that The target data source includes at least one of the following: Databases and storage services with data storage capabilities.

4. The method according to claim 1, characterized in that The target data source includes at least one of the following: Object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis.

5. A device for creating a data table, characterized in that: Applied to a data lake, the data lake itself does not store data, but reads data from the underlying data storage when the user needs it. The data lake corresponds to multiple data sources, and the data sources are used to store data. The multiple data sources include a target data source, including: A receiving module, configured to receive a first SQL table creation statement input by a user, wherein the first SQL table creation statement includes a data source identifier and a table name but does not include fields, and the first SQL table creation statement is used to indicate the creation of a data table with the table name for a target data source indicated by the data source identifier; A determination module, configured to extract at least one data record stored in the target data source from the target data source; and determine, based on the at least one data record, each data item of the data record and its corresponding first data type identifier; A creation module is used to uniformly map the data types of the first data type identifier corresponding to the data item to obtain the second data type identifier of the field corresponding to the data item, so as to generate a second SQL table creation statement, wherein the second SQL table creation statement includes the table name, the field and its second data type identifier; based on the second SQL table creation statement, a data table corresponding to the target data source is created so that the user can query the data in the target data source based on the data table in the data lake.

6. The device according to claim 5, characterized in that The second data type identifiers determined based on the first data type identifiers indicating the same data type are the same.

7. The device according to claim 5, characterized in that The target data source includes at least one of the following: Databases and storage services with data storage capabilities.

8. The device according to claim 5, characterized in that The target data source includes at least one of the following: Object storage service OSS, table storage service OTS, database MySQL, database SQL Server, database Postgres, relational database PolarDB, distributed document storage database MangoDB, and database Redis.

9. A device for creating a data table, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method for creating a data table according to any one of claims 1 to 4.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method for creating a data table according to any one of claims 1 to 4 is implemented.

Images