Wide table optimization method and device, electronic equipment and computer readable storage medium

Abstract

The present application relates to artificial intelligence technology, and discloses a wide table optimization method, comprising: obtaining metadata in a database, and generating a characteristic field of the metadata; generating an index field according to the characteristic field, generating an index data table based on the index field; obtaining a target attribute of the characteristic field according to the index data table, and establishing a first-level wide table about the target attribute according to a preset index mode; performing element mapping on a target field in the first-level wide table to obtain a mapping field; generating a mapping data table according to the mapping field and the characteristic field, and storing the mapping data table and the target field in the first-level wide table in association to obtain a target wide table. In addition, the present application also relates to blockchain technology, and the data list can be stored in a node of the blockchain. The present application also proposes a wide table optimization device, an electronic device and a storage medium. The present application can improve the performance of the wide table.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a wide table optimization method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the continuous expansion of the company's business, the data is numerous and the types are various, the generated data field increases significantly, and the generated wide table contains most of the fields required by business personnel, which is more intuitive for business personnel, and the wide table helps data management, code development, problem positioning, improves query performance, quickly responds to query tasks, reduces use cost, and improves user satisfaction, so it is necessary to establish a wide table.

[0003] Nowadays, the implicit index adopted by the wide table is easy to cause index invalidation and inaccurate data, and the wide table contains a single database type, which provides insufficient scenarios for data processing and analysis, and cannot meet the different needs of customers. Therefore, how to improve the performance of the wide table has become a problem to be solved. SUMMARY

[0004] The present application provides a wide table optimization method, device and computer readable storage medium, which mainly aims to solve the problem of insufficient performance of the wide table.

[0005] To achieve the above purpose, the wide table optimization method provided by the present application comprises:

[0006] Obtaining metadata in a database, generating a feature field of the metadata;

[0007] Generating an index field according to the feature field, and generating an index data table based on the index field;

[0008] Obtaining a target attribute of the feature field according to the index data table, and establishing a first wide table about the target attribute according to a preset index mode;

[0009] Element mapping is performed on the target field in the first wide table to obtain a mapping field;

[0010] A mapping data table is generated according to the mapping field and the feature field, the mapping data table and the target field in the first wide table are stored in association to obtain a target wide table.

[0011] Optionally, the index data table is generated based on the index field, comprising:

[0012] Obtaining an initial framework of the index data table, and performing field form configuration on the initial framework according to the index field to obtain a first framework;

[0013] The first-level framework is configured with a trigger event to obtain a second-level framework;

[0014] The second-level framework is configured with a jump page to obtain a third-level framework;

[0015] The third-level framework is configured with a query function to obtain an index data table.

[0016] Optionally, the index data table is generated based on the index field, including:

[0017] The index field required for defining a primary key and a node field required for creating an index data table are filtered from the index field by using a preset decision tree model;

[0018] A primary key index is created according to the filtered index field;

[0019] The primary key index and the node field are written into a blank framework of the index data table to generate the index data table.

[0020] Optionally, the index data table is generated based on the index field, including:

[0021] The index field is grouped according to a key value to obtain a hash column;

[0022] A hash code of each hash column is calculated one by one by using a preset storage engine;

[0023] A row pointer pointing to each index field is saved in a preset hash table, and the index data table is generated by using the row pointer and the hash code.

[0024] Optionally, the target field in the first-level wide table is mapped to obtain a mapping field, including:

[0025] The target field is defined as a mapping element, and the mapping element is uniquely valued to obtain a mapping value of the mapping element;

[0026] The mapping value and the mapping element are stored in association to obtain the mapping field.

[0027] Optionally, the mapping data table and the target field in the first-level wide table are stored in association to obtain a target wide table, including:

[0028] A unique identifier of the mapping data table is obtained, and a data interface parameter of the mapping data table is queried according to the unique identifier;

[0029] The data interface parameter is compiled into an interface connection statement;

[0030] The interface connection statement is used to store the mapping data table in association with the target field in the first wide table, so as to obtain a target wide table.

[0031] Optionally, the generating the feature field of the metadata comprises:

[0032] According to a preset data type, a first-level field is selected from the metadata.

[0033] Irregular data in the first-level field is filtered out, so as to obtain a second-level field.

[0034] The second-level field is subjected to format conversion, so as to obtain the feature field of the metadata.

[0035] In order to solve the above problems, the application further provides a wide table optimization device, the device comprises:

[0036] A feature field module is configured to obtain metadata in a database and generate a feature field of the metadata.

[0037] An index data table module is configured to generate an index field according to the feature field and generate an index data table based on the index field.

[0038] A first-level wide table module is configured to obtain a target attribute of the feature field according to the index data table and establish a first-level wide table about the target attribute according to a preset index mode.

[0039] An element mapping module is configured to perform element mapping on a target field in the first-level wide table, so as to obtain a mapping field.

[0040] A target wide table module is configured to generate a mapping data table according to the mapping field and the feature field, store the mapping data table in association with the target field in the first-level wide table, and obtain a target wide table.

[0041] In order to solve the above problems, the application further provides an electronic device, the electronic device comprises:

[0042] At least one processor; and,

[0043] A memory in communication connection with the at least one processor; wherein,

[0044] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the wide table optimization method described above.

[0045] To solve the above problems, the application further provides a computer readable storage medium, wherein at least one computer program is stored in the computer readable storage medium, and the at least one computer program is executed by a processor in an electronic device to implement the wide table optimization method.

[0046] In the embodiment of the application, the index field in the feature field is extracted, the index data table is generated based on the index field, the metadata obtained from multiple databases is implemented to realize the compatible mode of the multiple databases, the index data table is generated to help improve the data retrieval speed, the target attribute obtained from the index data table is used to establish a first wide table, the link between the index data table and the first wide table is formed, the explicit index is used to effectively avoid index invalidation, the target field in the first wide table is mapped, the mapping data table is generated according to the mapping field and the feature field, the mapping data table and the target field in the first wide table are stored in association, and the target wide table is obtained, the target wide table has an association relationship with the mapping data table and the index data table, the data interface is simplified, and the query performance is improved. Therefore, the wide table optimization method, device, electronic device and computer readable storage medium are provided, and the problem of insufficient performance of the wide table can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of a wide table optimization method provided by an embodiment of the application is shown in the figure.

[0048] Figure 2 A flowchart of generating an index data table provided by an embodiment of the application is shown in the figure.

[0049] Figure 3 A flowchart of associated storage table provided by an embodiment of the application is shown in the figure.

[0050] Figure 4 A functional module diagram of a wide table optimization device provided by an embodiment of the application is shown in the figure.

[0051] Figure 5 A structural diagram of an electronic device for implementing the wide table optimization method provided by an embodiment of the application is shown in the figure.

[0052] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0054] The wide table optimization method provided in the embodiments of the present application. The execution subject of the wide table optimization method includes, but is not limited to, at least one of electronic devices capable of being configured to execute the method provided in the embodiments of the present application, such as a server and a terminal. In other words, the wide table optimization method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, a cloud server cluster, and the like. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0055] Referring to Figure 1 FIG. 1 is a flowchart of a wide table optimization method provided in an embodiment of the present application. In the embodiment, the wide table optimization method includes:

[0056] S1, obtaining metadata in a database, and generating a feature field of the metadata.

[0057] In the embodiment of the present application, the metadata refers to unprocessed data, which can be obtained from various databases.

[0058] In the embodiment of the present application, the generation of the feature field of the metadata includes: selecting a first-level field from the metadata according to a preset data type; filtering out non-compliant data in the first-level field to obtain a second-level field; and performing format conversion on the second-level field to obtain the feature field of the metadata.

[0059] In detail, the selection of the first-level field from the metadata according to the preset data type needs to determine first the data management system running on the database server, whether there is manual data, the order of magnitude of the manual data, whether there is unstructured data, and the like, and then determine the linking function of the database according to the above information. The source data can be exported into a.txt or.xls file by using a DBMS, an ODBC mode, a program interface, or a tool, and then the source system file is imported into an operational data storage.

[0060] In detail, the task of filtering out non-compliant data in the first-level field is to filter out data that does not meet the requirements, and to determine whether to filter out such data that does not meet the requirements or to correct the data by a business unit and then extract the data. The data that does not meet the requirements generally includes three categories of incomplete data, incorrect data, and duplicate data.

[0061] Further, the task of the format conversion of the secondary field mainly performs inconsistent data conversion, data granularity conversion and some business rule calculation, the inconsistent data conversion is an integration process of unifying the same type of data of different business systems, such as the same supplier, the coding is XX0001 in the settlement system, and the coding is YY0001 in the CRM, so after extraction, it is uniformly converted into an encoding, the data granularity conversion is because the data in the data warehouse is used for analysis, and does not need very detailed data, generally, the business system data is aggregated according to the data warehouse granularity, the business rule calculation is different for different enterprises, different business rules and different data indicators, sometimes these indicators cannot be completed by simply adding and subtracting, at this time, the data indicators are calculated and stored in the data warehouse for analysis.

[0062] S2, generating an index field according to the characteristic field, and generating an index data table based on the index field.

[0063] In the embodiment of the application, the index data table is established to enable faster data query and effectively improve the query efficiency.

[0064] In the embodiment of the application, the index data table is generated based on the index field, comprising:

[0065] S21, obtaining an initial framework of the index data table, and performing field form configuration on the initial framework according to the index field to obtain a first-level framework;

[0066] S22, performing trigger event configuration on the first-level framework to obtain a second-level framework;

[0067] S23, performing jump page configuration on the second-level framework to obtain a third-level framework;

[0068] S24, performing query function configuration on the third-level framework to obtain the index data table.

[0069] In detail, the field form configuration includes but is not limited to configuring the font size and font color of the field, and the display mode, wherein the display mode includes but is not limited to text box display and pop-up window display.

[0070] In actual application, since there is correlation between data, the trigger event configuration can be performed on the data list initial framework, so that when a user triggers a specific event (such as clicking some specific field), another field or page can be jumped to according to the preset correlation. The embodiment of the application can perform trigger event configuration on the second framework through a function with a judging function, such as an IF function, or a computer program with an event listening function, so as to capture each trigger event in the list.

[0071] For example, the data list initial framework includes field A, and field A has a calling correlation with field B. When the preset function or program captures the click event of field A in the data list initial framework, the calling of field B can be realized.

[0072] Alternatively, the first data list of the data list initial framework includes field C, and field C has a calling correlation with the second data list in the data list initial framework. When the preset function or program captures the click event of field C, the calling of the second data list can be realized.

[0073] In the embodiment of the application, the page jump configuration can be realized by generating a hyperlink for the field with a correlation or generating a calling link. For example, field A has a calling correlation with field B. A hyperlink of field B is generated for field A, so that field B can be called when field A is clicked. Alternatively, a hyperlink of field A is generated for field B, so that field A can be called when field B is clicked.

[0074] In the embodiment of the application, the query configuration of the data list initial framework according to the query function in the target configuration includes: obtaining a query function corresponding to the query function, determining a query parameter corresponding to the query function; and generating a query entry corresponding to the query function in the data list initial framework according to the query parameter.

[0075] In detail, the query function can be multiple, and the query function includes but is not limited to a condition query function, a left fuzzy query function, a right fuzzy query function and an accurate query function.

[0076] Since the query modes of different query functions are different, the query parameters required by different query functions are also different. The query parameter is a field required by the query function when performing data query, such as a keyword and a similar word.

[0077] The embodiment of the present application determines the query parameter of each query function through a preset query parameter table.

[0078] Further, the embodiment of the present application generates a corresponding query entrance for different query functions according to different query parameters respectively, wherein the query entrance is a visual interface for realizing the query mode corresponding to the query function.

[0079] In the embodiment of the present application, the generating of the index data table based on the index field comprises: screening the index field required for defining the primary key and the node field required for creating the index data table from the index field by using a preset decision tree model; creating a primary key index according to the screened index field; and writing the primary key index and the node field into the blank framework of the index data table to generate the index data table.

[0080] In detail, the index field required for defining the primary key is used for uniquely identifying each piece of data in the table and has uniqueness, and the primary key index preferably selects one field as the index and the field is preferably updated least frequently.

[0081] In the embodiment of the present application, the generating of the index data table based on the index field comprises: grouping the index field according to the key value to obtain a hash column; calculating the hash code of each hash column one by one by using a preset storage engine; saving a row pointer pointing to each index field in a preset hash table; and generating the index data table by using the row pointer and the hash code.

[0082] In detail, the hash code represents a feature of an object, for example: we judge whether two strings are equal, if the hash codes are equal, then the two strings are equal; the row pointer points to an entire row of an object, not a specific element; the index data table is a data structure directly accessed according to the key code value, that is, it maps the key code value to a position in the table to access the record to speed up the search; and the key code value can be converted into a hash value by using a preset function conversion algorithm.

[0083] Further, the key value refers to a data item capable of playing an identification role in a data element, in a database, the key is composed of one or more attributes, for example: the key value of the primary key, the key value of the super key, the key value of the candidate key, the key value of the foreign key; the primary key refers to a column or a set of multiple columns, the value of which can uniquely identify each row in the table, and through it, the entity integrity of the table can be forced, the primary key can guarantee the integrity of the entity, the entity integrity requires that the primary key field in each table cannot be empty or repeated, can speed up the operation of the database, and when a new record is added to the table, the DBMS automatically checks the primary key value of the new record, and does not allow the value to be repeated with the primary key value of other records, and the DBMS automatically displays the records in the table according to the order of the primary key value; the super key can uniquely identify the attribute set of a tuple in a relation, such a key is called a super key of a relation mode, an attribute can be a super key, and multiple attributes combined together can also be a super key, for example, in a student table, assuming that there are "student number", "name", "gender" and the like fields, wherein the student number is unique, so the "student number" is a super key; the combination of "student number", "name" and "birthday" is also unique, so it can also be a super key, as long as the attributes can uniquely determine a student.

[0084] S3, obtaining the target attribute of the characteristic field according to the index data table, and establishing a first wide table about the target attribute according to a preset index mode.

[0085] In the embodiment of the application, after the target attribute of the characteristic field is obtained according to the index data table, the target attribute can be connected with the first wide table, and the target attribute can be obtained by field extraction.

[0086] In detail, if the index data table is a data table about the scores of students in a school, the target attributes can be "name", "mathematics", "grade", "Chinese", "English", "integrated science" and "subject", and the first wide table contains the rough items of the scores of the students, but does not contain the specific single subject scores of each student, for example, the first wide table displays the student name, class ranking, school ranking and the class in which the student is located.

[0087] In detail, the first wide table about the target attribute is established according to the preset index mode, that is, the target attribute is screened to obtain the classification category of the data table, the characters of the classification category are written into the rows of the data table, and other blank table boxes of the table filled with characters are filled, and the determination of the classification category can be performed by using the preset index mode.

[0088] S4, performing element mapping on the target field in the first wide table to obtain a mapping field.

[0089] In the embodiment of the present application, the element mapping of the target field in the first wide table to obtain a mapping field comprises: defining the target field as a mapping element, uniquely assigning the mapping element to obtain a mapping value of the mapping element; and storing the mapping value and the mapping element in association to obtain the mapping field.

[0090] In the embodiment of the present application, the target field can be "kitten", "puppy", "rabbit", and "tiger cub", and the "kitten", "puppy", "rabbit", and "tiger cub" are respectively assigned as "001", "002", "003", and "004", wherein "001", "002", "003", and "004" are mapping values of the mapping element, and the mapping field obtained is "kitten" associated with "001", "puppy" associated with "002", "rabbit" associated with "003", and "tiger cub" associated with "004".

[0091] In detail, the mapping refers to a special corresponding relationship, for example: a person's ID number corresponds to a person, a movie ticket corresponds to a person, one arrow shoots one bird in life, multiple arrows shoot one bird, and one arrow shoots multiple birds, but each arrow in the mapping must shoot a bird, and one arrow cannot shoot multiple birds. Mapping is to correspond two objects, the corresponding object is called image, and the object being corresponded is called original image.

[0092] S5, generating a mapping data table according to the mapping field and the feature field, storing the mapping data table in association with the target field in the first wide table to obtain a target wide table.

[0093] In the embodiment of the present application, the storing of the mapping data table in association with the target field in the first wide table to obtain a target wide table comprises:

[0094] S31, obtaining a unique identifier of the mapping data table, and querying a data interface parameter of the mapping data table according to the unique identifier;

[0095] S32, compiling the data interface parameter into an interface connection statement;

[0096] S33, storing the mapping data table in association with the target field in the first wide table by using the interface connection statement to obtain a target wide table.

[0097] In detail, the unique identifier includes but is not limited to a database number, a database address, and other identifiers that can be used to uniquely mark the preset database.

[0098] Specifically, the unique identifier can be used to query the data interface parameters corresponding to the preset database from a preset database interface parameter table, wherein the database interface parameter table contains a plurality of databases and data interface parameters corresponding to data interfaces in each database.

[0099] In the embodiment of the application, the data interface parameters can be compiled into an interface connection statement, and the target framework and the preset database are connected by using the interface connection statement, so as to realize database connection configuration of the target framework and obtain a data list.

[0100] In the embodiment of the application, the index field in the feature field is extracted, and an index data table is generated based on the index field. The metadata is obtained from a plurality of databases, a compatible mode of multiple databases is realized, and the generation of the index data table helps to improve the data retrieval speed. A first wide table is established by using target attributes obtained from the index data table, a link between the index data table and the first wide table is formed, index invalidation is effectively avoided by using explicit index, target fields in the first wide table are mapped, a mapping data table is generated according to the mapping fields and the feature fields, the mapping data table and the target fields in the first wide table are stored in association, a target wide table is obtained, the target wide table, the mapping data table and the index data table are in an associated relationship, the data interface is simplified, and the query performance is improved. Therefore, the wide table optimization method can solve the problem of insufficient performance of the wide table.

[0101] As shown in Figure 4 FIG. 1 is a functional module diagram of a wide table optimization device according to an embodiment of the application.

[0102] The wide table optimization device 100 can be installed in an electronic device. According to the functions to be implemented, the wide table optimization device 100 can include a feature field module 101, an index data table module 102, a first wide table module 103, an element mapping module 104 and a target wide table module 105. The modules of the application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, and are stored in the memory of the electronic device.

[0103] In this embodiment, the functions of each module / unit are as follows:

[0104] The feature field module is configured to obtain metadata in a database and generate feature fields of the metadata.

[0105] The index data table module is configured to generate an index field according to the feature field and generate an index data table based on the index field.

[0106] The first wide table module is configured to obtain target attributes of the feature fields according to the index data table, and establish a first wide table about the target attributes according to a preset index mode.

[0107] The element mapping module is configured to perform element mapping on the target fields in the first wide table to obtain mapping fields.

[0108] The target wide table module is configured to generate a mapping data table according to the mapping fields and the feature fields, store the mapping data table in association with the target fields in the first wide table, and obtain a target wide table.

[0109] As shown in Figure 5 FIG. 1 is a structural schematic diagram of an electronic device for implementing the wide table optimization method according to an embodiment of the present application.

[0110] The electronic device 1 can include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and can further include a computer program (such as a wide table optimization program) stored in the memory 11 and executable on the processor 10.

[0111] In some embodiments, the processor 10 can be composed of integrated circuits, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more combinations of central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control core of the electronic device, which connects all components of the electronic device through various interfaces and lines, executes or runs programs or modules stored in the memory 11 (such as executing a wide table optimization program), and calls data stored in the memory 11 to perform various functions and process data of the electronic device.

[0112] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device. The memory 11 can be used to store application software and various data installed in the electronic device, such as the code of the wide table optimization program, and can also be used to temporarily store data that has been output or will be output.

[0113] The communication bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory 11 and at least one processor 10, etc.

[0114] The communication interface 13 is used for communication between the electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (e.g., a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device and other electronic devices. The user interface can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device and to display a visualized user interface.

[0115] It should be appreciated that the electronic device shown in the figures is merely illustrative and does not limit the scope of the patent application. The electronic device can include fewer or more components than shown in the figures, or combine some components, or arrange the components differently.

[0116] For example, although not shown, the electronic device can further include a power supply (such as a battery) for powering the various components. Preferably, the power supply can be logically connected to the at least one processor 10 via a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0117] It should be appreciated that the embodiments are merely illustrative and do not limit the scope of the patent application.

[0118] The wide table optimization program stored in the memory 11 of the electronic device 1 is a combination of a plurality of instructions, which, when executed in the processor 10, can achieve:

[0119] Obtaining metadata in a database, generating a feature field of the metadata;

[0120] Generating an index field according to the feature field, and generating an index data table based on the index field;

[0121] Obtaining a target attribute of the feature field according to the index data table, and establishing a first wide table about the target attribute according to a preset index mode;

[0122] Mapping elements in the target field of the first wide table to obtain a mapping field;

[0123] Generating a mapping data table according to the mapping field and the feature field, and storing the mapping data table and the target field of the first wide table in association to obtain a target wide table.

[0124] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the related steps in the corresponding embodiments of the accompanying drawings, which is not described here.

[0125] Further, the modules / units integrated in the electronic device 1 are stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0126] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device:

[0127] obtaining metadata in a database, generating a characteristic field of the metadata;

[0128] generating an index field according to the characteristic field, generating an index data table based on the index field;

[0129] obtaining a target attribute of the characteristic field according to the index data table, and establishing a first wide table about the target attribute according to a preset index mode;

[0130] performing element mapping on a target field in the first wide table to obtain a mapping field;

[0131] generating a mapping data table according to the mapping field and the characteristic field, and storing the mapping data table and the target field in the first wide table in association to obtain a target wide table.

[0132] In several embodiments provided in the application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and the division of the modules is merely a logical function division, and there can be another division manner in actual implementation.

[0133] The modules illustrated as separate components can or can not be physically separate, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0134] In addition, each functional module in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software function module.

[0135] It will be obvious to a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments but can be implemented in other embodiments without departing from the scope of the application.

[0136] The embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the above description, all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No single feature or combination of features should be considered essential unless expressly stated in the claims.

[0137] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block containing a batch of network transaction information for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0138] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0139] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names and not to indicate any particular order.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method of wide table optimization, the method comprising: The method comprises: Obtaining metadata in a database, generating a characteristic field of the metadata; Generating an index field according to the characteristic field, obtaining an initial framework of a preset index data table, configuring a field form of the initial framework according to the index field to obtain a first-level framework, configuring a trigger event of the first-level framework to obtain a second-level framework, configuring a jump page of the second-level framework to obtain a third-level framework, configuring a query function of the third-level framework to obtain an initial index data table, screening out an index field required for defining a primary key and a node field required for creating an index data table from the index field by using a preset decision tree model, creating a primary key index according to the screened index field, and writing the primary key index and the node field into the initial index data table to generate an index data table; Obtaining a target attribute of the characteristic field according to the index data table, and establishing a first-level wide table about the target attribute according to a preset index mode; Mapping an element of a target field in the first-level wide table to obtain a mapping field; Generating a mapping data table according to the mapping field and the characteristic field, and storing the mapping data table and the target field in the first-level wide table in association to obtain a target wide table.

2. The wide table optimization method of claim 1, wherein, Generating an index data table based on the index field comprises: Grouping the index field according to a key value to obtain a hash column; Calculating a hash code of each hash column one by one by using a preset storage engine; Saving a row pointer pointing to each index field in a preset hash table, and generating an index data table by using the row pointer and the hash code.

3. The wide table optimization method of claim 1, wherein, The element mapping of the target field in the first-level wide table to obtain the mapping field comprises: Defining the target field as a mapping element, uniquely assigning a value to the mapping element to obtain a mapping value of the mapping element; and Storing the mapping value and the mapping element in association to obtain the mapping field.

4. The wide table optimization method of claim 1, wherein, The associated storage of the mapping data table and the target field in the first-level wide table to obtain the target wide table comprises: Obtaining a unique identifier of the mapping data table, querying a data interface parameter of the mapping data table according to the unique identifier; Compiling the data interface parameter into an interface connection statement; Storing the mapping data table and the target field in the first-level wide table in association by using the interface connection statement to obtain the target wide table.

5. The wide table optimization method of any one of claims 1 to 4, wherein, The generation of the characteristic field of the metadata comprises: Selecting a first-level field from the metadata according to a preset data type; Filtering out non-compliant data in the first-level field to obtain a second-level field; Converting the format of the second-level field to obtain the characteristic field of the metadata.

6. A wide table optimization apparatus, characterized by, The device comprises: A characteristic field module configured to obtain metadata in a database and generate a characteristic field of the metadata; An index data table module is configured to generate an index field according to the feature field, obtain an initial framework of a preset index data table, configure a field form of the initial framework according to the index field to obtain a first-level framework, configure a trigger event of the first-level framework to obtain a second-level framework, configure a jump page of the second-level framework to obtain a third-level framework, configure a query function of the third-level framework to obtain an initial index data table, filter out an index field required for defining a primary key and a node field required for creating an index data table from the index field by using a preset decision tree model, create a primary key index according to the filtered index field, and write the primary key index and the node field into the initial index data table to generate an index data table. A first-level wide table module is configured to obtain a target attribute of the feature field according to the index data table, and establish a first-level wide table about the target attribute according to a preset index mode. An element mapping module is configured to perform element mapping on a target field in the first-level wide table to obtain a mapping field. A target wide table module is configured to generate a mapping data table according to the mapping field and the feature field, store the mapping data table and the target field in the first-level wide table in association to obtain a target wide table.

7. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the wide table optimization method in any one of claims 1 to 5.

8. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the wide table optimization method in any one of claims 1 to 5.

Images