Data processing method and device, electronic equipment and computer readable storage medium

Abstract

The application provides a data processing method and device, electronic equipment and computer readable storage medium; relates to big data technology in cloud technology field; the method comprises the following steps: obtaining data constraint conditions for a data platform; wherein the data constraint conditions comprise index constraint conditions and data table constraint conditions; the index construction process is constrained and processed according to the index constraint conditions, and the constructed index is obtained; the data table construction process is constrained and processed according to the data table constraint conditions, and the constructed data table comprising a plurality of indexes is obtained; the data platform is processed for data acquisition according to the plurality of indexes in the data table, and the data obtained from the data platform is added to the data table; the metadata of the data table is constructed according to the data constraint conditions, and the query request for the data table is responded according to the metadata. Through the application, the standardization and uniformity of the data table can be improved, and the query efficiency of the data table can also be improved.

Description

Technical Field

[0001] This application relates to big data technology, and more particularly to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of computer technology, data has become increasingly important in the modern era. For example, enterprise data platforms store a large amount of data related to the enterprise's business. For purposes such as analytical reporting and decision support, data tables are usually constructed based on the data in the data platform to provide clear and effective data support for business operations.

[0003] In the process of building data tables, it is common practice to manually write specification documents and use these documents to guide developers in developing orderly development habits. However, due to differences in individual developers' understanding, the binding force of these specification documents is weak, which may result in siloed development outcomes. For example, the resulting data tables may have inconsistent formats and lack accurate and effective data query capabilities. Summary of the Invention

[0004] This application provides a data processing method, apparatus, electronic device, and computer-readable storage medium, which can ensure the standardization and uniformity of the obtained data table, and also improve the query efficiency of the data table.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a data processing method, including:

[0007] Obtain the data constraints for the data platform; wherein, the data constraints include indicator constraints and data table constraints;

[0008] The indicator construction process is constrained according to the aforementioned indicator constraints to obtain the constructed indicators.

[0009] The data table construction process is constrained according to the data table constraints to obtain a constructed data table that includes multiple of the aforementioned indicators.

[0010] The data platform is processed to acquire data based on multiple indicators in the data table, and the data acquired from the data platform is added to the data table.

[0011] The metadata of the data table is constructed based on the data constraints, and a query request for the data table is responded to based on the metadata.

[0012] This application provides a data processing apparatus, including:

[0013] The condition acquisition module is used to acquire data constraints for an independent data platform; wherein, the data constraints include indicator constraints and data table constraints.

[0014] The indicator constraint module is used to constrain the indicator construction process according to the indicator constraint conditions to obtain the constructed indicators.

[0015] The data table constraint module is used to constrain the data table construction process according to the data table constraint conditions to obtain a constructed data table including multiple indicators.

[0016] The data acquisition module is used to acquire data from the data platform based on multiple indicators in the data table, and add the data acquired from the data platform to the data table.

[0017] The query module is used to construct the metadata of the data table based on the data constraints, and respond to query requests for the data table based on the metadata.

[0018] This application provides an electronic device, including:

[0019] Memory, used to store executable instructions;

[0020] The processor, when executing executable instructions stored in the memory, implements the data processing method provided in the embodiments of this application.

[0021] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the data processing method provided in this application.

[0022] The embodiments of this application have the following beneficial effects:

[0023] By constraining the indicator construction process according to the indicator constraints in the data constraints, and by constraining the data table construction process according to the data table constraints, the standardization and consistency of the constructed data tables and the indicators in the data tables can be guaranteed. This ensures that after data from the data platform is added to the data tables, effective data support can be provided based on the data tables. At the same time, by constructing the metadata of the data tables according to the data constraints, the query efficiency of the data tables can be improved, and intelligent response to query requests can be achieved. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the architecture of the data processing system provided in the embodiments of this application;

[0025] Figure 2This is a schematic diagram of the architecture of the terminal device provided in the embodiments of this application;

[0026] Figure 3A This is a flowchart illustrating the data processing method provided in an embodiment of this application;

[0027] Figure 3B This is a flowchart illustrating the data processing method provided in an embodiment of this application;

[0028] Figure 3C This is a flowchart illustrating the data processing method provided in an embodiment of this application;

[0029] Figure 3D This is a flowchart illustrating the data processing method provided in an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of the indicator system provided in the embodiments of this application;

[0031] Figure 5 This is a schematic diagram illustrating the usage process of the data warehouse management platform provided in this application embodiment;

[0032] Figure 6 This is a schematic diagram of the data table query interface provided in an embodiment of this application;

[0033] Figure 7 This is a schematic diagram of the newly added dimension interface provided in the embodiments of this application;

[0034] Figure 8 This is a schematic diagram of the data warehouse layering provided in the embodiments of this application;

[0035] Figure 9 This is a schematic diagram of the newly added business process interface provided in the embodiments of this application;

[0036] Figure 10 This is a schematic diagram of the newly added service limitation interface provided in the embodiments of this application;

[0037] Figure 11 This is a schematic diagram of the newly added atomic index interface provided in the embodiments of this application;

[0038] Figure 12 This is a schematic diagram of the newly added derived index interface provided in the embodiments of this application;

[0039] Figure 13 This is a schematic diagram of the newly added derived index interface provided in the embodiments of this application;

[0040] Figure 14 This is a schematic diagram of the data table creation interface provided in an embodiment of this application;

[0041] Figure 15This is a schematic diagram of the architecture of the data warehouse management platform provided in the embodiments of this application;

[0042] Figure 16 This is a schematic diagram of the architecture of the data warehouse management platform provided in the embodiments of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0045] In the following description, the terms "first," "second," and "third" are used merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first," "second," and "third" may be interchanged in a specific order or sequence where permissible, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein. In the following description, the term "multiple" refers to at least two.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0047] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0048] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0049] 1) Data Platform: Related to business operations, it provides functions such as storage, computation, and presentation of data generated during business processes. For example, during the operation of a business, a large amount of data generated can be stored in a data platform, and the data platform's own computing power can be used to calculate business reports. Then, the visualization capabilities provided by the data platform can be used to present the business reports in a graphical interface. This application embodiment does not limit the type of data platform; for example, it could be a big data platform.

[0050] 2) Data constraints: These include indicator constraints and data table constraints. Indicator constraints are used to constrain the indicator construction process, while data table constraints are used to constrain the data table construction process. For example, data constraints may include some configured items, and these configuration items are used to implement the constraints.

[0051] 3) Metrics: These are the combination of business and data, the foundation of data statistics, and the basis for quantifying data. This application does not limit the applicable metric system. For example, the metric system may include dimensions, business processes, business constraints, atomic metrics, and derived metrics. The meaning of each type of metric will be explained later.

[0052] 4) Database: A collection of data stored together in a certain way, shared by multiple users, with minimal redundancy, and independent of applications. It supports user operations such as adding, querying, updating, and deleting data. Database types include Oracle and MySQL. Databases are primarily used for basic, routine transaction processing, while data warehouses are mainly used for data analysis. Data warehouses are created for analytical reporting and decision support purposes, providing guidance for business process improvement, monitoring time, cost, quality, and control. Essentially, a data warehouse can be considered a special type of database. This application does not limit the type of data warehouse; for example, it could be a Hive data warehouse.

[0053] In this embodiment, the data platform can store data based on a database or a data warehouse. Within the data platform, data can be stored in the form of data tables, which include concepts such as table names and fields.

[0054] 5) Metadata: Also known as intermediary data or relay data, in this embodiment, it is used to describe data tables. Based on metadata, query capabilities for the corresponding data tables can be provided. Metadata can be constructed based on data constraints; for example, it can be constructed based on configuration data for at least some configuration items.

[0055] 6) Big Data: Refers to data sets that cannot be captured, managed, and processed within a certain timeframe using conventional software tools. It represents massive, rapidly growing, and diverse information assets that require new processing models to achieve stronger decision-making, insight discovery, and process optimization capabilities. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to effectively process large amounts of data within a tolerable timeframe. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems. In this application embodiment, a big data platform can be built based on big data technologies.

[0056] This application provides a data processing method, apparatus, electronic device, and computer-readable storage medium, which can ensure the standardization and uniformity of the constructed data tables and indicators, and also improve the query efficiency of the data tables. The following describes exemplary applications of the electronic device provided in this application. The electronic device provided in this application can be implemented as various types of terminal devices or as a server.

[0057] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the data processing system 100 provided in this application embodiment. Terminal device 400 connects to server 200 via network 300-1, and server 200 connects to server 500 via network 300-2. Network 300-1 can be a wide area network (WAN), a local area network (LAN), or a combination of both; the same applies to network 300-2. Server 500 can be a server of a data platform. Here, we take a data platform with only one server as an example, but it should be understood that a data platform can also include multiple servers and several terminal devices.

[0058] In some embodiments, taking the electronic device as a terminal device as an example, the data processing method provided in this application can be implemented by the terminal device. For example, the terminal device 400 runs a client 410, which obtains data constraints for the data platform. These data constraints can be pre-stored locally on the client 410 or obtained in real time, such as by real-time input from the user. The client 410 performs constraint processing on the indicator construction process according to the indicator constraints in the data constraints to obtain the constructed indicators. Simultaneously, it performs constraint processing on the data table construction process according to the data table constraints in the data constraints to obtain a constructed data table including multiple indicators.

[0059] Then, client 410 retrieves and processes data from server 500 (i.e., the data platform) based on multiple metrics in the data table, and adds the data (i.e., metric values) obtained from server 500 to the data table, completing the data table production. Production refers to the process of adding metric values ​​to the data table. Simultaneously, client 410 constructs the metadata of the data table based on data constraints and responds to query requests for the data table based on the metadata. It's worth noting that client 410 can store the produced data table locally or on server 500, and the constructed metadata can be stored similarly.

[0060] In some embodiments, taking the electronic device as a server as an example, the data processing method provided in this application embodiment can also be implemented by a server. For example, the server 200 can obtain data constraints for the data platform and perform constraint processing on the indicator construction process and the data table construction process respectively, and finally obtain a data table including multiple indicators.

[0061] Then, server 200 retrieves and processes data from server 500 based on multiple metrics in the data table, and adds the retrieved data to the data table to produce it. Simultaneously, server 200 constructs the metadata for the data table based on data constraints and responds to query requests for the data table based on the metadata. Similarly, server 200 can store the produced data table locally or on server 500, and the constructed metadata is stored similarly.

[0062] In some embodiments, the data processing method provided in this application can also be implemented collaboratively by a terminal device and a server. For example, the client 410 can obtain data constraints (such as data constraints input by the user) and send the data constraints to the server 200, so that the server 200 can perform the construction of indicators, the construction of data tables, the production of data tables, and the construction of metadata according to the data constraints.

[0063] When client 410 receives a query request, it can send the query request to server 200, so that server 200 can respond to the query request based on metadata. As an example, in... Figure 1 The query data in the query request is shown, as well as metadata 1, metadata 2 and metadata 3, where metadata 1 to 3 are used as a response to the query request.

[0064] In some embodiments, the electronic device provided in this application can be used to build a data table management platform to achieve data processing through interaction between the data table management platform and the data platform. The data table management platform and the data platform can have a non-strongly bound relationship, meaning a communication connection can be established through a common interface, thus improving applicability to different data platforms.

[0065] In some embodiments, such as the data processing method provided in the embodiments of this application, various data involved (such as data constraints, indicators, data tables, data used to generate data tables, and metadata, etc.) can be stored in a blockchain. Because the blockchain has the characteristic of being immutable, the accuracy of the stored data can be guaranteed. For example, the data platform can be a platform built based on blockchain principles, server 500 can be a native node in the blockchain network, and server 200 and / or client 410 can be client nodes in the blockchain network.

[0066] In some embodiments, the terminal device 400 or server 200 can implement the data processing method provided in this application embodiment by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run; it can also be a small program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin.

[0067] In some embodiments, server 200 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The cloud services may be data processing services that can be invoked by terminal device 400. Terminal device 400 may be a smartphone, tablet, laptop, desktop computer, smart TV, smartwatch, etc., but is not limited to these. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0068] Taking the example of a terminal device provided in this application embodiment, it can be understood that in the case where the electronic device is a server, Figure 2 Some parts of the structure shown (such as the user interface, presentation module, and input processing module) can be omitted. See also Figure 2 , Figure 2 This is a schematic diagram of the structure of the terminal device 400 provided in the embodiments of this application. Figure 2The terminal device 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0069] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0070] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0071] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0072] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0073] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0074] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0075] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0076] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with user interface 430 (e.g., a display screen, a speaker, etc.).

[0077] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0078] In some embodiments, the data processing apparatus provided in this application can be implemented in software. Figure 2 A data processing device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a condition acquisition module 4551, an indicator constraint module 4552, a data table constraint module 4553, a data acquisition module 4554, and a query module 4555. These modules are logically connected and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0079] The data processing method provided in this application will be described in conjunction with exemplary applications and implementations of the electronic devices provided in the embodiments of this application.

[0080] See Figure 3A , Figure 3A This is a flowchart illustrating the data processing method provided in the embodiments of this application, which will be combined with... Figure 3A The steps shown are explained.

[0081] In step 101, the data constraints for the data platform are obtained; wherein, the data constraints include indicator constraints and data table constraints.

[0082] During business operations, relevant data generated by the business is typically stored in a data platform. This data is raw, unprocessed data, often scattered and difficult to understand. Therefore, in this embodiment, for purposes such as analytical reporting and decision support, data tables can be constructed based on the data in the data platform to provide clear and effective data support for the business, such as guiding business development.

[0083] First, data constraints for the data platform can be obtained. This application embodiment does not limit the form of the data constraints. For example, data constraints may include several configuration items to implement constraint processing. These configuration items may include necessary and unnecessary configuration items. For necessary configuration items, it is required that corresponding configuration data (user-inputted configuration data) be received; otherwise, subsequent steps cannot be executed. For unnecessary configuration items, it is not required that corresponding configuration data be received; that is, the user can choose whether to input configuration data for unnecessary configuration items. Furthermore, data constraints may also include configuration data conditions corresponding to the configuration items (e.g., less than 10 characters). When the received configuration data does not meet the configuration data conditions, a prompt for re-entry can be displayed to encourage the user to enter configuration data that meets the conditions.

[0084] It's worth noting that data constraints can be pre-set based on the characteristics of the data platform (or the characteristics of the business), or they can be derived by performing common analysis on multiple existing data tables (or metrics) to obtain data constraints that all the existing data tables satisfy. Of course, the premise is that the data tables undergoing common analysis meet the actual data table construction requirements. For different data platforms, data constraints can be the same or different.

[0085] In this embodiment, data constraints may include indicator constraints and data table constraints, used to constrain the indicator construction process and the data table construction process, respectively. Of course, the objects of data constraints are not limited to these; for example, data constraints can also be used to constrain the database structure or data warehouse structure in a data platform, or to constrain related code in the data processing process to ensure the code conforms to specific coding standards. After obtaining the data constraints, they can be updated according to actual needs.

[0086] In step 102, the indicator construction process is constrained according to the indicator constraints to obtain the constructed indicators.

[0087] Here, "metric" refers to statistical indicators used to analyze or integrate data within the data platform. For example, a metric could be the sum of order amounts over the past three days, while the data platform stores the amount of each individual order.

[0088] Metric constraints can include multiple configuration items used to construct the metric. During the metric construction process, these configuration items are presented, and the metric is constructed based on the received configuration data for each item, thus implementing constraint processing for the metric construction process. Of course, the form of metric constraints and the method of constraint processing are not limited to this.

[0089] In some embodiments, after step 102, the method further includes: when the indicator is in an active state, prohibiting the updating of the indicator; when the indicator is in an inactive state, presenting the update option corresponding to the indicator, and updating the indicator when a trigger operation for the update option is received.

[0090] In this embodiment of the application, the indicator may have a specific state, such as an effective state and an ineffective state.

[0091] When a constructed metric is in an active state, it proves that the metric has been enabled, and updates to the metric can be prohibited. For example, updates to the metric's configuration data (also known as metric configuration data) can be prohibited, or the metric can be deleted. This effectively avoids adverse chain reactions caused by updating the metric, such as the data table for the metric becoming unavailable, or the inability to build a new data table based on the metric.

[0092] When a constructed metric is in an inactive state, indicating that it is not yet enabled, an update option for that metric can be presented. Upon receiving a trigger action for that update option, the metric will be updated. This update process can involve updating the metric's configuration data, such as modifying one or more configuration data points; it can also refer to deleting the metric. Of course, the update process is not limited to these methods. By using the above approach, the security and availability of the constructed metrics can be improved.

[0093] It is worth noting that the embodiments of this application do not limit the type of received operation. For example, it can be a contact operation, such as a click or long press; or it can be a non-contact operation, such as voice input or gesture input. For example, the trigger operation for the update option here can be a click.

[0094] In some embodiments, after step 102, the method further includes performing any of the following processes: updating the state of the metric to an effective state in response to an effective configuration operation for the metric; updating the state of the metric to an effective state when a dependent metric that has a configuration dependency relationship with the metric is in an effective state.

[0095] In this embodiment, the state of the constructed metric can be configured based on the received operation. For example, when an operation to activate the configuration of the metric is received, the state of the metric is updated to activated; when an operation to deactivate the configuration of the metric is received, the state of the metric is updated to deactivated.

[0096] The status of the constructed metrics can be automatically configured based on existing configuration dependencies. For example, dependent metrics that have a configuration dependency relationship with the constructed metric can be identified. For instance, if metric A is built based on metric B, then metric A is considered a dependent metric of metric B; that is, the configuration dependency relationship in this embodiment can be a downstream dependency relationship. If metric A is in an active state, the status of metric B can be maintained as active, while updating the status of metric B to inactive is prohibited. This is because if the status of metric B is updated to inactive, metric A may become unavailable, thus adversely affecting business stability. This approach improves the flexibility of status configuration, allowing the selection of a specific method based on the actual application scenario.

[0097] In step 103, the data table construction process is constrained according to the data table constraints to obtain a constructed data table that includes multiple indicators.

[0098] Here, data table constraints can include multiple configuration items used to construct the data table. During the data table construction process, these configuration items are presented, and the data table is constructed based on the received configuration data for these items, thereby implementing constraint processing for the data table construction process. It is worth noting that among the multiple configuration items used to construct the data table, at least some of them are used to select the metrics included in the data table.

[0099] In some embodiments, after step 103, the method further includes: when the data table is in an active state, prohibiting update processing of the data table; when the data table is in an inactive state, presenting update options corresponding to the data table, and updating the data table upon receiving a trigger operation for the update option. Here, the state of the constructed data table can be configured according to the received operation (such as an active configuration operation or an inactive configuration operation), or other configuration methods can be applied. This approach improves the security and stability of the data table.

[0100] In step 104, the data platform is processed to acquire data based on multiple indicators in the data table, and the data acquired from the data platform is added to the data table.

[0101] The data table constructed in step 103 only defines indicators, without including actual values ​​under those indicators. Therefore, in this step, a data table is generated based on multiple indicators in the data table. For example, data is acquired and processed from the data platform based on multiple indicators in the data table, and the data obtained from the data platform (i.e., indicator values) is added to the data table to complete the data table production. For instance, if an indicator in the data table is the payment amount of an order, the specific value of the payment amount for that order (e.g., 100 yuan) can be obtained from the data platform and added (populated) to the corresponding position in the data table.

[0102] In some embodiments, after adding data obtained from the data platform to the data table, the method further includes storing the data table on the data platform. For the generated data table, the electronic device can store it locally or on the data platform, such as in the data platform's database or data warehouse, thereby making better use of the data platform's storage resources. This approach enhances the flexibility of storing data tables.

[0103] In step 105, the metadata of the data table is constructed based on the data constraints, and the query request for the data table is responded to based on the metadata.

[0104] For the data table constructed in step 103, metadata can also be constructed based on data constraints. For example, the received configuration data for at least some of the configuration items in the data constraints can be used as the metadata of the data table. Of course, the method of constructing metadata is not limited to this. Metadata is data used to describe the data table and can reflect the characteristics of the data table. Therefore, accurate and effective query capabilities for the data table can be provided based on metadata, that is, metadata can be used to respond to query requests for the data table.

[0105] For example, upon receiving a query request, the query data in the query request can be matched with multiple historical metadata entries, and the successfully matched historical metadata entries can be presented in response to the query request. Here, historical metadata refers to the metadata of a pre-constructed data table. When presenting successfully matched historical metadata entries, some or all of the data in the successfully matched historical metadata entries can be presented, or the data table corresponding to the successfully matched historical metadata entries can be presented directly. In this embodiment, the matching criteria can be set according to the actual application scenario. For example, the similarity between the query data and each historical metadata entry can be determined, and historical metadata entries with a similarity ranking in a set position (e.g., top 5) can be considered successfully matched historical metadata entries. Alternatively, historical metadata entries with a similarity greater than a similarity threshold (e.g., 70%) can also be considered successfully matched historical metadata entries. This embodiment does not limit the algorithm used to determine the similarity; for example, it can be a Term Frequency–Inverse Document Frequency (TF-IDF) algorithm. When the number of successfully matched historical metadata entries includes multiple entries, they can be presented in descending order of their corresponding similarity, and the same applies below.

[0106] It is worth noting that the execution order of steps 104 and 105 is not limited in this embodiment; they can be executed simultaneously or sequentially, for example, step 105 can be executed before step 104. Furthermore, the metadata of the data table can be constructed based on data constraints during the data table construction process; that is, it is not necessary to wait until the data table construction is complete.

[0107] In some embodiments, the metadata includes multiple types of indexes; after constructing the metadata of the data table according to data constraints, the method further includes: matching the indexes of the data table that conform to a set type with historical indexes that conform to a set type, and presenting the historical indexes that have successfully matched; wherein, the set type is at least one of multiple types; the historical index is an index of the data table that has been constructed and is different from the data table.

[0108] In this embodiment, metadata may include various types of indexes. For example, different types of indexes can be constructed based on the configuration data corresponding to different configuration items in the data constraints. In this case, after obtaining the metadata of the data table, the indexes in the metadata that conform to a set type can be matched with multiple historical indexes that conform to the set type, and the successfully matched historical indexes can be presented. The set type can be at least one of several types. For example, when the metadata includes table name indexes, table description indexes, and field indexes, the set type index can be a table name index. Furthermore, historical indexes are indexes of other constructed data tables that are distinct from this data table.

[0109] It's worth noting that when multiple index types are specified, matching can be performed separately for each type. Furthermore, when presenting successfully matched historical indexes, partial or full metadata of the corresponding data table can also be displayed. This allows users to easily determine whether successfully matched historical indexes meet their needs, thus deciding whether to stop building the data table, effectively saving computational resources (such as storage resources for storing the data table).

[0110] In some embodiments, after step 103, the method further includes: deleting any one of the data tables in response to a stop construction operation for any of the constructed data tables.

[0111] Here, when a stop build operation is received for any of the constructed data tables, the table can be deleted to save computing resources. The stop build option for each data table can be displayed to facilitate receiving stop build operations for that table. It's worth noting that if a stop build operation is received during the data table construction process, the configuration data received during the data table construction process can be deleted, effectively stopping the data table construction.

[0112] In some embodiments, after constructing the metadata of the data table according to data constraints, the method further includes: storing the metadata of the data table in the storage service; synchronously storing the metadata in the storage service in the query service; wherein the query service is used to respond to query requests based on the stored metadata; perform metadata update processing on the storage service in response to an update operation on the metadata; and when the query service meets the metadata update conditions, perform metadata update processing on the query service based on the new metadata obtained from the metadata update processing in the storage service.

[0113] Here, after obtaining the metadata of the data table, the metadata of the data table can be stored in a dedicated storage service, and the metadata in the storage service can be synchronously stored in the query service. The storage service is used to store the latest metadata, and the query service is used to respond to query requests based on the stored metadata, that is, to provide query functionality.

[0114] When an update operation for metadata is received, the storage service can be updated accordingly, such as modifying the metadata stored in the storage service to ensure the timeliness and accuracy of the stored metadata. When the query service meets the metadata update conditions, it is updated based on the new metadata obtained from the metadata update process in the storage service. The metadata update conditions can be set according to the actual application scenario, such as setting it to periodic updates (e.g., once every day), or triggered by the metadata stored in the query service.

[0115] For example, after responding to a query request based on the metadata A1 of table A stored in the query service (i.e., presenting the metadata A1 of table A stored in the query service), if a trigger operation is received regarding the presented metadata A1, then the metadata A1 of table A in the query service is updated based on the metadata A2 of table A in the storage service. Specifically, metadata A2 and metadata A1 can be compared first. If they match, no metadata update is needed for metadata A1 in the query service; if they do not match, then metadata A1 in the query service is updated to metadata A2.

[0116] Here, the presented metadata A1 can refer to a portion of the data in presented metadata A1, and the triggering operation for presented metadata A1 can be to view the full data (i.e., complete data) of metadata A1. When there is no need to update metadata A1 in the query service, the full data of metadata A1 can be presented; when metadata A1 in the query service is updated to metadata A2, the full data of metadata A2 can be presented.

[0117] It is worth noting that the aforementioned storage and query services can be provided by electronic devices, i.e., independent of the data platform, or they can be provided by the data platform. By having storage and query services provide storage and query functions respectively, the computational burden can be effectively reduced, facilitating load balancing. Simultaneously, updating the metadata of the query service based on specific metadata update conditions ensures the accuracy of the metadata in the query service while saving computational resources.

[0118] like Figure 3A As shown, this application embodiment applies constraints to the indicator construction process and the data table construction process according to data constraints, which can ensure the standardization and uniformity of the constructed data table and the indicators in the data table. Thus, after adding data from the data platform to the data table, the data table can provide effective data support, such as providing clear guidance for business. At the same time, the metadata of the data table is constructed according to the data constraints, which can improve the query efficiency of the data table and realize intelligent response to query requests.

[0119] In some embodiments, see Figure 3B , Figure 3B This is a schematic flowchart of a data processing method provided in an embodiment of this application. Figure 3A Step 103 shown can be implemented through steps 201 to 204, which will be explained in conjunction with each step.

[0120] In step 201, in response to the data table construction operation, data table configuration items and field configuration items are presented.

[0121] For example, data table building options can be presented, and the triggering operation for these options can be considered the data table building operation. When a data table building operation is received, the data table building process is constrained according to data table constraints, such as presenting data table configuration items and field configuration items. Data table configuration items are used for configuration at the data table level, such as configuring the physical type of the data table; field configuration items are used for configuration at the field level, such as configuring which metrics will be used as the corresponding metrics for each field. There is no limit to the number of data table configuration items or field configuration items.

[0122] In step 202, data table configuration data for the presented data table configuration items is received.

[0123] For the displayed data table configuration items, configuration data input by the user can be received. For easy distinction, the configuration data received here will be named data table configuration data, and the same applies below.

[0124] In step 203, the field configuration data for the presented field configuration items is received, and the indicators that match the field configuration data are selected from the multiple constructed indicators.

[0125] The system can receive user-input field configuration data for the presented field configuration items. Then, it filters out metrics that match the field configuration data from a pool of pre-built metrics. For example, the field configuration data can include identifiers for one or more metrics; the corresponding metrics can be filtered based on this data. Of course, the function of the field configuration data is not limited to filtering metrics; it can also include data table field names, field types, and field descriptions.

[0126] In step 204, the data table is constructed based on the data table configuration data, field configuration data, and the selected indicators to obtain the constructed data table.

[0127] Here, the data table is constructed based on the data table configuration data, field configuration data, and the selected indicators, thus achieving an effective definition of the data table and its fields.

[0128] exist Figure 3B middle, Figure 3A Step 105 shown can be implemented through steps 205 to 209, which will be explained in conjunction with each step.

[0129] In step 205, a table name index is constructed based on the data table attribute configuration data.

[0130] In this embodiment, the data table configuration item may include a data table attribute configuration item and a data table description configuration item. The data table attribute configuration item is used to configure the overall attributes of the data table, such as physical type, refresh cycle, storage strategy, and storage period. The data table description configuration item is used to configure a description of the data table. Correspondingly, the received data table configuration data may include data table attribute configuration data for the data table attribute configuration item and data table description configuration data (e.g., some descriptive statements) for the data table description configuration item.

[0131] Here, you can configure some or all of the data in the data table attribute configuration as the table name index, thus realizing the construction of the table name index.

[0132] In step 206, a table description index is constructed based on the data table description configuration data.

[0133] Here, some or all of the data in the data table description configuration data can be used as the table description index, that is, the table description index is constructed.

[0134] In step 207, a field index is built based on the field configuration data.

[0135] Here, some or all of the data in the field configuration data can be used as field indexes, thus realizing the construction of field indexes.

[0136] In step 208, the metadata of the data table is constructed based on the table name index, table description index, and field index.

[0137] Here, metadata includes various types of indexes, specifically table name indexes, table description indexes, and field indexes. It's worth noting that metadata may include other data besides table name indexes, table description indexes, and field indexes, depending on the specific application scenario.

[0138] In step 209, the query request for the data table is responded to based on the data table's metadata.

[0139] Here, query requests for data tables can be responded to based on metadata including various types of indexes.

[0140] like Figure 3B As shown in the embodiments of this application, an exemplary data table construction process is provided, which can realize the rapid and efficient construction of data tables; by constructing metadata including various types of indexes, the comprehensiveness and effectiveness of data table queries can be improved, fully meeting the diverse query needs of users.

[0141] In some embodiments, see Figure 3C , Figure 3CThis is a schematic flowchart of a data processing method provided in an embodiment of this application. Figure 3A Step 102 shown can be implemented through steps 301 to 302, which will be explained in conjunction with each step.

[0142] In step 301, in response to the indicator building operation, indicator configuration items for building indicators are presented.

[0143] For example, metric construction options can be presented, and the triggering operation for the metric construction option can be used as the metric construction operation. When the metric includes multiple types, metric construction options corresponding to each type of metric can be presented, such as dimension construction options, business process construction options, business-limited construction options, atomic metric construction options, and derived metric construction options.

[0144] When a metric construction operation is received, the metric construction process is constrained according to the metric constraints. For example, metric configuration items used to construct the metric can be presented. Here, there is no limit to the number of metric configuration items; the metric configuration items used to construct the metric can be determined based on the type of metric.

[0145] In step 302, indicator configuration data for the presented indicator configuration items is received, and indicator construction processing is performed based on the received indicator configuration data to obtain the constructed indicators.

[0146] Here, the system receives indicator configuration data (such as indicator configuration data input by the user) for the presented indicator configuration items, and performs indicator construction processing based on the received indicator configuration data to obtain the constructed indicators.

[0147] In some embodiments, before performing indicator construction processing based on the received indicator configuration data to obtain the constructed indicator, the method further includes: matching the received indicator configuration data with historical indicator configuration data, and presenting the historical indicator configuration data that has been successfully matched; wherein, the historical indicator configuration data is the indicator configuration data of the constructed indicator.

[0148] Here, the indicator configuration data received during the indicator construction process can be matched with multiple historical indicator configuration data, and the successfully matched historical indicator configuration data can be displayed. The historical indicator configuration data refers to the indicator configuration data of already constructed indicators. The conditions for successful matching can also be set according to the actual application scenario. By displaying the successfully matched historical indicator configuration data, users can easily check whether the matched historical indicator configuration data meets their needs, thereby determining whether to stop the indicator construction process and effectively saving computing resources.

[0149] In some embodiments, after receiving metric configuration data for the presented metric configuration items, the method further includes: deleting the received metric configuration data in response to a stop build operation on the received metric configuration data.

[0150] During the metric construction process, in response to a stop construction operation on received metric configuration data, the received metric configuration data is deleted, i.e., metric construction is stopped. Here, corresponding stop construction options can be presented to facilitate receiving stop construction operations.

[0151] In some embodiments, metrics include dimensions, business processes, business constraints, atomic metrics, and derived metrics, and the metric configuration items corresponding to different types of metrics are different; wherein, atomic metrics are metric indicators for business processes; the metric configuration items corresponding to derived metrics include dimensions, business constraints, and atomic metrics.

[0152] In this embodiment of the application, the indicator system may include five types of indicators, namely, dimension, business process, business limitation, atomic indicator and derived indicator. Different types of indicators have different indicator configuration items. Of course, different types of indicators may also correspond to one or more of the same indicator configuration items.

[0153] For ease of explanation, the embodiments of this application provide, as follows: Figure 4 The diagram of the indicator system shown will be combined with Figure 4 Please provide an explanation.

[0154] 1) Business Line: Also known as a business module, it is a system-level conceptual object that can be divided according to actual business needs. For example, a business line may include e-commerce business, information flow business, and instant messaging business.

[0155] 2) Data domain: also known as subject domain, is the boundary of a subject determined after analyzing a specific topic. For example, a data domain may include a product domain, a transaction domain, and a membership domain.

[0156] 3) Dimension: The perspective from which business is analyzed, corresponding to the business objects in the business process. For example, dimensions may include users, products, and departments.

[0157] 4) Business Process: This refers to business activity events, which are usually indivisible events. For example, an e-commerce order can include multiple indivisible events such as placing the order, payment, shipping, and confirmation of receipt. Each of these events can be considered a business process.

[0158] 5) Business constraints: The scope of the statistics is used to filter out records that meet the business rules. Business constraints are similar to the conditions defined by the WHERE clause in Structured Query Language (SQL) (excluding time conditions).

[0159] 6) Atomic metrics: These define the measurement and statistical methods for business analysis, similar to the aggregate expressions qualified by the SELECT statement in SQL, such as the SUM expression. Atomic metrics are metric indicators of business processes and can be used to measure business processes.

[0160] 7) Statistical period: The time range for statistics, also known as the time period, such as the most recent 1 day or the most recent 30 days. The statistical period is similar to the time condition specified by the WHERE statement in SQL.

[0161] 8) Derived metrics: Metrics used to analyze business operations, which can be defined by dimensions, atomic metrics, statistical periods, and business constraints. For example, if the atomic metric is payment amount, then a derived metric could be the payment amount made by overseas buyers in the most recent day. Here, the most recent day is the statistical period, overseas is the business constraint, and buyers are the dimension.

[0162] For example, in a real-world application scenario, the business line is e-commerce, the data domain is the transaction domain, the dimension is the product category, the business process is order placement and purchase, the business is limited to dried fruits, the statistical period is the most recent day, the atomic metric is the total sales amount, and the derived metric is the total sales amount of dried fruit products in the most recent day.

[0163] For dimensions, business processes, business constraints, atomic metrics, and derived metrics, the same metric configuration items can correspond to each other, such as business lines and data domains. Different types of metrics can also have different metric configuration items. For example, derived metrics may have configuration items including dimensions, business constraints, and atomic metrics, meaning that derived metrics have configuration dependencies on dimensions, business constraints, and atomic metrics. For instance, after a derived metric is built based on a certain atomic metric, that derived metric becomes a dependent metric of that atomic metric. Through this approach, a complete and effective metric system can be established, enabling multi-level descriptions of the business and improving the rationality and relevance of the constructed metrics.

[0164] like Figure 3C As shown, this application provides an exemplary indicator construction process that enables the rapid and effective construction of indicators.

[0165] In some embodiments, see Figure 3D , Figure 3D This is a schematic flowchart of a data processing method provided in an embodiment of this application. Figure 3AStep 105 shown can be implemented through steps 401 to 404, which will be explained in conjunction with each step.

[0166] In step 401, the metadata of the data table is constructed based on the data constraints.

[0167] In step 402, the target type of the query request is determined; wherein the metadata includes multiple types of indexes, and the target type is at least one of multiple types.

[0168] Here, when the metadata includes multiple types of indexes, a query request can be received, and the target type of the query request can be determined, wherein the target type is at least one of multiple types. For example, the target type can be included in the query request; for example, a selection operation for multiple types can be received, and the type selected by the selection operation can be used as the target type of the query request; for example, the target type can also be predefined.

[0169] In step 403, the query data is matched with historical indexes that match the target type, and the successfully matched historical indexes are presented in response to the query request; wherein, the historical indexes are the indexes of the data tables that have been constructed.

[0170] Here, the query data in the query request is matched against multiple historical indexes that match the target type, and the successfully matched historical indexes are presented as a response to the query request. These historical indexes are indexes in the metadata of data tables that have been built historically. When presenting successfully matched historical indexes, partial or full data from the metadata of the corresponding data table can also be displayed.

[0171] It's worth noting that if the presented historical indexes with successful matches do not meet the user's needs, the user can also perform an update operation on the target type. When the electronic device receives an update operation for the target type, it can update the target type to obtain a new target type. For example, the original target type might correspond to a table name index, and the new target type might correspond to a table description index. Then, the query data can be matched against multiple historical indexes that match the new target type, and the historical indexes with successful matches can be presented.

[0172] In some embodiments, when presenting a successfully matched historical index, the method further includes: determining the index object of the successfully matched historical index; wherein the index object includes any one of a data table and a field; presenting the popularity data of the index object; wherein the popularity data includes at least one of call popularity and query popularity.

[0173] Here, the index objects for different types of indexes can be predefined. For example, the index objects for table name indexes and table description indexes are both data tables, while the index objects for field indexes are fields, such as one or more fields. When a successfully matched historical index is obtained, the index objects of the successfully matched historical indexes can also be determined, and the popularity data of the index objects can be presented. The successfully matched historical indexes and the popularity data of their index objects can be presented in a bound manner, such as being presented in the same row or column, thus demonstrating the relationship between the two at the visualization level. Furthermore, different types of index objects can be presented in different ways, including at least one of the following parameters: font, icon, and color. For example, when the index object is a data table, the first font can be used to present the successfully matched historical indexes and / or popularity data corresponding to that index object; when the index object is a field, the second font can be used to present the successfully matched historical indexes and / or popularity data corresponding to that index object, where the first font is larger than the second font.

[0174] Popularity data can include at least one of call popularity and query popularity. Call popularity refers to the frequency with which an index object is called in business operations, such as the number of times it is called. Query popularity refers to the frequency with which an index object is triggered after a query. For example, for a specific index object, the number of trigger operations received after the historical index corresponding to that index object was displayed can be counted, and this number of triggered operations can be used as the query popularity of that index object. The trigger operations for the displayed historical index can be used to view some or all of the metadata in the data table corresponding to that historical index. This approach can further improve query performance. For example, historical indexes of index objects with higher popularity are more likely to meet user needs. Thus, combining popularity data can help users quickly determine query results that meet their requirements.

[0175] like Figure 3D As shown, the embodiments of this application can perform targeted matching processing by determining the target type of the query request, thereby improving the targeting and accuracy of the query process.

[0176] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario. For ease of understanding, the data platform will be described as a big data platform. The data processing method provided by the embodiments of this application can realize a data warehouse management platform (corresponding to the data table management platform mentioned above). This data warehouse management platform can be used to build standardized data tables and provide accurate and efficient data table query capabilities.

[0177] As an example, embodiments of this application provide, for instance, the following: Figure 5The diagram shown illustrates the usage flow of the data warehouse management platform. Figure 5 In this context, the data warehouse management platform can provide access control functionality, which verifies the permissions of users accessing the platform (e.g., users entering the platform's interface). When a user has access rights, the platform determines that the verification is successful and provides data warehouse management functions; when a user does not have access rights, the platform determines that the verification has failed and displays an access failure message on the interface, prompting the user to request access.

[0178] The data warehouse management platform provides three main functions: data table query, indicator system, and data table construction, which will be explained later.

[0179] 1) Data table query.

[0180] The data table query function provides the ability to query data tables within the data warehouse. As an example, embodiments of this application provide, for instance... Figure 6 The diagram shown illustrates the data table query interface. Figure 6 In the data table query interface 61 shown, query data (i.e., keywords entered by the user) can be received through the query box 62. Here, the query data is exemplified by "transfer number". After receiving the query data, the data warehouse management platform matches the query data with the historical index of the target type. Here, the historical index of the target type is exemplified by the table name (or table name index). Then, the successfully matched table names are presented in the query results list. In addition to presenting the successfully matched table names, some or all of the metadata of the corresponding data table can also be presented. Furthermore, the matching conditions can be set according to the actual application scenario. For example, the similarity between the query data and each table name can be determined, and the table names with a similarity ranking in a set position (such as the top 5) can be used as successfully matched table names. Alternatively, table names with a similarity greater than a similarity threshold (such as 70%) can also be used as successfully matched table names. The number of successfully matched table names may be zero or at least one.

[0181] If the query results presented in the query results list do not match the user's query intent, the user can trigger... Figure 6Query option 63 allows for more granular and multi-dimensional queries. When the data warehouse management platform receives a trigger for query option 63, it matches the query data against various types of historical indexes (such as table name indexes, table description indexes, and field indexes), and displays the successfully matched historical indexes in the query results list. Alternatively, it can display partial or full data from the metadata of the corresponding tables for the successfully matched historical indexes. The matching criteria described above can also be applied. For example, for each type of historical index, the similarity between the query data and each historical index matching that type can be determined, and historical indexes ranking in a set similarity position can be considered successfully matched. Alternatively, historical indexes with similarity greater than a similarity threshold can also be considered successfully matched.

[0182] It's worth noting that when presenting successfully matched historical indexes in the query results list, the popularity data of the index objects of those successfully matched historical indexes can also be displayed simultaneously. The index objects can include any type of data table or field. Popularity data can include at least one of the following: call popularity and query popularity. Call popularity refers to the call popularity of upstream computing logic, i.e., the frequency with which the index object is invoked in the business logic; query popularity refers to the frequency with which the index object is triggered after a query.

[0183] Furthermore, successfully matched historical indexes can be displayed differently depending on the index object they belong to. For example, when the index object of a successfully matched historical index is a field, it can be displayed using the icon "F"; when the index object of a successfully matched historical index is a table, it can be displayed using the icon "T".

[0184] As an example, in Figure 6 The image shows two successfully matched field indexes. For the field index named "FieldName1", the shown tags "Full Author Accounts", "Author Attributes", "Daily Updates", and "Snapshot Table" are the tag description data for the corresponding data table. The shown tags "1=Repost Number; 2=Original Number; 3=Repost Number; 4=Character Design Number" are the field description data within this field index. The shown tag "t_dim_fcc_b_puin_acc_d" is the table name for the corresponding data table. Furthermore, in... Figure 6The document also shows detail option 64. When the data warehouse management platform receives a trigger operation for detail option 64, it can present detailed metadata of the data table corresponding to that field index (here referring to the field index whose field name is field name1). The detailed metadata is similar to the full metadata. The popularity (e.g., number of times) of the trigger operation for detail option 64 can be used as the query popularity of the index object corresponding to that field index (here referring to the field index whose field name is field name1).

[0185] 2) Indicator system.

[0186] In this embodiment of the application, the indicator system framework is subdivided into five types of indicators, starting with derived indicators, which are: dimensions (also known as dimension definitions), business processes, business constraints, atomic indicators, and derived indicators. These will be explained separately later.

[0187] ① Dimensions. For dimensions, the data warehouse management platform provides a dimension list interface and a new dimension interface. The dimension list interface is used to display the dimensions that have been built, and the new dimension interface is used to build new dimensions.

[0188] As an example, Figure 7 The interface for adding a new dimension is shown in 71, which includes multiple indicator configuration items for constructing the dimension, such as the indicator configuration item 72 for configuring the primary key of the dimension. In addition, the business line, data domain, data warehouse layer, dimension English (i.e., the English name of the dimension), dimension Chinese, dimension type (such as ordinary dimension, hierarchical dimension, enumerated dimension, and virtual dimension), and dimension description can be configured according to the indicator configuration items.

[0189] exist Figure 7 In this context, the default layering of the data warehouse is the dimension layer, i.e., the DIM layer. It is worth noting that this application does not limit the layering method of the data warehouse in the big data platform, such as... Figure 8 As shown, a data warehouse can be divided into an access layer, a warehouse layer, and an application layer. The warehouse layer can include an Operation Data Store (ODS) layer, a Data Integration Layer (DIL) layer, a Data Market Layer (DML) layer, a dimension layer, and a temporary table layer (also known as a TMP layer). The application layer can be a Data Application Layer (DAL).

[0190] After presenting the indicator configuration items, the data warehouse management platform can receive the indicator configuration data for those items, such as the indicator configuration data entered by the user. When the data warehouse management platform receives a trigger operation for the submit option in the "Add Dimension" interface 71, it can construct the dimension based on the indicator configuration data for the multiple indicator configuration items used to construct the dimension. Thus, it can provide query capabilities for the dimension based on the dimension's indicator configuration data (again, through matching processing). The data warehouse management platform can present the constructed dimension's indicator configuration data in the "Add Dimension" interface 71, such as in the list below the submit option, or it can be presented in the dimension list interface.

[0191] ② Business Processes. The data warehouse management platform also provides a business process list interface and a new business process interface. The business process list interface displays existing business processes, while the new business process interface is used to create new business processes.

[0192] As an example, Figure 9 The newly added business process interface 91 is shown, which includes multiple indicator configuration items for building the business process. These are used to configure the business line, data domain, data warehouse layer, English name, Chinese name, and description of the business process. Among them, the data warehouse layer of the business process can be set to the Data Warehouse Detail (DWD) layer by default.

[0193] When the data warehouse management platform receives a trigger operation for the submit option in the new business process interface 91, it can construct the business process based on the indicator configuration data for multiple indicator configuration items used to construct the business process. Similarly, it can provide query capabilities for the business process based on the indicator configuration data of the business process.

[0194] ③ Business constraints. Business constraints are modifiers used to configure derived metrics. They can be obtained by selecting a specific value from the dimension. For example, business constraints can be obtained from the DIM layer of the data warehouse. If the dimension is "user", the business constraint can be "male user" or "female user". Alternatively, business constraints can be obtained by calculating specific data in the data warehouse (such as data in the DWD layer) through set calculation logic. For example, the user's frequently used login locations can be extracted from the user's transaction logs and used as the business constraint.

[0195] For business constraints, the data warehouse management platform also provides a business constraint list interface and a new business constraint interface. The business constraint list interface is used to display the existing business constraints, while the new business constraint interface is used to create new business constraints.

[0196] As an example, Figure 10The newly added business constraint interface 101 is shown, which includes multiple indicator configuration items for building business constraints. These are used to configure the business line, data domain, data warehouse layer, source table (referring to the source table in the big data platform, and the source field in the following text is similar), source field, English name, Chinese name, description, and calculation logic of the business constraint. The source table of the business constraint can come from the DIM layer or the DWD layer.

[0197] When the data warehouse management platform receives a trigger operation for the submit option in the new business limitation interface 101, it can construct the business limitation based on the indicator configuration data for multiple indicator configuration items used to construct the business limitation. Similarly, it can provide query capabilities for the business limitation based on the indicator configuration data of the business limitation.

[0198] ④ Atomic metrics. For atomic metrics, the data warehouse management platform also provides an atomic metric list interface and a new atomic metric interface. The atomic metric list interface is used to display the atomic metrics that have been built, and the new atomic metric interface is used to build new atomic metrics.

[0199] As an example, Figure 11 The interface for the newly added atomic metric 111 is shown, which includes multiple metric configuration items for constructing atomic metrics. These items are used to configure the business line, data domain, data warehouse layer, source field, English name, Chinese name, description, data type, field, field format, calculation logic, and additivity of the atomic metric. Additivity refers to whether the atomic metric has an additive property. For example, an atomic metric used to count the number of people does not have additivity; that is, the same user will only be counted once.

[0200] When the data warehouse management platform receives a trigger operation for the submit option in the interface 111 for adding an atomic indicator, it can construct the atomic indicator based on the indicator configuration data of the multiple indicator configuration items used to construct the atomic indicator. Similarly, it can provide query capabilities for the atomic indicator based on the indicator configuration data of the atomic indicator.

[0201] ⑤ Derived metrics. For derived metrics, the data warehouse management platform also provides a derived metric list interface and a new derived metric interface. The derived metric list interface is used to display the derived metrics that have been built, and the new derived metric interface is used to build new derived metrics.

[0202] This application provides embodiments such as Figure 12 The newly added derived indicator interface 121 shown, and as shown in the example Figure 13 The newly added derived metric interface 131 is shown. The newly added derived metric interface 121 displays multiple metric configuration items, which are used to configure the business line, data domain, data warehouse layer, and atomic metrics of the derived metrics.

[0203] The newly added derived indicator interface 131 also shows several indicator configuration items, which are used to configure the dimensions (also known as statistical dimensions), statistical periods, and business limitations of the derived indicators. Among them, the statistical periods include the most recent day, the most recent three days, the most recent seven days, and the most recent fourteen days.

[0204] When the data warehouse management platform receives a trigger operation for the "Generate Derived Indicator" option in the new derived indicator interface 131, it can construct derived indicators based on the indicator configuration data received in the new derived indicator interface 121 and the new derived indicator interface 131. Similarly, it can provide query capabilities for derived indicators based on the indicator configuration data of the derived indicators.

[0205] like Figure 5 As shown, during the construction of derived metrics, an identity document (ID) can be configured for each derived metric, i.e., it can be coded or numbered to facilitate the differentiation of different derived metrics. Furthermore, the metric configuration data of all constructed derived metrics can be added to the metric dictionary so that users can view the metric configuration data for each derived metric.

[0206] It's worth noting that for each type of indicator in the indicator system, before building the indicator of that type based on the received indicator configuration data, the received indicator configuration data can be matched with historical indicator configuration data, and the successfully matched historical indicator configuration data will be presented. This historical indicator configuration data refers to the indicator configuration data of that type of indicator that has been built in the past. In this way, users can check whether the successfully matched historical indicator configuration data meets their needs and decide whether to continue building the indicator. In response to the stop-build operation for received indicator configuration data, the data warehouse management platform can stop building indicators based on the received indicator configuration data, for example, by deleting the received indicator configuration data, thereby reducing resource waste. In response to the continue-build operation for received indicator configuration data, the data warehouse management platform can build indicators normally based on the received indicator configuration data. In addition to presenting successfully matched historical indicator configuration data, when no successfully matched historical indicator configuration data exists, a "Successfully built indicator" message can be displayed; when successfully matched historical indicator configuration data exists, a "Successfully built indicator" message can be displayed. This also helps users decide whether to continue building indicators.

[0207] It's worth noting that the constructed metrics can have specific states, such as active and draft (corresponding to the inactive state mentioned above). When a constructed metric is in an active state, updating that metric can be prohibited; when a constructed metric is in a draft state, update options corresponding to that metric can be presented, and when a trigger operation is received for that update option, the metric will be updated. These update options include, but are not limited to, deletion and modification options.

[0208] The status of an indicator can be configured based on the received operations. For example, the data warehouse management platform can update the status of an indicator to active status based on the received active configuration operation, or update the status of an indicator to draft status based on the received draft configuration operation (i.e., inactive configuration operation).

[0209] The status of metrics can also be automatically configured. For example, a data warehouse management platform can identify dependent metrics that have configuration dependencies on the constructed metrics. When the dependent metric is active, the status of the constructed metric will also remain active. Specifically, if metric A is built on top of metric B, then metric A will be considered a dependent metric of metric B; that is, configuration dependencies can be downstream dependencies.

[0210] 3) Creating data tables.

[0211] In this embodiment, the components of a data table can be abstracted, for example, divided into statistical periods, dimensions, and atomic metrics. Therefore, the data warehouse management platform can present corresponding field configuration items to configure the statistical periods, dimensions, and atomic metrics in the data table, i.e., to configure the fields of the data table. Furthermore, multiple data table attribute configuration items can be provided for constructing the data table, respectively used to configure the data source (referring to the data source in the big data platform), database (referring to the database in the big data platform), physical type, data warehouse layering, business line, data domain, custom attributes, refresh cycle, storage strategy, and storage cycle, such as... Figure 14 The data table creation interface is shown in Figure 141. When the data warehouse management platform receives data table attribute configuration data for data table attribute configuration items, it can construct the table name based on the data table attribute configuration data. For example, the table name can include physical type, data warehouse layer, business line, data domain, business description, period (such as refresh period or storage period), and update method, such as "t_dim_fcc_b_puin_acc_d" above. The business description and update method are not specified in the table name. Figure 14 As shown in the image.

[0212] After constructing a table name, the data warehouse management platform can match this table name with multiple historical table names and display the successfully matched historical table names. This allows users to determine whether a data table matching their needs already exists and whether to continue constructing the data table. Historical table names refer to the names of data tables that have been constructed in the past.

[0213] Furthermore, the data warehouse management platform can present data table description configuration items and receive table description data (corresponding to the data table description configuration data mentioned above) for these configuration items. These configuration items can include Comment and Tag description items, corresponding to Comment and Tag description data, respectively. Comment description data is used to describe the purpose and function of the data table in the form of a title, while Tag description data is used to describe the data table in a more concise way. Comment and Tag description data can combine to form table description data, essentially abstracting the data table into an article describing facts, thus forming an information summary system for the data table and improving the query capabilities. Simultaneously, the data warehouse management platform can determine the corresponding field data (also called Col data) based on the configured fields in the data table. For example, field data can include the field name, field type, and field description data.

[0214] Then, based on the table name, table description data, and field data, table name indexes, table description indexes, and field indexes can be constructed, i.e., multiple types of indexes can be built. Further, the obtained table name indexes, table description indexes, and field indexes can be used to compose table index data, also known as schema data. Metadata for the table can be constructed based on the schema data. This application embodiment does not limit the form of metadata; that is, metadata may include other data besides schema data.

[0215] It is worth noting that, such as Figure 5 As shown, after constructing the metadata of a data table, the data warehouse management platform can match this metadata with historical metadata and present the successfully matched historical metadata, which is the step of executing the verification logic. Historical metadata refers to the metadata of other constructed data tables besides the current data table. This allows users to easily determine whether the data table corresponding to the successfully matched historical metadata meets their needs, thereby deciding whether to proceed with subsequent data acquisition processing through the data warehouse management platform.

[0216] When constructing a data table, the data warehouse management platform can acquire and process data from the big data platform based on multiple metrics (such as derived metrics) defined in the data table, and add the acquired data to the data table to produce the data table. Then, the data warehouse management platform can store the produced data table within the big data platform, such as in the data warehouse maintained by the big data platform.

[0217] At the underlying implementation level, the embodiments of this application provide, as follows: Figure 15 The diagram shown illustrates the architecture of the data warehouse management platform. Figure 15 In a data warehouse management platform, four main components can be identified: data warehouse specifications (corresponding to the data constraints mentioned above), an indicator system, interaction with the big data platform, and query capabilities. The data warehouse specifications can include indicator specifications (corresponding to the indicator constraints mentioned above, involving indicator configuration items used to build indicators), data table specifications (corresponding to the data table constraints mentioned above, involving data table configuration items and field configuration items used to build data tables), data warehouse layering specifications, and code writing standards. The indicator system involves the construction of dimensions, business processes, business constraints, atomic indicators, and derived indicators. The interaction with the big data platform involves the selection (or configuration) of atomic indicators, statistical periods, and dimensions required for data tables, i.e., configuring the fields in the data tables; it also involves the generation of Comment description data, Tag description data, and Col data; and it involves acquiring and processing data from the big data platform and adding the acquired data to the data tables to produce the data tables. The query capabilities section allows the construction of schema data based on Comment description data, Tag description data, and Col data, thereby constructing the metadata of the data tables (metadata may also include table names), and providing query capabilities for the data tables based on the metadata. It is worth noting that, based on the existing data tables, it is possible to add, delete, query, and modify data tables, depending on the actual business scenario.

[0218] As an example, embodiments of this application also provide, for example, Figure 16 The diagram shown illustrates the architecture of the data warehouse management platform. Figure 16The diagram illustrates client 1, client 2, client 3, server, indicator service, storage service, and query service. The data warehouse management platform provided in this embodiment does not require specifying a particular big data platform. The data warehouse management platform can establish a communication connection with the big data platform through a representational state transfer style application programming interface (API), that is, through a general RESTful API, to perform data acquisition and processing, and to add, delete, query, and modify data tables.

[0219] Here, the types of metric services, storage services, and query services are not limited. For example, the metric service can be provided by a MySQL service, the storage service by a Metastore service, and the query service by an Elasticsearch service. The metric service stores the metric configuration data for the built metrics; the storage service stores the metadata of the built data tables and synchronizes this metadata to the query service. The storage service also responds to metadata update operations by updating the stored metadata, ensuring that the metadata stored in the storage service is up-to-date and accurate; the query service responds to query requests based on the stored metadata. It is worth noting that the metric service, storage service, and query service can be provided by a data warehouse management platform or a big data platform.

[0220] The following will be explained step by step. Figure 16 The data warehouse management platform shown provides the following functions.

[0221] 1) Client (referring to...) Figure 16 Any one of the clients shown (1 to 3) can present a data table query interface and, upon receiving query data input by the user, initiate a query request including the query data to the server. Upon receiving the query request, the server matches the query data in the query request with the table name indexes in the query service and returns the successfully matched table name indexes to the client for presentation in the data table query interface. The user can choose to perform a trigger operation on one of the presented successfully matched table name indexes, or perform further query operations (such as triggering an operation on the query options presented in the data table query interface).

[0222] 2) When the client receives a trigger operation for any successfully matching table name index, it can notify the server. The server then requests the metadata (i.e., the latest metadata) of the table corresponding to the triggered table name index from the storage service. It compares the retrieved metadata with the metadata of the same table stored in the query service. If they are inconsistent, the server updates the metadata of the table in the query service based on the metadata in the storage service. Simultaneously, the server can send the metadata of the table retrieved from the storage service to the client for display in the table query interface.

[0223] 3) When the client receives a further query request, it can notify the server. The server then matches the query data with the table description index and field index in the query service, and returns the successfully matched table description index and field index to the client for display in the data table query interface.

[0224] When a client receives a trigger action targeting any index (referring to various types of indexes), it can notify the server. The server then requests the metadata (i.e., the latest metadata) of the table corresponding to the triggered index from the storage service. It compares the retrieved metadata with the metadata of the same table stored in the query service. If they are inconsistent, the server updates the metadata of the table in the query service based on the metadata in the storage service. Simultaneously, the server can send the metadata of the table retrieved from the storage service to the client for display in the table query interface.

[0225] 4) When the client receives a metric build operation, it can present the metric configuration items used to build the metric. The client then sends the received metric configuration data to the server. The server matches the received metric configuration data with historical metric configuration data in the metric service and sends the successfully matched historical metric configuration data back to the client. Historical metric configuration data refers to the metric configuration data already stored in the metric service; the different names are only for easy identification. Furthermore, the server can also send a prompt to the client indicating whether the metric can be successfully built based on the existence of successfully matched historical metric configuration data, helping the user decide whether to build the metric.

[0226] 5) When the client receives a table construction operation, it can present the table attribute configuration items used to construct the table. Then, the client can send the table name to the server. The server matches this table name with the table name indexes in the query service and sends the successfully matched table name indexes (such as the top 10 similarity indexes) to the client. Simultaneously, the server can also send some or all of the metadata from the corresponding tables to the client for presentation. In this way, the user can decide whether to stop building the table based on their needs.

[0227] When the client receives a request to continue building, it can present data table description configuration items and field configuration items. The client can send the received field configuration data to the server, allowing the server to perform data acquisition and processing on the big data platform based on the field configuration data (such as the configured statistical period, dimensions, and atomic metrics). The server can also request the storage service to build the data table. After the storage service completes the data table construction and obtains the table's metadata, it can synchronously store the metadata in the query service, enabling the query service to provide query capabilities for the data table.

[0228] Through the embodiments of this application, at least the following technical effects can be achieved: 1) By materializing the data warehouse specifications through the data warehouse management platform, the constraint effect on the data development process can be improved, avoiding poor development quality caused by frequent changes in developers, and also avoiding siloed development caused by inconsistent understanding of the specifications by developers; 2) By materializing the data warehouse specifications through the data warehouse management platform, the data warehouse specifications can be updated at any time, adapting to the rapid iteration and frequent changes of Internet businesses, and developers can also adapt to business changes more quickly; 3) Powerful and effective query capabilities can be provided based on various types of indexes in the metadata, helping users quickly find data tables or fields that meet their needs, thereby improving query efficiency; 4) There is no strong binding relationship between the data warehouse management platform and the big data platform. The communication connection between the two can be established through a general RESTful API. The embodiments of this application do not require explicit binding of the type, version, etc. of the big data platform used, which can improve the universality and applicability to different big data platforms.

[0229] The following continues to describe an exemplary structure of the data processing apparatus 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2As shown, the software modules stored in the data processing device 455 of the memory 450 may include: a condition acquisition module 4551, used to acquire data constraints for an independent data platform; wherein the data constraints include indicator constraints and data table constraints; an indicator constraint module 4552, used to constrain the indicator construction process according to the indicator constraints to obtain the constructed indicators; a data table constraint module 4553, used to constrain the data table construction process according to the data table constraints to obtain the constructed data table including multiple indicators; a data acquisition module 4554, used to acquire data from the data platform according to the multiple indicators in the data table, and add the data acquired from the data platform to the data table; and a query module 4555, used to construct the metadata of the data table according to the data constraints, and respond to query requests for the data table according to the metadata.

[0230] In some embodiments, data table constraints include data table configuration items and field configuration items for constructing the data table; the data table constraint module 4553 is further configured to: in response to a data table construction operation, present the data table configuration items and field configuration items; receive data table configuration data for the presented data table configuration items; receive field configuration data for the presented field configuration items, and filter out indicators that conform to the field configuration data from a plurality of constructed indicators; perform data table construction processing based on the data table configuration data, the field configuration data, and the filtered indicators to obtain the constructed data table.

[0231] In some embodiments, the data table configuration items include data table attribute configuration items and data table description configuration items; the data table configuration data includes data table attribute configuration data for the data table attribute configuration items and data table description configuration data for the data table description configuration items; the query module 4555 is further configured to: construct a table name index based on the data table attribute configuration data; construct a table description index based on the data table description configuration data; construct a field index based on the field configuration data; and construct the metadata of the data table based on the table name index, the table description index, and the field index.

[0232] In some embodiments, the indicator constraints include indicator configuration items for constructing the indicator; the indicator constraint module 4552 is further configured to: in response to the indicator construction operation, present the indicator configuration items for constructing the indicator; receive indicator configuration data for the presented indicator configuration items, and perform indicator construction processing based on the received indicator configuration data to obtain the constructed indicator.

[0233] In some embodiments, the indicator constraint module 4552 is further configured to: perform matching processing on the received indicator configuration data and historical indicator configuration data, and present the historical indicator configuration data that has been successfully matched; wherein, the historical indicator configuration data is the indicator configuration data of the constructed indicators; and delete the received indicator configuration data in response to the stop construction operation on the received indicator configuration data.

[0234] In some embodiments, metrics include dimensions, business processes, business constraints, atomic metrics, and derived metrics, and the metric configuration items corresponding to different types of metrics are different; wherein, atomic metrics are metric indicators for business processes; the metric configuration items corresponding to derived metrics include dimensions, business constraints, and atomic metrics.

[0235] In some embodiments, the metadata includes multiple types of indexes; the query module 4555 is further configured to: determine the target type of the query request; wherein the target type is at least one of multiple types; match the query data with historical indexes that match the target type, and present the historical indexes that match successfully in response to the query request; wherein the historical indexes are indexes of the constructed data tables.

[0236] In some embodiments, the query module 4555 is further configured to: when presenting a historical index that has successfully matched, determine the index object of the historical index that has successfully matched; wherein the index object includes any one of a data table and a field; present the popularity data of the index object; wherein the popularity data includes at least one of call popularity and query popularity.

[0237] In some embodiments, the query module 4555 is further configured to: store the metadata of the data table in the storage service; synchronize the metadata in the storage service to the query service; wherein the query service is configured to respond to query requests based on the stored metadata; perform metadata update processing on the storage service in response to an update operation on the metadata; and perform metadata update processing on the query service based on the new metadata obtained from the metadata update processing in the storage service when the query service meets the metadata update conditions.

[0238] In some embodiments, the metadata includes multiple types of indexes; the query module 4555 is further configured to: match the indexes of the data table that conform to a set type with historical indexes that conform to the set type, and present the historical indexes that have successfully matched; wherein, the set type is at least one of multiple types; the historical index is an index of the data table that has been constructed and is different from the data table.

[0239] In some embodiments, the data table constraint module 4553 is further configured to: delete any data table in response to a stop construction operation on any constructed data table.

[0240] In some embodiments, the indicator constraint module 4552 is further configured to: prohibit updating the indicator when the indicator is in an active state; and present the update option corresponding to the indicator when the indicator is in an inactive state, and update the indicator when a trigger operation for the update option is received.

[0241] In some embodiments, the indicator constraint module 4552 is further configured to: perform any of the following processes: update the state of the indicator to an effective state in response to an effective configuration operation for the indicator; update the state of the indicator to an effective state when a dependent indicator that has a configuration dependency relationship with the indicator is in an effective state.

[0242] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing method described in this application.

[0243] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 3A , Figure 3B , Figure 3C and Figure 3D The data processing method is shown.

[0244] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0245] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0246] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0247] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0248] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method, characterized by, The method includes: Obtain data constraints for the data platform; wherein, the data constraints include indicator constraints and data table constraints, the data table constraints include data table configuration items and field configuration items for constructing the data table, and the data table configuration items include data table attribute configuration items and data table description configuration items; The indicator construction process is constrained according to the aforementioned indicator constraints to obtain the constructed indicators. Receive data table attribute configuration data for the data table attribute configuration item, data table description configuration data for the data table description configuration item, and field configuration data for the field configuration item; The data table is constructed based on the data table configuration data, the field configuration data, and the selected indicators to obtain the constructed data table; wherein, the data table configuration data includes the data table attribute configuration data and the data table description configuration data, and the selected indicators refer to the indicators that conform to the field configuration data among the multiple constructed indicators; The data platform is processed to acquire data based on multiple indicators in the data table, and the data acquired from the data platform is added to the data table. Build a table name index based on the data table attribute configuration data; build a table description index based on the data table description configuration data; build a field index based on the field configuration data. The table index data is composed of the table name index, the table description index, and the field index; the metadata of the data table is constructed based on the table index data. The system responds to query requests for the data table based on the metadata.

2. The method of claim 1, wherein, The indicator constraints include indicator configuration items used to construct the indicators; The process of constraining the indicator construction process according to the indicator constraints to obtain the constructed indicators includes: In response to the indicator building operation, the indicator configuration items used to build the indicator are presented; Receive indicator configuration data for the presented indicator configuration items, and perform indicator construction processing based on the received indicator configuration data to obtain the constructed indicators.

3. The method according to claim 2, characterized in that, After receiving the indicator configuration data for the presented indicator configuration item, the method further includes: The received indicator configuration data is matched with historical indicator configuration data, and the successfully matched historical indicator configuration data is presented. The historical indicator configuration data refers to the indicator configuration data of the constructed indicators. In response to the stop build operation for the received indicator configuration data, the received indicator configuration data is deleted.

4. The method according to claim 2, characterized in that, The metrics include dimensions, business processes, business constraints, atomic metrics, and derived metrics, and the metric configuration items corresponding to different types of metrics are different; Wherein, the atomic metric is a metric for the business process; the metric configuration items corresponding to the derived metric include the dimension, the business limitation, and the atomic metric.

5. The method according to any one of claims 1 to 4, characterized in that, The metadata includes various types of indexes; responding to a query request for the data table based on the metadata includes: Determine the target type of the query request; wherein the target type is at least one of the multiple types; The query data in the query request is matched with historical indexes that match the target type, and the successfully matched historical indexes are presented in response to the query request. The historical index is an index of the data table that has already been constructed.

6. The method according to claim 5, characterized in that, When presenting the historical index that has successfully matched, the method further includes: Identify the index object of the historical index that has successfully matched; wherein, the index object includes any one of the data table and fields; Present the popularity data of the indexed object; wherein the popularity data includes at least one of call popularity and query popularity.

7. The method according to any one of claims 1 to 4, characterized in that, After constructing the metadata of the data table based on the data table index data, the method further includes: Store the metadata of the data table in the storage service. The metadata in the storage service is synchronized and stored to the query service; wherein the query service is used to respond to the query request based on the stored metadata; In response to the update operation on the metadata, the storage service performs metadata update processing; When the query service meets the metadata update conditions, the query service is updated according to the new metadata obtained from the metadata update process in the storage service.

8. The method according to any one of claims 1 to 4, characterized in that, The metadata includes various types of indexes; after constructing the metadata of the data table based on the data table index data, the method further includes: The indexes of the data table that conform to the set type are matched with the historical indexes that conform to the set type, and the historical indexes that are successfully matched are presented. Wherein, the set type is at least one of the multiple types; the historical index is an index of an already constructed data table that is distinct from the data table; In response to a stop build operation on any of the constructed data tables, delete the aforementioned data table.

9. The method according to any one of claims 1 to 4, characterized in that, After constraining the indicator construction process according to the indicator constraints to obtain the constructed indicators, the method further includes: When the indicator is active, updating the indicator is prohibited. When the indicator is inactive, the update option corresponding to the indicator is presented, and when a trigger operation for the update option is received, the indicator is updated.

10. The method according to any one of claims 1 to 4, characterized in that, After constraining the indicator construction process according to the indicator constraints to obtain the constructed indicators, the method further includes: Perform any of the following processes: In response to the configuration operation that applies to the metric, update the status of the metric to an active status; When a dependent metric that has a configuration dependency relationship with the metric is in the active state, the state of the metric is updated to the active state.

11. A data processing apparatus, characterized in that, The device includes: The condition acquisition module is used to acquire data constraints for an independent data platform; wherein, the data constraints include indicator constraints and data table constraints, the data table constraints include data table configuration items and field configuration items for constructing the data table, and the data table configuration items include data table attribute configuration items and data table description configuration items; The indicator constraint module is used to constrain the indicator construction process according to the indicator constraint conditions to obtain the constructed indicators. The data table constraint module is used to receive data table attribute configuration data for the data table attribute configuration items, data table description configuration data for the data table description configuration items, and field configuration data for the field configuration items; and to perform data table construction processing based on the data table configuration data, the field configuration data, and the selected indicators to obtain the constructed data table; wherein, the data table configuration data includes the data table attribute configuration data and the data table description configuration data, and the selected indicators refer to the indicators that conform to the field configuration data selected from the multiple constructed indicators; The data acquisition module is used to acquire data from the data platform based on multiple indicators in the data table, and add the data acquired from the data platform to the data table. The query module is used to construct a table name index based on the data table attribute configuration data; construct a table description index based on the data table description configuration data; construct a field index based on the field configuration data; compose data table index data based on the table name index, the table description index, and the field index; construct metadata of the data table based on the data table index data; and respond to query requests for the data table based on the metadata.

12. The apparatus according to claim 11, characterized in that, The indicator constraints include indicator configuration items used to construct the indicators; The indicator constraint module is also used for: In response to the indicator building operation, the indicator configuration items used to build the indicator are presented; Receive indicator configuration data for the presented indicator configuration items, and perform indicator construction processing based on the received indicator configuration data to obtain the constructed indicators.

13. The apparatus according to claim 12, characterized in that, The indicator constraint module is also used for: The received indicator configuration data is matched with historical indicator configuration data, and the successfully matched historical indicator configuration data is presented. The historical indicator configuration data refers to the indicator configuration data of the constructed indicators. In response to the stop build operation for the received indicator configuration data, the received indicator configuration data is deleted.

14. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the data processing method according to any one of claims 1 to 10.

15. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing the data processing method according to any one of claims 1 to 10 when executed by a processor.

16. A computer program product, characterized in that, The computer program product includes computer instructions for implementing the data processing method according to any one of claims 1 to 10 when processed and executed.

Images