Data management method, apparatus, device, and storage medium

Abstract

A data management method, device, equipment and computer readable storage medium are provided. The method comprises: in response to receiving a data definition request, mapping data definition metadata based on data definition information in the data definition request; in response to receiving an operation definition request, mapping operation definition metadata based on operation definition information in the operation definition request, wherein the operation definition metadata contains identification information of the data definition metadata; and generating an operation processing result based on the operation definition metadata.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of big data, and in particular, to a data management method, a data management apparatus, an electronic device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] Big data is a new stage of information development. With the convergence of information technology and human production and life, the rapid popularization of the Internet, and the explosive growth and massive aggregation of global data, big data has a significant impact on economic development, social governance, national management, and people's lives. The amount of data collected in different fields has reached an unprecedented level. At the same time, the way of data generation, storage, and processing has undergone revolutionary changes. People's work and life can basically be represented digitally, and data usage and queries are very frequent.

[0003] With the continuous development of time, the quantity and variety of data are increasing, and the amount of stored data is also increasing. Various data are fused together, which makes the current data storage and query method unable to meet the needs of users. SUMMARY

[0004] The present disclosure provides a data management method, a data management apparatus, an electronic device, a computer readable storage medium, and a computer program product.

[0005] According to an aspect of the present disclosure, a data management method is provided. The method includes: in response to receiving a data definition request, mapping data definition metadata based on data definition information in the data definition request; in response to receiving an operation definition request, mapping operation definition metadata based on operation definition information in the operation definition request, wherein the operation definition metadata contains identification information of the data definition metadata; and generating an operation processing result based on the operation definition metadata.

[0006] According to another aspect of the present disclosure, a data management apparatus is provided. The apparatus includes: a data definition module configured to, in response to receiving a data definition request, map data definition metadata based on data definition information in the data definition request; an operation definition module configured to, in response to receiving an operation definition request, map operation definition metadata based on operation definition information in the operation definition request, wherein the operation definition metadata contains identification information of the data definition metadata; and an operation processing module configured to generate an operation processing result based on the operation definition metadata.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any of the above aspects.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method described in any of the above aspects.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in any of the foregoing aspects.

[0010] According to one or more embodiments of this disclosure, users' personalized data management needs can be efficiently met, thereby improving data management efficiency.

[0011] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0012] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0013] Figure 1 This is a schematic diagram illustrating an example system in which various methods described herein may be implemented according to exemplary embodiments;

[0014] Figure 2 This is a flowchart illustrating a data management method according to an exemplary embodiment;

[0015] Figure 3 This is a flowchart illustrating a data management method according to another exemplary embodiment;

[0016] Figure 4 This is a schematic block diagram illustrating a data management apparatus according to an exemplary embodiment;

[0017] Figure 5 This is a block diagram illustrating an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0018] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0019] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0020] With the continuous development of big data, its statistical analysis applications are becoming increasingly widespread across various industries. For example, demonstrating marketing effectiveness or improving processes requires data-driven verification. However, traditional big data statistical analysis platforms often employ a generic model for data collection and storage, calculating and displaying predefined indicators according to fixed logic for a specific industry. While these platforms may initially meet user needs, as users gain a deeper understanding, they develop more specific and detailed requirements tailored to their business needs. This necessitates customized development of the big data statistical analysis platform. However, existing customized development methods not only increase communication costs but are also time-consuming and labor-intensive, and may not even meet the user's specific requirements.

[0021] To alleviate, mitigate, or eliminate at least one of the aforementioned problems, this disclosure provides a data management method. This method supports user-defined data models, efficiently meeting users' personalized data management needs, thereby improving data management efficiency.

[0022] Before introducing exemplary embodiments of this disclosure, several terms used herein will first be explained.

[0023] 1. Dataset

[0024] A dataset is a collection of data, which can be in any form, such as collected data, data files, database tables, etc. A dataset can include information such as its name, identifier, field types, and composition rules.

[0025] 2. Physical storage information

[0026] Physical storage information is information about the physical storage of data, which may include storage type, access address, partitioning method, and authentication information.

[0027] 3. OLAP model

[0028] OLAP (Online Analysis Processing) models are statistical analysis models for data, which can include information such as dimensions, metrics, and hierarchies / levels.

[0029] Exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0030] Figure 1 This is a schematic diagram illustrating an example system 100 in which various methods described herein may be implemented according to exemplary embodiments.

[0031] refer to Figure 1 The system 100 includes a client device 110, a server 120, and a network 130 that communicatively couples the client device 110 and the server 120.

[0032] Client device 110 includes a display 114 and a client application (APP) 112 that can be displayed on the display 114. Client application 112 can be an application that needs to be downloaded and installed before running, or a lightweight application (liteapp). If client application 112 is an application that needs to be downloaded and installed before running, client application 112 can be pre-installed on client device 110 and activated. If client application 112 is a mini-app, user 102 can directly run client application 112 on client device 110 without installing it, by searching for client application 112 in the host application (e.g., by the name of client application 112) or scanning the graphic code of client application 112 (e.g., barcode, QR code, etc.). In some embodiments, client device 110 can be any type of mobile computing device, including mobile computers, mobile phones, wearable computing devices (e.g., smartwatches, head-mounted devices including smart glasses, etc.), or other types of mobile devices. In some embodiments, the client device 110 may alternatively be a fixed computer device, such as a desktop computer, server computer, or other type of fixed computer device.

[0033] Server 120 is typically a server deployed by an Internet Service Provider (ISP) or Internet Content Provider (ICP). Server 120 can represent a single server, a cluster of multiple servers, a distributed system, or a cloud server providing basic cloud services (such as cloud databases, cloud computing, cloud storage, and cloud communications). It will be understood that, although... Figure 1 The diagram shows that server 120 communicates with only one client device 110, but server 120 can provide background services to multiple client devices simultaneously.

[0034] Examples of network 130 include combinations of local area networks (LANs), wide area networks (WANs), personal area networks (PANs), and / or communication networks such as the Internet. Network 130 can be wired or wireless. In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to process data exchanged through network 130. Furthermore, encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some of the links. In some embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0035] For the purposes of this disclosure's embodiments, Figure 1 In the example, client application 112 can be a data management application that provides various data management-based functions, such as data collection, data visualization, etc. Correspondingly, server 120 can be a server used in conjunction with the data management application. Server 120 can provide online data services, such as online data collection and online data visualization, to client application 112 running on client device 110. Alternatively, server 120 can also provide management data to client device 110, and client application 112 running on client device 110 can provide data management services based on this management data.

[0036] Figure 2 This is a flowchart illustrating a data management method 200 according to an exemplary embodiment. Method 200 can be implemented on a client device (e.g., ...). Figure 1 The execution is performed at the client device 110 shown, that is, the execution entity of each step of method 200 can be... Figure 1 The client device 110 shown. In some embodiments, method 200 can be performed on a server (e.g., Figure 1The method 200 is executed at server 120 (as shown in the diagram). In some embodiments, method 200 may be executed in combination by a client device (e.g., client device 110) and a server (e.g., server 120). Hereinafter, the steps of method 200 will be described in detail with the client device 110 as the executing entity.

[0037] In step S201: In response to receiving a data definition request, the data definition metadata is mapped based on the data definition information in the data definition request.

[0038] According to some embodiments, a data definition request can be information corresponding to the business planning data sent by a user to the client device 110 based on their own business planning data. The data definition request includes data definition information. In the example, when the user's business planning data is streaming data, the data definition information includes the field names and data types of the data; when the user's business planning data is batch data, the data definition information includes the column names of the database table (or data file) and the type of the database table (or data file). Data definition metadata may include the name of the data definition metadata, the identification information of the data definition metadata, and the field types of the data definition metadata.

[0039] According to some embodiments, data definition information may include data logic information and data storage information, and data definition metadata may include data logic metadata and data storage metadata. In response to a received data definition request, client device 110 maps the data definition metadata based on the data definition information in the data definition request. Here, mapping can refer to mapping the data definition information in the data definition request to the data definition metadata. In the example, the mapping process may include mapping data storage information to data storage metadata and mapping data logic information to data logic metadata.

[0040] In step S202: In response to receiving an operation definition request, the operation definition metadata is mapped based on the operation definition information in the operation definition request, wherein the operation definition metadata includes the identification information of the data definition metadata.

[0041] According to some embodiments, an operation definition request can be information sent by a user to a client device 110 based on their own business planning data, corresponding to a specific operation process corresponding to the business planning data. The operation definition request includes operation definition information. In response to the received operation definition request, the client device 110 maps the operation definition metadata based on the operation definition information included in the operation definition request. Here, mapping can refer to mapping the operation definition information in the operation definition request to the operation definition metadata. This allows the operation definition metadata to be associated with the data definition metadata, thereby associating the operation definition information with the data definition information to facilitate the generation of operation processing results.

[0042] According to some embodiments, the operation definition information includes process information and processing logic information, and the operation definition metadata includes process metadata and processing logic metadata. In the example, the process metadata can be directed acyclic graph (DAG) metadata, and the processing logic metadata can be operator definition metadata. In response to a received operation definition request, the client device 110 can map the process information in the operation definition information to the DAG metadata, and simultaneously map the processing logic information to the operator metadata.

[0043] In step S203: Based on the operation definition metadata, generate the operation processing result.

[0044] Since the operation definition metadata contains the identification information of the data definition metadata, through this step, the client device 110 can generate the operation processing result based on the operation definition metadata.

[0045] In the data management method 200 according to the embodiments of the present disclosure, data definition information is mapped to data definition metadata, and operation definition information is mapped to operation definition metadata. At the same time, by including the identification information of data definition metadata in the operation definition metadata, the operation metadata is associated with the data definition metadata. This allows users to customize data models, thereby efficiently meeting users' personalized needs for data management and improving data management efficiency.

[0046] According to some embodiments, data definition information includes data logic information and data storage information, and data definition metadata includes data logic metadata and data storage metadata. Mapping the data definition metadata based on the data definition information in the data definition request includes:

[0047] Based on data storage information, map the data storage metadata; and

[0048] Based on data logic information, data logic metadata is mapped. The data logic metadata includes the identification information of data storage metadata, and the operation definition metadata includes the identification information of data logic metadata.

[0049] According to some embodiments, data definition information includes data logic information and data storage information, and data definition metadata includes data logic metadata (Dataset) and data storage metadata (Storage / DataSource). In response to a received data definition request, client device 110 maps the data definition metadata based on the data definition information in the data definition request. Here, mapping can refer to mapping data storage information to data storage metadata (Storage / DataSource) and mapping data logic information to data logic metadata (Dataset).

[0050] According to some embodiments, the data logic metadata (Dataset) contains identification information of the data storage metadata (Storage / DataSource), and the operation definition metadata contains identification information of the data logic metadata (Dataset). The data logic metadata (Dataset), data storage metadata (Storage / DataSource), and operation definition metadata can call each other based on the identification information, simplifying user-defined data models.

[0051] According to some embodiments, data storage information includes data source information and physical storage information, and data storage metadata includes data source metadata and physical storage metadata. Mapping the data storage metadata based on the data storage information includes:

[0052] Mapping physical storage metadata based on physical storage information; and

[0053] Based on the data source information, the data source metadata is mapped. The data source metadata includes the identification information of the physical storage metadata, and the data logic metadata contains the identification information of the data source metadata.

[0054] According to some embodiments, data storage information includes data source information and physical storage information; data storage metadata includes data source metadata (DataSource) and physical storage metadata (Storage). Among them, data source metadata (DataSource) includes the division of data storage rules, such as whether the partition information is divided according to specific values ​​or according to the range of values, format attributes, and storage file format, etc.; physical storage metadata (Storage) is the specific physical implementation method of physical storage, data access permissions, and data storage path, etc.

[0055] According to some embodiments, in response to a received data definition request, the client device 110 maps physical storage metadata (Storage) based on the physical storage information in the data definition request (here, mapping may refer to mapping physical storage information to physical storage metadata (Storage)), and maps data source metadata (DataSource) based on the data source information in the data definition request (here, mapping may refer to mapping data source information to data source metadata (DataSource)).

[0056] In some embodiments, the data logical metadata (Dataset) contains identification information for the data source metadata (DataSource) and physical storage metadata (Storage). In an example, the data logical metadata (Dataset) includes the attribute "DataSourceRef," where the value of "DataSourceRef" is a unique identifier for the data source metadata (DataSource). The system can retrieve the data source metadata (DataSource) using the value of "DataSourceRef." Similarly, the data source metadata (DataSource) includes the attribute "StorageRef," where the value of "StorageRef" is a unique identifier for the physical storage metadata (Storage). The system uses this to retrieve the specific physical storage metadata (Storage).

[0057] According to some embodiments, operation definition information includes process information and processing logic information, and operation definition metadata includes process metadata and processing logic metadata. In response to receiving an operation definition request, mapping the operation definition metadata based on the operation definition information in the operation definition request includes:

[0058] In response to receiving an operation definition request, the process metadata is mapped based on the process information; and

[0059] Based on the processing logic information, the processing logic metadata is mapped. The process metadata includes the identification information of the processing logic metadata and the identification information of the data definition metadata.

[0060] According to some embodiments, the data processing process can be a directed acyclic graph (DAG), where each data processing step corresponds to a node in the DAG, and the nodes are interdependent. A DAG can contain a set of nodes, a set of inputs, and a set of outputs, where the inputs and outputs are the identifiers of the logical metadata (DataSet) in the aforementioned embodiments.

[0061] According to some embodiments, a node includes inputs, outputs, and processing logic, where the processing logic is an operator definition or a data processing task definition. Furthermore, a node may also include dependencies, indicating the names of other nodes that the current node depends on. An operator definition includes information such as the input parameters, context, language, platform, and category of the data processing step.

[0062] According to some embodiments, operation definition information includes process information and processing logic information, and operation definition metadata includes process metadata (DAG) and processing logic metadata. The logic metadata can be operator definitions or data processing task definitions, which can be selected according to the specific use case. Operator definitions are generally used for data processing steps, while data processing task definitions are generally used for scheduling system orchestration.

[0063] According to some embodiments, in response to a received operation definition request, the client device 110 maps the process metadata (DAG) based on process information (here, mapping may refer to mapping process information to process metadata (DAG)); and maps the processing logic metadata (Operator or Task) based on processing logic information (here, mapping may refer to mapping processing logic information to processing logic metadata (Operator or Task)).

[0064] According to some embodiments, the process metadata (DAG) includes identification information for processing logic metadata (Operator or Task) and identification information for data definition metadata. Processing logic metadata (Operator or Task) can be associated with data definition metadata based on the process metadata (DAG), thereby enabling the generation of operation processing results based on the process metadata (DAG).

[0065] According to some embodiments, process metadata includes one or more node metadata, process input metadata, and process output metadata. Node metadata includes node input metadata, node output metadata, node logical metadata, and node association metadata. Each of the process input metadata and process output metadata includes identification information for data definition metadata. For each node metadata, each of the node input metadata and node output metadata includes identification information for data definition metadata, the node logical metadata includes identification information for processing logic metadata, and the node association metadata includes identification information for other node metadata.

[0066] According to some embodiments, since each of the process input metadata and process output metadata in the process metadata (DAG) includes identification information of data definition metadata, the process metadata (DAG) can be associated with the data definition metadata. Similarly, since each of the node input metadata and node output metadata in the node metadata (Node) includes identification information of data definition metadata, the node metadata (Node) can be associated with the data definition metadata, thereby achieving the association between the process metadata (DAG), node metadata (Node), and data definition metadata. Furthermore, since the node logical metadata includes identification information of processing logical metadata, and the node association metadata includes identification information of other node metadata, the node metadata (Node) can be interconnected.

[0067] According to some embodiments, generating an operation processing result based on operation definition metadata includes: in response to receiving a data operation request, performing the operation corresponding to the processing logic metadata based at least on the identification information of the processing logic metadata in the data operation request.

[0068] According to some embodiments, the data manipulation request includes processing logic metadata (Operator). As mentioned above, the processing logic metadata (Operator) includes information such as the input parameters, context, language, platform, and category of the data processing steps.

[0069] According to some embodiments, the client device 110 receives a data operation request sent by the user, and can perform the operation corresponding to the processing logic metadata (Operator) based at least on the identification information of the processing logic metadata (Operator) in the data operation request, making the operation of data simpler.

[0070] According to some embodiments, in response to receiving a dimension definition request, the dimension definition information in the dimension definition request is mapped to dimension definition metadata, wherein the dimension definition metadata includes identification information of data definition metadata.

[0071] According to some embodiments, users can utilize client device 110 to perform data exploration within the generated operation processing results. The scope of the explored data can be based on a user-defined OLAP model. The OLAP model includes dimension definition information, which may include data selection rules for that dimension. For example, for the country dimension, it can be specified that only Asian countries are selected. The dimension definition information may also include attributes of the dimension values, which may include: mType, indicating whether the dimension value is numeric or character; isTime, indicating whether the dimension value is a time dimension; physicalRef, indicating a reference to the dimension information's mapping relationship to physical storage; and cardinality, indicating the cardinality information of the dimension value.

[0072] According to some embodiments, in response to a received dimension definition request, the client device 110 maps the dimension definition information in the dimension definition request to the dimension definition metadata. Since the dimension definition metadata includes the identification information of the data definition metadata, the dimension definition metadata can be associated with the data definition metadata, thereby limiting the dimensions of the data definition metadata based on the dimension definition metadata.

[0073] According to some embodiments, the data definition metadata includes multiple data member metadata, and the dimension definition metadata also includes multiple dimension member metadata and dimension mapping metadata. The dimension mapping metadata defines the mapping between the multiple dimension member metadata and the multiple data member metadata.

[0074] According to some embodiments, data definition metadata includes multiple data member metadata, and dimension definition metadata includes multiple dimension member metadata. Since dimension definition metadata also includes dimension mapping metadata, the mapping between multiple dimension member metadata and multiple data member metadata can be defined through dimension mapping metadata. Here, the mapping between multiple dimension member metadata and multiple data member metadata can be multiple dimension member metadata corresponding to one data member metadata, one dimension member metadata simultaneously corresponding to multiple data member metadata, or one dimension member metadata corresponding to one data member metadata.

[0075] According to some embodiments, in response to receiving an indicator exploration request, the dimension definition metadata corresponding to the indicator exploration request is determined. Based on the indicator exploration information in the indicator exploration request, indicator calculation is performed on the dimension definition metadata corresponding to the indicator exploration request. Based on the indicator exploration information, the indicator definition metadata is mapped, wherein the indicator definition metadata includes the identification information of the data definition metadata.

[0076] According to some embodiments, users can explore data through client device 110. The results obtained after exploration can be saved as custom indicators and mapped to indicator metadata. The scope of the explored data is based on a user-defined OLAP model, and all events and attributes depend on the content of the OLAP model. During the exploration process, in addition to adding various filtering conditions to the attributes of individual events, grouping operations can be performed on one or more attributes. Multiple events can be used to construct complex calculation logic through formulas to form complex indicators.

[0077] In some implementations, users can set filter conditions, groupings, and formulas for data members and their corresponding dimensions. These data members, along with their corresponding dimensions, can form a query request.

[0078] According to some embodiments, information in the indicator exploration request is mapped to the indicator definition metadata (Indicator) based on indicator exploration information. Since the indicator definition metadata (Indicator) includes identification information of the data definition metadata, the indicator definition metadata (Indicator) can be associated with the data definition metadata, further simplifying the complexity of user-defined indicators.

[0079] According to some embodiments, the calculation of metrics based on the metric exploration information in the metric exploration request and the metric definition metadata corresponding to the metric exploration request includes:

[0080] Based on the dimension definition metadata corresponding to the indicator exploration request, determine the data storage metadata corresponding to the indicator exploration request;

[0081] Based on the data storage metadata corresponding to the indicator exploration request and the exploration condition information in the indicator exploration request, an indicator query statement is generated;

[0082] Execute the indicator query statement in the dimension definition metadata corresponding to the indicator exploration request; and

[0083] In response to the query results returned by the indicator query statement, the indicator exploration results are determined based on the query results and the calculation information in the indicator exploration request.

[0084] According to some embodiments, the indicator definition metadata (Indicator) may include:

[0085] - Query logic, such as the SQL body generated above;

[0086] - Parameters represent parameters required at runtime, such as time;

[0087] - Dataset, representing the dataset involved in calculating this metric;

[0088] - Implementation language, such as SQL;

[0089] - Platform, referring to the execution platform and environment, such as Spark;

[0090] - Result, including the form of the returned result, such as Table / Map / List; whether caching is required; and the cache expiration strategy, etc.

[0091] According to some embodiments, the implementation language of the exploration request can be any of the implementation languages ​​such as SQL and DQL.

[0092] According to some embodiments, the client device 110 can generate different types of indicator query statements based on the implementation language of the exploration request and the exploration condition information in the exploration request.

[0093] According to some embodiments, the client device 110 can execute an indicator query statement in the dimension definition metadata corresponding to the indicator exploration request; and determine and return the indicator exploration result based on the query result and the calculation information in the indicator exploration request.

[0094] In some implementations, if a metric query does not return a result, the metric exploration request is determined to be abnormal. Users can then resend the metric exploration request based on this result.

[0095] Figure 3 This is a flowchart illustrating a data management method 300 according to another exemplary embodiment. (e.g.) Figure 3 As shown, data management method 300 includes:

[0096] Step S301: Data model definition.

[0097] According to some embodiments, in response to receiving a data definition request, data logic metadata is mapped based on data logic information; physical storage metadata is mapped based on physical storage information; and data source metadata is mapped based on data source information, wherein the data source metadata includes identification information of physical storage metadata, and the data logic metadata includes identification information of data source metadata.

[0098] Step S302: Define data exploration metrics.

[0099] According to some embodiments, in response to receiving an operation definition request, process metadata is mapped based on process information, and processing logic metadata is mapped based on processing logic information, wherein the process metadata includes identification information of processing logic metadata and identification information of data definition metadata.

[0100] According to some embodiments, in response to receiving a dimension definition request, the dimension definition information in the dimension definition request is mapped to dimension definition metadata, wherein the dimension definition metadata includes identification information of data definition metadata. Based on the dimension definition metadata corresponding to the indicator exploration request, the data storage metadata corresponding to the indicator exploration request is determined; based on the data storage metadata corresponding to the indicator exploration request and the exploration condition information in the indicator exploration request, an indicator query statement is generated; based on the indicator exploration information, the indicator definition metadata is mapped, wherein the indicator definition metadata includes identification information of data definition metadata.

[0101] Step S303: Execute metadata service.

[0102] According to some embodiments, in response to a received request, metadata is mapped based on the information in the request.

[0103] Step S304: Perform data processing.

[0104] According to some embodiments, the raw data is cleaned and processed.

[0105] Step S305: Call the reporting system.

[0106] According to some implementations, the results of operations and indicator exploration can be presented through a reporting system. Reports can display the same indicator in different formats (charts) and sizes (large, medium, small). Users can also perform secondary explorations on the reports to obtain more specific or macro-level indicator data.

[0107] Step S306: Call the query engine.

[0108] According to some embodiments, an indicator query statement is executed in the dimension definition metadata corresponding to the indicator exploration request; and in response to the return of query results by the indicator query statement, the indicator exploration result is determined based on the query results and the calculation information in the indicator exploration request.

[0109] Step S307: Call the storage system.

[0110] According to some embodiments, the operation processing results, indicator exploration results, intermediate data during the operation processing, and data during the indicator exploration are stored.

[0111] Although the operations are depicted in the accompanying drawings in a specific order, this should not be construed as requiring that the operations be performed in the specific order shown or in chronological order, nor should it be construed as requiring that all the operations shown be performed to obtain the desired result.

[0112] According to embodiments of this disclosure, data definition information is mapped to data definition metadata, and operation definition information is mapped to operation definition metadata. At the same time, the operation definition metadata includes the identification information of the data definition metadata, which associates the operation metadata with the data definition metadata. This allows users to customize data models, thereby efficiently meeting users' personalized needs for data management and improving data management efficiency.

[0113] Figure 4 This is a schematic block diagram illustrating a data management apparatus 400 according to an exemplary embodiment. The apparatus 400 includes: a data definition module 401 configured to: in response to receiving a data definition request, map data definition metadata based on data definition information in the data definition request; an operation definition module 402 configured to: in response to receiving an operation definition request, map operation definition metadata based on operation definition information in the operation definition request, wherein the operation definition metadata includes identification information of the data definition metadata; and an operation processing module 403 configured to: generate an operation processing result based on the operation definition metadata.

[0114] It should be understood that Figure 4 The various modules of the device 400 shown can be connected to the reference. Figure 2 The steps in method 200 correspond to each other, and therefore, the operations, features, and advantages described above for method 200 also apply to apparatus 400 and its included modules. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0115] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action may include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module. For example, the operation definition module 402 / operation processing module 403 described above may be combined into a single module in some embodiments.

[0116] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 4 The various modules described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, in some embodiments, one or more of the data definition module 401, operation definition module 402, and operation processing module 403 can be implemented together in a System on Chip (SoC). The SoC may include an integrated circuit chip (which includes a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more components of other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0117] According to one aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0118] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0119] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0120] In the following text, combined with Figure 5 Illustrative examples describing such electronic devices, non-transitory computer-readable storage media, and computer program products. Figure 5 This is a block diagram illustrating an electronic device according to an embodiment of the present disclosure. For example, Figure 1 The server 120 and / or client device 110 shown may include an architecture similar to electronic device 500. The aforementioned electronic device may also be implemented wholly or at least partially by electronic device 500 or similar devices or systems.

[0121] Electronic device 500 can be a variety of different types of devices. Examples of electronic device 500 include, but are not limited to: desktop computers, server computers, laptop or netbook computers, mobile devices (e.g., tablet computers, cellular or other wireless phones (e.g., smartphones), notebook computers, mobile stations), wearable devices (e.g., glasses, watches), entertainment devices (e.g., entertainment appliances, set-top boxes communicatively coupled to a display device, game consoles), televisions or other display devices, automotive computers, and so on.

[0122] Electronic device 500 may include at least one processor 502, memory 504, communication interfaces(s) 506, display device 508, other input / output (I / O) devices 510, and one or more mass storage devices 512 capable of communicating with each other, such as via system bus 514 or other suitable connections.

[0123] Processor 502 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 502 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, processor 502 may be configured to acquire and execute computer-readable instructions stored in memory 504, mass storage device 512, or other computer-readable media, such as program code of operating system 516, program code of application program 518, program code of other program 520, etc.

[0124] Memory 504 and mass storage device 512 are examples of computer-readable storage media for storing instructions executed by processor 502 to perform the various functions described above. For example, memory 504 may generally include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, mass storage device 512 may generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Both memory 504 and mass storage device 512 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by processor 502 as a specific machine configured to perform the operations and functions described in the examples herein.

[0125] Multiple programs can be stored on mass storage device 512. These programs include operating system 516, one or more applications 518, other programs 520, and program data 522, and they can be loaded into memory 504 for execution.

[0126] Although Figure 5 The modules 516, 518, 520, and 522, or portions thereof, are illustrated as being stored in memory 504 of electronic device 500; however, modules 516, 518, 520, and 522 may be implemented using any form of computer-readable medium accessible by computer device 500. As used herein, “computer-readable medium” includes at least two types of computer-readable media: computer-readable storage media and communication media.

[0127] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD, or other optical storage devices, magnetic cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computer device. In contrast, communication media can embody computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms. Computer-readable storage media as defined herein do not include communication media.

[0128] One or more communication interfaces 506 are used for exchanging data with other devices, such as via a network, direct connection, etc. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), wired or wireless (such as IEEE 802.11 Wireless LAN (WLAN)) wireless interface, Wi-MAX interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth. TM Interfaces include near-field communication (NFC) interfaces. Communication interface 506 facilitates communication across various network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. Communication interface 506 can also provide communication with external storage devices (not shown) such as storage arrays, network-attached storage, storage area networks, etc.

[0129] In some examples, a display device 508, such as a monitor, may be included for displaying information and images to the user. Other I / O devices 510 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and so on.

[0130] The technologies described herein can be supported by these various configurations of electronic device 500, and are not limited to specific examples of the technologies described herein. For example, the functionality can also be implemented wholly or partially on a “cloud” using a distributed system. A cloud includes and / or represents a platform for resources. The platform abstracts the underlying functionality of the cloud’s hardware (e.g., servers) and software resources. Resources may include applications and / or data that can be used when performing computational processing on a server remote from electronic device 500. Resources may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks. The platform can abstract resources and functionality to connect electronic device 500 to other computer devices. Therefore, the implementation of the functionality described herein can be distributed throughout the cloud. For example, the functionality can be implemented partly on electronic device 500 and partly through a platform that abstracts the functionality of the cloud.

[0131] Although this disclosure has been described and illustrated in detail in the accompanying drawings and the foregoing description, such description and illustration should be considered illustrative and suggestive, not restrictive; this disclosure is not limited to the disclosed embodiments. By studying the drawings, the disclosure, and the appended claims, those skilled in the art will be able to understand and implement variations of the disclosed embodiments in practice with respect to the claimed subject matter. In the claims, the word "comprising" does not exclude other elements or steps not listed, the indefinite article "a" or "an" does not exclude a plurality, the term "a plurality" means two or more, and the term "based on" should be interpreted as "at least partially based on". The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be beneficial.

Claims

1. A data management method, comprising: In response to receiving a data definition request, data definition metadata is mapped based on the data definition information in the data definition request. The data definition information includes data logic information and data storage information, and the data definition metadata includes data logic metadata and data storage metadata. The mapping of data definition metadata based on the data definition information in the data definition request includes: Based on the data storage information, the data storage metadata is mapped. The data storage information includes data source information and physical storage information, and the data storage metadata includes data source metadata and physical storage metadata. Furthermore, the mapping of the data storage metadata based on the data storage information includes: Based on the physical storage information, the physical storage metadata is mapped; and Based on the data source information, the data source metadata is mapped. Wherein, the data source metadata includes the identification information of the physical storage metadata, and the data logical metadata contains the identification information of the data source metadata; and Based on the data logic information, the data logic metadata is mapped, wherein the data logic metadata includes the identification information of the data storage metadata. The operation definition metadata includes the identification information of the data logic metadata; In response to receiving an operation definition request, the operation definition metadata is mapped based on the operation definition information in the operation definition request. The operation definition metadata includes identification information of the data definition metadata. The operation definition information includes process information and processing logic information. The operation definition metadata includes process metadata and processing logic metadata. Furthermore, the mapping of the operation definition metadata based on the operation definition information in the operation definition request includes: In response to receiving the operation definition request, the process metadata is mapped based on the process information, and Based on the processing logic information, the processing logic metadata is mapped. The process metadata includes the identification information of the processing logic metadata and the identification information of the data definition metadata. The process metadata also includes one or more node metadata, process input metadata, and process output metadata. The node metadata includes node input metadata, node output metadata, node logic metadata, and node association metadata. Each of the process input metadata and the process output metadata includes the identification information of the data definition metadata, and For each node metadata, each of the node input metadata and the node output metadata includes the identification information of the data definition metadata; the node logical metadata includes the identification information of the processing logic metadata; and the node association metadata includes the identification information of other node metadata. Based on the operation definition metadata, the operation processing result is generated.

2. The method as described in claim 1, wherein, The generation of operation processing results based on the operation definition metadata includes: In response to receiving a data operation request, the operation corresponding to the processing logic metadata is performed, at least based on the identification information of the processing logic metadata in the data operation request.

3. The method of claim 1, further comprising: In response to receiving a dimension definition request, the dimension definition information in the dimension definition request is mapped to the dimension definition metadata, wherein the dimension definition metadata includes the identification information of the data definition metadata.

4. The method of claim 3, wherein, The data definition metadata includes multiple data member metadata, and the dimension definition metadata also includes: Multi-dimensional member metadata; and Dimension mapping metadata, wherein the dimension mapping metadata defines the mapping between the plurality of dimension member metadata and the plurality of data member metadata.

5. The method of claim 3, further comprising: In response to receiving an indicator exploration request, determine the dimension definition metadata corresponding to the indicator exploration request; Based on the indicator exploration information in the indicator exploration request, the indicator is calculated on the dimension definition metadata corresponding to the indicator exploration request; as well as Based on the indicator exploration information, the indicator definition metadata is mapped, wherein the indicator definition metadata includes the identification information of the data definition metadata.

6. The method of claim 5, wherein, The step of calculating the metrics based on the metric exploration information in the metric exploration request and the metric definition metadata corresponding to the metric exploration request includes: Based on the dimension definition metadata corresponding to the indicator exploration request, determine the data storage metadata corresponding to the indicator exploration request; Based on the data storage metadata corresponding to the indicator exploration request and the exploration condition information in the indicator exploration request, an indicator query statement is generated; Execute the indicator query statement in the dimension definition metadata corresponding to the indicator exploration request; and In response to the return of query results by the indicator query statement, the indicator exploration result is determined based on the query results and the calculation information in the indicator exploration request.

7. The method of claim 6, further comprising: If the query statement for the indicator does not return any results, the indicator exploration request is determined to be abnormal.

8. A data management device, comprising: A data definition module is configured to: in response to receiving a data definition request, map data definition metadata based on the data definition information in the data definition request, wherein the data definition information includes data logic information and data storage information, and the data definition metadata includes data logic metadata and data storage metadata; and the data definition module includes: A data storage module is configured to: map data storage metadata based on the data storage information, wherein the data storage information includes data source information and physical storage information, and the data storage metadata includes data source metadata and physical storage metadata; and the data storage module includes: The physical storage module is configured to: map the physical storage metadata based on the physical storage information; and The data source mapping module is configured to map the data source metadata based on the data source information. Wherein, the data source metadata includes the identification information of the physical storage metadata, and the data logical metadata contains the identification information of the data source metadata; and The data logic mapping module is configured to: map the data logic metadata based on the data logic information, wherein the data logic metadata includes the identification information of the data storage metadata. The operation definition metadata includes the identification information of the data logic metadata; An operation definition module is configured to: in response to receiving an operation definition request, map operation definition metadata based on the operation definition information in the operation definition request, wherein the operation definition metadata includes identification information of the data definition metadata, wherein the operation definition information includes process information and processing logic information, and the operation definition metadata includes process metadata and processing logic metadata; and the operation definition module includes: The process mapping module is configured to: in response to receiving the operation definition request, map the process metadata based on the process information, and The processing logic mapping module is configured to: map the processing logic metadata based on the processing logic information. The process metadata includes the identification information of the processing logic metadata and the identification information of the data definition metadata. The process metadata also includes one or more node metadata, process input metadata, and process output metadata. The node metadata includes node input metadata, node output metadata, node logic metadata, and node association metadata. Each of the process input metadata and the process output metadata includes the identification information of the data definition metadata, and For each node metadata, each of the node input metadata and the node output metadata includes the identification information of the data definition metadata; the node logical metadata includes the identification information of the processing logic metadata; and the node association metadata includes the identification information of other node metadata. The operation processing module is configured to generate operation processing results based on the operation definition metadata.

9. An electronic device, comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-7.

11. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method of any one of claims 1-7.

Images